Generative AI and The Elusive Turing Machine

In simple terms, a Turing Machine is an abstract computational model that performs computations using one algorithm by reading and writing to an infinite tape or memory. A Turing machine’s compute capacity is improved by improving the instructions given to the algorithm and not the hardware it sits on.

Implementing a Turing Machine on the existing von Neumann Architecture Machines has been impossible till now because they do not have infinite Memory. A von Neumann Machine’s compute capacity is improved by improving the hardware clock speed and memory capacity. All of today’s computers (laptops, phones, servers etc) are von Neumann Machines as their performance is improved by altering their capacity to compute.

von Neumann stretched Turing’s machine concept to support more than one algorithm in shared, centralized, unprotected physical memory storage. This led away from Alonzo Church’s Lambda Calculus and functional programming. Lambda calculus is a formal system in mathematical logic for expressing computation based on function abstraction and application using variable binding and substitution. It is a universal model of computation that can be used to simulate any Turing machine and is what is commonly used to create logic in software programs by developers.

All software is basically an attempt to emulate a Turing Machine on von Neumann Hardware with finite memory using Lambda Calculus.

To Improve our comsumption of computing capacity, we need to be creating creating Virtual Functions (VFs) and agents, instead of Virtual Machines (VMs). Our achievement of a true Turing Machine will be an illusion for as long as we use VMs.

Can Generative AI create a true Turing Machine?

The rapid adoption and interest in Generative AI systems trained through Large Language Models has caught many off-guard with many a skeptic saying that its too early to tell how generative AI use cases will pan out.

Other than the perceived near future improvement in productivity, one of the biggest benefits I see from Generative AI is the ability to use it to simulate a true Turing Machine. I will explain how below.

Our best attempts so far in creating a Turing Machine have been through the use of software programming languages that are all different implementations of of Lambda Calculus (C#, Python, Java, etc). Software program code has no infinite memory from hardware and this limits its ability to closely emulate Turing Machines, whose capacity to compute is only limited by the instruction quality. One can create a VF such as a Gen AI GPTs to perform computing as Turing foresaw.

In Generative AI, the output quality is a direct function of the quality of instruction you give it. This realization has led to the growth of the field known as Prompt Engineering: where we improve a VF or GPTs outcomes based on how well we structure our instructions to it. In short, improve computing by instruction and not by resources thrown at it.

The fact that Generative AI can also understand computer code, means it can also virtually emulate the outcome of the code without actually running/compiling it in von Neumann hardware.

This makes it possible to create a Generative AI tool/system/thing/assistant that, with precisely engineered prompts/inputs, can emulate a true Turing Machine. The difference with this AI emulated Turing Machine and the one made directly from Software code is that the AI Machine uses threads/sessions/context as memory whereas the software code uses actual physical memory.

We’ve done well in as far as processing data using von Neumann Machines are concerned. However, to process information (not data) efficiently, we need to use a Turing machine ( i.e., you need AI (a Turing Machine) to gain insights from data)

The recent launch of ChatGPT-4Turbo with 128K of context (up from 2K a year ago), 300pages of prompt text input capacity, and advances in prompt engineering means that we can now increase computing outcomes without a massive and corresponding increasing in hardware capability, bringing us close to creating true Turing Machines. This was Allan Turing’s dream