YCP Logo Lecture 22: Turing Machines, Decidability, Halting Problem

Turing Machines

A Turing machine (TM) is the most powerful kind of automaton that we will discuss.  In fact, Turing machines are capable of solving any problem that can be solved by computation.  (There are some problems that are not solvable by a computation, as we will see shortly.)

Turing machines are named after Alan Turing, the mathematician who invented them.

Turning machines are similar to PDAs: they consist of states and transitions, and use an infinite tape for storage.  Unlike PDAs, however, the use of the tape by a Turing machine is not limited to pushing and popping symbols: it can move the tape either left or right after writing a symbol on the tape.  Unlike a PDA, the tape is used as BOTH the input string and the temporary storage.  In addition, when the Turing machine terminates we can consider the contents of the tape to be output.  In this way, a Turning machine is more than a recognizer for strings in a language.

A Turing machine, like a PDA, has a "tape head" indicating the current location on the tape that the Turing machine is looking at.  The symbol underneath the tape head is used to determine which transition will be followed.

Each transition in the state diagram of a Turing machine is labeled with three symbols

(i, o, d)

"i" is an input symbol.  The transition will be taken if the symbol underneath the tape head matches this symbol.

"o" is an output symbol.  If the transition is taken, then this symbol is written to the location underneath the tape head, overwriting whatever symbol was there previously.

"d" is a direction: L (left) or R (right).  If the transition is taken the tape head is moved one position in the specified direction.

The Turing machine completes its computation if it reaches a state labeled "Halt".

Turing machine that can recognize the language anbncn

Here is a description of a Turing machinen that can recognize the language  anbncn, which we have noted is not a context-free language.

The Turing machine will operate by scanning from left to right, replacing one set of a,b,c symbols with upper case symbols A,B,C.  When it reaches the Δ symbol marking the end of the string, it will "rewind" from right to left to work on the next set of a,b,c symbols.  If no more a symbols remain in the string, it will scan from left to right to verify that no more b or c symbols remain.  If the verification succeeds (no b or c symbols are encountered), then the original string has been accepted as a member of the language.

Example: we will start out with a tape that looks like this, representing the string "aabbcc".  (The tape head is positioned at the underlined symbol.)

aabbccΔ

Here is how the TM progresses:

AabbccΔ

AabbccΔ

AaBbccΔ

AaBbc

AaBbCcΔ

AaBbCcΔ

("rewind" by moving left until the Turing machine encounters an A symbol)

AaBbCcΔ    (move right)

AaBbCcΔ    (replace one more a,b,c set with A,B,C)

AABbCcΔ

AABbCcΔ

AABBC

AABBCcΔ

AABBCCΔ

("rewind" by moving left until the Turing machine encounters an A symbol)

AABBCCΔ    (move right)

AABBCCΔ    (because a B was encountered, there are no more a symbols, so we verify that no b or c symbols remain)

AABBCCΔ

AABBC

AABBCCΔ

AABBCCΔ    (Halt)

If at any step in the process the Turing machine encounters an unexpected symbol, then it does not halt (and is considered to have rejected the original input string.)

Here is a state diagram of this Turing Machine:


Turing Completeness

It may seem surprising, but Turing machines have been shown to be at least as powerful as every "reasonable" known model of computation.  For example, if we wanted to we could translate a C++ or Java program into a Turing machine.

Any model of computation that can be translated into an equivalent Turing machine is said to be Turing complete.

Decidability and the Halting Problem

The problem of decidability may be stated roughly as follows: is it possible for an algorithm to correctly answer a yes/no question for all possible input?

For example:

Is there an algorithm that will tell us whether or not two arbitrary DFAs recognize the same language?

Is there an algorithm that will tell us whether or not two arbitrary context-free grammars generate the same language?

Given an arbitrary Turing machine and initial tape, will the Turing machine reach the Halt state?

A problem is decidable if such an algorithm exsits.  The first problem (deciding whether or not two DFAs are equivalent) is decidable.  The second two problems are undecidable: there is no algorithm that can correctly answer these questions for all possible input.  The last problem (whether or not a Turing machine will reach the Halt state for some initial tape) is known as the Halting Problem, and is a very famous problem in the theory of computation.

The Halting Problem is Undecidable

Here is a sketch of how you can prove that the Halting problem is undecidable.  (This is a summary of the proof presented in the Wikipedia article on the Halting Problem: see the section entitiled "Sketch of proof".)

Assume that a TM capable of solving the halting problem exists.  Call this TM halt.  It takes, as input, the encoded representation of a TM and an input tape on which the encoded TM will run.  Let's call this input pair (p,i): p for "program" (the encoded TM) and i for "input" (the input tape).  If input TM p will halt on input i, then halt halts and outputs "true".  If input TM p will not halt on i, then halt halts and outputs "false".  So, we can view halt as a function that takes parameters p and i and produces the output "true" or "false" depending on whether or not TM p halts on input i.


Based on halt, we can trivially construct a new TM, which we can call trouble, which will work as follows.  It first duplicates the entire input tape q, creating two copies of the original input tape.  It then runs halt, using one copy of the original input q as parameter p and the other copy of the original input q as parameter i.  If halt(q,q) outputs "false", then trouble TM halts.  If halt(q,q) outputs "true", then trouble goes into an infinite loop (and thus does not halt).


Any TM can be encoded as an initial tape.  So, let's assume that the encoding of trouble as a tape is called t.

Consider what will happen when trouble is executed with t as its input tape.  Does trouble halt?

If trouble halts, that means that halt(t,t) answered "false".  However, that means that trouble does not halt when given t as input.

If trouble does not halt, that means that halt(t,t) answered "true".  However, that means that trouble does halt when given t as input.

In either case, halt gave the wrong answer.  Therefore, given any TM that claims to solve the halting problem, it is possible to construct an program/input pair for which it will answer incorrectly.  So, any claim that a particular TM solves the Halting problem can be proved false, meaning that the Halting Problem is undecidable.

The undecidability of the halting problem has several important consequences.  The main consequence is that, in general, it is impossible to predict exactly what a program will do when it is executed.  This problem may be stated as follows:

Nontriivial properties of programs are undecidable.