CMPU 102, Fall 2005 |
Lecture 16 |

The Fibonacci Sequence may be defined recursively by the following formula:

f(0) = 0

f(1) = 1

f(n) = f(n-2) + f(n-1)for n>1

This leads to a very simple recursive implementation of a
method to compute the *n*'th member of the Fibonacci sequence:

public static int fib(int n) { if (n < 0) throw new IllegalArgumentException(); if (n == 0 || n == 1) return n; return fib(n - 2) + fib(n - 1); }

This method is *correct*, in the sense that it will compute the
correct result. However, it is not *efficient* because it
violates one of the rules of recursion: specifically, it recomputes
the answer to subproblems more than once.

Here is the "tree" of recursive calls that will result from calling
the **fib** method with the argument 5:

Note that the subproblem fib(3) is computed twice, fib(2) is computed three
times, fib(1) is computed 5 times, and fib(0) is computed 3 times.
As n increases, the size of the tree (the number of recursive calls required)
grows *exponentially*. Exponential functions grow very, very fast;
so fast that algorithms with exponential running time are generally
intractable, even for relatively small input sizes. In an algorithm with exponential running
time, even a small input may require a very large number of steps to solve.

Here is a Java program that computes members of the Fibonacci sequence recursively using the naive approach:

public class Fib { public static void main(String[] args) { int n = Integer.parseInt(args[0]); long begin = System.currentTimeMillis(); int result = fib(n); long end = System.currentTimeMillis(); System.out.println("fib("+n+")=" + fib(n) + "[" + (end-begin) + " ms]"); } public static int fib(int n) { if (n == 0 || n == 1) return n; else return fib(n-2) + fib(n-1); } }

If you try running this program, you will notice that the number of milliseconds required to compute the requested Fibonacci number grows very quickly. On my laptop, the 40th Fibonacci number takes about 6 seconds to compute, and the 41st takes 9.5 seconds! Here is a graph showing the number of milliseconds requried for n=1 to n=41:

Obviously, this is not a good implementation of **fib**.

*Dynamic programming* is a technique for avoiding recomputing the
answers to subproblems in a recursive algorithm. The idea
is very simple: record the answers to subproblems in a table.
Each time an answer to a subproblem is needed, consult the table.
If no answer exists in the table, perform the computation *and
save the answer in the table*. Future occurrences of the
subproblem can be "solved" by just loading the answer from the table.
The technique of storing answers to subproblems is often
referred to as *memoization*.

Here is a simple dynamic programming version of the same program.
Answers to subproblems are saved in an array called **memo**.

public class FibMemo { public static void main(String[] args) { int n = Integer.parseInt(args[0]); long begin = System.currentTimeMillis(); int result = fib(n); long end = System.currentTimeMillis(); System.out.println("fib("+n+")=" + fib(n) + "[" + (end-begin) + " ms]"); } public static int fib(int n) { return fibMemo(n, new int[n + 1]); } public static int fibMemo(int n, int[] memo) { if (n == 0 || n == 1) return n; int answer = memo[n]; if (answer == 0) { answer = fibMemo(n-2, memo) + fibMemo(n-1, memo); memo[n] = answer; } return answer; } }

Because each subproblem is computed only once, this version of
**fib** has linear (O(n)) running time.

This dynamic programming implementation is a trivial modification of the naive recursive implementation. Another way to apply dynamic programming is to start with the smallest subproblem, and then to iteratively work up towards larger problems, until the overall problem is solved. This kind of solution can be written using a loop of the general form

for j = 1 to n memo[j] = solve subproblem j return memo[n]

The step that reads "solve subproblem j" may be implemented by looking up the answers to previously solved subproblems.

A tree is a data structure consisting of *nodes* and *edges*.
There is a single *root node* of the tree. Every node in the
tree has 0 or more *children*: an edge exists between the *parent*
node and each child node. Each node has exactly one parent,
except the root, which does not have a parent. Another property
of trees is that there are no *cycles*: by following
edges from parent to child, it is not possible to visit the
same node more than once. Nodes which have no children are
called *leaf nodes*.

Example tree:

In this example, the root is colored red and the leaves are colored green.

A tree is a *recursive data structure* because each child of a node
in the tree is a tree in its own right:

Because of this property, many of the important algorithms to access and manipulate trees are most easily expressed using recursion.

When representing a tree which will have a small, fixed number of
children per node, the edges of the tree are generally
represented as a direct reference from the parent node to
the child node. For example, trees with at most two
children per node are called *binary trees*. Here is
one way that a binary tree might be represented in Java:

class BinaryTreeNode<E> { E payload; BinaryTreeNode<E> left, right; } class BinaryTree<E> { BinaryTreeNode<E> root; }

Each node in the tree is an instance of the **BinaryTreeNode** class.
References to a node's children are stored in the **left** and
**right** fields. The **BinaryTree** class simply
stores a reference to the root node of the tree.

For trees in which the nodes may have a large number of children,
it is often better to represent the tree using a *first child/next sibling*
representation. In this representation, each node contains a link
to its first child and its next *sibling*:

The solid and dashed black lines represent the tree edges.
The solid lines (black and blue) represent the actual direct links
from parent to child or sibling to sibling.
The advantage of this representation is that a node can have
any number of children using only two fields per node.
The disadvantage is that adding children or finding the
*n*'th child requires more work, since the children of
each node are essentially stored in a singly-linked list.

Usually, the empty tree is represented using the value **null**.
Therefore, if a child or sibling is **null**, that means that the
child or sibling does not exist.

Sometimes it is advantageous to use a special *sentinel* node
to represent the empty tree. Use of a sentinel node can
simplify the implementation of some of the balanced search tree
algorithms we will see later.

A node's *ancestors* include the node itself, its parent,
its parent's parents, and so forth up to the root of the tree.
Its *descendents* include the node itself, its children,
its children's children, and so forth down to the leaves.
It should be obvious that the root of a tree is the ancestor of
every node in the tree.

The *proper ancestors* and *proper descendents* of a node
are defined the same way, except that they do not include the node
itself.

A *traversal* of a tree is an algorithm that *visits* each
node in the tree exactly once. Visiting a node just means that
some action will be performed on the node; the exact nature of the
visitation will generally be performed using a *functor*.
(Functors were considered in Lecture 7.)

Traversals are generally specified (and often implemented) recursively. Two common traversals are pre-order and post-order:

both pre and post order traversals start at the rootpreorderTraversal(node) { if (node does not exist) return; visit(node); for each child of node preorderTraversal(child); } postorderTraversal(node) { if (node does not exist) return; for each child of node postorderTraversal(child); visit(node); }

A level-order traversal visits each *level* of the tree
in order: first the root, then each child of the root, then
each grand-child, etc. down to the lowest level of the tree.
It is defined using a queue:

q = new queue of nodes q.enqueue(root); while (q is not empty) { node = q.dequeue(); visit(node); for each child of node q.enqueue(child) }

This traversal is also known as a *breadth-first* traversal.

The *in-order* traversal is a special traversal defined only for
binary trees. In a binary tree, each node has a left subtree and
a right subtree (each or both of which may be empty).

in-order traversal starts at the rootinorderTraversal (node) { if (node does not exist) return; inorderTraversal(left child of node); visit(node); inorderTraversal(right child of node); }