CMPU 102, Spring 2006 Lecture 15

Dynamic Programming, Trees, Tree Traversals

Dynamic Programming

The Fibonacci Sequence may be defined recursively by the following formula:

f(0) = 0
f(1) = 1
f(n) = f(n-2) + f(n-1)    for n>1

This leads to a very simple recursive implementation of a method to compute the n'th member of the Fibonacci sequence:

public static int fib(int n) {
    if (n < 0) throw new IllegalArgumentException();
    if (n == 0 || n == 1)
        return n;
    return fib(n - 2) + fib(n - 1);
}

This method is correct, in the sense that it will compute the correct result.  However, it is not efficient because it violates one of the rules of recursion: specifically, it recomputes the answer to subproblems more than once.

Here is the "tree" of recursive calls that will result from calling the fib method with the argument 5:

Note that the subproblem fib(3) is computed twice, fib(2) is computed three times, fib(1) is computed 5 times, and fib(0) is computed 3 times.  As n increases, the size of the tree (the number of recursive calls required) grows exponentially.  Exponential functions grow very, very fast; so fast that algorithms with exponential running time are generally intractable, even for relatively small input sizes.  In an algorithm with exponential running time, even a small input may require a very large number of steps to solve.

Here is a Java program that computes members of the Fibonacci sequence recursively using the naive approach:

public class Fib {
  public static void main(String[] args) {
    int n = Integer.parseInt(args[0]);
    long begin = System.currentTimeMillis();
    int result = fib(n);
    long end = System.currentTimeMillis();
    System.out.println("fib("+n+")=" + fib(n) + "[" + (end-begin) + " ms]");
  }

  public static int fib(int n) {
    if (n == 0 || n == 1)
      return n;
    else
      return fib(n-2) + fib(n-1);
  }
}

If you try running this program, you will notice that the number of milliseconds required to compute the requested Fibonacci number grows very quickly.  On my laptop, the 40th Fibonacci number takes about 6 seconds to compute, and the 41st takes 9.5 seconds!  Here is a graph showing the number of milliseconds required for n=1 to n=41:

Obviously, this is not a good implementation of fib.

Dynamic programming is a technique for avoiding recomputing the answers to subproblems in a recursive algorithm.  The idea is very simple: record the answers to subproblems in a table.  Each time an answer to a subproblem is needed, consult the table.  If no answer exists in the table, perform the computation and save the answer in the table.  Future occurrences of the subproblem can be "solved" by just loading the answer from the table.  The technique of storing answers to subproblems is often referred to as memoization.

Here is a simple dynamic programming version of the same program.  Answers to subproblems are saved in an array called memo.

public class FibMemo {
  public static void main(String[] args) {
    int n = Integer.parseInt(args[0]);
    long begin = System.currentTimeMillis();
    int result = fib(n);
    long end = System.currentTimeMillis();
    System.out.println("fib("+n+")=" + fib(n) + "[" + (end-begin) + " ms]");
  }

  public static int fib(int n) {
    return fibMemo(n, new int[n + 1]);
  }

  public static int fibMemo(int n, int[] memo) {
    if (n == 0 || n == 1)
      return n;

    int answer = memo[n];
    if (answer == 0) {
      answer = fibMemo(n-2, memo) + fibMemo(n-1, memo);
      memo[n] = answer;
    }

    return answer;
  }
}

Because each subproblem is computed only once, this version of fib has linear (O(n)) running time.

This dynamic programming implementation is a trivial modification of the naive recursive implementation.  Another way to apply dynamic programming is to start with the smallest subproblem, and then to iteratively work up towards larger problems, until the overall problem is solved.  This kind of solution can be written using a loop of the general form

for j = 1 to n
    memo[j] = solve subproblem j

return memo[n]

The step that reads "solve subproblem j" may be implemented by looking up the answers to previously solved subproblems.

Trees

A tree is a data structure consisting of nodes and edges.  There is a single root node of the tree.  Every node in the tree has 0 or more children: an edge exists between the parent node and each child node.  Each node has exactly one parent, except the root, which does not have a parent.  Another property of trees is that there are no cycles: by following edges from parent to child, it is not possible to visit the same node more than once.  Nodes which have no children are called leaf nodes.

Example tree:

In this example, the root is colored red and the leaves are colored green.

A tree is a recursive data structure because each child of a node in the tree is a tree in its own right:

Because of this property, many of the important algorithms to access and manipulate trees are most easily expressed using recursion.

Representing trees

When representing a tree which will have a small, fixed number of children per node, the edges of the tree are generally represented as a direct reference from the parent node to the child node.  For example, trees with at most two children per node are called binary trees.  Here is one way that a binary tree might be represented in Java:

class BinaryTreeNode<E> {
    E payload;
    BinaryTreeNode<E> left, right;
}

class BinaryTree<E> {
    BinaryTreeNode<E> root;
}

Each node in the tree is an instance of the BinaryTreeNode class.  References to a node's children are stored in the left and right fields.  The BinaryTree class simply stores a reference to the root node of the tree.

For trees in which the nodes may have a large number of children, it is often better to represent the tree using a first child/next sibling representation.  In this representation, each node contains a link to its first child and its next sibling:

The solid and dashed black lines represent the tree edges.  The solid lines (black and blue) represent the actual direct links from parent to child or sibling to sibling.  The advantage of this representation is that a node can have any number of children using only two fields per node.  The disadvantage is that adding children or finding the n'th child requires more work, since the children of each node are essentially stored in a singly-linked list.

Representing the empty tree

Usually, the empty tree is represented using the value null.  Therefore, if a child or sibling is null, that means that the child or sibling does not exist.

Sometimes it is advantageous to use a special sentinel node to represent the empty tree.  Use of a sentinel node can simplify the implementation of some of the balanced search tree algorithms we will see later.

Tree terminology

A node's ancestors include the node itself, its parent, its parent's parents, and so forth up to the root of the tree.  Its descendents include the node itself, its children, its children's children, and so forth down to the leaves.  It should be obvious that the root of a tree is the ancestor of every node in the tree.

The proper ancestors and proper descendents of a node are defined the same way, except that they do not include the node itself.

Tree Traversals

A traversal of a tree is an algorithm that visits each node in the tree exactly once.  Visiting a node just means that some action will be performed on the node; the exact nature of the visitation will generally be performed using a functor.  (Functors were considered in Lecture 7.)

Traversals are generally specified (and often implemented) recursively.  Two common traversals are pre-order and post-order:

both pre and post order traversals start at the root

preorderTraversal(node) {
    if (node does not exist) return;
    visit(node);
    for each child of node
        preorderTraversal(child);
}

postorderTraversal(node) {
    if (node does not exist) return;
    for each child of node
        postorderTraversal(child);
    visit(node);
}

A level-order traversal visits each level of the tree in order: first the root, then each child of the root, then each grand-child, etc. down to the lowest level of the tree.  It is defined using a queue:

q = new queue of nodes
q.enqueue(root);
while (q is not empty) {
    node = q.dequeue();
    visit(node);
    for each child of node
        q.enqueue(child)
}

This traversal is also known as a breadth-first traversal.

The in-order traversal is a special traversal defined only for binary trees.  In a binary tree, each node has a left subtree and a right subtree (each or both of which may be empty).

in-order traversal starts at the root

inorderTraversal (node) {
    if (node does not exist) return;
    inorderTraversal(left child of node);
    visit(node);
    inorderTraversal(right child of node);
}