Programming Languages: Chapter 13: Control



Outline


Recursive control behavior



  • for instance, consider the following definition of a factorial function which naturally reflects the mathematical definition of factorial (i.e., n!=n*(n-1)!
    (define factorial
       (lambda (n)
          (cond
             ((zero? n) 1)
             (else (* n (factorial (- n 1)))))))
    
    ;; depiction of the control context:
    (factorial 5)
    (* 5 (factorial 4))
    (* 5 (* 4 (factorial 3)))
    (* 5 (* 4 (* 3 (factorial 2))))
    (* 5 (* 4 (* 3 (* 2 (factorial 1)))))
    (* 5 (* 4 (* 3 (* 2 (* 1 (factorial 0))))))
    (* 5 (* 4 (* 3 (* 2 (* 1 1)))))
    (* 5 (* 4 (* 3 (* 2 1))))
    (* 5 (* 4 (* 3 2)))
    (* 5 (* 4 6))
    (* 5 24)
    120
    
    )
  • requires an ever-increasing amount of memory to store the control context as the depth of the recursion increases
  • question: can we define a version of factorial which does not cause the control context to grow?
  • answer: yes


Tail recursion



  • for instance,
    ;; a is called an `accumulator'
    (define factorial
       (lambda (n)
          (letrec ((fact
             (lambda (n a)
                (cond
                   ((zero? n) a)
                   (else (fact (- n 1) (* n a)))))))
             (fact n 1))))
    
    ;; depiction of the control context:
    ;; the world is flat!
    (factorial 5)
    (fact 5 1)
    (fact 4 5)
    (fact 3 20)
    (fact 2 60)
    (fact 1 120)
    (fact 0 120)
    120
    
  • when fact calls itself it does so at the tail end of the call to fact
  • such an invocation is referred to as a tail call
  • a call is a tail call if there is no promise to do anything with the returned value but return from the lambda expression
  • this version of factorial is said to use tail recursion
  • code using tail recursion only requires a bounded amount of memory to store the control context
  • contrast with first version where the recursive call is in an operand position
  • important principle: ``a procedure call that does not grow control context is the same as a jump'' [EOPL2] p. 268



  • tail calls make recursion iterative and thus efficient
  • now the code no longer naturally reflects the mathematical definition of factorial


Continuations

  • one of the most powerful concepts in all of programming languages
  • a continuation is a promise to do something
  • a continuation represents the pending control context
  • a continuation is a pair of (program counter, environment) pointers
  • thunks and continuations are both closures


Continuation-passing style

  • make all recursive calls tail calls by packaging up any work remaining after the would be recursive call into an explicit continuation and passing it to the recursive call
  • make the implicit continuation capture by call/cc explicit by packaging it as an additional procedural argument passed in every call
  • this is called continuation-passing style
  • for instance,
    (define add
       (lambda (x y k)
          (k (+ x y))))
    
    (define multiply
       (lambda (x y k)
          (k (* x y))))
    
    ;; non-CPS
    (* 3 (+ 1 2))
    
    ;; CPS
    (add 1 2 (lambda (rtnval) (multiply 3 rtnval (lambda (x) x))))
    
  • re-wrote product in CPS
  • what does CPS give that call/cc does not?
    • a procedure can accept multiple continuations (e.g., success and failure continuations passed to the integer-divide procedure)
    • now the continuation can take more than one argument (because we are defining it) (e.g., the success continuation passed to integer-divide accepts two values; success was bound to list at the time of the call)
  • any program written using call/cc can be mechanically re-written in CPS without call/cc:
    ``Unfortunately, the procedures resulting from the conversion process are often difficult to understand. The argument that [first-class] continuations need not be added to the Scheme language is factually correct. It has as much validity as the statement that the names of the formal parameters can be chosen arbitrarily. And both of these arguments have the same basic flaw: the form in which a statement is written can have a major impact on how easily a person can understand the statement. While understanding that the language does not inherently need any extensions to support programming using [first-class] continuations, the Scheme community nevertheless chose to add one operation [(i.e., call/cc)] to the language to ease the chore'' [MS].
  • CPS makes recursion as efficient as iteration
  • CPS transformation, now we can have our cake and eat it too!


Full circle




    call-by-name(delay ...)(force ...)
    continuations(call/cc ...)(k ...)
    coroutinessuspend/yieldresume

    problems side-effects cause for call-by-name akin to synchronization problem for shared memory [PLPP] p. 512


First-class continuations

  • all languages manipulate continuations internally, but only some (e.g., Scheme, Ruby, give the programmer first-class access to them)
  • the Scheme function call-with-current-continuation allows the programmer to capture the continuation at any point in a program, store it in a variable, and then use it to replace a continuation elsewhere in a program
  • call-with-current-continuation is canonically abbreviated call/cc (i.e., (define call/cc call-with-current-continuation))
  • note: the letcc construct used in The Seasoned Schemer [TSS] is the call/cc construct used here, without the lambda


Graphical depiction of general call/cc continuation capture process


Graphical depiction of a specific example of a call/cc continuation capture process


Graphical depiction of the stack during a contination replacement process

Note: unlike goto in C, continuation replacement in Scheme is not just a transfer of control, but also a restoration of the environment to what it was when the continuation was captured.


Codes developed in class

    Note: these codes are an amalgamation of codes from [TSS] Chapter 13 and [TSPL3] § 3.3
    (define call/cc call-with-current-continuation)
    
    (define product
      (lambda (lon)
        (cond
          ((null? lon) 1)
          (else (* (car lon) (product (cdr lon)))))))
    
    ;; test cases:
    (product '(1 2 3 4 5)) ; works
    ;;> (product '(1 2 3 4 5))
    ;;(* 1 (product '(2 3 4 5)))
    ;;(* 1 (* 2 (product '(3 4 5))))
    ;;(* 1 (* 2 (* 3 (product '(4 5)))))
    ;;(* 1 (* 2 (* 3 (* 4 (product '(5))))))
    ;;(* 1 (* 2 (* 3 (* 4 (* 5 (product '())))))))
    ;;(* 1 (* 2 (* 3 (* 4 (* 5 1)))))
    ;;(* 1 (* 2 (* 3 (* 4 5))))
    ;;(* 1 (* 2 (* 3 20)))
    ;;(* 1 (* 2 60))
    ;;(* 1 120)
    ;;120
    
    (product '(1 2 3 0 4 5)) ; works inefficiently
    ;;> (product '(1 2 3 0 4 5))
    ;;(* 1 (product '(2 3 0 4 5)))
    ;;(* 1 (* 2 (product '(3 0 4 5))))
    ;;(* 1 (* 2 (* 3 (product '(0 4 5)))))
    ;;(* 1 (* 2 (* 3 (* 0 (product '(4 5))))))
    ;;(* 1 (* 2 (* 3 (* 0 (* 4 (product '(5))))))))
    ;;(* 1 (* 2 (* 3 (* 0 (* 4 (* 5 (product '())))))
    ;;(* 1 (* 2 (* 3 (* 0 (* 4 (* 5 1))))
    ;;(* 1 (* 2 (* 3 (* 0 (* 4 5)))))
    ;;(* 1 (* 2 (* 3 (* 0 20))))
    ;;(* 1 (* 2 (* 3 0)))
    ;;(* 1 (* 2 0))
    ;;(* 1 0)
    ;;0
    
    ;; the continuation stored in break is bound to
    ;; (lambda (rtnval) rtnval)
    (define product2
      (lambda (lon)
        (call/cc
         ;; break stores the current continuation
         (lambda (break)
           ;; why is this letrec necessary?
           (letrec ((P (lambda (lat)
                         
                         (cond
                           ((null? lat) 1)
                           ((zero? (car lat)) (break 0))
                           (else (* (car lat) (P (cdr lat))))))))
             (P lon))))))
    
    ;;test cases:
    (product '(1 2 3 4 5)) ; still works
    (product '(1 2 3 0 4 5)) ; works efficiently now
    
    ;; consider another application of continuations: backtracking
    
    (define retry "ignore")
    
    (define factorial
      (lambda (n)
        (cond
          ((zero? n) (call/cc (lambda (k) (set! retry k) 1)))
          (else (* n (factorial (- n 1)))))))
    
    ;; after (factorial 5), the continuation retry is bound to
    ;; (lambda (rtnval)
    ;;    (* 5 (* 4 (* 3 (* 2 (* 1 rtnval)))))))
    
    ;; effectively we can change the base case at "run-time"!
    
    ;; big deal...so what?  this is exactly how breakpoints in debuggers work;
    ;; the continuation of the breakpoint is saved so that the computation may
    ;; be restarted from the breakpoint (more than once, if desired, and with
    ;; different values)!!
    
    ;; a simple implementation of threads (for multi-tasking) in Scheme
    
    (define ready-queue '())
    
    (define create-thread
      (lambda (thunk)
        (set! ready-queue (append ready-queue (list thunk)))))
    
    (define start-next-ready-thread
      (lambda ()
        (let ((thunk (car ready-queue)))
          (set! ready-queue (cdr ready-queue))
          ;; invoke thunk
          (thunk))))
    
    (define pause-thread
      (lambda ()
        (call/cc
         (lambda (k)
           (create-thread (lambda () (k "ignored")))
           (start-next-ready-thread)))))
    
    ;; create seven threads and starts the first
    (create-thread (lambda () (let f () (pause-thread) (display "h") (f))))
    (create-thread (lambda () (let f () (pause-thread) (display "e") (f))))
    (create-thread (lambda () (let f () (pause-thread) (display "l") (f))))
    (create-thread (lambda () (let f () (pause-thread) (display "l") (f))))
    (create-thread (lambda () (let f () (pause-thread) (display "o") (f))))
    (create-thread (lambda () (let f () (pause-thread) (display ".") (f))))
    (create-thread (lambda () (let f () (pause-thread) (newline) (f))))
    (start-next-ready-thread)
    
    ;; violates 12th commandment;
    ;; set2 never changes and therefore need not be passed
    (define intersect
      (lambda (set1 set2)
        (cond
          ((null? set1) (quote ()))
          ((member (car set1) set2)
           (cons (car set1) (intersect (cdr set1) set2)))
          (else (intersect (cdr set1) set2)))))
    
    ;; test cases:
    (intersect '(peanut butter and) '(jelly and butter)) ; works inefficiently
    (intersect '(peanut butter and) '(jelly)) ; works inefficiently
    (intersect '() '(peanut butter)) ; works efficiently
    (intersect '(peanut butter) '()) ; work inefficiently
    
    ;; that's better
    (define intersect
      (lambda (set1 set2)
        (letrec ((I
                  (lambda (set1)
                    (cond
                      ((null? set1) (quote ()))
                      ((member (car set1) set2)
                       (cons (car set1) (I (cdr set1))))
                      (else (I (cdr set1)))))))
          (I set1))))
    
    ;; test cases:
    (intersect '(peanut butter and) '(jelly and butter)) ; works efficiently now
    (intersect '(peanut butter and) '(jelly)) ; works efficiently now
    (intersect '() '(peanut butter)) ; still works efficiently
    (intersect '(peanut butter) '()) ; still works inefficiently
    
    ;; has 3 problems
    (define intersectall
      (lambda (lst)
        (cond
          ((null? (cdr lst)) (car lst))
          (else (intersect (car lst) (intersectall (cdr lst)))))))
    
    ;; test cases:
    
    ;; first problem
    ;;(intersectall '()) ; fails
    
    ;; second problem
    ;; works, but inefficiently
    (intersectall '((3 mangoes and) () (3 diet bacon cheeseburgers)))
    
    ;; third problem
    ;; works, but inefficiently
    (intersectall '((3 mangoes and) (forget the) (3 bacon cheeseburgers)))
    
    ;; works
    (intersectall '((3 mangoes and) (3 tomatoes and) (3 oranges)))
    
    ;; fixed first problem --- when intersectall is passed an empty list
    (define intersectall2
      (lambda (lst)
        (letrec ((IA
                  (lambda (l)
                    (cond 
                      ((null? (cdr l)) (car l))
                      (else (intersect (car l) (IA (cdr l))))))))
          (cond
            ((null? lst) (quote ()))
            (else (IA lst))))))
    
    ;; test cases:
    (intersectall2 '()) ; works now
    
    ;; what are the other two problems?
    
    ;; first is: if intersectall's argument contains an empty list,
    ;; we immediately know the intersection is empty and therefore
    ;; need not return through all the levels of recursion
    
    ;;>(intersectall '((3 mangoes and) () (3 diet bacon cheeseburgers)))
    ;;()
    
    ;;(intersectall '((3 mangoes and) () (3 diet bacon cheeseburgers)))
    
    ;;(intersect '(3 mangoes and) (intersectall '(() (3 diet bacon cheeseburgers))))
    
    ;;(intersect '(3 mangoes and) (intersect '() (intersectall '((3 diet bacon cheeseburgers)))))
    
    ;;(intersect '(3 mangoes and) (intersect '() '(3 diet bacon cheeseburgers)))
    
    ;;(intersect '(3 mangoes and) '())
    
    ;;()
    
    ;; the continuation stored in hop is bound to
    ;; (lambda (rtnval) rtnval)
    
    ;; still plagued by one more problem
    (define intersectall3
      (lambda (lst) 
        (call/cc
         (lambda (hop)
           (letrec ((IA
                     (lambda (l)
                       (cond
                         ((null? (car l)) (hop "an empty list was in the original list"))
                         ((null? (cdr l)) (car l))
                         (else (intersect (car l) (IA (cdr l))))))))
             (cond 
               ((null? lst) (quote ()))
               (else (IA lst))))))))
    
    ;; test cases:
    ;; still works
    (intersectall3 '())
    
    ;; works efficiently now
    (intersectall3 '((3 mangoes and) () (3 diet bacon cheeseburgers)))
    
    ;; still works, but inefficiently
    (intersectall3 '((3 mangoes and) (forget the) (3 bacon cheeseburgers)))
    
    ;; what problem is still remaining?
    
    ;; if the intersection of any two sets in the argument of intersectall
    ;; is empty, then the result of intersectall is also empty
    
    ;; this problem actually resides in the intersect function
    
    ;; when set1 is finally empty, it could be because
    ;;  - it was always empty, or
    ;;  - because intersect has examined all of its arguments.
    
    ;; but when set2 is empty, intersect should not look at any
    ;; elements in set1 at all; it knows the result is empty
    
    ;; is it correct now?
    (define intersect
      (lambda (set1 set2)
        (letrec
            ((I
              (lambda (set1)
                (cond
                  ((null? set1) (quote ()))
                  ((member (car set1) set2)
                   (cons (car set1) (I (cdr set1))))
                  (else (I (cdr set1)))))))
          (cond 
            ((null? set2) (quote ())) 
            (else (I set1))))))
    
    ;; test cases:
    (intersect '(peanut butter and) '(jelly and butter)) ; still works efficiently
    (intersect '(peanut butter and) '(jelly)) ; still works efficiently
    (intersect '() '(peanut butter)) ; still works
    (intersect '(peanut butter) '()) ; works efficiently now
    
    ;; now intersect does return immediately,
    ;; but it still does not work with intersectall
    
    ;; when intersect returns () in intersectall, we know the result of intersectall!
    
    ;;>(intersectall '((3 mangoes and) (forget the) (3 bacon cheeseburgers)))
    ;;()
    
    ;;(intersectall '((3 mangoes and) (forget the) (3 bacon cheeseburgers)))
    
    ;;(intersect '(3 mangoes and) (intersectall '((forget the) (3 bacon cheeseburgers))))
    
    ;;(intersect '(3 mangoes and) (intersect '(forget the) (intersectall '((3 bacon cheeseburgers)))))
    
    ;;(intersect '(3 mangoes and) (intersect '(forget the) '(3 bacon cheeseburgers)))
    
    ;;(intersect '(3 mangoes and) '())
    ;;()
    
    ;; so we need a version of intersect that hops all the way over _all_ the
    ;; remaining intersects in intersectall
    
    ;; the continuation stored in hop is still bound to
    ;; (lambda (rtnval) rtnval)
    (define intersectall4
      (lambda (lst) 
        (call/cc
         (lambda (hop)
           (letrec ((IA
                     (lambda (l)
                       (cond
                         ((null? (car l)) (hop "an empty list was in the original list"))
                         ((null? (cdr l)) (car l))
                         (else (intersect (car l) (IA (cdr l)))))))
                    (intersect
                     (lambda (set1 set2)
                       (letrec
                           ((I
                             (lambda (set1)
                               (cond
                                 ((null? set1) (quote ()))
                                 ((member (car set1) set2)
                                  (cons (car set1) (I (cdr set1))))
                                 (else (I (cdr set1)))))))
                         
                         (cond
                           ((null? set2) (hop "one of the intersects returned empty"))
                           (else (I set1)))))))
             
             (cond 
               ((null? lst) (quote ()))
               (else (IA lst))))))))
    
    ;;test cases:
    
    ;; still works
    (intersectall4 '())
    
    ;; still works efficiently
    (intersectall4 '((3 mangoes and) () (3 diet bacon cheeseburgers)))
    
    ;; works efficiently now
    (intersectall4 '((3 mangoes and) (forget the) (3 bacon cheeseburgers)))
    
    ;; still works
    (intersectall4 '((3 mangoes and) (3 tomatoes and) (3 oranges)))
    
    ;; final version:
    (define intersectall
      (lambda (lst) 
        (call/cc
         (lambda (hop)
           (letrec ((IA
                     (lambda (l)
                       (cond
                         ((null? (car l)) (hop (quote ())))  
                         ((null? (cdr l)) (car l))
                         (else (intersect (car l) (IA (cdr l)))))))
                    (intersect
                     (lambda (set1 set2)
                       (letrec
                           ((I
                             (lambda (set1)
                               (cond
                                 ((null? set1) (quote ()))
                                 ((member (car set1) set2)
                                  (cons (car set1) (I (cdr set1))))
                                 (else (I (cdr set1)))))))
                         
                         (cond
                           ((null? set2) (hop (quote ())))
                           (else (I set1)))))))
             
             (cond 
               ((null? lst) (quote ()))
               (else (IA lst))))))))
    
    
    ;; more examples:
    
    (let ((x (call/cc (lambda (k) k))))
      (x (lambda (ignore) "hello world")))
    
    ;; continuation k is bound to
    ;; (lambda (rtnval)
    ;;    (let ((x rtnval))
    ;;       (x (lambda (ignore) "hello world"))))
    
    ;; x gets bound to this expression in the let
    
    ((lambda (rtnval)
       (let ((x rtnval))
         (x (lambda (ignore) "hello world")))) (lambda (ignore) "hello world"))
    
    (let ((x (lambda (ignore) "hello world")))
      (x (lambda (ignore) "hello world")))
    
    ((lambda (ignore) "hello world") (lambda (ignore) "hello world"))
    
    ;; another example
    
    (((call/cc (lambda (k) k)) (lambda (x) x)) "hello again")
    
    ;; the literal continuation here is bound to
    ;; (lambda (rtnval)
    ;;     ((rtnval (lambda (x) x)) "hello again"))
    
    ;;((k (lambda (x) x)) "hello again")
    ;;(((lambda (x) x) (lambda (x) x)) "hello again")
    ;;((lambda (x) x) "hello again")
    ;;"hello again"
    
    ;; another backtracking example: simulating a triple for loop
    ;; printing from 000 to 999 in a triple-nested for loop
    ;; example courtesy Marc Feeley '(Montreal Scheme/Lisp User Group)
    ;; from `The 90 minute Scheme to C compiler' with minor modifications
    (define fail
     (lambda () 'end))
    
    (define in-range
     (lambda (a b)
       (call/cc
        (lambda (cont)
          (enumerate a b cont)))))
    
    (define enumerate
     (lambda (a b cont)
       (if (> a b)
           (fail)
           (let ((save fail))
             (set! fail
                   (lambda ()
                     ;; restore fail to its immediate previous value
                     (set! fail save)
                     (enumerate (+ a 1) b cont)))
             (cont a)))))
    
    
    (let ((x (in-range 0 9))
         (y (in-range 0 9))
         (z (in-range 0 9)))
     (write x)
     (write y)
     (write z)
     (newline)
     (fail))
    
    


Support for restoring the control context in C

    setjmp and longjmp is somewhere between the chaos of gotos and the generality of call/cc [PLP2] p. 451.
    #include<stdio.h>
    #include<setjmp.h>
    
    jmp_buf env;
    
    int factorial(int n) {
       int x;
       if (n == 0) {
          x = setjmp(env);
          /*
          printf ("a");
          longjmp(env, 7);
           */
          if (x == 0)
             return 1;
          else
             return x;
       } else
          return n*factorial(n-1);
    }
    
    main() {
       printf ("%d\n", factorial(5));
       longjmp(env, 7);
    }
    
    Why does not Scheme suffer from this problem? Because local variables in Scheme have unlimited extent.

    For more information see coverage of signals, and especially sigsetjmp and siglongjmp in the CPS 445/545 lecture notes, Joe Morrison's blog page covering the relationship between continuations and sigsetjmp and siglongjpm, and [PLP2] pp. 451-452.


Power of first-class continuations

First-class continuations allow the programmer to define any new control flow construct.

We can define ``any desired sequential control abstraction'' (e.g., iteration, conditionals, repetition, co-routines, threads, lazy-evaluation, gotos) using first-class continuations ([OCWC]). The corollary of this is that continuations are yet another primitive, such as lambda, from which to build language features or, in other words, new (specialized) languages, from which to solve the particular computing problem at hand.


Applications of continuations

  • non-local (abnormal) exits (for exceptional handling) for efficiency (i.e., to prevent returning through several layers of recursion)
  • backtracking
  • breakpoints (as used in debuggers)
  • thunks (call-by-name parameters)
  • multi-threading and multi-tasking (e.g., co-routines)
  • iterators (see [TSS] Chapter 19)
  • human-computer dialogs


End of Scheme (for this class)

  • take away: exotic programming languages are powerful (we have only seen the tip of the iceberg in this class) and poorly understood
  • they are an ideal lens through which to study the core concepts of programming languages (and computer science in general),
  • and they can be your claim to fame and fortune, there are too many success stories to mention (e.g., Viaweb (Paul Graham and Robert Morris), Orbitz, emacs, and AutoCAD are just some)


References

    [EOPL2] D.P. Friedman, M. Wand, and C.T. Haynes. Essentials of Programming Languages. MIT Press, Second edition, 2001.
    [MS] J.S. Miller. Multischeme: A Parallel Processing System Based on MIT Scheme. Ph.D. dissertation, Massachusetts Institute of Technology, 1987.
    [OCWC] C.T. Haynes, D.P. Friedman and M. Wand. Obtaining Coroutines With Continuations. Computer Languages, 11(3/4), 143-153, 1986.
    [PLP2] M.L. Scott. Programming Language Pragmatics. Morgan Kaufmann, Amsterdam, Second edition, 2006.
    [PLPP] K.C. Louden. Programming Languages: Principles and Practice. Brooks/Cole, Pacific Grove, CA, Second edition, 2002.
    [TSPL3] R.K. Dybvig. The Scheme Programming Language. MIT Press, Cambridge, MA, Third edition, 2003.
    [TSS] D.P. Friedman and M. Felleisen. The Seasoned Schemer. MIT Press, Cambridge, MA, 1996.

Return Home