Many Arc compilers and interpreters

The introduction of the Arc language has spawned a mini-boom of Arc compilers and interpreters implemented in everything from JavaScript to .NET. Because Arc is a fairly simple Lisp dialect, it is relatively straightforward to implement, and many people are using it to learn about compiler and interpreter design, and to showcase different implementation techniques and languages.

Paul Graham's official distribution started it all, of course. This version is implemented in MzScheme and translates Arc into Scheme code. The writeup on Arc Internals provides more information on how it is implemented.

The Arc Forum is the site of most of the unofficial Arc development effort. The "Anarki" branches of Arc on github consist of a "stable" bug-fixed version of official Arc, and a more exploratory version with many modifications. (For information on using github, see Brief Guide to Anarki and Git.

Anarki includes "ac.sbcl.lisp", an implementation of Arc in SBCL. It implements closures using CPS (Continuation-passing style) transformations, but doesn't include threads or networking according to the brief description.

Perhaps the first reimplementation of Arc was ArcLite, which is Arc ported to JavaScript. This version of Arc is unique because it runs in the browser. The running version can be accessed online. It provides a subset of functionality, omitting tail-call optimization, continuations, and various operations.

Rainbow (l'arc en ciel in French) is an implementation of Arc in Java. It is fairly complete, providing continuations and tail-call optimization according to the description. It is available at github.

An Arc compiler for .NET has been developed, based on the MBase compiler prototyping framework. The description states that it implements a subset of Arc, but is much faster than standard Arc. It is available from Meta Alternative.

Currently under active development is "arc2c", which compiles Arc to C. Current status is at arclanguage.org. This compiler is intended to be a full, high-performance implementation of Arc. It uses CPS transformations to implement continuations. It uses the Boehm garbage collector. arc2c is available at github.

CL-Arc is a proposal to compile Arc to Common Lisp; work hasn't started on it yet.

Many of these Arc implementations actively welcome participation. The future is likely to hold continuing improvement in the quality, functionality, and performance of Arc implementations, as well as surprising and unusual variants.

Ajax and Arc

Ajax is surprisingly easy to use with the Arc language. This article shows how to implement Ajax-based autocompletion and dynamic page updates using the script.aculo.us and Prototype JavaScript frameworks. These frameworks allow Ajax applications to be implemented with just a few lines of additional code. The hard work is done by the script.aculo.us and Prototype libraries. All the Arc code needs to do is provide the list of autocompletion candidates, and the dynamic page content.

The example

This example implements a simple Ajax input autocompleter and dynamic page update. For the autocompleter, as soon as you enter a few letters into the input, it provides a list of matching terms. As soon as you select a term, the page is dynamically updated with associated data, as in the following screenshot:
screenshot
For the purposes of the demonstration, the content is information on the countries of the world, obtained from the CIA World Factbook.

To summarize how it all works: When you enter characters in the input field, the script.aculo.us autocompleter JavaScript sends the charaters to the Arc server, which returns the autocomplete suggestions. The autocompleter JavaScript displays the suggestions on the page. When you select a country, the updater JavaScript does three separate requests to the Arc server, to request the population, area, and capital. When the responses arrive, they are inserted into the page.

In more detail, the autocompletion is performed by Ajax.Autocompleter, and the dynamic updating is performed by Ajax.Updater. The control flow is:

  • The Ajax.Autocompleter associated with the autocomplete input and autocomplete_choices div sends inputs to /auto_complete?prefix=chars.
  • The Arc handler sends back a formatted list of autocomplete candidates, e.g. <ul><li>Iran</li><li>Iraq</li><li>Ireland</li></ul>.
  • The Ajax.Autocompleter displays the candidates in the autocomplete div.
  • When an autocompletion is selected, the update JavaScript function starts three Ajax.Updater instances, which send an Ajax request to URL /getcontents?field=population&name=value, and similarly for area and capital in parallel.
  • The Arc handler for getcontents returns the desired contents.
  • Ajax.Updater puts the contents into the contents div.

Running the example

First set up the necessary files.
  • Download ajax.zip and uncompress it into the same directory that Arc runs in. This zip file provides ajax.html, ajax.arc, data.arc, and autocomplete.css.
  • Download the script.aculo.us library files from script.aculo.us. Copy the .js files from lib and src into the same directory that Arc runs in.
Next, start up the Arc interpreter, load the ajax.arc file, and start the web server.
arc> (load "ajax.arc")
arc> (thread (serve 8080))
arc> ready to serve port 8080
Then go to http://localhost:8080/ajax.html. (Unfortunately I don't have a live demo on a public machine. If anyone sets one up, let me know and I'll add a link here.)

Start typing in the box, and you should see a dropdown with autocompleted choices. Click one, and the code will be displayed below asynchronously.

The Arc code

The Arc server code is in ajax.arc. Two Arc handlers are implemented to provide the client-side Ajax support. The first, auto_complete receives the current input contents and returns a list of up to 10 autocompletion candidates formatted in HTML. The second handler, getcontents returns the dynamic content for the page, from the database table. Note that the handlers do nothing particularly special to make them "Ajax"; they are standard Arc web handlers based on defop and can be accessed directly through the browser. See Arc Web Server for more information on web serving with Arc.
(defop auto_complete req
  (let prefix (downcase (arg req "prefix"))
    (prn (to-html-list (cut (startselts prefix keylist) 0 10)))))

(defop getcontents req 
  (with (field (arg req "field")  name (arg req "name"))
    (if (is field "population") (prn ((database name) 1))
        (is field "area") (prn ((database name) 2))
 (is field "capital") (prn ((database name) 3)))))
The handlers use a couple helpers; startselts returns the elements that match the autocomplete prefix and to-html-list wraps the elements in HTML list tags.
; Returns a list of elements that start with prefix
(def startselts (prefix seq) (rem [no (begins (downcase _) prefix)] seq))

; Wraps elts in HTML list tags
(def to-html-list (elts) (tostring
  (prall elts "<ul><li>" "</li><li>")
  (pr "</li></ul>")))
The actual content is obtained from data.arc, which contains the information on each country as a list of lists.
("United States" "301,139,947" "9,826,630" "Washington, DC")
Some simple code converts the list into a table called database indexed by country for easy lookup, and generates a sorted list of countries for use in autocompletion:
(= database (table))
(w/infile inf "data.arc"
  (let datalist (sread inf nil)
    (each elt datalist
      (= (database (elt 0)) elt))))
(= keylist (mergesort < (keys database)))

For some reason, the default Arc web server srv.arc doesn't support .js files. The ajax.arc file modifies the server slightly to support these files:

(= (srv-header* 'text/javascript) 
"HTTP/1.0 200 OK
Content-Type: text/javascript; charset=UTF-8
Connection: close")

(def static-filetype (sym)
  (let fname (string sym)
    (and (~find #\/ fname)
         (case (last (check (tokens fname #\.) ~single))
           "gif"  'gif
           "jpg"  'jpg
           "css"  'text/css
           "txt"  'text/html
           "html" 'text/html
           "js"   'text/javascript
           ))))

Debugging

Several things can go wrong when trying to run the example. If the initial page doesn't load at all, something is probably wrong with the Arc server. If no autocompletion happens, the JavaScript may not be loading, or the Arc server may have problems. If the autocompletion shows up as a bulleted list, the CSS file is probably not loading.

The Arc code can be debugged in several ways. The first is to access the handlers directly. The autocomplete handler http://localhost:8080/auto_complete?prefix=a should return a bullet list of autocomplete suggestions. The contents handler: http://localhost:8080/getcontents?field=area&name=Algeria should return the dynamic contents value.

The Arc code can also be debugged by strategically inserting print statements such as:

(write req (stderr))
This will display the request on the Arc console.

To debug Ajax, you can use Firefox plugins Live HTTP Headers and Firebug. Live HTTP Headers lets you see the headers between the browser and the server, and is very helpful to determine if something is failing. Firebug allows much more in-depth JavaScript debugging. The browser's JavaScript console is also useful to see JavaScript errors.

This article has just scratched the surface of script.aculo.us and Ajax. More information is available at the script.aculo.us website or in books.

Documentation of HTML generation in the Arc language

I've created documentation of Arc's HTML generation operations (from arc.arc). There are a lot of different functions there, but I've tried to cover them all.

Continuations made difficult

For a challenge, I translated the "mondo-bizarro" Arc Continuation Puzzle into Java.

Why would I do that? Because I can :-) But seriously, using continuations in a language entirely unsuited for it is a good way to experience the tradeoffs of different styles of languages, as well as a way to learn more about how continuations work.

This code is entirely different from my Arc version, mainly because in the Arc version I decided to see if the throw/catch idiom could replace call/cc; the Java code is much closer to the original Scheme. Because Java doesn't have first-class continuations, I used the Continuation-passing style of explicitly passing continuations around. Then call/cc is simply replaced with use of the current continuation.

Because Java doesn't support first-class functions, every continuation function needs to be wrapped in a class that implements an interface, which I unimaginatively call Continuation. The "let" also turns into an object creation, resulting in another class. This results in a fair bit of boilerplate to handle all these classes compared to the Scheme code, but the Java code maps recognizably onto the Scheme code.

On the whole, mondo-bizarro worked better in Java than I expected; no Greenspunning required. It produces the expected 11213 output, proving it works. I think the Java code is actually easier to understand, since everything is explicit.

I have also found it entertaining to implement some of the complex SICP exercises in Java; maybe I'll post details later.

(The title of this article is, of course, a homage to Mathematics Made Difficult.)

Here's the code.

/**
 * Mondo-Bizarro ported to Java.
 * Based on mondo-bizarro by Eugene Kohlbecker
 * ACM SIGPLAN Lisp Pointers, Volume 1, Issue 2 (June-July 1987) pp 22-28
 */

/* Original Scheme code:
(define mondo-bizarro
  (let (
        (k (call/cc (lambda (c) c)))
        )
    (write 1)
    (call/cc (lambda (c) (k c)))
    (write 2)
    (call/cc (lambda (c) (k c)))
    (write 3)))
*/


interface Continuation {
  public void run(Continuation c);
}

public class Mondo implements Continuation {

  public static void main(String argv[]) {
    Continuation c = new Mondo();
    c.run(c);
  }

  public void run(Continuation c) {
    Continuation let = new Let();
    let.run(c);
  }
  
  class Let implements Continuation {
    Continuation k;

    public void run(Continuation c) {
      k = c;
      System.out.println("1");
      k.run(new C2());
    }

    class C2 implements Continuation {
      public void run(Continuation c) {
        System.out.println("2");
        k.run(new C3());
      }
    }

    class C3 implements Continuation {
      public void run(Continuation c) {
        System.out.println("3");
      }
    }
  }
}

Continue reading...

Documentation for (almost) all of Arc

I've got almost all of the Arc core documented with an alphabetical index. I've covered ac.scm, arc.arc, strings.arc, but am currently omitting the web server and applications.

For each function, I include a description, examples, a link to the code, and a link to the wiki. I'll fill in a few gaps as time permits. I'll appreciate any feedback on the organization.

Arc: popular since 2004!

Did you know that Arc has been a popular computer language since 2004? Pretty good for a language released in January 2008. I discovered Arc's longstanding popularity in Computerworld's review of Paul Graham's Hackers and Painters. The review from July 2004 says, "Paul Graham, well known for developing the popular Arc computer language and other Internet technologies...". I guess they didn't realize Arc hadn't been released at the time. Hopefully we won't see them publishing job listings looking for 4 years of Arc experience.

"Even tried multi" error explained

If you were around during the early days of the web, you surely remember the message
Error 404 Not found - file doesn't exist or is read protected [even tried multi]
But what is "multi"? And why does the web server act like it's doing you a big favor trying it? Is it trying multiple times, maybe?

These days, 404 messages are still common, but the "multi" message is fairly rare. The message was once so common, though, it spawned a large number of parodies.

I decided to get to the bottom of this, and find out what this "multi" is. It didn't take long to discover that this message comes from the CERN httpd server, written by Tim Berners-Lee and others back in the 1990's.

A bit of poking around found the source code that generates the infamous multi message. It turns out that "multi" is short for multi-format documents, and was a cool new feature back in 1993. It is a technique for the server to have multiple formats of a file and provides the best one your browser can handle. For instance, suppose you try to fetch an image /cat. Back in the olden days, browsers weren't so good with images, so your browser might only display gifs and not jpegs. Your browser asks for /cat and says that it can handle gifs. Now for the multi magic. The server can have multiple files, say /cat.gif, /cat.jpg, and /cat.png. It looks at what your browser can accept, looks at what it has, and returns the best choice, in this case /cat.gif. Likewise, a file could have versions in text and PDF, and the server would return the best type for your browser. Web servers still support content negotiation, but they generally don't mention it in 404 messages any more.

So there you have it, the definitive answer to what "even tried multi" means, and a bit of web history.