LispODROID, quad-core processor.
CL-HTTP 70.218 s512rj1 on an ODROID-XU computer.
Clozure Common Lisp on Linux/Ubuntu.

Ergonomics of the Symbolics Lisp Machine - Reflections on the Developer Productivity

Context: the article describes a historical system initially designed starting in 1973 at the MIT AI Lab (Massachusetts Institute of Technology Artificial Intelligence Laboratory) for AI software development with LISP. The specific hardware / software combination described here was further developed and sold by Symbolics, Inc. starting in 1981/82 until 1993. The specific user interface (Dynamic Windows) described here was released in 1986 by Symbolics, Inc.

Often the Lisp Machine is said to be very productive. (Example: John Lewis in a Tweet in 2014: '30 years ago 100k+ LOC p.a. on a Symbolics').

To discuss this productivity and the reasons for it, let's look a bit into some basic factors.

Meet the Life Hackers - The NYT article talks about the problems of the modern computer-based workplace when it comes to keep you focused and productive. The productivity degradation due to modern work environment (Internet, Mobile Phones, Multi-tasking operating systems, too many options to choose from, tailoring to usersÂ’ needs is difficult (too many different ways)...) is all too familiar. What are the problems and what are some solutions?

Productivity increases when the current work context is clear, reasonably complete and not too complex, when enough information is presented in a visually easy to understand way. We have lots of visual processing power in the brain and we should use that. It is also important not to overload the memory and to 'swap in' enough information, so that retrieval takes little time. You want to have very fast response times from your tools and near zero switch time between tasks using those tools.

Documentation should be organized in small pieces that can be easily understood within its small context and which can be shown/hidden on demand. The work environment should reduce the number of interrupts and after an interrupt the original work context should be recreated (retrieved) easily. It also should be possible to decomposed tasks into smaller tasks, where executing these smaller tasks should create only little 'cognitive load'. Work contexts can be layered, so when you switch your work context, it might be useful sometimes to still be able to access the objects from another, related work context (meta-contexts, learning from contexts, apply a context and see the results). Reusing and manipulating already existing objects lets you stay in context. It is also often easier to fill out a form, than to work in a free form.

The Symbolics Lisp Machine is as much a physical machine as it was software.


XL1200

Lisp programmers often claim very high software development productivity due to the use of Lisp. In my view the single most productive development platform for Lisp was and still is the Symbolics Lisp Machine. Though that platform has long been forgotten and won't come back soon, it is still useful to learn from it and to see which of its lessons can be applied to today's development environments. By platform I mean the combination of hardware and software. The Symbolics Lisp Machine is as much a physical machine as it was software. Unfortunately only a few people have ever seen it been used by a Lisp wizard for (complex) development tasks. In this text I'll try to give some ideas what some of the main building blocks for increased productivity are. I will concentrate more on the environment and not on the Lisp programming language. Of interest is how the development environment supports the programmer dealing with tasks like: edit code, test code, find documentation, run commands, inspect output and so on.

The Lisp Machine was a real workstation for the AI software developer. It was not a personal computer to play games, browse the web, watch DVDs or copy music from your friends. It was also not a multi-user server. It was initially designed as a powerful single-user workstation supporting the development of complex Artificial Intelligence software. Its main task was to enable the software developer to write the most complex programs possible. It was also grounded in the believe that software development is best done by small groups of very good software engineers working in a networked environment. Users that demanded and got the best tool.

During its roughly one decade of commercialization (from 1981 to 1993), it went from being a commercialized laboratory prototype through several stages of development and improvement. It got smaller and cheaper. Up to versions which could work embedded into other systems (Macivory, Microexplorer, UX400, UX1200, ...). Its later versions also had real microprocessors (the TI Megachip or the Symbolics Ivory) and standard ports to SCSI disks, Ethernet, Nubus, etc.

Its users then were not only AI developers, but actual end users. Symbolics for example sold Lisp Machines and their Graphics suite into the high-end graphics markets: TV companies, Sony, 3d gaming studios like Nintendo, ... Special Lisp Machines types then were built for application delivery: smaller and cheaper. Power plants ran multiple Macivories for maintenaince planning purposes, American Express ran an automated credit card transaction checking expert system. The MCC research center developed Chip design, object-oriented databases and new human-interface technologies using them. The Cyc project developed a huge common-sense knowledge-representation system using them.

The base system originally started with $70,000. A tool whose keyboard was as expensive as a PC today. It didn't come with lots of 'media' keys on the keyboard, the operating systems wasn't dumbed down to the least educated user. It was from experts for experts. It was completely programmable with all source code. It was also not for 'playing' around - for that it was too expensive. As a software developer you could focus on your task and the whole operating system was supporting you. There was no piece of software that was not accessible in a few mouse clicks. Everything could be inspected, everything was up for modification. Software was live and dynamic. Not dead and static like today. There was no boundary between software development and software usage. The software was directly connected to the hardware. A keypress runs a Lisp function - written in an high-level language. The Lisp function talks to the chips and interfaces.

The NYT article talks about computer-supported work in general. I will explain how the Symbolics Lisp environment made the software developer more productive. The Symbolics Lisp Machine was a high-tech solution. There are also useful lower-tech solutions. Many Lisp programmers like the relative 'simplicity' of just an Emacs window with a 'slave' Lisp process (say, SLIME + Emacs). Some of these developers are using just a terminal and Emacs. It gives them an integrated environment that is both simple and effective. The Symbolics Lisp Machine environment is more complex. It has been designed to support the development of novel and complex software, especially AI software. The whole point was to make an individual or a small group very productive and let them manage high complexity. Artificial Intelligence software often was very complex (Cyc is an example for that).

The Symbolics Lisp Machine user interface is based on a few simple ideas

  • Lisp is the base language throughout the system. Only the hardware is below Lisp.

  • There is a single Lisp system running sharing all data and code as objects.

  • Changes to the software are immediately active and promoted to all tools.

  • All source code was available for immediate and incremental changes.

  • The screen is large, so you can see a lot of information in one context.

  • Window contents are buffered, so switching windows is extremely fast.

  • Applications have a full-screen UI with subpanes. You usually would use one full-screen activity at any one time.

  • Switching between applications is easy and supported by a special key on the keyboard (SELECT).

  • The layout of the window's subpanes adapts to the window size.

  • Everything on the screen is an object, mouse-sensitive and reusable.

  • Every output is recorded.

  • You interact with the system via commands. Commands can be entered via the keyboard or the mouse. Commands are also objects. Commands are recorded in histories.

The Lisp Machine keyboard

There was a relatively simple keyboard where keys on the outside were larger (so still easy to reach). These larger keys had special purpose functions, - they were task oriented - for the developer (like forcing the front process into a debugger and continuing from there) and act more like 'buttons' immediately controlling a machine. There were several layers of keyboard interaction: special characters, network operations, window operations, process scheduler operations, Lisp operations, switching applications, console interactions and context sensitive help. The key mechanism was excellent, not comparable to the cheap keyboards in use nowadays. The keyboard layout also was optimized for writing and editing Lisp programs. Often used characters (for example the parentheses) are placed to be more accessible, without the need for shift. Deleting characters is done with the Rubout key to the left. For the completion of words, symbols, etc. a special COMPLETE key is provided.


Symbolics new style keyboard

The Lisp Machine is basically a networked single user workstation for small teams. If there was ever an object-oriented environment (operating system and user interface) then it was the Symbolics Lisp Machine. Built on top of several thousand software classes, several ten thousand software methods and millions of objects. It was built for one user, one user in a small team, with one programming language, a central administration server and distributed resources (files, documentation, printers, databases, ...) .

Using only one programming language increases the complexity of this language. Lisp suddenly became a systems programming language with all the necessary features. But using only one language also avoids the complexity of interaction between multiple languages. The data objects are suddenly directly reusable between all applications. There was no guessing what these objects are. Each object's class and identity is known.

Screen and Windows

The screen was at least a 'Mega Pixel' screen. The main screen was usually black and white (which was 'cheaper' and faster at the time), but many installations had at least one additional color screen with a color framebuffer to display color images and data. Large screens are now common, but at the time when these machines appeared on the market (1981) this was very unusual. Especially as a minimum requirement.


Document Examiner

The windows were mostly full screen. They were thought to bundle all visual information for an activity (writing email, drawing a picture, editing text, reading documentation, chatting, administering the mail server, ...). One could switch between the windows with the keyboard in a logical fashion. Select-E gets you to the editor. Select-L gets you to the Lisp Listener, Select-D gets you to the documentation reader - you get the idea. Another Select-E would get you to the next editor window. Select-Control-E would open another fresh editor window. These keystrokes are hardwired in the brain after a short period of time. Especially since the time to switch between activities was mostly zero. Each activity window had a few panes: title, menu, options, command interpreter, application data. Each user interaction also left an entry in the command interpreter pane. This layout was very similar in all applications. There are overviews about available commands, completion for commands, input forms for commands, help for command options and so on.

So the activity windows keep the context and switching between these contexts is very fast. This is crucial. Switching needs to be very fast - otherwise the user feels uncomfortable. You are not juggling around windows and screens. Access to the activities is hardwired into the brain with a simple keystroke mechanism.

Sometimes you need to display temporary information, like a documentation snippet. Sometimes the software will ask questions. The Symbolics Lisp Machine has little temporary windows which expand from the top of a window. You would display there some information or have a short interaction with Lisp - with the press of a key the window goes away, and you get your context back.

It is also possible to create windows that have several applications as sub-windows. There was a simple dialog to configure such windows. This allows you to assemble a few windows to do some task and bundle them into one window.

The developer gets help designing these activities. There is a user interface designer which the designer uses to create the window and pane layout, sets some options for the panes, assigns the Select key and more. The tool generates the Lisp code, which the developer uses then in his application as a starting point for the user interface.

In modern software we know see a lot of these kinds of windows which bundle the information for a task. Apple is using this in iTunes, iMovie, iPhoto and other applications.

There are alternative user interface concepts. Instead of the Symbolics activity-based user interface, the Xerox InterLisp-D Lisp Machine had a very fancy window manager called ROOMS.

The Symbolics Lisp Machine also has a modeline on the bottom of the screen which displays information about things like time, the login user, the Lisp package. But it also displays the state of the front process (which is visually on a computer not clear) and the state of system using 'runbars'. These runbars blink on disk access, paging and network access. Since the Lisp Machine itself typically was running in some machine room and the console screen could be far away, the user might not 'hear' the machine. Some users had long console cables (30 meters and more). All they had on their desk was a large screen, the special keyboard and a three button mouse. So the user needs some visual clues if the machine is active and what it is doing. The modeline also displays a progress bar whenever the front application needs one.


Symbolics Console of Boris Katz

So a typical user was a researcher in his office, with a Lisp Machine console, noise-free and always running. With such an environment, the Lisp Machine could be running for months without rebooting.

The Presentation-based User Interface and the role of the Mouse

The next thing the modeline displays are the current actions that the mouse can do with the thing under the mouse cursor. The mouse has three buttons and these button commands can also be modified by some shift keys. The modeline displays the the commands accessible from the three buttons and gives hints how more commands could be accessed. Is the mouse over special objects like a scrollbar, the modeline displays a hint how this object can be manipulated with the mouse.

The mouse with its three buttons and the modifier keys, allowed dozens of gestures. These gestures were for selecting, context-menu, inspecting, UI debugging, editing operations. Some developers like to work keyboard-only. In Symbolics Genera the role of the mouse was much more prominent and useful, since most text and graphics on the screen could be treated as objects and could be used for various gesture-based interactions.

Above is extremely crucial. Everything on the screen is an object with a type. Every object has context-specific and general commands. The whole user interface gets suddenly extremely dynamic. There is no dead text. Since the objects belong to a type, the Lisp Machine user interface calculates on the fly while the mouse moves over the screen the possible commands and updates the modeline. For the user this means two extremely important things: the objects don't loose context and they can be reused in different contexts when they fit into the new context. A form displaying some options could be filled out with objects that are already on the screen. As you are prompted for input, the system knows what kind of objects are acceptable. Possibly these objects are already on the screeen. We are talking not about text, but about the real Lisp object. The program gets passed the objects, not text.

But the user interface does even more for you. Whenever you are asked for some type of object - say, the system asks you to enter a user - it not only recognizes user objects on the screen, but it provides you completion over the users (by search or by menu), it rejects invalid users and it allows you to choose a user from a history of previously entered users. For the user of the Symbolics Lisp Machine this means that the user interface is very powerful in reducing the complexity. It searches in the right contexts and provides all this in-place. No switching, no searching around. When you need to enter some option, the Lisp Machine knows all the possibilities in this special context and helps you to find the right option without switching context.

This means you have a dynamically updated user interface that remembers the semantics of the objects. By semantics it does not only mean that the system knows the class of objects, but it records also a type. For example a number displayed on the screen is not just a number, but it could be a year, a social security number, a temperature or something else. When the user interface asks you to enter a number, it does not simply ask for a number, but for the type. So when you were asked for a social security number, it only accepts those. But it accepts also previously displayed or entered social security numbers. So it can filter also the input possibilities by context. Whenever you move the mouse over the screen the operating system not only looks what kind of object there is (and computes what it can do with it) - it also looks if the object fits in the current context or even if it can be morphed some way to fit into the current context. For example it could ask for an email address and a person with an email address could be an acceptable input, where the system would extract the email address and reuse it.

We have a fluid and semantic aware user interface.

For the developer this is even cooler. Since these contexts and the resulting user interface possibilities are computed on the fly, a change in the software gets active immediately. Add a new command for an object type? It is immediately available throughout the whole system in the right contexts. You add a superclass to some class which adds new user interface commands? Immediately active. Not just on new objects on the screen, but also for old. This allows the developer to incrementally develop dynamic user interfaces. You don't have to switch between writing your interface and testing it out by a long process. The objects are there, you write a new command or modify an existing one and you can test it out immediately. The Lisp compiler will compile these code snippets in a second without noticable delay for the developer. If you have incremental development capabilities you want those widely used, because they speed up your development. You can complete developing a command and using it at the same time. There is no time lag between these two things. Even while developing very complex software.

Compared to other Lisp Machines

How does this compare to other Lisp Machines?

The Xerox Interlisp-D Lisp Machines had a very different user interface and hardware. Interesting, but very different.

LMI, like Symbolics, commercialized the MIT Lisp Machine. LMI improved the UI and hardware of the MIT Lisp Machine. Texas Instruments (TI) got the rights to the LMI technology in 1983 by buying a 25% stake. TI further improved the hardware - for example they developed the first microprocessor-based Lisp Machine. The UI used a lot of the earlier technology. Later TI developed a more standard UI toolkit (called CLUE and CLIO, on top of CLX). The hardware of TI was slightly more office-like.

Symbolics has introduced Dynamic Windows with Genera 7 in 1986. This Presentation-based user interface is unique to Symbolics Genera 7 and 8.

Background

An Information Presentation System, Zdybel/Greenfeld/Yonke/Gibbons - The idea of Presentation-Based User Interfaces was developed at BBN, using the KL-ONE knowledge representation system running on Interlisp.

Presentation Based User Interfaces, Eugene C. Ciccarelli IV - Work at MIT delivered the direct ideas for Symbolics' Dynamic Windows

Genera 8.0 Common Lisp Interface Manager (CLIM) - The portable Common Lisp version of the Symbolics user interface libraries.

The End. The article is a bit work in progress. I have a bit more topics to write about. So much for now.