1 Introduction

Most people don’t get to hear their obituaries. I feel very lucky to have eavesdropped on these thoughtful pre-mortems, and I want to thank all the authors for their understanding, kindness, wit, and forbearance. I feel much better understood than I knew.

First let me thank several people: Daniele Struppa and Doug Dechow for the gift of the event and this book; My son Erik and his mother Deborah Stone for their understanding and great moral support over the years; and my collaborator, ex-IBMer and systems angel Marlene, who has organized me across many oceans and continents, my dear wife-waft.

2 What It Was Like from the Inside

Others have presented many perspectives on my life and work, and now I’ll tell how it’s been from the inside. I want you to know the whole story of the ideas I have tried to carry out.

People now call me a “computer scientist.” I did not think of myself as a computer scientist until recently, when people started calling me that, and Chapman University made it official with their honorary PhD.

For most of my life I have thought of myself as a philosopher and a film-maker.

So I’ll talk about philosophy and filmmaking—and media in general—before I talk about computers.

3 Philosophy of Intertwingularity

Let me begin at the philosophy end. Let’s talk about intertwingularity.

This book, like the conference, is called Intertwingled. It’s a word that expresses a philosophical position about cross-connection. I said in Computer Lib [2, 6], “Everything is deeply intertwingled.” I meant that all subjects and issues are intertwined and intermingled.

But intertwingled subjects are not what computers usually represent. From the beginning, people have set computers up to be hierarchical. Hierarchy is not in the nature of the computer. It is in the nature of the people who set computers up.

If you say, “everything is hierarchical,” as many computer people do, that is not science, it’s a metaphysical position. It can’t be proven true or false, it can only be proven inconvenient.

Hierarchy maps only some of the relationships in the world, and it badly maps the rest. You cannot represent history hierarchically, but as cross-connecting threads of narrative and relationship.

Unfortunately, the computer world has traditionally imposed hierarchy on everything. Most of the computer world is committed to a metaphysic of hierarchy. Files, directories, and now XML are hierarchical.Footnote 1 This is not just a philosophical position. It’s an IMposition.

Now, it’s also a metaphysic to say, “everything is deeply intertwingled,” since the sentence cannot be proven true or false. But it is computer science to say that we need to represent cross-connection, and I’m expressing a computer science opinion when I say that intertwingularity is a better form of representation—for everything—than hierarchy. For things that overlap, shade off, and entwine, hierarchy does not work. Hierarchy is less and less appropriate as we try to represent more and more of the world.

Aristotle is often cited to support hierarchy. But intertwingularity has its philosopher too. His name was Heraclitus, and it was he who said you can never step in the same river twice, because of the constant flux of change and interconnection. Alas, none of his writing has survived, but his view of interconnection has.

My main designs, which I will discuss, are examples of intertwingularity. My document structure is cross-connective on the literary level, my data structure is cross-connective internally, and my viewing system is cross-connective on the screen.

This I see as practical intertwingularity.

4 Movies and the Other Presentational Arts

That is my brief on philosophy. Now let’s talk about movies and media and presentational arts. This story is told at much greater length in POSSIPLEX [8].

The 1940s, the years of my boyhood, were media-rich. Usually, you experienced one medium at a time: magazines, radio, comics, stage and screen, and, of course, books. They all interested me much more than school or other kids. I drank in every aspect of every medium.

I adored the movies. (We lived in a very sophisticated part of Manhattan, so we saw more foreign movies than American.) I avidly studied the details of my comic books, from the language and visual angles to the dots of the color. And I listened to radio programs with every fiber of my brain.

I had four main media heroes in my first 10 years, and they are my heroes now: Walt Disney, Leonardo da Vinci, Frank Lloyd Wright, and Buckminster Fuller. They worked in different media, but in much the same way. Each was independent, visionary and original. All these years I have tried to be like them: independent, able to see what others could not, and creating new designs others could not imagine.

I also learned a lot about show business; I happened to have inside connections. I rarely saw my parents, who were divorced when I was born, but I learned a lot when I saw them. My mother became a star on Broadway in her twenties, and after she would take me to a Broadway play, she would take me backstage to meet the actors. My father was successful in another direction of show business. When I was ten, a new medium came along called “television,” and he became a top director in that new medium. I got to sit behind him in control rooms at NBC and CBS.

I got to see how all that magic was made: on stage and TV, the technicalities and tricks, the pressure on the actors and crew, and the bravery in real time. I took some of that bravery with me when I started giving my radical speeches in the computer world, telling computer experts how their field should be conducted.

By the time I got to college my father had put me on TV, radio, and the professional stage—not much, but enough to be confident.

At Swarthmore College I became a media innovator. I had my own little magazine. The first issue I did jointly with my friend Len Corwin. I did the others by myself. I figured out how to use the new offset presses to print a magazine for 32 dollars. I commissioned the cartoons from a great cartoonist, Russ Ryan.

Figure 17.1 shows Nothing #3, a very mischievous design. It was kite-shaped, and it had to be rotated as you went from page to page. I did it when I was 19. It cost more than thirty-two dollars to print, but not much more. The printer, my friend Ned Pyle, approved the mockup, but he was astounded when he saw the result. I had done it on my own without realizing.

Fig. 17.1
figure 1

Issue #3 of Nothing magazine

Full size image

Later that year I wrote and directed what I believe was the first rock musical, Anything & Everything. It was a rock musical (a play in which actors would burst into song), not a “rock opera”. But it had rock songs and a plot, and it came long before Bye Bye Birdie and Hair. Few have heard of it, but it ran at Swarthmore for two nights (as scheduled) in November of 1957. It is not in the official rock histories, but I think it should be.

My last year in college, I shot a 30-min comedy film, The Epiphany of Slocum Furlow (Fig. 17.2), which I think is the best thing I ever did. It is now available on YouTube. Because of the methods of that time, it took years to put the sound track on—not too well—but it tells a story and audiences laugh. I believe it shows that I was a competent film director from the start. I have never enjoyed any form of work so much.

Fig. 17.2
figure 2

Still from The Epiphany of Slocum Furlow

Full size image

Through all these lessons I came to learn that the presentational arts and media are all the same—writing, layout, diagrams, essays, poetry, and brochures; stage, screen, and radio. All these arts present ideas to the mind and heart with a variety of mechanisms, tricks of emphasis, sequence, and overview. And when we say “media” we simply mean the presentational arts as they get to be distributed in the world.

And in all these arts and media, the processes of designing and detailing are the same. Every part of every detail, in whatever medium, involves imagining how it will affect the heart and mind of the viewer (or reader, or participant, or user).

Movies are the pinnacle of the presentational arts because they bring together all the other modes—theater, graphics, sound, and more—with many, many mechanisms. Designing interaction was to be an inevitable new medium, requiring the same talents.

5 Loner

By the time I graduated from college, I was fearless and very ambitious. I expected to be a film director, but I also intended to be a True Renaissance Man, meaning a serious intellectual as well as a media guy. My professors had made it clear that I was good at philosophy, which I could not leave behind. So I graduated from college thinking of myself as a philosopher and filmmaker. Putting these together, I believed I could analyze anything, design anything, and see things others could not.

Collaboration was not my style. I’m not saying there’s anything wrong with collaboration, but it has many drawbacks, especially if you have a large, precise vision. I have been criticized for citing mainly my own work. But I have found the work of others to be less and less relevant to my own.

6 My Plans

My plan out of college was to get a PhD and then go to Hollywood. Little did I know that grad school would be abrasive and boring, with no chance to do anything else.

But in graduate school I had a considerable epiphany (below), and I made a new and much bigger plan. I would found the personal computing industry and world-wide hypertext. I figured this might take until 1967, when I would be thirty; at which point I would get back to my original plan. (Note that Steve Jobs and Tim Berners-Lee were both 5 years old at this time, and they would have been eleven when I was thirty.)

I expected to make a lot of money in the computer field by that time. Meanwhile I would simply accumulate notes for my other projects, which I could then pay to have typed into the software I was designing. I would also have enough money to finance my own movies. And so I set aside—temporarily, I was sure—the one thing I really loved to do.

7 My Epiphany

Like Slocum Furlow, the hero of my college movie, I had an Epiphany. His was somewhat garbled. Mine was very clear (described in more detail in POSSIPLEX).

Sometime in the fall of 1960, I believe, I had the following premonitions:

  • there would be a vast personal-computer industry;

  • the future of human life and work would be at the interactive computer screen;

  • the design of media for the interactive computer screen was in itself a worthy goal (“Screens!” I thought, “I can do THAT!”);

  • there would be a new medium of interactive text, which I envisioned as the true generalization of writing and literature (as humanity had known them for thousands of years), extending the medium far beyond the boundaries of paper; and

  • it was my job to design this new medium, with whatever insights I already had about the overlap of subjects, the nature of the publishing industry, the sociology of readership, the different sides of copyright, and the nature of writing.

8 The Long March

I have worked hard on these matters for over 50 years since then, with great difficulty and little accomplished. Those who went after simpler goals, like Gates and Jobs and many lesser-known successes, had an easier time of it because they swam in a world of mutual agreement on conventional concepts. For instance, “word processing,” a glorified typewriter, was simply for preparing conventional paper documents.

I see the purpose of computers as giving us new and better worlds, not simulating the old. The others were content to build conventional tools, not radical ones, and they were not hell-bent to use them for radical new media, as I was.

This is not the place to talk about adventures or people or badly chosen fights, partly enumerated in my autobiography. I have worked on overarching designs very different from what others have done, and I have disagreed with almost everybody about almost everything in the fields of personal computing and electronic documents.

Meanwhile, I have hundreds of thousands of notes from these 54 years, possibly a million. Figure 17.3 shows a tray of my file cards. It contains computer notes from the 1960s, sorted by topics of my own devising. I kept all my notes on file cards until the 1980s, and then I went to various forms of chronological pages and books. It would be nice to work out a chronology of my work, but very difficult and time-consuming.

Fig. 17.3
figure 3

A tray of file-card notes from the 1960s

Full size image

Because I have disagreed with almost everybody about almost everything, it was a special miracle to find my five collaborators in 1979, discussed below.

9 Seeking the Magic

In design, I believe in magic. That is to say, there can be magical combinations and configurations that are not obvious—simple ideas that extend to create an elegant unified system. An example that moved me as a boy was Wright’s legendary house, Fallingwater. In high school I was similarly moved by several electronic designs: heterodyning, the Theremin, and the Hammond Organ. In the computer field, I was greatly inspired by several different pieces of software and hardware: the APL computer programming language; the PDP-8 computer; Ivan Sutherland’s Sketchpad; Ken Knowlton’s L6 language; Nassi-Shneiderman diagrams; and Sinden’s film, Force, Mass and Motion.

Their designers had all found simple constructs that generated all the results they wanted. This was a clear lesson for the design of software—finding the cleanest and most powerful constructs.

Constructs are the heart of the computer. In general, computers do not deal with reality. Computers deal with constructs. The constructs we put into the computer then become the computer’s model the world. Files and directories are constructs. Screen windows are constructs. Word processor and spreadsheet are constructs.

As I said above, I see the purpose of computers as giving us new and better worlds, not simulating the old. This meant envisioning new worlds and finding new constructs to generate these new worlds cleanly.

Designing constructs—what I call Construct Logic—is for me the center of software design. Trying to find the magical, minimal structure is the highest goal. As in those examples that moved me in my youth, I have sought construct designs that had elegance, minimalism, and generative power. This was a new kind of philosophical design—the design of abstraction.

Over this half century of work, I believe I have found the cleanest and most general designs for interactive systems. These satisfy a lot of people’s wish lists, but with very simple and unifying structures.

Each of these designs is nonhierarchical and intertwingled.

I believe I have found a system of documents far deeper, and a system of data far richer, and a system of visualization far more sweeping, than proposed by those others who imitated the past. To work on anything else seems wrong to me, for I still must get these things working.

10 Preamble to Xanalogical Documents: What’s Wrong with the Web?

People keep asking me, “What’s wrong with the World Wide Web?”

What’s wrong with the Web is—to begin with—the same thing that’s wrong with Microsoft Word and paper simulation in general. You can’t show parallel pages, visibly connected (Fig. 17.4).

Fig. 17.4
figure 4

What can’t be shown on the Web or with other electronic documents

Full size image

This is not just a complaint about the Web. It’s a complaint about the simulation of paper by computer. It is my complaint about textfiles, Microsoft Word, and PDF: all are systems of paper simulation that cannot show parallel pages visibly connected!

11 Seeking the True Generalization of Literature: Translit/Xanalogical Structure (Trademarked Xanadu)

As I said before, I see the purpose of computers as giving us new and better worlds, not simulating the old.

I asked this in 1960: If we could interact with documents on screens, what would be the truest, finest generalization of literature? This was a philosophical question with a powerful kick, for it defined the way we might be able to think in the future. Indeed, “As We May Think” [1] was the title of an article that had influenced me as a boy.

But what abstractions and generalizations to choose? The new document design should be elegant. It should be comprised of the simplest possible constructs.

I then realized that the screens of the whole world would be connected, and my aspirations exploded further. To redefine the whole of literature, with new capabilities making it grander and far better, seemed to me the truly noble ambition to which I must turn.

My document design began with the idea of managing my notes on a computer screen, and setting it up so that the same note could combine with others in different ways, the different combinations visible side by side. This meant indirect addressing, now called transclusion, which I believe was one of my earliest ideas.

And I immediately imagined the document structure of the future, including the jump-links of today’s Web, but with certain key differences.

While most of this design came to me in 1960–1961, it took a long time—18 years—and working with five brilliant collaborators (Roger Gregory, Mark S. Miller, Stuart Greene, Roland King, and Eric C. Hill), to reconcile all the parts into a clean internal structure,Footnote 2 which we call xanalogical hypertext, or Translit. The system has been described in the original conference paper [11] and in the various editions of Literary Machines [3, 5].

The document structure turned out to be very difficult for people to understand. Many people read Literary Machines, even the simple classic edition [3], and couldn’t understand it at all. In fact, without a visible example, most people of that time couldn’t even imagine jump-links.

12 Transpointing Windows

In hindsight, I should have simply emphasized a non-abstract aspect that people could visualize: “parallel pages, visibly connected on the screen.” Even at that time, a few people might have been able to imagine it. But I left it out of my earlier papers, thinking it was obvious. I first published the concept in a 1972 paper for a conference I couldn’t afford to attend [10]. I didn’t have a computer at the time—no individuals did—so I simulated it on top of a Selectric typewriter, to demonstrate parallel pages and windows visibly connected (Figs. 17.5 and 17.6). In more recent writings I have called these transpointing windows. [4, 7].

Fig. 17.5
figure 5

Simulated computer screen created to demonstrate, “parallel pages, visibly connected”

Full size image
Fig. 17.6
figure 6

Simulated computer screen built by the author, ca. 1971

Full size image

But the world started to go in a different direction. In the 1970s, the current windowing system was adopted by Macintosh and Windows. While the current windowing system is often referred to as “the GUI” (Graphical User Interface) or “the PUI” (PARC User Interface), it is highly restrictive. As implemented at Apple and Microsoft, it completely prevents the kind of connection illustrated in Figs. 17.5 and 17.6 and those that follow.Footnote 3

I must stress this to listeners over and over. Today’s operating systems rule out interconnection between windows at the system level—only within a restricted “application” can such windows be interconnected, and only if you create a new set of windows internal to the application. Of course, one sneaky method is to seize the whole screen as a transparent canvas, but this still does not give access to the windows provided by the operating system.

It is still hard for many people to understand that I mean actually showing visible connections between pages on the screen, and hard for them to imagine writing based on this capability, though it is the only kind of writing I wish to do. For instance, I would like to rewrite my autobiography into parallel, visibly connected pages, in case I have the time and tools to do so (Fig. 17.7).

Fig. 17.7
figure 7

Notional rearrangement of my autobiography, POSSIPLEX, into parallel connected pages

Full size image

13 Xanalogical Structure

The document structure we designed in 1979, xanalogical structure, has been greatly set back by events. However, several interactive demos of xanalogical documents have been implemented.

The first interactive version we can now show was “The Ping Demo,” done by Ka-Ping Yee in 1999 (Fig. 17.8). It shows (and scrolls) two versions of Jefferson’s Declaration of Independence. Fed by the Xanadu Green server, it shows transclusions but no xanalinks.

Fig. 17.8
figure 8

The Ping Demo: Interactive xanalogical implementation by Ka-Ping Yee

Full size image

The most recent instantiation of xanalogical structure is OpenXanadu, implemented by Nicholas Levin in April 2014 (Fig. 17.9). It runs in a browser, and it also shows only transclusions but no xanalinks. The document illustrated is “Origins,” by Moe Juste. Note that the entire King James Bible is in the left-hand column.

Fig. 17.9
figure 9

Parallel connected pages in the OpenXanadu browser implementation

Full size image

Our most general and vivid xanalogical presentation is the XanaduSpace demo, which was programmed by Robert Adamson Smith in 2007. This demonstration shows connected, parallel pages in a 3D space (Fig. 17.10). The document illustrated is again, “Origins,” by Moe Juste. This version shows both transclusions and xanalinks.

Fig. 17.10
figure 10

Parallel connected pages in the XanaduSpace demo, implemented by Robert Adamson Smith, 2014

Full size image

Another version of transpointing windows is CosmicBook from 2003 (Fig. 17.11). This version is not xanalogical. It is a simple hypertext with visible links.

Fig. 17.11
figure 11

CosmicBook, programmed by Ian Heath

Full size image

These different versions of transpointing windows show the implementability of the concept, though they are still regrettably far from product.

14 Our Other Intertwingled Software

While xanalogical/transliterary documents have been the center of my concern, I have also worked on other forms of cross-connected software, two in particular.

14.1 Spreadsheet and Database Intertwingled in a Single Construct (Hyperthogonal Structure, Trademarked ZigZag)

In 1982, I realized that spreadsheet cells and database fields could be reduced to a single, minimalist construct—a cell connectable into crossed lists, or zzcell.

Conventional spreadsheets and databases can be built from zzcells, but so can other powerful structures harder to describe, crisscrossed in multiple ways at right angles (“hyperthogonal”).

This cell, the zzcell, is a curious construct. It is a unit that can be cross-connected in orthogonal dimensions. But these are not spatial or Cartesian dimensions, they are just listing dimensions, and the links are merely precedence links (Fig. 17.12).

Fig. 17.12
figure 12

Hyperthogonal cells shown in 3D. The internal mechanisms of the XanaduSpace demo are shown

Full size image

This has many consequences, consequences that are presented at the ZigZag home page [9]. That page also enumerates the different versions of ZigZag and the people who deserve credit for building them.

14.2 The Most Generalized Mutidimensional Graphics Engine

Conventional software deals with 2D objects (conventional electronic documents and tables) and 3D objects (“virtual reality” and CAD models).

Based on these other mechanisms, we have a graphics engine that does animated tweening in a multidimensional coordinate space. Thus, it is in principle the most general viewer. We call such tweening sworfing, since it can do either swooping or morphing.

This viewer (called Zzogl in its one instantiation) appears to work very well. However, it has only been exercised in three dimensions.

The viewer creates—that is, interactively presents—a viewing space of which all other viewing spaces are subsets. We may think of such N-dimensional Cartesian spaces as a generalization of “virtual reality.” But since it can also do front-to-back occlusions like conventional PUI windows, which are called 2½D (a term for which I thank Ron Baecker), this viewer may be said to allow N½D viewing. This viewer, then, may and should be used as a visualization substrate for everything else.

15 Merging the Generalizations: Generalized Documents, Data Cells, and Viewer

I have enumerated three fundamental designs, each of which I believe is the cleanest and most general possible design in its field:

  • Documents: xanalogical documents

  • Database and Spreadsheet: hyperthogonal data cells

  • Visualization: a multidimensional engine for arbitrary new spaces

How to fit these systems together is not determinate. Each of them is a construct system that generates a universe with its own rules.

Each of them is useful on its own, but I envision a single user environment built from all three—an everyday workspace offering documents and visualizations not otherwise possible. The hardest design problem I have faced is making these constructs fit together into a single clean system. I believe I have succeeded, but there is no room for the solution here. The problem now is to make it work.

16 Not in the Box

Thinking out of the box never meant anything to me because I never got into any box. I grew up in Greenwich Village and conventionality never appealed to me. I have tried to skip the obvious and find the magic.

In the computer world I have from the beginning considered myself an alternative school of thought, as if I were a university on another planet, not trapped in the ideas of the rest of the computer world. I’ve been a Free Range intellectual, outside the chicken-wire of academic departments and traditions that powerfully shape the thoughts of those who seek tenure.

Why? From an early age I was accustomed to having insights others did not dare to imagine. It has been my job to tell the truths people don’t want to hear.

17 Today’s Prison

My religion is human freedom and human creativity.

I dreamt of a liberating system of personal computing; instead we now have a computer world of imprisonment and imposition. I dreamt of a new liberating medium of hypertext that would make people smarter; that dream has turned into a flapping, screaming mess that slathers content with special effects and panders to the lowest minds.

I consider today’s computer world a nightmare honkytonk prison. From boyhood until college, school to me was imprisonment and imposition, and these same issues now define the computer world—imprisonment and imposition. So my course has been unchanged straight on till now.

18 Conclusion

What I intended to do in 1961 was done by others, but divided among Jobs, Gates, and Berners-Lee, all of whom did it wrong.

I inspired a lot of people with my book Computer Lib, but this gives me no joy, since what they did, and the way they did it, would have happened anyway. My anomalous position in the computer revolution parallels that of Hugh Hefner in the sexual revolution—each of us triggered, idealized, and publicized a revolution that was inevitable. Except Hefner was really conventional, though he didn’t know it.

Whereas my vision for the computing revolution has been far from the conventional. In decades of thinking and searching, I believe I have found the cleanest and most general designs for all interactive systems, each nonhierarchical and intertwingled—a system of documents far deeper, a system of data far richer, and a system of visualization far more sweeping, than proposed by those others who imitated the past. To work on anything else seems wrong to me, for I still must get these things working.

Control freak? I prefer the term “artist.” In the computer world I consider myself an artist of construct design, and I believe my constructs still hold great promise.

I believe this would be a much better world if I had succeeded. But I ain’t dead yet.