Ken Jordan
DEFINING MULTIMEDIA
(3 of 4)

[Note: This is part 3 of a paper-in-progress that grew out of my
collaboration with Randall Packer, Multimedia: From Wagner to Virtual
Reality (W.W. Norton, 2001, and on ArtMusuem.Net). Part 1 proposed a
definition of digital multimedia based on five core characteristics. Part 2
compared our definition to the one proposed in Lev Manovich's The Language
of New Media. Part 4 will be posted soon. Comments are welcome.]


3. The Modernist Thread

For the purpose of our project, Randall and I felt that the term "digital
multimedia" seemed to be the most appropriate – rather than "new media,"
"digital media," etc. – because it emphasizes the form's continuity with
efforts in the arts that came before. The word "multimedia" was coined by
artists in the 1960s to describe avant-garde practices that not only mix
diverse media, but also emphasize audience participation, non-linear
narrative structures, and indeterminacy. There is a line in the development
of computer-based media that runs parallel to an important trajectory in
modernism. We want to make that connection explicit.

This is not to say that digital multimedia grew out of a cohesive, carefully
coordinated strategy. But looking back, you can identify a few consistent
themes that drove the medium's development over a half century. These themes
were pursued concurrently with other, at times conflicting, objectives. But
in retrospect the extent of a consistent vision shared by the scientists and
artists who pioneered multimedia is quite profound – as is the mutual
influence between science and art (with conceptual and technological
breakthroughs feeding one another) that led to the computer-based media we
know today. Eventually these diverse efforts coalesced into a meta-medium,
to borrow a phrase from Alan Kay. [1] Kay is the man who tied the loose
threads of digital multimedia together in the late 1960s, by designing the
prototype for the first true multimedia computer, the Dynabook.

Vannevar Bush began it all by proposing a mechanical device that operated
literally "as we may think." [2] The challenge, as he discussed it in his
famous article of 1945, was to create a machine that supported the mind's
process of free association in the act of creation. This aspect of Bush's
hypothetical machine, which he dubbed the memex, tends to get overlooked
today. What gets attention instead are the many ways the memex foreshadows
the personal computer – particularly its ability to call up media objects
from a database. Bush did not use the word "database," because the memex, as
he described it, was not a digital device. It was analog: a desktop and
storage space that gave access to microfilm, audio recordings, photographs,
and movies. It was, in a way, a kind of library – but with a crucial
difference. Libraries arrange information linearly. Bush, however, was
interested in rearranging information according to the idiosyncratic paths
of personal association that each individual invents during the creative
process. He wanted a machine that encouraged spontaneous, associative,
stream-of-consciousness thinking, and then left a trail of that thought
process behind so that it could be retrieved, not only by the individual who
created it, but by others as well. In this way, the memex would allow people
to share their private, unconsidered thoughts as they leap between ideas
moment by moment.

Bush was interested in identifying a central aspect of consciousness, and
making a device that effectively expanded consciousness through mechanical
means. If you look at the history of the personal computer from this
perspective – as an ongoing project to create a media machine that enhances
the intuitive, associative tendencies of consciousness – it connects
digital media inextricably to important currents that run through modernism.

Bush had taken, essentially, an esthetic position – an esthetic position
that shares remarkable qualities with some unexpected bedfellows. These are
contemporaries with whom Bush is never associated, particularly as he was
FDR's chief science advisor and the architect of the military industrial
complex. Still, as the person who proposed that information should be
organized and saved mechanically in a way that captures the spontaneous
movement of the mind, it is inevitable that he should be linked to others
who shared similar interests in mid-century.

For example, during the 1940s Charlie Parker was pioneering a new musical
vocabulary based on spontaneous improvisation – one that went far beyond
the method established by Louis Armstrong. Parker's radical approach to
improvisation, the charts be damned, placed non-linear associative thinking
above all else in jazz, and led to the free jazz of John Coltrane, Ornette
Coleman, and others in the 1960s and 70s. In painting, Jackson Pollock was
taking a similar approach at the time, dripping paint in loops following the
dictates of his spirit, never following a plan or a sketch. The privileging
of spontaneous action was central to Pollock's practice. In literature,
during these same years, Jack Kerouac pursued a method of "spontaneous bop
prosody" – as he called it – that led him to write novels that captured
the movement of his mind moment-by-moment in the act of creation; a steady
stream of honest personal observation that used associative thinking as its
central organizing principle.

The prim bureaucrat Vannevar Bush might have been surprised to find himself
in such unkempt, but august, company. However, looking back the similarities
between Bush and the mid-century American avant-garde are obvious. They
shared an esthetic that treats the individual's private impulse as primary,
and that gives people permission to act in a non-linear, irrational way, as
society would define it. Bush's interest was to enable each of us to shape
data into the form that serves us best, rather than to conform our private
thought process to an organization set by others. This opposition between
self and society is not absolute, of course (though in mid-century the
tension between private impulse and social conformity was an intellectual
flash point, especially because of the threats of Fascism and Stalinism, on
the one hand, and the theories of Freud, on the other). That digital media
can trace its birth to the intent to mine this opposition, however, is
significant.

Bush's vision inspired a generation of computer pioneers in the 1960s, and
led directly to the personal computer. Douglas Engelbart, for one, was
famously inspired by "As We May Think," and dedicated himself to building a
working model of Bush's association machine – this during the same years
that Coltrane, Pollock, and Kerouac (not to mention their many cohorts, and
the legions of young artists they inspired) had broken through to the
mainstream. The assumption that "great art" was made through the formal
arrangement of spontaneous impulses was not only the mantra of cultural
bohemians; it was a notion hotly debated in the popular press. The birth of
the personal computer belongs to this moment.

Engelbart expanded on Bush's premise by designing an oNLine System that
would "augment human intellect," as he put it, [3] based on the insight that
the open flow of ideas and information (as represented by texts and
pictures) between collaborators was as important to creativity as private
free association. At the same time, J.C.R. Licklider envisioned universal
networked access to the full "library" of human knowledge. This idea led him
to spearhead the early development of the Internet while he ran a research
program for the Defense Department, ARPA. Soon after, Ted Nelson followed
with a proposal for a "hypermedia" system (he coined the term) that would
fulfill Bush's objective to arrange materials from this "library" in a
manner that reflects how the mind moves freely from one thought to another.
[4]

Central to all these efforts was the notion that the user should not only
have access to media objects, so she can organize them as she pleases, but
that the computer user should also be able to interact with media objects,
and change them to suit the needs of the moment. Editing and recombining
digital media was seen as essential to the utility of the computer.
Licklider, in his seminal article "Man-Computer Symbiosis," [5] proposed
that the computer should act as an extension of the human capabilities for
cognition and communication – which includes, of course, the manipulation
of media. Engelbart's oNLine System was designed specifically for the
collaborative manipulation of digital media over a wired network. In keeping
with Bush's vision of the memex as a way to enhance creativity, these
pioneers insisted that the computer user's ability to interact with and
change media should be as great as possible. Tim Berners-Lee has often said
that he considered the edit function in the first Web browser to be just as
important as the ability to link between Web pages; for the Web to be
successful, he felt it essential that each reader could also be an author,
able to annotate Web pages by adding "private links." [6]

This approach to interactivity paralleled currents in the avant-garde,
particularly in performance. In 1948, John Cage introduced the idea of live
performance as unscripted event, in which the audience encounters people,
objects, and activities within a defined space, in surprising juxtaposition
to one another. The audience is encouraged to become creative participants
in the work of art as it occurs. [7] This type of performance, which Allan
Kaprow later named Happenings [8], shared many concerns with the way
engineers were shaping online interactive environments. Both engineers and
artists were addressing the question: how do you encourage the appropriate
dynamic encounter between people within a framed situation? And they reached
a similar conclusion: give the user/participant as much freedom to act as
possible.

Implicit in Bush's memex is the suggestion that a mechanical device can
replicate the intimate movement of the mind at play, by representing media
objects of all kinds in any order, as the user desires. From this, it
follows that a computer might one day effectively mimic the encounter of
consciousness with the world through the senses, by arranging media objects
in a way that mimics reality. Though Bush himself did not make this leap,
engineers influenced by his vision in the early 1960s did, and none more
profoundly than Ivan Sutherland.

Sutherland was the first person to propose that bits and bytes could be
represented as three-dimensional virtual environments. In his article from
1965, "The Ultimate Display," [9] he began with the idea that by digitizing
information – transforming it into ones and zeros – all data became
subject to the graceful manipulations made possible by mathematics. This, in
turn, invites the computer programmer to shape data into a three-dimensional
form that mimics the way we encounter information in the physical world.
Like Bush, Sutherland's approach to the formal arrangement of information is
essentially an esthetic stance. This particular esthetic stance can be
traced back to the mid-19th century writings of Richard Wagner, which
declared that art should do its best to recreate the full, multi-sensory
engagement between the self and the world. To facilitate his vision, Wagner
reinvented the conventions of the opera house, and in 1876 opened the
Festpielhaus Theater in Bayreuth, Germany. It was the first modern theater
to employ Greek amphitheatrical seating, surround-sound accoustics, the
darkening of the house, and the placement of musicians in an orchestra pit
– all to focus the audience's attention on the dramatic action, and
transport them into an illusionary world staged within the proscenium arch.
Wagner's call for an immersive "collective artwork" that fuses all the arts
into a single expression [10] – his "Gesamtkunstwerk" – is echoed in the
last paragraph of Sutherland's 1965 paper:

"The ultimate display would, of course, be a room within which the computer
can control the existence of matter. A chair displayed in such a room would
be good enough to sit in. Handcuffs displayed in such a room would be
confining, and a bullet displayed in such a room would be fatal. With
appropriate programming such a display could literally be the Wonderland
into which Alice walked." [11]

Sutherland presented this paper at an engineering conference, and it was
first published in a technical journal. But it is hard to ignore how much it
reads like a manifesto written by an Italian Futurist. There is, in fact, a
remarkable similarity between the tone and intention of articles by certain
computer media engineers and fiery artistic manifestos. The modernist
imperative to "make it new" (in Pound's famous phrase), and the belief that
society will be transformed as a result, is very much present in writing by
computer scientists. Digital multimedia may well force us to reconsider the
entire historic arc of modernism, including its supposed end, since the
esthetic stance of modernism has become increasingly relevant in response to
digital media.

When Alan Kay designed the prototype for the Dynabook, in the late 1960s,
the intellectual foundation was in place for a digital multimedia that
synthesized all existing art forms, and presented them in an environment
that enabled meaningful interactivity and hyperlinks. With the requisite
processing power, it would eventually incorporate Sutherland's experiments
with three dimensional representations. This meta-medium, to use Kay's term,
carried with it specific, idealistic attitudes and intentions about human
creativity and communications. It reflected a commitment to media forms that
are nonhierarchical, open, collaborative, and emulate the free movement of
the mind at play. It is, in sum, an extraordinary vision.

—–

Notes:

[1] Alan Kay and Adele Goldberg, "Personal Dynamic Media," in Multimedia:
From Wagner to Virtual Reality, Randall Packer and Ken Jordan, eds. (New
York: W.W. Norton, 2001), p. 167

[2] Vannevar Bush, "As We May Think," ibid, p. 135

[3] Douglas Engelbart, "Augmenting Human Intellect: A Conceptual Framework,"
ibid, p. 64

[4] Ted Nelson, "excerpt from Computer Lib/Dream Machines," ibid, p. 154

[5] J.C.R. Licklider, "Man-Computer Symbiosis," ibid, p. 55

[6] Tim Berners-Lee, "Information Management: A Proposal," ibid, p. 189

[7] John Cage, "Diary: Audience 1966," ibid, p. 91

[8] Allan Kaprow, "Untitled Guidelines for Happenings," ibid, p. 279

[9] Ivan Sutherland, "The Ultimate Display," ibid, p. 232

[10] Richard Wagner, "Outlines of the Artwork of the Future," ibid, p. 3

[11] Ivan Sutherland, ibid, p. 236


————
Ken Jordan
ken@kenjordan.tv
212-741-6173

"Be as if." - Andrew Boyd