This is an annotated script for a desktop-performance-lecture that was presented on May 12, 2020 as a culmination of prior research conducted between 2018 and 2020 on the history of the computer mouse. The text below was written after the recording of the performance, and is meant to accompany future viewings. The text formatted in italics is describing what's happening on the screen, including descriptions of the images and files being opened. The unformatted plain text inside quotation marks is what's being spoken. The text formatted in bold describes the mouse's actions. Complication of the Computer Mouse (Annotated) is a work in progress, and will continue to be. Through writing it, I've learned that the project of describing images is ongoing, potentially never-ending. While descriptions can appear concrete, they're also highly subjective and ultimately allow the person writing them to take part in what tech educator and designer Chancey Fleet calls a “reflective practice.” 1 Additionally, by attempting to name many things, one of them being the embedded structures of whiteness 2 and gendered divisions of labor within the history of computing, this script is meant to engage with Ruth Starr's point that "image description inherently intersects with questions of race, gender, and identity." 3 Finally, the text below is meant to explode what would otherwise be an 11-minute single channel video that depends heavily on images and visual cues; it's meant to contextualize what's being said, and to more deeply engage with the act of description.
INTRODUCTION
A view of a MacBook Pro laptop screen. The desktop wallpaper is a photo of Doug Engelbart, a white man, in the mountains. His picture is superimposed on a default Apple background called “Desert Rock”. The desert photo is included with the Mojave operating system. It shows a horizon in the Mojave Desert with gradients of pinks and blues in the sky and browns and grays in the earth. A video titled 1-IF.mov is already open and playing on the screen. Emma is centered in the frame of this video. She is a white woman with a navy blazer and blue collared shirt on. Behind her on the right there is an outer space themed mousepad hanging on the wall. There is another video in the lower right-hand corner of the screen. This is a live stream with a birds-eye view of Emma’s hand on the computer mouse, a black Apple Pro Mouse from the year 2000, which she uses throughout the performance. Underneath her hand and the mouse is a rainbow mousepad which is also the cover of a spiral bound book. There is a USB chord woven through top of the book. It is not visible in the video, but one end of the USB is plugged into the computer. The other end is where the Pro Mouse is plugged in. The mousepad book, if anything, is a dongle. Video 1-IF.mov takes up about 20% of the desktop area and is positioned to the upper left. Scattered around the desktop screen are all of the icons for the images and text files that Emma will open throughout the duration of the performance. Open text file titled hello.rtf. There are two lines in this file. The first line reads: emma rae norton. 4 The second line reads “Complication of the Computer Mouse.” Highlight the title and make the font size bigger. “I hope you’ll go along with this rather usual setting and the fact that I remain seated and the fact that I’m going to come to you mostly” Emma lifts the Apple Pro mouse from the desk and holds it to her chin, as if it were a walkie talkie. “through this medium here for the rest of the show.” 5 Close hello.rtf. “So, let’s get started.” Open images A.png, B.png, C.png, D.png, E.png, F.png, and G.png. “I’d first like to start with a manual for the Macintosh computer from 1984.” Each of these images is a page from the first six pages of the 1984 manual. On the cover page there is a drawing of the Macintosh computer. There are tables of contents on the next, graphics of dropdown menus in the graphical user interface and the hand on the mouse, and long descriptions for how to use the mouse. Each file opened from here on out takes up about 10% of the screen, some are a bit bigger, some a bit smaller. “Inside there are six pages whole pages which describe how to use a computer mouse. In the present touch screens allow us to navigate computers directly with our hands. The mouse, on the other hand, has become a kind of optional device. Its use assumed to be intuitive.” Open 0.jpg. An image of the Dépraz mouse. Made in 1980, this mouse is in the shape a dome and is bright red with three black buttons sitting at the bottom edge. It’s pretty unique as far as mouse designs go. “As the mouse becomes a less a necessary element” Open 1.jpg. An image from eBay of 5 white Apple Pro Mouse’s for sale. They are sitting in a circle with their chords tangled up in the middle. It looks kind of like a rat king. “within emerging ensembles of ubiquitous networked computing” Open 2.jpg. An image of a 1988 manual titled "Logitech Mouse User's Manual: Serial Mouse, Bus Mouse, Series/2 Mouse." The mouse pictured is gray and square and has three large dark grey buttons on the top. “I would like here to ask questions” Open 4.png. A drawing from a manual on how to clean the rubber ball of a mechanical mouse. The drawing depicts two hands. One hand is holding the mouse so that the bottom side meant for the desk is facing up. The other hand, between the index finger and the thumb, is twisting a black ball into the bottom of the mouse. There is an arrow pointing right to indicate the direction in which the cover which holds the ball in place should be pushed. “not only of how humans and computers shape one another but of what is surfaced when the mouse is taken on its own terms?” Open 3.png. A photo of the 2016 Evoluent VerticalMouse C Right Wired mouse. This mouse looks like a spaceship and it is quite tall. Half of its body is silver with a black concave area for the thumb and the other half is black with the Evoluent logo in bright blue letters positioned at the bottom. “what is surfaced when the mouse is taken on its own terms?” Move 1-IF.mov to another corner of the desktop once the images start to cover it. “what is surfaced when the mouse is taken on its own terms?” Video 1-IF.mov ends and closes automatically. Open coming-into-being.rtf. This is a text file which reads “Part 1 of 4, Coming into Being.”
PART I
COMING INTO BEING
Close coming-into-being.rtf. Open 2-YOU.mov. “What is elucidated” Emma begins slowly following the perimeter of her face with the mouse so that the red light coming out of the bottom highlights the parts of her face that it is traveling over. “when we see the mouse” Open 5.png. This is a photo from the 1960s of a white man in a jacket and tie, Bill English, using the oN-Line System. 6 He has one hand on the mouse, the other on the chordset. English was a part of Engelbart’s lab the Augmentation Research Center and is responsible for the first prototype of the mouse. “the object that quite literally” Open 6.png. This is a photo from the 1970s of Beau Hardeman, a Black man wearing a patterned shirt and vest, using a later version of the oN-Line system at a desk. He has one hand on the mouse, the other on the chordset. Hardeman was a computer scientist and grandmaster chess player who worked closely with Engelbart at the ARC. “sits in between your body and the screen, as an agent” Open 7.jpg. A close-up picture of Doug Engelbart’s hand on the first computer mouse. “with computers.” At this point Emma’s mouse has landed full circle, towards the bottom of her chin. She takes the mouse to the mousepad behind her on the wall but is still facing the camera. She slowly moves the mouse back and forth. Open 8.jpg. An Apple advertisement from 1984. Pictured is a white man’s hand with a leather glove on. The hand is sitting on top of Apple’s one button mouse. The text above the hand and mouse reads “Take Macintosh out for a test drive.” The image of the leather glove advertisement, in step with Emma’s hand on the hanging mouse pad, moves back and forth across the screen. “WITH WHICH particular modes of seeing and knowing does the mouse operate? MODES of seeing and knowing. With which particular modes of SEEING and KNOWING does the mouse operate?” Take the mouse off the mousepad and bring it back into microphone mode. “Let’s begin at MacWorld,” Open mac-world.mov. The first frame of the video is a still of the late Steve Jobs, a white man and founder of Apple, facing left so that we can see the profile of his face and upper body. He has his hands clasped together under his nose; it looks like he is praying. “the annual conference held by Apple for the release of their latest products. In the year 2000 Steve Jobs spent four whole minutes” Open 28.jpg. This is an image of U.S. patent 4,464,652 from August 7th, 1984. The patent belongs to Apple and shows a line drawing of their one button mouse in a kind of exploded view where you can see all of the individual mechanics inside. The title of the patent is “Cursor control device for use with display systems” “unveiling a new computer mouse. The Apple Pro Mouse was marketed as worlds away” Open 29.jpg. This is an image of U.S. patent 4,628,755 from December 16th, 1986. It has another line drawing of a square mouse body with three rectangular buttons on top. The title of the patent is “X-Y position controller having axially-inclined transducer members” “from its hockey puck-shaped predecessor.” Move 2-YOU.mov and the Mac World video to the forefront and place next to one another. “Jobs describes the Pro Mouse as innovative because it used optical technology which allowed for precision without the need for a mouse pad.” Emma puts the microphone mouse down. She faces right, towards Steve Jobs. She puts her hands into a prayer just like he is doing in the first frozen frame of the MacWorld video. She takes several seconds in this prayer position and then turns and presses play on the MacWorld video. The Steve Jobs video is playing, and she is facing the camera now, watching the video until he says, ‘Farewell mouse pad.’ Then she turns to look at the mousepad behind her. “Farewell mouse pad.” She takes the mousepad down from the wall. “Farewell mouse pad. Farewell mouse pad.” Video 2-YOU.mov ends. There is some movement in the lower left-hand corner video. The desk mouse on top of the mousepad-book is removed, and the book is opened to an image of Doug Engelbart in Oregon looking out at a mountain range, just like the mountain ranges offered as background images on our desktops. The Pro mouse is put back on top of Engelbart’s image.
PART II
WHOLES THROUGH PARTS
Open 3-GIVE.mov. Emma is turned again to the left so that her profile is in view. She is using the mouse as if it were a walkie talkie like Scotty does in Star Trek IV: The Voyage Home. "This removal of the need for the mouse pad" Open wholes-through-parts.rtf. This text file reads “Part 2 of 4: Wholes through Parts.” “was a step towards the mouse becoming less dependent on its environment,” Close wholes-through-parts.rtf. Close mac-world.mov. “a less necessary element” Open 10.png. This image is from Apple Times, Volume 2, No. 1, January 1982 of 12 Southeast Asian people sitting on a floor in a circle, each of them has a personal computer in front of them, and most of them are smiling and looking at the camera. The printed caption for the image reads: These are the first twelve Apple Singapore employees to obtain their Loan-To-Own computers. They did it by taking software classes during their lunch and tea breaks. Technician Johnson Cho, 82, says it is "...fantastic! I can calculate my Income tax and type letters to my friends with it." The employees pictured here are, clockwise from top: Mavis Soh, Caroline Lim, Alice How, Sheen Kan, Agnes Lee, M.T. Chan, H.C. Chiam, Herbert Lim, Irene Lim, Mike Lee, Wendy Ng, and Johnson Cho. “within the larger wholes it creates and is created by.” Emma turns to the wall behind her, places the mouse on the wall and starts very slowly dragging it in a clockwise circle, just like she did on her face earlier. “Perhaps an UNDERSTANDING” Move 3-GIVE.mov in a clockwise circle, mirroring the one being made by the mouse on the wall. “of the computer mouse’s coming into being, the computer mouse’s COMING into BEING, the computer mouse’s coming INTO being, would prove to be immediately concerned with its larger wholes.” End circle. The 3-GIVE.mov comes full circle. Emma switches back to a profile view walkie talkie view. “The question then becomes” Open 20.png. An image from the 1940s or 50s. A white woman holding a large cathode ray tube, it looks like an oversized lightbulb and is probably around 2-3 feet wide. “which larger wholes allow the mouse to emerge? I’m thinking radar, artificial intelligence,” Open 12.png. An image of the one button Apple mouse. “office work, global computing assembly lines” Open 11.png. An advertisement for an IBM mouse that reads “Why you should buy a mouse with no moving parts. Now.” Underneath there is an image of a 20-dollar bill and two mice, the two-button IBM mouse with a bunch of small electronic components next to it, and a non-descript three-button mouse which ostensibly has a bunch of moving parts inside. “and integrated circuits.” Move all of the open images around. “And, how has the mouse re-oriented us to those wholes? We click to like. Click to buy. Click to see. Click to know.” Open ali-na.rtf. This text file reads 'continually supplementing, deferring, and redirecting that desire.' 7 “In step with media scholar Ali Na we can understand the click” Highlight the following words as Emma says them. “as continually supplementing, deferring, and redirecting our desires.” Close ali-na.rtf. 3-GIVE.mov ends. Open 24.png. A line drawing of a hand on a mouse in front of a desktop computer with a circle graphic on the screen.
PART III
MOUSE USERS AND TYPISTS
Open 4-A.mov. Open mouse-users-and-typists.rtf. Now Emma is facing left, holding the mouse to her ear like a telephone and speaking into it. “I wonder which worlds have been opened up the by the mouse and which have been closed.” Close mouse-users-and-typists.rtf. “Literary theorist Gayatri Spivak” Open spivak.rtf. “wrote ‘The globe is on our computers. No one lives there.'" 8 “With the mouse we move through this globe, this abstraction of the world,” Open 13.png. An advertisement for a personal computer. One side of the page is a drawing of a desktop computer with a keyboard. Underneath it reads “Era one.” On the other side of the page there is a drawing of earth with a keyboard plugged into it. Underneath it reads “Era two.” Close spivak.rtf. “this singular representation of the planet. How does the mouse sustain a view of the world as global?” Open 14.jpg. Three people in an office cubicle with Apple computers. One white woman is sitting at the computer, a white man is standing next to her, pointing to something on the screen while saying something to another white woman who is standing across from the white other woman. “And, how does it also remind us of embodied and planetary realities? It’s important to note that the mouse assumes a lot about your body,” Open 25.gif. An animated gif of Doug Engelbart’s hands using the chordset and mouse in the oN-Line System. “that you are able to make disciplined movement with your arm.” Slowly move 25.gif around the screen. “that you can follow with your eyes a tiny cursor on a screen.” Again, slowly move 25.gif around the screen. “disciplined movement with your arm, that you can follow with your eyes a tiny cursor on a screen.” Open 19.png. An image of a manual cover for the Apple Lisa Personal Computer. The title reads Questions and Answers (Feb 83). Below the title is a graphic of the Lisa computer. “The mouse, in coordination with the graphical user interface, simultaneously made computers” Shuffle images around. “easy to use and incredibly inaccessible.” Open 15.png. Three people in an office. A white woman is sitting at a terminal, looking at a piece of paper on her desk. A Black woman is sitting at another terminal pointing to something on the screen. There is a white man standing behind her looking at her screen with a cup of coffee in his hand. “And, throughout the 1980’s, as the computer and the mouse became more dependent on each other, the computer -- no longer a glorified typewriter was beginning to be seen as a tool for higher-ups in the office.” Try to find the image of Engelbart’s hand on the mouse, or open 27.png. A tear sheet from a computing magazine that reads ‘Early mouse users were mostly men.’ in enlarged text. “This often meant managerial men…who in the previous decades…would not be caught in the feminized act of typing at a computer…were now the primary users of computers and therefore the mouse.” Re-open Engelbart’s hand on the mouse image. “I think it’s worth noting that the inventor of the mouse, Douglas Engelbart,” Remove hand from the mouse on desk, let images stand still. “like many men of his time, did not know how to type.” Video 4-A.mov ends. Open 5-MOUSE.mov. Emma is facing front, using the mouse as a microphone. “The movements and actions made with the mouse are and have always been about dragging” Drag 5-MOUSE.mov up, down, left and right. “up, down, left or right. The technology within the mouse evolved from a collection of small separately working mechanical parts” Move mouse off mouse-pad book which is still open to Engelbart’s photo. Flip through pages to find the image of the radar technician pointing a light gun at a video display. “into to a streamlined architecture of a kind of optic system. All of the elements within the shell of the optical mouse rely on encoded light, the ability to differentiate between 1 to 2-pixel images of the surface below.” Bring mouse back on top of mouse-pad book. “This tracking and capturing of light goes all the way back to histories of radar technology and the use of input devices that looked like guns.” Move live video stream of the mousepad to the left, bring second live video stream of the mouse-as-camera and slowly hover back and forth over the image of the radar technician’s light gun. “The mouse is unlike its predecessor, the light gun. There is nowhere to point it. No target. Nothing for the mouse to go into and likewise nothing to go into it. If found alone on a desk the intuition to pick the mouse up might be as far as you go, it is not asking to be pointed and projected although this is complicated by what it is meant to do once connected to a computer. What happens in the embodied space before the computer is a slow and steady movement of the hand a few centimeters at a time. 5-MOUSE.mov ends. Minimize the live video feed of the mouse-as-camera to remove it from view.
PART IV
ELEMENTS AND ENSEMBLES
Open elements-and-ensembles.rtf. This file reads “Part 4 of 4, Elements and Ensembles.” Open 6-GILBERT.mov. Emma is facing the computer at a 45-degree angle, using the mouse as a microphone, the red light from the optical sensor is reflected on her shirt. “What happens when we see the mouse not only as a part of a much larger of ensembles of networked computing but also to see it as an ensemble itself.” Open 57.png. A close-up image of the inside of a mechanical mouse, the roller ball is surrounded by tiny transistors and resistors and other small electronic components. “to SEE the computer mouse” Close elements-and-ensembles.rtf. “as an ENSEMBLE itself. When we look closely” Open 16.png. An image of an early trackball device. Surrounded by rotary encoders and other electronic components is a five-pin bowling ball. “at the mouse, as an ensemble, we can see that it is looking closely at us, that it is ensuring that we see just like it sees,” Open 17.png. A page from an instruction manual for the 1980s arcade game Missile Command which featured a track ball in its console. The instructions are for taking the console apart and replacing the track ball. “the world as one whole, made up of parts with clear boundaries, a way of seeing that does not reflect the messy world the slowly moving hand” Open 22.png. A line drawing of the inside of an early optical mouse. “and the disciplined arm is moving within.” Open simondon.rtf. This text file reads 're-situated in the past, in order to grasp the technicity of ensembles in their reality.' 9 “To reflect on Gilbert Simondon’s philosophy of technical objects, it can be a generative act when we understand technical elements as “re-situated in their past” in order to understand the larger ensembles of their present reality.” Close simondon.rtf. “That is to understand an object through its coming into being” Open 18.png. A line drawing from Doug Engelbart’s first mouse. There are two figures drawn, both showing the inner mechanics, one from the top view and another from the side view. The drawings are relatively simple compared to later patent drawings of computer mouses. “allows us to understand it as an object on its own. To have a philosophy of the computer mouse” Move open images around the screen. “is to move towards a philosophy of technological objects from their elements all the way up to their ensembles.” 6-GILBERT.mov ends.
CONCLUSION
Open 7-SIMONDON.mov. Move all open images randomly around, with a bit more fervor than before. Uncover images from much earlier in the video, move the video stream of the mouse on the desk to the center of the screen. Emma is positioned much closer to the camera now and is using the mouse as a walkie-talkie. “Coming back to the hand, the mouse centers the hand, it holds the hand, it’s a handshake, a moment of entanglement, between the body and the computer, and if we stop to pay attention to it, we begin to see its power as a technological object. A staying with the mouse is a staying with the present in order to simultaneously imagine past and future worlds of computing. It’s a way in. To the screen, to deepen its flatness, to complicate its representationalism with performative and embodied movement. We move the mouse through our world and the mouse moves us through its world, it stays between these two worlds and by looking closely, holding the mouse, we can feel that it holds us.” Emma pauses here for a few seconds, nothing else is moving. Thank you. 7-SIMONDON.mov ends. Screen fades to black, a few more clicks can be heard in the background.
1. Chancey Fleet, https://twitter.com/ChanceyFleet/status/12527152868825866252. For a closer look into the embedded whiteness of the screen and Silicon Valley at large see American Artist, BLACK GOOEY UNIVERSE, https://unbag.net/end/black-gooey-universe3. Ruth Starr, Cooper Hewitt Guidelines for Image Description, https://www.cooperhewitt.org/cooper-hewitt-guidelines-for-image-description4. I've been using different last names in different places. And sometimes, as is the case in this text file, I write my name all lower case. This is something I learned from danah michele boyd's writing on capitalization, https://www.danah.org/name.html5. This is what Doug Engelbart says at the very beginning of “The Mother of All Demos”6. The oN-Line System was developed by Engelbart and his team at the Augmentation Research Center at Stanford in the 1960s. It is an amalgamation of devices which demonstrated some of the earliest versions hyperlinks, cross-computer collaboration, video conferencing, and the use of a computer mouse.7. Ali Na, "The Fetish of the Click: A Small History of the Computer Mouse as Vulva," Feminist Media Studies 18, no. 2 (March 4, 2018): 221–34.8. Gayatri Chakravorty Spivak, Imperatives to Re-Imagine the Planet (Vienna: Passagen Verlag, 1999), 44.9. Gilbert Simondon, On the Mode of Existence of Technical Objects (Minneapolis, MN: Univocal Publishing; Minneapolis, Minn.: University of Minnesota Press, 2017), 238.