Picture this: while gazing into the sky watching the clouds drift by you notice a couple of clouds merge, and as they merge they clouds seem to transform into something else. As you watch the clouds mold each other you get the feeling that something familiar is appearing right there in the clouds; the form slowly takes a shape, and you can barely make out the contours of something resembling a human face, vaguely but definitely there. Wondering what kind of phenomenon this is you pick up your smartphone in order to make a quick search online, though not knowing exactly what to search for. You start typing: “clou”… you stop typing as a list of words and phrases appears to inform you of possible searches you probably wish to perform; you tap a proposal adequate to your original intentions.
Whether watching the passing clouds form into a familiar object in the sky or interacting with a digital search algorithm, patterns emerge in the compositional prehension of seemingly random data; which models the data are filtered through, however, determine the emerging pattern. Meaning-making processes are an intuitive and cognitive part of the perceptual experience of the world around us, and human perception has a tendency to seek, or impose, meaningful patterns in a ‘chunking’ of sensorial experience, thus providing a stable foundation in a world of chaos.
In Matter and Memory philosopher Henri Bergson distinguishes between two kinds of perceptual recognition: automatic (or habitual) recognition and attentive recognition. Bergson argues that modes of recognition emerges from and extends into movement, but that the difference lies in how memory interferes. Automatic recognition continues and prolongs perception in an anticipation of the immediate future (e.g. assuming that the search engine will make proposals as you type based on prior experience with the same search engine). Attentive recognition makes a cut in perception by dwelling on the perceived object and analyze it by projecting resembled memories onto it (e.g. the contours of clouds resembling a face) (2011). In the following, Bergson’s distinction between different modes of recognition will be useful in regards to thinking through the capture and processing of data in humans and machines alike in terms of the potential for both control and creativity offered by contemporary mnemotechnical apparatus’.
Control and Creativity: Archival Representation to Anarchival Modulation
The move from analogue archives of motion to digital archives in motion (Røssaak, 2010; Ernst, 2014) entails a transformation in archival practices effectively affecting how we experience the world. Representational meaning-making processes previously induced by traditional print culture have been replaced by the perpetual and modulatory processing of digital code in omnipresent, internetworked technology. In many ways we already live in societies of control (Deleuze, 1992) in which new technologies transform our lives, as we not only use but live with technology (Derrida, 1995, McCarthy & Wright, 2004). The anticipation for digital technology to offer new modes of thought and multiple access points to the interpretation and transformation of archived knowledge, and the socio-economical promise of equality and liberation imbued herein, is yet to be fulfilled. Previously appointed to the few, the construction and consignation of archived information today is delegated to the masses, as we participate like never before in mapping, tracking, and tracing our thoughts, bodies, and movements. This increase in participatory practices has paradoxically not sparked a new paradigm of individual expression, mutual understanding, and collective enunciation in which novelty and new relations can emerge, but has rather given way to neoliberal capture and reproduction of the same. So, if the capture and processing of data is automatically performed by means of networked protocols (Galloway, 2004), how is meaning then generated in data?
In what Bergson terms attentive recognition, memory might enrich perception to a great extent at times when the semblance between memory and sense-data do not form any immediate resemblance. Such interference by memory in perception will happen “until other details that are already known come to project themselves upon those details that remain unperceived” (2011: 123). This projection of memory upon perception can result in minor details to be blown out of proportion in the vivid imposing of meaning on random data (e.g. seeing faces in clouds), a tendency in human perception known as ‘apophenia’. However, superimposing a model on a set of data until a match is found, or generated, may result in exaggerations of minor details ending up defining the compositional whole. Such ‘overfitting’ can be compared to apophenia in machines with the example of Google’s ambition to automatically classify images through their Deep Dream initiative (Steyerl, 2016). Much like human apophenia it can result in unexpected findings, and Google’s Deep Dream has been popularized due to its psychedelic output resembling abstract art, a user-friendly interface, and open source access (open source becomes outsource by participatory cloaking).
In this sense, contemporary computational processing is moving from an automatic recognition of data only to also encompassing a certain degree of attentive recognition, as neural networks automatically and attentive transform data to fit the model they operate by. Always processing and always calculating probabilities this new form of governance is truly modulatory and preemptive in its algorithmic reproduction of the same patterns in new data.
While also sounding a warning about these anticipatory architectures of control, Luciana Parisi in Contagious Architecture proposes an inclusion of the speculative power of digital algorithms to generate unlived computational events. Parisi argues for a new mode of thought inherent in digital algorithms, a soft(ware) thought, by which she offers an supplementary alternative to the anthropocentric orientation of glitch art and affective interaction design in terms of their inclination towards human sensorial and perceptual effects. Through the generation of computational events, that have not yet been lived (and perhaps never will be), and if, following Brian Massumi’s argument, “[t]he overseen is unseen potential” (Massumi, 2011: 97), this paper raises the following question:
How might digital technology be designed to generate novelty and enable alternative perceptual patterns to emerge by making the imperceptible accessible, rather than engaging in a habitual reproduction of the same?
When Etienne Jules Marey operationalized his chronophotographical technique to capture a bird’s flight in a series of overlapping images 150 years ago, he was not only depicting the movement of a bird, he was also mapping the durational change in movement. Marey was in a way picturing time and making the differential duration between the discrete data felt, as the chronophotography is not a tracing of time attempting to account for the bird’s path, but a means to represent and thus make possible an analysis of change in motion.
Though often labeled as a predecessor to cinema, Marey’s work is recognized by Stephen Mamber in its own right in terms of his capture of discrete data as means of analyzing intervals in movement thus being able to measure the forces that determine the movement rather than describing the total movement (2006). In other words, although the individual images are captured chronologically, the lines drawn by the chronophotographical graphing can be accessed from multiple entry points. Rather than situating Marey’s work within cinema’s chronological continuity, Mamber recognizes the capture of discrete data as “a mode of digital thinking in that, by its very nature, it breaks down a continuous, on-going activity into a set of measurable, discrete components” (2006: 87).
In Mamber’s link between chronophotography and digital thinking he emphasizes data capture as a key concept, as neither intent to capture data as a realistic reproduction, but rather in a translational quantification “offering up an alternative vision” (2006: 89). This alternative vision is the analytic power of the diagrammatic capture of time, enriching perception by bringing the potential in the overseen into attentive recognition. Building from such diagrammatic capture of data, this paper further examines the potential in digital technology to create new relational patterns and generate events imperceptible to the human sensorium through the processing of data.
Montage Interdit is a web-based documentary platform by filmmaker Eyal Sivan and computational artist/researcher Robert M. Ochshorn. The platform consists of a number of different video clips, each represented by a thumbnail, which, when viewed in their totality, take the appearance of a mosaic. This particular mosaic, however, is not statically combined based on the visual appearance of the individual videos as parts merged into a whole, but in terms of how the videos, the data, function together in dynamic assemblages. Montage Interdit allows people to access an archive consisting of Jean-Luc Godard films and to interpret and transform the archive by rearranging the data according to editable and navigable metadata (e.g. “Africa” as seen below). The videos are grouped via annotated tags made by human editors. Opening up the mapping of the data through metadata creates multiple pathways to the interpretation of the archive as opposed to stabilizing and maintaining patterned relations.
Similar to Marey’s chronophotographic images Montage Interdit explores the potential in what Mamber calls “all-at-a-glance” in which the imaginative impulse in human perception is activated by excessively “showing you more than you can handle at the same time that you see all you need to know.” (Mamber, 2006: 90). In other words, the platform leaves it to the perceiving subject to participate in the attentive selection of data without imposing order by overfitting data and showing probable linkages between data. It does not order and compress the data’s complexity, but instead it builds complexity from which new movements of thought can rise.
Montage Interdit thus challenges the habitual way of watching a film or navigating an archive by showing all frames of the video archive at once and offering the possibility of zooming in and out of the chronological order of time thus perceiving the relational assemblages in either a single video or between multiple videos. By attentively dwelling on, for instance, a video in one contextual grouping, then clicking a tag annotated to that video and thus shifting the contextual order in which the video is played, it is possible to gain different perspectives on the same data. Montage Interdit not only offers alternive visions to how meaning is made when watching the videos in an “all-at-a-glance” perspective, the platform also allows the user to co-compose the technical substrate affecting these meaning-making processes by zooming in and out and reorganizing the archive.
Montage Interdit is a speculation into the power of digital computation to not only capture durational change in motion, as with Marey’s chronophotography, but also process motion in order to embrace and experiment with a non-linear configuration of the archive (both in terms of the archived frames in the video and the platform). In this sense, the platform can be thought of as an anarchive in which complexity breeds complexity instead of being suppressed by a certain order, as the metadata immanently edits the videos by restructuring the archival order and allowing the video(s) to be informed by an archive in motion as it translates from one relational context to another. In sum, Montage Interdit pushes towards novelty in archival practices that are not rooted in the tracings of the past, but open up for creative futurity and becoming.
Bergson, Henri (2011, org. 1911): Matter and Memory. Translation: Nancy Margaret Paul and W. Scott Palmer. Martino Publishing.
Deleuze, Gilles (1992): Postscript on the Societies of Control; in October, vol. 59, pp. 3-7.
Derrida, Jacques (1995): Archive Fever: A Freudian Impression. Translation: Eric Prenowitz. The University of Chicago Press.
Ernst, Wolfgang (2013): Digital Memory and the Archive. Editing and introduction: Jussi Parikka. University of Minnesota Press.
Galloway, Alexander (2004): Protocol: How Control Exists After Decentralization. The MIT Press.
Mamber, Stephen (2006): Marey, the analytic, and the digital; in Allegories of Communication: Intermedial Concerns from Cinema to the Digital. Editors: John Fullerton and Jan Olsson. John Libbey Pub.
Massumi, Brian (2011): Semblance and Event: Activist Philosophy and the Occurent Arts. The MIT Press.
McCarthy, John & Wright, Peter (2004): Technology as Experience. The MIT Press.
Parisi, Luciana (2013): Contagious Architechture: Computation, Aesthetics, and Space. The MIT Press.
Røssaak, Eivind (2010): The Archive in Motion: New Conceptions of the Archive on Contemporary Thought and New Media Practices. Novus Press.
Sivan, Eyal: Montage Interdit – web based documentary practice [in progress]. http://www.eyalsivan.info/index.php?p=elements1&id=8#&panel1-11
Steyerl, Hito (2016): A Sea of Data: Apophenia and Pattern (Mis-)Recognition; in e-flux, vol. 72.