“Myths, folktales, fairy tales– these are the prototypes of all narrative, the ancestors and the models of later fictional developments. In studying the history of narrative, we find that in modern times forms have developed which elaborate and transform the basic constituents of primitive fiction almost beyond recognition, but we also find that modern fictional forms have never lost touch with the primitive entirely and have frequently returned to their sources to draw upon the almost magical power they possess.” (Scholes, 1978)

It is an innate tendency of humans to tell or follow stories. From stories that describe the creation of the universe to patterns we identify in clouds, we try to impose order on our experience of the world, unable to perceive a universe without a purpose (Yorke, 2014). Even the most abstract figures can be interpreted as human characters of a story, such as the movements of two triangles and one circle in the Heider-Simmel animation, an experimental study of the two psychologists in 1944 (Heider & Simmel, 1944).

Cinema shaped 20th century storytelling by moving the story from the written word to the screen. In similar fashion, contemporary methods of intermedia storytelling transform the cinematic into a hybridised art that integrates theatre, architecture, or even computer science (Manovich, 2001). In our design project , named Neural Kubrick, we computationally deconstruct classic films and regenerate them into a spatial installation which introduces the viewer to the way the computer interprets cinema. Through this process several questions are raised regarding the boundaries between cinema and contemporary immersive installation art, architecture and filmmaking, the stories we are told and the meaning we make.
Filmmaker Hito Steyerl introduces the term “apophenia”, in her article about machine intelligence with the name Apophenia and Pattern (Mis-)Recognition (2016). She defines it as “the perception of patterns within random data”, a proclivity of humans to connect unconnected information, a tactic often employed by analysts (ibid). In this report, I am using Christopher Booker’s theory that there are several governing patterns in the narratives which people have been constructing since time immemorial, and identify where these patterns are met in the narratives that are made by the nascent technologies of new media[1].  In the first part of this dissertation, the patterns and their subversions are briefly analysed, along with their connection to architectural composition. In the third chapter, I investigate how the screen-based linear narrative of cinema is expressed, while the last chapter taps into the new media realm, examining whether Booker’s patterns can still be identified in hybrid media installation or in videos generated by a machine learning algorithm. Is this another case of analytic apophenia, or do these patterns — hardwired into our unconscious as they are — reoccur in the arts of computer age? Are there patterns in the narratives of today’s exploded screens? If we can find traces of Booker’s patterns in 3-dimentional space, does artificial intelligence understand the same patterns or does it generate its own?

Our experiments focused around three types of machine learning algorithms : the Generative Adversarial Networks (GANs)[1], Recurrent Neural Networks (RNNs) [2]  and Reverse Image Search[3]. Due to the nature of this report, I will not delve into detailed explanation of how the networks operate, but only give further details on how our choices affected the results.

2. Patterns of Stories

A characteristic example of the universality of stories can be traced in the variations of folktales such as the Cinderella story, which is found in different countries across Europe (Booker, 2016). Several writers have tried to scrutinise the patterns that govern stories across centuries and cultures. Aristotle(1907) was the first one who, in Poetics, identified the three stages of a complete story , beginning, middle and end, while recognised two main categories depending on the story’s ending which can be summed up as the “comedy“ and the “tragedy“. The first type begins with complications in the hero’s situation that worsen and appear to be inescapable, but a “reversal of fortune“ brings happy ending. Tragedy on the other hand, begins with a seemingly happy situation that turns into a disaster. Carl Jung(1988) analysed how the mental images in ancient mythology and art are embedded in the collective unconscious of today’s humans, while Joseph Campbell (2008) in his book The Hero with a Thousand Faces uses the term “monomyth” describes the archetypical journey of a hero through an adventure, crisis, a series of trials, transformation and return. Different attempts for classifications of storytelling have been made, most notably Georges Polti’s(1960) The 36 Dramatic Situations, Ronald Tobias’s (2003) 20 Master Plots or Christopher Booker’s (2016) The Seven Basic Plots. For the purpose of this thesis only the latter classification is examined further, it being one of the most concise recent approaches.

 

2.1 The seven basic plots
According to Christopher Booker’s theory, the seven basic archetypical plots to which most of the known stories can align with are the following: Overcoming the Monster, Rags to Riches, the Quest, Voyage and Return, Comedy, Tragedy, and Rebirth (2016). The following diagram briefly summarises the basic attributes of each plot:

Booker adds two more types which have not been as popular until recently:

A characteristic common for all types of plot is an archetypical 5-act structure which is observed by Booker in 5 stages : (1) anticipation, or call to adventure, (2) dream stage, where hero meets initial success (3) frustration stage, where things start to go wrong and the hero comes across his enemy (4) nightmare stage, where everything seems to go wrong and disaster seems unavoidable (5) resolution, that comes with victory and union with ‘perfect love’ or destruction, in the case of tragedy. This structure constitutes the main pattern that, for Booker, governs most types of stories.
All stories begin with the protagonist being incomplete and in some state of “limited consciousness”. During the story she encounters “light figures”, characters that play a guiding role, but most importantly “dark figures”, that usually have the qualities that the hero lacks and are her main rivals. To overcome the darkness, the hero has to go through a transformation and by the end of the story become a mature person. This formula is inverted in the case of tragedy, in which the hero is completely taken over by darkness, thus becoming herself the monster.
Abstracting the structure typologies from their details and observing their structure paths and their recurring figures, one can identify a lot of similarities with architectural paths. The dipole between light and darkness or the linear progression either upwards or downwards, are a few of the patterns that can also be traced in spatial narratives of everyday life. This parallelism will be explored further in following chapters.

 

2.2 Detachment from the plot
The author recognises a shift in storytelling over the last two centuries which was triggered during the age of Romanticism. The most important of the transformations that have occurred over the last two hundred years is that the story has lost its archetypical form due to the projection of the author’s dreams and ego onto it. Subsequently, no character is only light or dark, but appears in ‘twilight’ and is not transformed at the end of the story, thus not leading the plot to a resolution. For Booker, this is due to the modernisation of life which brought a detachment between the ego and the Self and a “cosmic and spiritual dead end”. As a consequence, the stories either appear as “going nowhere”, or are focused on sex and violence. In Samuel Becket’s Waiting for Godot, the two heroes are waiting for some mysterious character —the name of whom resembles God — whose arrival could bring meaning to their lives. But Godot never arrives and even though the characters decide to commit suicide, they do nothing. This relates to Christian Metz’s account on the phenomenology of narrative in cinema, in which he identifies narrative as a closed sequence of events and therefore contends that “traditional narratives, with their definite conclusions, are closed sequences of closed events; the trick-ending narratives which cultural modernism enjoys are closed sequences of unclosed events” (Metz, 2007).
The above indicates that storytelling is a lot more complicated than what a classification in seven types can offer. A plot can combine two or more types, can start as one and end as another or have no resolution at all. Nonetheless, Booker’s theory can function as a scheme in order to study how a contemporary story works. Media culture has undeniably transformed the concept of storytelling in unprecedented ways. The computer, along with the screen, are our main diodes to information (Manovich, 2001). Our everyday lives are surrounded by screens and penetrated by networks that transmit multiple stories, leading us to perceive the world through a frame more often than we do directly (Friedberg, 2009). How did that radical change affect the narratives we create?

 

2.3 The narratives of architecture
Bernard Tschumi (1996) in Architecture and Disjunction draws parallels between space and narratives. The author finds the analogous of cinematic sequences in spatial transformations, according to several techniques which direct the movement of users in space, such as sets of rules and variations, fusions or repetitions. Tschumi asserts that for all methods of transformation in spatial arrangement, one can locate their relevant method in the cinematic composition. A baroque element in a modern building takes the role of the flashback in a film (ibid). Tschumi’s main influence on the ideas between architecture and film was Sergei Eisenstein’s theories on montage (Bruno, 2002). The connecting point between the two arts is for Eisenstein the promenade, in the sense that the path inside an “architectural ensemble” is the embodiment of the imaginary path of the film spectator (ibid). Through enacting a promenade in the Acropolis of Athens, Eisenstein explains the way the user’s path constructs meaning, like the montage does in every film shot (ibid). The difference to cinema comes due to the fact that architecture is lived and thus the sequences are ever-changing, as a series of events which transform the fixed architectural programme.
If there is a correlation between the sequence in space and in the timeframe of film, would it be possible to apply Booker’s set of patterns to architecture? Tschumi distinguishes closed sequences of transformation from open ones, the former being predictable and having exhausted its transformational rules, the latter being open-ended and accepting new elements (ibid). One could argue that the closed sequences are analogous to the 7 basic plots and the open to their detached subversions. At the same time, the events of architecture bring a new level of complexity to the initial plot that is the programme. Tschumi points out what he calls “conflict”, that is, when an event contradicts the programmatic rule of space, when for instance the user sleeps in the kitchen.
The ritual, the quintessential example of control and unchanged order, is used by Tschumi to examine a linear sequence of events, where “successive challenges await the new candidate”(ibid). In The Book of Tea, Kakuzo Okakura (1906) describes the configuration of the tea room, a ritualistic piece of architecture defined by purism and absolute order, taking the participant through distinct stages in space, according to the stages of tea ceremony. The experience of each of the spaces reflects various Zen doctrines. For instance, the garden path that leads from the portico to the tea room signifies “the passage into self-illumination” (ibid). Yet, in the midst of complete order, the very architecture of the tea room manifests impermanence and incompleteness. As Okakura (ibid) states, the humble and ephemeral structure with simplistic decoration signifies the view of “house only as a temporary refuge for the body”, whilst also shows that the tea-room is designed for the present moment and not for the eternal, expressing the individuality of each guest’s choice. Zen philosophy aims at the incomplete and fears repetition, hence the simple and asymmetrical design, where no object or design appears more than once in the setting. The composition leaves the completeness to emanate from the user’s imagination.

Here one can observe a contradiction. What is seemingly a closed system par excellence, a narrative with distinct stages and rhythm, is in fact an open system that invites the guest’s mental images to fill in the gaps of an almost empty space. It is this very subtle transition from the ordered to the open-ended that the tea room achieves, that is useful for our design. Through the careful curation of an experience, the tea ceremony leaves space for the mind to wander.

 

2.4 Narratives in our research
Having introduced the theoretical background, it is important to identify in which ways these rules are associated to our design. The thread that connects theory and design is divided in two key questions. First, since patterns of stories – linear or fragmented- can be traced in architectural space, in what form do these plots appear in our own new media installation? Similarly, the next question refers to the comparison between the machinic eye and the filmmaker’s eye in the production of time-based art. In other words, can the machine make meaning? To answer these questions, Booker’s theory is used as a roadmap with which we observe the generated stories. With the use of simple diagrams, I will notate the potential or definite paths of the spectator or participant inside the stories.
Before unfolding the computationally generated narratives, I would like to tell the story of our design research process. Starting our research with an experimental approach rather a design-oriented one, our purpose was often left to chance and to serendipitous discovery. Therefore, the plot of our investigation, while non-linear, lies somewhere between the confusion of the comedy, the riddles of mystery, the voyage of venturing into new mediums without genres and defined boundaries, as well as the quest for novelty.

The diagram below depicts the progress of our thesis chronologically, following the paths of each one of its members:

 

3. The story on the Screen

The precursor of the screen can be traced in the frame of the painting, which creates a second virtual space separated from the physical (Manovich, 2001). When the screen was first established as an apparatus for the display of images in the 19th century Phantasmagorias, a show where the plot would be illustrated through image projection, its purpose was to create an atmosphere of mystery while the technology would be hidden behind the screen (Kutlubasis-Krajewska and Krajewski, 2009). The element of visual illusionism that played a central role in Phantasmagorias, has been equally important in media arts (Manovich, 2001).
Patricia Pisters, in her book the Neuro-Image (Pisters, 2012) examines the connection between Deleuze’s film theory and recent neuroscientific experimentation on the effect of the experience of film particularly and of the screen generally, on the brain. Like visual illusions, the screen shows how the brain can perceive the same image as two different things: the two-dimensional surface that is the screen and the three-dimensional world, from which the spectator receives and interprets information (ibid). The second world of the screen creates a “direct and embodied screen effect in our brains”, as mirror neurons in our brain show the same reaction in actions they see in real life and on the screen (ibid.). “The brain is the screen” is Deleuze’s central thesis the two books he wrote about cinema (Deleuze, 1997a; 1997b). That is, through “simulation mechanisms” inside our brain, we become what we see (Pisters, 2012). As Christian Metz pinpoints, the film’s appeal emerges from its ability to render the impossibility of the narrated story believable, by evoking “affective and perceptual participation” (Metz, 2007).
Focusing on the spatial aspect of cinematic composition, I will further investigate the effect of space on the visual story. Giulana Bruno (2002), in her book Atlas of Emotion : Journeys in Art, Architecture, and Film points out that “as a practice of narrative space, film inherits art’s historical concern with visual dynamics, especially in the realms of set design, stage setting, and the picturing of townscapes”. It follows the 18th century topographic aesthetics in terms of representation and offering the viewer a “panoramic vision” of scenic spaces (ibid). Thus, the spectator becomes a tourist, as her gaze traverses across a framed architectural space (ibid.). She stays immobile while the virtual space of the screen is constantly transforming and moving so that it keeps her always in the best viewpoint, holding the same tradition that started with Alberti’s window (Manovich, 2001). This is the “mobilized virtual gaze” of cinema, as film-historian Anne Friedberg (2000) phrases it, in which the imprisoned passive body of the spectator comes in contrast to her mobile gaze that traverses space and time, in the virtual world of the screen. Whereas Baudelaire’s 19th century flaneur, the walking observer in the modernist city, experienced spatial mobility, the spectator of cinema experienced temporal mobility as well as the virtual, some form of “imaginary flaneurie” (ibid). Both architecture and film create what Bruno (2002) calls “(e)motional space”, as film takes the viewer through a tour of narratives, of architectural spaces across cultures and geography.
If this is the case, how does this illusionary role of the screen affect the cinematic and post-cinematic ways of storytelling, both in terms of content and structure?

 

3.1 The medium and the content
For Booker, it was the transition to modernity that caused the divergence of the plot from its original didactic purpose, into an expression of the creator’s ego. Marshal McLuhan (1967) in his book the Medium is the Message has stressed out the pervasive influence of media in all aspects of human life, to the extent that the content of the medium is another medium. According to McLuhan, our technological inventions alter and extend our relationship with our environment. The invention of writing gave humans order and direction, making visual perception instead of acoustic the prevailing transmitter of knowledge (McLuhan, 1967). Likewise, with the introduction of film, a slew of information could be imparted in an instant compared to the linear way in which the written word is expressed (McLuhan, 2013). Yet narrative cinema kept the linearity in the way information is transferred (Manovich, 2001). A single image would be communicated per frame, while experimentation with multiplication of spatiotemporal representation would only be done by the avant-garde, and expressed in different mediums, such as transparency in architecture, cubist painting or photographic collage (Friedberg, 2009). It was the computer interface that broke the linearity of the moving image and brought a “paradigm shift in the vernacular system of visuality”, by introducing multiple windows on the same screen, carrying parallel and non-related meanings (ibid).
Meanwhile, the coexistence of different screen-based mediums; namely film, video, television, computer, and the interactions between these, as well as across all forms of artistic practices, has brought the questions of medium specificity back to the fore (ibid). I elaborate further on this convergence between mediums in the fourth chapter of this thesis. The motto of this new age of non-dominant medium was formulated best by Nicholas Negroponte when he postulated that “the medium is no longer the message” (2000). The reciprocity or non-reciprocity between the medium and the work of art has been a driving force for our project. Throughout the series of experiments conducted, a few relevant questions kept reappearing : What is the identity of the artwork when transferred to another medium? Is it cinema anymore, a deviation from it or a new hybrid form?

 

3.2 The structure of visual story
Notwithstanding that cinema lost its supremacy as a medium, the cinematic modes of representation are still embedded in contemporary visual culture of the interface (Friedberg, 2009). It is meaningful to comprehend the ways in which a story is transferred from one medium to the other; in this case from text to film. Due to the extended literature on the topic, which stretches beyond the aims of this thesis, I will only delve into certain of Gilles Deleuze’s theories in his books Cinema 1: The Movement-Image (Deleuze, 1997a) and Cinema 2:The Time-Image (Deleuze, 1997b), focusing on his commentaries on Bergson’s (1907) Matter and Memory as well as Christian Metz’s (2007) Film Language: A Semiotics of Cinema.
Christian Metz applies semiotic theory to analyse filmic narrative structure (2007). As Metz underpins, the “basic unit” of the narrative is the event (Metz, 2007). The chain of events is expressed, in the case of language-based narrative with the use of sentences, and otherwise through images[5]. However, the cinematic story is not manifested as a single medium but as a “collective art form with different individuals directing color, lighting, sound, acting, speaking” (McLuhan, 2013). As Metz(2007) postulates, storytelling is intrinsically linked to cinema to such an extent that even non-narrative films follow the same mechanisms as story based ones. Such is the prevalence of narrativity, that what one remembers from a film after some time is the storyline and a number of images (ibid).
Cinematic composition is articulated through a series of connotations , or signs, of a denoted meaning, that is the plot. In the case of the Shining (1980), the tracking shots of Danny cycling around the corridors of the Overlook hotel, taken from a low-angle, portray the menacing effect that the hotel has on the kid, and subsequently transfer this sense of fear to the audience (García Mainar, 1999) . His maze-like wanderings connect with the maze outside of the hotel, as much as with the maze in Jack’s (Jack Nicholson) brain (ibid). It is the balanced composition between scene, shooting techniques and plot that gives a specific impression, and if the same scene was shot in a different story, it would hold a considerably different meaning.
Contrary to this, Deleuze argues that it is the images themselves and their combination that shape narration and no underlying structure or language can be applied (Deleuze, 1997b). In his third commentary on Bergson’s theory, he makes a distinction between the representation of the actual and virtual in the cinematic present, as happens in life. He recognises three types of images, or time-images, beyond the actual image.[6] These are, recollection-images, dream-images and crystal-images. “A zone of recollections, dreams, or thoughts corresponds to a particular aspect of the thing: each time it is a plane or a circuit, so that the thing passes through an infinite number of planes or circuits which correspond to its own ‘layers’ or its aspects”, as Deleuze points out (ibid). Plethora of instances can be located for each of these virtual images. A flashback is a form of recollection, dream-like states such as the sequences of Un Chien Andalou (1929) are dream-images , while mirror is a form of crystal-image. All these

are cases of images that are not what they seem to be. As they are assimilated within the actual images, these images constitute an actualization of the virtual, and for Deleuze (1997b) they show the forking of time. What the time-images depict is how non-linearity of time percolates into the linearity of narration, while the past and future come into play inside the plot and in the viewer’s mind.
Going back to the Shining (1980), if Booker’s theory is followed, as a horror film it follows the pattern of Overcoming the Monster. Conversely, if one looks it from the perspective of Jack (Jack Nicholson), the story is a Tragedy, depicting how a person descends to the level of a monster. The diagram of the of the plot in both cases would look like the following :

 

 

Even though this is the case in general terms of narrative progression, Deleuze’s time-image interferes. Several scenes throughout the film break the spatial and temporal continuity, bringing the subjectivity of Jack’s brain to the objective line of narration. For instance, the use of dissolve between the ending scenes of the movies links — in the same way a punctuation mark would – the maze with a view of Jack in a 1920’s party in the hotel, inviting the spectators to speculate that the whole film is a fantasy created by Jack’s inner life (García Mainar, 1999). As Deleuze (1997b) suggests, for the film “the world itself is a brain”, thus is making it impossible to distinguish the internal from the external world, perception from hallucination. This shows that the viewer’s experience goes beyond the apparent linearity towards a maze-like system open to imagination.

What does this mean for our spatialized installation? Is it possible to use techniques of linear narrativity, as cinema does, and still evoke feelings to the spectator, even though her whole body is mobilised and her gaze unframed?

 

3.3 From cinema to new media
For Manovich (2001) “the visual culture of a computer age is cinematographic in its appearance, digital on the level of its material, and computational (i.e., software driven) in its logic”. As mentioned before, the importance of understanding the ways in which cinema functions lies in the fact that the elements of cinematic perception have also dominated the aesthetics of computer media. Keeping in mind the theories and techniques that govern the cinematic experience, we did a series of experiments, exploring the ways in which the barriers of one medium can be broken and the features of film can be transferred, altered or vanished when interpreted by the computer.

experiment 1.1

In our first set of experiments, we imported frames of cinematic scenes to photogrammetry software, which gave us a 3D point cloud model of the scene. The resulting shadows often appeared “glithcy” with many dark spots at areas where the camera would not go. Therefore, the recreated space was not the actual space but what the filmmaker’s intention was for the viewer to see.
Photogrammetry is a technique often used in cultural heritage restoration (Cultural Heritage Imaging, n.d.). We equally treated the point clouds as artifacts of movies, as remnants of places bygone, since it is common that movie sets are specifically designed for the film shoot and are not existent anymore. Experimenting with different types of movies, varying outcomes emerged. Movies with fast-changing sequences such as Trainspotting (1996) or with darker colours, like Enter the Void (2009) result to photogrammetric models with less data, compared to movies with longer shots, such as The Shining (1980) or Russian Ark (2002), a movie filmed in a single 96 minute shot, therefore providing more complete spatial results.

The significance of the findings of our experimentation lies on the fact that we broke the rules of the filmmaker’s intentions on how the architectural space should be experienced. Taking the Shining (1980) for instance, the viewer’s gaze is no longer “mobilized” and directed to traverse the Overlook Hotel corridors within the confines of the cinematic frame. The hotel becomes a virtual ruin that one can experience as would in a visit to the Acropolis of Athens, or a piece of data, an architectural model that can be scaled or transformed to a new piece of architecture that no longer facilitates the narrative of a horror movie.

 

A walkthrough of the Overlook Hotel of the Shining, as understood by photogrammetry software

 

Experiment 2.1

In a different set of experiments, we implemented Deep Convolutional Generative Adversarial Network(DCGAN) algorithms using movie frames as inputs. On the first iteration, we imported 10000 frames from the Shining (1980) for 180 epochs. Due to the fact that the dataset was very diverse, the generated images were abstract hallucinations of the movie. The generated content is far from making meaning, however it depicts the general ambience of the film, again working as an artifact or a carte-postale from the movie set. Multiple visual patterns emerge in the frames, such as human figures and details of architectural spaces. We conducted the same experiment in several other movies, such as The Matrix (1999), The Shawshank Redemption (1994) and Godfather II (1974), each of which created an output of what the machine interpreted of the movie.

On a subsequent iteration, we set the input to be frames in which Jack Nicholson appears, using 15 movies from his filmography, which gave a dataset of 6500 frames. Due to the smaller amount of frames, the algorithm ran for 300 epochs. The more consistent dataset resulted in a more precise output, with characteristics of the actor’s face appearing in most generated frames. However, the experiment needs to be reiterated with a larger dataset, and after it runs through a computer vision algorithm that recognises the face and crops the frame in a uniform manner.
The DCGANs, in all their failings and loss of detail, reveal the features which the machinic eye considers important. Such features are vertical and horizontal lines, blobs, splices, as well as the colour that governs the visual story. The filmic frame, striped from its meaning becomes a set of visual cues which the algorithm recognises and mimics in order to generate a new set of images. Of these images, the human eye can only recognise patterns and the mood which differs from movie to movie. In chapter 4.2 of this report, I will explain how Generative Adversarial Networks can be used to create new videos.

https://vimeo.com/234761147

Training process of DCGANs on frames with Jack Nicholson’s face

 

4. Narratives in the age of new media

In the end of the 20th century, Lev Manovich(2000) recognised the obsolescence of media typology in art with the evolution of hybrid artistic forms , such as installation, assemblage, performance or time-based art. At the same time, the digitisation and the use of mass culture distribution systems creates a conflict between art and mass media (ibid.). As an answer to this, the theorist suggests a new form of categorization to be created, that is not linked to a particular medium but to user’s experience. Thus, art would move from the dichotomy between content and form to one between content and interface (Manovich, 2001).
What, for Lev Manovich (2001), constitutes one of the defining differences between old and new media, is the battle between database and narrative. While artists and spectators expect from new media to determine new forms of “interactive narratives”, the main intrinsic attribute of contemporary interfaces is that of the database, which offers multiple information in different sets of menus and lists (ibid).
One of the first database films according to Manovich (ibid) is Dziga Vertov’s 1929 experimental film Man with a Movie Camera, which does not follow a script but serves as a catalogue of the filmmaker’s techniques. The unfolding of each technique from the previous is the film’s main narrative, while meaning stems from the visual and spatial arrangement of sequences as opposed to the text-based meaning conveyed in classic filmmaking.

This relates to Peter Greenaway’s argument that cinema ought to break free from four of its elements : the text, the actor, the camera and the frame (Greenaway, 2010). In the case of the text, he contends that since cinema is an image-based medium, it is preposterous to create moving image based on a written script (ibid). Perhaps his propositions for a new cinema can find a space to nurture in new media installations. Greenaway himself has created numerous artworks that reflect his ideas. One example is Vertical Features Remake (1978), in which the filmmaker instead of following a text-based storyline, organizes frames of landscapes with vertical features, accomplishing an image-based narration according to the structuralist film theory (Greenaway, 1978).

In the case of new media, what is the structure to be followed, if it is not the one that the written word uses? Manovich (2001) proposes a fractal structure. This idea relates to the concept of rhizome, introduced by Gilles Deleuze and Felix Guattari in A Thousand Plateaus (Deleuze and Guattari, 2005). That is, a structure best represented as a tree or root, in which any point can be connected to another, has multiple dimensions and when fragmented, it does not lose its shape, but starts again from its old lines (ibid). Manovich also suggests that in a hybrid or computer-based environment, some part of the artist’s control over the experience is lifted as both the user and computer share some of the choices being made on the content (Manovich, 2001). The once spectator has now the role of participant, which leads her to be constantly shifting between different mental states, multitasking between the illusionary of cinema and the interactive of new media (ibid).

Looking back at the patterns of stories from the scope of this post-digital turn, can we identify the same archetypical plots in the new multimedia narratives? In the following chapters, two emerging forms of narration are scrutinized : the transmedia storyscapes and the art created by artificial intelligence.

 

4.1 Storyscapes[7]

In his 1970’s book Expanded Cinema, Gene Youngblood (1970) stressed out the need for a new cinematic language to be discovered, one that reflects a fast changing world. Expanded cinema, a movement that emerged in the late 1960’s explores the moving image spatially through “an explosion of the frame outward towards immersive, interactive and interconnected forms of culture” (Lord and Marchessault, 2015). This broad experimental phenomenon encompasses different forms of art installations from virtual reality to performance art and overlaps with various screen-based practices which have emerged since the 1960’s, such as structural film, intermedia environments or projected-image installations (Mondloch, 2010). These hybrid artworks obfuscate the boundaries between the cinematic and the sculptural, thus destroying the specificity of the medium and the limits between disciplines (ibid). In the words of Kate Mondloch (ibid), what renders them evocative is that “as environmental, experiential sculptures, they stage temporal and spatialised encounters between viewing subjects and technological objects, between bodies and screens.”
Several artists from the 1960’s up until today have experimented with the dissolution of the frame. Eija-Liisa Ahtila’s Anne, Aki and God(1998) invites the viewers to become part of a theatre staging, as they are surrounded by video projections of narrative elements across genres, from documentary to music videos and commercials; an experience of multiple simultaneous meanings (Dinkla, 2004). Other examples of installations that encompass the viewer are Bruce Nauman’s video corridor works (1967-72), in which the viewer walks into a narrow corridor to watch a video, or Doug Aitken’s Metallic Sleep/Electric Earth (1999), a walk-in installation with illusionary imagery aiming at the user’s total sensual immersion (Frohne, 2004). In all these cases, the artists manipulate the psychological effect of cinema, combining it with the spatialized experience of architecture.

An example of spatialization of film footage is the 1996 project The Invisible Shape of Things Past, by the studio ART+COM, in which the artists used documentary footage shot in Berlin streets, and by stacking the film frames one after another they created digital and physical models in various mediums and scales, which they set up at the location that the film was initially taken (Manovich, 2001). This was, for the artists, an exercise on finding architectural form through a generative process (artcom.de, n.d.).

The way the user experiences the artwork is unique each time, as she moves around a navigable space, which for Manovich (2001) is “a subjective space, its architecture responding to the subject’s movement and emotion”, a notion that brings to mind the flaneur, the urban explorer lost in the crowd. The “virtual flaneur” is an explorer of interface, happy on the fast pace of information and lost in a crowd of data (ibid.). As the user’s trajectory drives the path of the narrative, Manovich suggests that this type of narrative development is linked to navigation through computer games, a space where the user “builds character” through exploration of unknown places. This consequently connects to the American novel, where narrative is defined by the movement of the character in the outside space — such as in books of Twain or Hemingway-, compared to the psychological space of 19th century European stories (ibid).
Another main influence of such space, however, can be traced in modernist literature. Authors were engaged with the simultaneity of urban life in the turn of the 20th century, and expressed it with non-uniform storytelling. In both Ulysses (1939) and in Finnegans Wake (2010), James Joyce breaks the norms of objective reality by altering his subject according to the perspective from which it is viewed each time (Dinkla, 2004). The forking of time is demonstrated in hypertext novels, such as Borges’s The Garden of Forking Paths (1941), or Michael Joyce’s afternoon (1987-1990), a work of electronic fiction published on diskette. The hypertext, the text that branches, is the most accurate expression of the concept of rhizome (Dinkla, 2004).

A different narrative structure which breaks completely the linearity is the circular one. Deriving from the loop of feedback systems, circular structure never progresses but traps the artwork in a spiral of action and reaction (ibid). Jeffrey Shaw’s, The Narrative Landscape (1985 — 1995) is a projection of images on the exhibition floor, in three different layers, depicting three distinct narratives. The user is given a joystick, with which she can pan and zoom in the images, going around the artwork in circles with no resolution or progress (Shaw, Klotz and Abel, 1997).
In all these cases the programme of architecture comes into play, as the artist becomes the architect of multiple series of events, that might be followed or might be ignored, skipped or contradicted. The meandering experience of the user can only be directed but not controled.

 

Experiment 1.2
To experience the photogrammetric point clouds in virtual space we spatialised audio from documentaries in which the director and actors discuss the filmmaking process of each particular movie. The audios can be activated as the user moves across different spaces. In a second iteration we reshot the film with computationally generated camera paths. This became feasible after we extracted the camera coordinates (x, y, omega, phi, kapa) for every frame of the movie in a text file through the photogrammetry software. Afterwards, we trained a Recurrent Neural Network(RNN) with the coordinates and the output was new camera paths, according to what the network interpreted. We imported the camera coordinates of three different scenes of the Shining (1980) and executed an iteration per scent. In ths way, the generated coordinates were relative to each 3D point cloud, and acheived the highest level of accuracy possible for the small dataset. This allowed us to re-experience the movie through the machinic lens in the role of cinematographer, after importing the generated text file into the game engine.
The process of computing cinematography gave us some important conclusions in the distinction between the artist and the machine. Where the cinematographer sees close-ups, long shots, panning shots and an extended list of techniques to help the progress of the narrative and the evoking of emotion, the machine sees numeric and alphabetical characters, recognises patterns and generates new characters, which often give unpredicted changes in camera and jagged movement. For example, while in the corridor scene the camera starts with navigating the corridor similar to Kubrick’s tracking shot, after 3 seconds it stops following the director’s path or the spatial arrangement. Instead, it breaks the one point perspective, turns to the opposite side of the narrative focus —in this case the twin girls — and starts moving backwards while focusing on the details of the walls. Despite being influenced by Kubrick’s techniques, the machine appears to take its own initiatives, carving a different and often uncanny path, and thus subverting the filmmaker’s intentions on how space should be experienced. This behavior can have a two-way interpretation. One could argue that this can possibly be a new way of directing freed from the constraints of human intentionality, one that breaks the rules of cinematography to make its own. Or the opposite direction would be that this is a meaningless navigation through space, that proves the incapability of artificial intelligence to grasp the director’s view.

But besides the ethical dilemmas, what is the structure of the story that the machine is telling? In my view, the folding paths that it follows, have profound similarities with the turns and multiplicities of the rhizome, deriving from the inner workings of the structure of neural networks. Future experiments can help us map this relationship more systematically.

 

4.2 The story of the machine

The age of artificial intelligence is here, not in the form of Singularity that is being feared by the western privileged male, but in the mode of news bubbles and amplification of social discrimination (Kogan, 2016). This uncanny form of artificial life has become “animism for the rich”, as Matteo Pasquinelli (2016) posits. “What was once attributed to the obscure and infinite night is now projected onto the abstract abyss of computation, data centers, and machine learning” (ibid) he writes. Is this a monster that our social narrative imposes to us, by shifting the question from the current injustices towards some potential bleak future?
Our interest in machine learning developed from our intention to bridge the gap between this unknown entity that scares with its mystical power and the artistic endeavour. We decide to figure out what is the nature of the monster, instead of fighting it. Hito Steyerl (2016) asserts that contemporary perception is a machinic one and the human eye can only sense a small fraction of it. If there is something that the machine can fathom and humans cannot, perhaps it would be helpful for artists to uncover it. What are the new aesthetics that machinic art generates and what stories can it create? And since this new artform is based on pattern recognition from preexisting art, is it really new?
Researchers have trained Recurrent Neural Networks(RNNs) to spell English words and generate text in the style of different historic writers (Karpathy, 2015). The excerpt below is Andrej Karpathy’s trained network writing Shakespearean monologues:
‘KING LEAR:
O, if you were a feeble sight, the courtesy of your law,
Your sight and several breath, will wear the gods
With his heads, and my hands are wonder’d at the deeds,
So drop upon your lordship’s head, and your opinion
Shall be against your honour.’
Although on a cursory read this piece of prose is rather convincing, it has no consistency and no real meaning. Can we still identify the same patterns that apply to narratives created by humans? Is the RNN’s King Lear still a comedy in similar fashion to the Shakespearean one, or is it a whole new type of story? Given the confusion that dominates the text, one could argue that this is still a comedy, one without resolution.
Another example is Sunspring, a 2016 film directed by Oscar Sharp and written by a neural network named Benjamin, which was trained by AI researcher Ross Goodwin on a list of 1980’s and 1990’s science fiction movies (Newitz, 2016). The actors played as instructed by the script, which often gave nonsensical directions such as “He is standing in the stars and sitting on the floor” (Benjamin, 2016). Even though the story appears to have no meaning, Goodwin recognises several recurring patterns followed by science fiction movies, for instance characters wondering where they are, unable to identify their environment (Newitz, 2016). The algorithm is unable to make sense of the content of stories, giving results that resemble a dada piece of artwork, but as an ultimately unoriginal author that generates through studying precedents, follows the average pattern of science fiction storytelling. Could it be that the machine is on its dada phase?

Sunspring(2016), a film written by an AI.
This leads back to the 1960’s experimental movements. In Allan Kaprow’s book Assemblage, Environments and Happenings, he proposes an application of chance in four areas of the artwork: the creator, the material used, its form and its purpose (Kaprow, 1966). Thus the artist becomes a co-creator of the assemblage, along with forces of nature or users’ participation. Could natural forces find their analogy in the inner workings of neural networks? Taking a different
path, Hito Steyerl purports that the equivalent of today’s machinic perception can be traced further back in the Neolithic age, in which “magical thinking” would prevail (Steyerl, 2016). Like our ancestors, we “mine” and “harvest” data instead of soil, while we use empiricism to interpret phenomena (ibid).
Either way, what is certain is that the output of machinic art is still unstructured and void of meaning or emotion. The artist is still the governing figure who curates the content and makes sense of it.

experiment 2.2
Moving on to visual narratives, a series of experiments with neural networks complements the ones described previously. One implementation of generative adversarial networks is videogans, which extend the research conducted on GANs to video, thus generating one seconds long videos of future predictions of given static images  (Vondrick, Pirsiavash and Torralba, 2016). On a first iteration of the experiment, we divided the Shining (1980) into small videos of 3-5 minutes which we used as a dataset for the training. Due to the very small dataset and the diversity of data, the output after 20 hours of training was still very primitive. In spite of that, it is showing potential for producing a whole generated movie from the edited clips in future iterations with a larger and better curated dataset, so that all the scenes are divided into categories relevant to the visual content. Could the result be a structural film in similar fashion to Vertov’s or Greenaway’s?

For the production of the final generated films due to time limitation, we used the DCGAN implementation generating static frames, which we then interpolated using a script to create a new series of frames along a vector z.

Making of video explaining the process of creating new clips out of generated frames

Experiment 3

Another set of experiments involves Reverse Image Search. We first trained the network with our dataset of frames from the top 100 IMDB movies (2017 edition). Next, we ran the classification importing a scene from the Shining (1980) in which Danny (Danny Lloyd) cycles around the corridors of the Overlook Hotel, only to come across the two dead girls that had been living in the hotel some time back. The algorithm gave the top results depending on the percentage of visual matching in each frame from the Shining (1980) to the movies contained in the dataset, an output of images which we limited to 25. We then made a selection of the most fitting of scenes and created a new video in which a split screen shows two scenes in parallel, one from the Shining and one from its visually similar.
The resulting video is one that more resembles a structural film, rather than a narrative one. The connection between the two scenes is made according to similarity in visual features that the machine considers, which in some cases is colour, in others vertical lines, faces or human figures. The final video was curated by us, thus making it a cooperative intervention of human and machine to the film.
Future iterations will include a larger dataset of 500 movies as well as alterations in the algorithm so that the output gives preference to one feature at a time. We are also planning on creating an interface where the user can set the input and their features of preference and watch a video progressing from one scene to the other, in accordance with their choices. This type of narrative would have signs of a hypertext story, in which the spectator has partial control of the storyline, and different choices lead to different plots, as well as to loops and folds within the plot. The rhizomatic and feedback systems are again present in this type of narrative.

Making of video explaining the process of co-editing a video with the use of Reverse Image Seach

5. Final Piece

Our extended sets of experiments with different types of neural networks and their effect on how films are created and viewed has provided us with some important conclusions while at the same time it is an open ground for experimentation. Our final installation is composed by three large screens, in which the three versions of a movie will be displayed in videos. The three categories that occupy the three screens respectively are: the Generative Adversarial Networks — both in image formats and also the generated videos -, the 3D point clouds as directed by RNNs and finally the structural videos produced by Reverse Image Search. The three screens are positioned in a spatial arrangement that encompasses the viewer and breaks her focus. In order to curate the experience and to observe the comparison between movies, our installation has as a starting point three of Stanley Kubrick’s movies; namely The Shining (1980), 2001: A space Odyssey (1968), and A Clockwork Orange(1971).
The three parallel screens showing the deconstruction and interpretation of the same movie by the machinic eye is a way of breaking the linearity of film and offering the spectator a viewing of three different possibilities for alternative meaning to classic movies. Another presence of our installation will be online, through a website (neuralkubrick.com) where these different perspectives on film will live as a database and a vestige of this moment in time.

Introduction video of our website, neuralkubrick.com

6. Conclusion

To inscribe spatial and machinic narratives into and definite margins, would be anachronistic. Something has changed in the way we tell and receive stories, similarly to the way that technological advancements have altered who we are. The fragmented plots of our computer culture, the partial control of the user to navigate herself through narratives, along with the partial control of the computer’s unconscious intentions; all suggest that there is a level of authorship that has been taken away from the artist. We are “lost in a sea of data”, as Steyerl (2016) notes. Does this mean that artists should expect to lose complete authority, and what we will be left with is the chance encounters of the user as she goes to a promenade in a generated world? I argue that not. Throughout this journey into the secret world of machines, we took a step towards the infinite possibilities that the symbiosis between the artist and the machine can bring. As technology progresses, this need for collaboration will be even more imperative.
Perhaps what will be lost —if it is not lost already – is the five-act structure, the linearity and the notion of the artist as the individual with a didactic purpose to tell a tale that goes one way, either up or down. But at the same time, within this disjunction and multiplicity of meanings, we can still identify the same stories we did in previous times, even in parts. There is always a monster, there are always influential dark and light forces, there is the confusion on comedy, something to discover and some mystery to solve. There are also emerging patterns, namely the fork and the loop, which do not only have flexible non-linear structure , but are also intrinsic to the way the algorithms function.
In shortage of a great amount of artworks with similarities we cannot yet define genres and metapatterns in the way the new narratives are communicated. Whether their style is structuralist, dada, Neolithic, or combinations of these, the advent of new media has already transformed perception in the everyday. As architects, artists and inquisitive people, it is in our hands to evaluate their social as well as cultural implications, and finally create the visual culture of the future.

 

 

 

[1] There is a debate around the terminology of the art of information age. Domenico Quaranta (2013) addresses this issue, presenting several terms that fail to describe it properly, such as digital art, interactive art or electronic art. I am using the term New Media Art, with the definition of Beryl Graham and Sarah Cook (2010), as the “art that is made using electronic media technology and that displays any or all of the three behaviors of interactivity, connectivity, and computability in any combination”.

[2] Neural networks, a programming paradigm inspired by the structure of the human brain is used for problems such as image or speech recognition as well as generation of content. Generative Adversarial Networks consist of two distinct networks, a Generator  and a Discriminator. Whilst the generator creates fake content, the discriminator is constantly trying to tell apart what is generated and what is real (Nag, 2017)

[3]  Recurrent Neural Networks, unlike the feedforward ones, are models of Neural Networks that work with feedback loops (Nielsen, 2017)

[4] Reverse Image Search uses Convolutional Neural Networks (CNNs) to extract features from the given image, and locates its nearest neighbors from a dataset of images (Kogan, 2016)

[5] The author also acknowledges a third means of expression,  that is gesture, in art forms such as ballet or pantomime.

[6] In his first book of the two, Cinema 1: The Movement-Image, Deleuze draws from Henri Bergson’s theory that each perception of the world causes affections and consequently affections cause actions, to establish his thesis on three types of the movement-image : perception-image, affection-image and action-image (Deleuze, 1997a). He further develops this theory in his second book (Deleuze, 1997b), which is my main focus as it depicts the non-linearity of cinematic affect.

[7] Storyscape, a form of interactive storytelling, here is used as a term for a spatial story expressed in 3 dimensions, encompassing VR worlds and transmedia installations. Tribeca Film Festival introduced the ‘storyscapes award’ in 2013 which includes VR and interactive installations.
Bibliography
Key references

Booker, C. (2016). The seven basic plots. 1st ed. London: Bloomsbury.

Deleuze, G. (1997b). Cinema 1: The Time-Image. Minneapolis: Univ. of Minnesota Press.

Manovich, L. (2001). The Language of New Media. 1st ed. MIT Press.
Citations

Aristotle., and Butcher, S. (1907). The poetics of Aristotle. 1st ed. London: Macmillan.

Artcom.de. (n.d.). ART+COM Studios | The Invisible Shape of Things Past. [online] Available at: https://artcom.de/en/project/the-invisible-shape-of-things-past/ [Accessed 12 Jul. 2017].

Borges, J. L., “El jardín de senderos que se bifurcan.” 1941. Nueva antología personal. Mexico, DF: Siglo Veintiuno, 1968. 100-111.

Brown, B. (2012). Cinematography: Theory and Practice. 1st ed. Oxford: Focal Press.

Bruno, G. (2002), Atlas of Emotion Journeys in Art, Architecture, and Film. New York :Verso, 2002

Campbell, J. (2008). The hero with a thousand faces. 1st ed. Novato: New World Library.

Daniels, D. (2004). Television–Art or Anti-art? Conflict and cooperation between the avant-garde and the mass media in the 1960s and 1970s. [online] Medienkunstnetz.de. Available at: http://www.medienkunstnetz.de/themes/overview_of_media_art/massmedia/scroll/ [Accessed 12 Jul. 2017].

Deleuze, G. (1997a). Cinema 1: The Movement-Image. Minneapolis: Univ. of Minnesota Press.

Deleuze, G. and Guattari, F. (2005). A Thousand Plateaus. 11th ed. Minneapolis: University of Minnesota Press, pp.6-13.

Dinkla, S. (2004). Virtual Narrations, From the crisis of storytelling to new narration as a mental potentiality. [online] Medienkunstnetz.de. Available at: http://www.medienkunstnetz.de/themes/overview_of_media_art/narration/scroll/ [Accessed 12 Jul. 2017].

Friedberg, A. (2000). Window shopping. Berkeley, Calif. [u.a.]: Univ. of California Press.

Friedberg, A. (2009). The virtual window. Cambridge, Mass.: MIT Press. pp. 1-12

Frohne, U. (2004). Immersion und Participation in Video-Installations by Dan Graham, Steve McQueen, Douglas Gordon, Doug Aitken, Eija-Liisa Ahtila, Sam Taylor-Wood. [online] Medienkunstnetz.de. Available at: http://www.medienkunstnetz.de/themes/art_and_cinematography/immersion_participation/print/ [Accessed 12 Jul. 2017].

García Mainar, L. (1999). Narrative and stylistic patterns in the films of Stanley Kubrick. Rochester, NY [u.a.]: Camden House, pp.34-35.

Graham, B. and Cook, S. (2010). Rethinking curating. Cambridge, Massachusetts – London, England 2010,: Mit Press, p.10.

Greenaway, P. 1978. Peter Greenaway- Vertical Features Remake. [online] Available at: http://petergreenaway.org.uk/vfr.htm. [Accessed 13 July 2017].

Greenaway, P. (2017). New Possibilities: Cinema is Dead, Long Live Cinema. [image] Available at: https://www.youtube.com/watch?v=u6yC41ZxqYs&t=1739s [Accessed 11 Jul. 2017].

Heider, F., & Simmel, M. (1944) An experimental study in apparent behavior. The American Journal of Psychology, 57, pp. 243-259.

Joyce, J. (1939). Ulysses. Hamburg [u.a.]: Odyssey Pr.

Joyce, J., (2010). Finnegans wake. Mousehole [England]: Houyhnhnm.

Joyce, M. (1990). Afternoon. Eastgate.

Jung, C., Henderson, J., Franz, M., Jaffé, A. and Jacobi, J. (1988). Man and his symbols. 1st ed.

Karpathy, A. (2015). The Unreasonable Effectiveness of Recurrent Neural Networks. [online] Karpathy.github.io. Available at: http://karpathy.github.io/2015/05/21/rnn-effectiveness/ [Accessed 3 May 2017].

Kaprow, A. (2010). Assemblage, Environments & Happenings. In: M. Iversen, ed., Chance, 1^st ed. London: Whitechapel: Documents of Contemporary Art, pp.52-57.

Kogan, G. (2016). Convolutional neural networks. [online] Machine Learning for Artists. Available at: https://ml4a.github.io/ml4a/convnets/ [Accessed 4 Jul. 2017].

Kutlubasis-Krajewska, V. and Krajewski, P. (2009). WIDOK. WRO Media Art Reader, Issue 1, From Absolute Cinema to Future Film. 1st ed. [ebook] Wroclaw. Available at: http://wrocenter.pl/pl/widok/ [Accessed 26 Apr. 2017].

Lord, S. and Marchessault, J. (2015). In: O. Kholeif, ed., Moving image, 1st ed. London: Whitechapel: Documents of Contemporary Art, pp.56-57.

Manovich, L. (2000). Post-Media Aesthetics. [online] Available at: http://manovich.net/index.php/projects/post-media-aesthetics [Accessed 1 May 2017].

McLuhan, M. and Fiore, Q. (1967). The medium is the massage. 1st ed. New York: Bantam Books.

McLuhan, M. and Gordon, W. (2013). Understanding media. 1st ed. New York: Gingko Press.

Metz, C. and Taylor, M. (2007). Film language: A Semiotics of Cinema 1st ed. Chicago, Ill: Univ. of Chicago Press

Mondloch, K. (2010). Interface Matters: Screen-Reliant Installation Art. In Screens: Viewing Media Installation Art (pp. 1-19). University of Minnesota Press. Retrieved from http://www.jstor.org/stable/10.5749/j.ctttsj4b.5

Nag, D. (2017). Generative Adversarial Networks (GANs) in 50 lines of code (PyTorch). [online] Medium. Available at: https://medium.com/@devnag/generative-adversarial-networks-gans-in-50-lines-of-code-pytorch-e81b79659e3f [Accessed 13 Jul. 2017].

Negroponte, N. (2000). Being digital. New York: Knopf.

Nielsen, M. (2017). Neural Networks and Deep Learning. [online] Neuralnetworksanddeeplearning.com. Available at: http://neuralnetworksanddeeplearning.com [Accessed 3 May 2017].

Okakura, K. (2014). The book of tea. e-book: Global Grey, pp.20-27.

Pasquinelli, M. (2017). Abnormal Encephalization in the Age of Machine Learning – Journal #75 September 2016 – e-flux. [online] E-flux.com. Available at: http://www.e-flux.com/journal/75/67133/abnormal-encephalization-in-the-age-of-machine-learning/ [Accessed 10 Jul. 2017].

Pisters, P. (2012). The Neuro-Image: A Deleuzian Film-Philosophy of Digital Screen Culture. Stanford University Press.

Polti, G. (1960). The thirty-six dramatic situations. 1st ed. Boston: The Writer.

Quaranta, D. (2013). Beyond new media art. Brescia: LINK Editions. p. 23-35

Scholes, R. (1978). Structuralism in literature. New Haven: Yale Univ. Press, p.60.

Shaw, J., Klotz, H. and Abel, M. (1997). Jeffrey Shaw, a user’s manual. Ostfildern-Ruit: Cantz, p.106.

Steyerl, H. (2016). A Sea of Data: Apophenia and Pattern (Mis-)Recognition – Journal #72 April 2016 – e-flux. [online] E-flux.com. Available at: http://www.e-flux.com/journal/72/60480/a-sea-of-data-apophenia-and-pattern-mis-recognition/ [Accessed 10 Jul. 2017].

Tobias, R. (2003). 20 Master plots (and how to build them). Cincinnati, Ohio: Writer’s Digest Books.

Tschumi, B. (1996). Architecture and disjunction. 1st ed. Cambridge, Mass.: MIT Press.

Vondrick, C., Pirsiavash, H. and Torralba, A. (2016). Generating Videos with Scene Dynamics. Barcelona: 29th Conference on Neural Information Processing Systems (NIPS 2016).

Weibel P. (2012). The Post-Media Condition | Mute. [online] Available at: http://www.metamute.org/editorial/lab/post-media-condition [Accessed 28 Mar. 2017].

Wilson, B. (2012). The Machine Learning Dictionary. [online] Cse.unsw.edu.au. Available at: http://www.cse.unsw.edu.au/~billw/mldict.html#weight [Accessed 14 Jul. 2017].

Yorke, J. (n.d.). Into the woods: How Stories Work and Why We Tell Them. 1st ed. London :Penguin Books Ltd.

Youngblood, G. (1970). Expanded Cinema. New York: P. Dutton & Co, pp.59-65.
Filmography

 

2001, a space odyssey. (1968) [film] Directed by S. Kubrick. USA: Warner Home Video

A Clockwork Orange (1971) [film] Directed by S. Kubrick. USA: Warner Bros

Enter The Void. (2009). Directed by G. Noé. France: Fidélité Films.

Eternal Sunshine of the Spotless Mind. (2004). Directed by M. Gondry. USA: Focus Features, Anonymous Content, This Is That Productions.

Full Metal Jacket. (1987). [film] Directed by S. Kubrick. UK / USA: Natant, Stanley Kubrick Productions, Warner Bros.

Man with a Movie Camera (1929) [film] Directed by D. Vertov

Russian Ark. (2002). Directed by A. Sokurov. Russia Germany Canada Finland: Seville Pictures.

The Great Escape. (1963). [film] Directed by J. Sturges. USA: Mirisch Company.The Godfather part II. (1975). [film] Directed by F. Coppola. USA: Paramount Pictures, the Coppola Company.

The Shawshank Redemption. (1994). [film] Directed by F. Darabont. USA: Castle Rock Entertainment.uctions TSG Entertainment.

The Shining. (1980). [film] Directed by S. Kubrick. Elstree Studios: Warner Bros.

Trainspotting. (1996). [film] Directed by D. Boyle. United Kingdom: Channel Four Films.

Un Chien Andalou, (1929). [film] Directed by Luis Buñuel, France: Luis Buñuel.

Vertical Features Remake. (1997). [film] Directed by P. Greenaway. England, UK: Arts Council of Great Britain.
List of figures

Fig. 1 Marouda, Hattab and Iyengar, (2017) Generated images by a Generative Adversarial Network trained on frames from the Shining
Fig.2 Thommen, E. (2011). Images from the Heider-Simmel Animation. [image] Available at: https://www.researchgate.net/publication/258030158_De_l%27interaction_sociale_a_la_theorie_de_l%27esprit_aspects_du_developpement_typique_et_atypique [Accessed 13 Jul. 2017].
Fig. 3 Marouda (2017) , Table showing the basic types of plots according to Christopher Booker
Fig. 4 Marouda (2017), Table showing the two additional plots in Christopher Booker’s theory
Fig. 5 Chikanobu, Y. (1888). Tea Ceremony (Chanoyu no zu), from the series Etiquette for Ladies (Onna reishiki no uchi). [Woodblock print (nishiki-e); ink and color on paper] Boston: Museum of Fine Arts Boston. Available at: http://www.mfa.org/collections/object/tea-ceremony-chanoyu-no-zu-from-the-series-etiquette-for-ladies-onna-reishiki-no-uchi-491183 [Accessed 13 Jul. 2017].
Fig. 6 Marouda, (2017) diagram showing the progress of our research
Fig. 7 Bunuel, (1929), Still from Un chien andalou [ONLINE]. Available at: Un Chien Andalou, 1929. [DVD] Luis Buñuel, France: Luis Buñuel. [Accessed 12 July 2017].
Fig. 8 Marouda, (2017) diagrams showing the plot structure of the Shining – Overcoming the Monster or Tragedy -depending on the perspective from which one sees them
Fig. 9 Marouda, (2017) Diagram showing the plot structure of the Shining including the extra-sensory phenomena that the heros encounter
Fig. 10 Marouda, Hattab and Iyengar, (2017) photogrammetric model for Enter the Void
Fig. 11 Marouda, Hattab and Iyengar, (2017) photogrammetric model for Trainspotting
Fig. 12 Marouda, Hattab and Iyengar, (2017) photogrammetric model for Russian Ark
Fig. 13 Marouda, Hattab and Iyengar, (2017) sample of generated images from the movie the Godfather II
Fig. 14 Marouda, Hattab and Iyengar, (2017) sample of generated images from the movie the Matrix
Fig. 15 Vertov, D. (1929). Still from the opening scene of man with a Movie Camera. [image] Available at: https://unaffiliatedcritic.com/2016/02/man-with-a-movie-camera-1929/ [Accessed 14 Jul. 2017].
Fig.16 Marouda, (2017) Diagram depicting one instance of the rhizomatic structure
Fig.17 Media Art Net (2004). Eija-Liisa Ahtila – “Anne Aki and God”, installation view. [image] Available at: http://www.medienkunstnetz.de/works/anne-aki-and-god/ [Accessed 14 Jul. 2017].
Fig. 18 ART + COM (1995) The Invisible shape of things past https://artcom.de/en/project/the-invisible-shape-of-things-past/ [Accessed 4 May. 2017].
Fig. 19 Marouda, (2017) Diagram depicting one instance of the loop structure
Fig.20 Media Art Net, (2004), Jeffrey Shaw – The Narrative Landscape [ONLINE]. Available at: http://www.medienkunstnetz.de/works/the-narrative-landscape/images/2/ [Accessed 13 July 2017].
Fig. 21 Marouda, Hattab and Iyengar (2017) Diagrams of the original camera path from the corridor scene of the movie the Shining
Fig. 22 Marouda, Hattab and Iyengar (2017) Diagrams of the generated camera path from the corridor scene of the movie the Shining