Giving character to space generated by redirected Walking
Virtual Reality, as an emerging medium, has been explored in a variety of ways. Researchers have come up with their own interpretations of Virtual Realities that include the Mixed Reality, Unreality, Virtual Reality, etc. Nevertheless, neither differentiating all kinds of Virtual Realities nor defining a new version of VR is in the scope of this thesis.
Driven by the desire to offer users an advanced, more selective and non-linear experience of space, this piece focuses on how to turn the homogeneous and serialized spaces, generated by redirected walking, into a series of spaces that entail different styles and are assigned particular characters. Such a practice would offer users the freedom to interpret their individual experience and come up with a purpose, as constructed by the redirected walking and its staged narratives.
The narratives that run in the background of the experience relate to the housing problem, as an issue that has drawn and still draws great concerns and more specifically, the thesis looks at the case of the Kowloon Walled City.
The Kowloon Walled City in Hong Kong, sets an extreme case of a densely populated neighborhood, that is however, regarded as being stunning and charming in its own ways.
Despite the extreme poverty, poor sanitary conditions and the dystopian appearance, the Kowloon Walled City has,Â provokes feelings of nostalgia to many of the inhabitants. Most of them depict an intimate community spirit emerged from shared poverty and enclosure (Rackard, 2013). The ways that individuals and the community use limited and highly- dense spaces can also be seen in other areas around the globe, such as the Migingo Island, where a market, the government, the overall community, all production, and entertainment spaces, they all co-exist in an area of only 0.002 square kilometers with 131 residents. In other words, whether an individual is physically or mentally satisfied or a community is vigorous or not cannot be simply measured based on the size of space.
In some cases, even extremely small spaces might meet people’s varying levels of satisfaction. In fact, Jane Jacobs, a specialist on American urban issues, holds the idea that the vitality of cities comes out of high density and diversification (Jacobs J., 2016)
Despite the fact that people in slums like the Kowloon Walled City suffer from poor conditions, they are being provided with plenty of low-cost spaces for young people who are passionate and creative. These spaces have become the foundation of their development and help them find their meaning in life.
Physical spaces have become increasingly limited and people have to work extremely hard to afford a place of their own. For this reason, it is of vital importance to research how technology and design can meet people’s needs in various dimensions. The author believes that with the advancement and development of technology, virtual reality will ultimately be one of the possible mediums that can ultimately change people’s perception of space and what reality could mean to each individual.
Redirected walking techniques aim, to some extent, at expanding physical experience in the virtual environment. Experiments conducted by using these techniques are research-driven, which means that they are dedicated to testing if various gains used in these techniques, such as rotational, translational and curvature gains can “mislead”Â viewers’ sensation (Walker, 2013). Spaces in the experience become visual cues and references to guide viewers’ routes to be redirected.
By deploying the work of pioneers of the field and by running our own set of tests and demos, the design team intends to develop an algorithm that can make viewers perceive an infinity of spaces physically in limited spaces, and construct a narrative of enlargement in those spaces, using VR storytelling as such, each space can be assigned a character that alongside the redirected walking techniques would establish a deeper and more realistic relation with the way people perceive this space.
The Needs to Narrate : Homogeneous Spaces in Redirected Walking
2.1 Overview of Redirected Walking
Before discussing the necessity of telling a story, the author would like to briefly introduce the concept of the redirected walking, the algorithm and the way it operates, to help readers to establish a basic understanding of what the redirected walking is and what more is it that the author and his team have managed to developed that moves beyond the original redirected walking techniques.
Briefly speaking, the design team developed a redirected walking technique that can map any regular physical spaces into a virtual environment, it enlarges its boundaries and offers to the viewer an added layer of sensations when in the virtual world. Moreover, users can walk freely in the virtual scene without feeling that they are being manipulated. This is realized by applying not a singular gain but all three gains mentioned above, into remapping the scale and orientation between the physical and virtual boundaries.
As an initial algorithm, the team uses a 4 x 4 meters room as the physical space to remap it into a hexagon-shaped space. By getting the position and orientation of the viewers in real time, the relative view of the camera from which viewers see the virtual space will be rotated and translated according to the remapping rules set by the team. As for the size of the visual space, when the rectangular space is mapped to a hexagonal space, the size of the hexagon is about 1.5 times larger than the rectangular one. By doubling the physical space into 4 x 8 meters and connecting them with a door between the two rooms, the algorithm can allow viewers to walk between the two rooms repeatedly while being able to enter numerous of different rooms. So the space viewers perceive is duplicated infinitely while their physical body has been always in the same two rooms. The algorithm is highly technical and it is not in the scope of this thesis to explain in detail how exactly this can be realized. Please refer to “Problem Solving During Mixed Reality Project Development Process” written by Jiasheng Huang to learn more about it.
2.2 Why Do We Need to Tell Stories of Spaces
One problem with the spaces generated by the redirected walking algorithm is that the hexagons look exactly the same creating plenty of homogeneous spaces. Two shared attributes of these spaces are efficiency and functionality. However, people would like to discover new ways of using a space according to their own interests and habits. For example, a bed is originally designed for sleeping. But many people would like to read or work on their computers in their beds. Another example which could be more convincing is decorating a house. Houses, before being furnished, are only a combination of floors, walls, and ceilings. These shape the basic look of what a dwelling space commonly looks like. However, a story will be told as soon as the owners start to decorate their houses. Whether the style is European, American or Chinese and what objects the owners put in the living room, all tell us a story of who the owner is and what the owner is like. ItÂ is the same with storytelling in virtual reality.
Virtual Reality, by its nature, is a platform and a medium with which people can tell their own stories in different dimensions. The algorithm of redirected walking and the hexagonal spaces it generates are like the houses that are built for the users. It is totally up to the viewers to decide how to furnish the spaces based on their own understanding and will.
At this point, the design team, formed by researchers and developers, started to question what would happen when a technical experiment is given a character and a narrative. What would this space mean to the viewers and, what kind of feedback would the team get from those who participated in the narrative? With these questions in mind, we started to design a demo scene using the hexagonal space to provide viewers with a series of immersive experiences, with the team consciously assigning the space a meaning, so that the viewers would gradually immerse themselves in the virtual world, without feeling they are being exposed to a scientific experiment.
Walking in Narratives and the Multi-Sensory Experience
Stories are all about imagining the audience themselves to be in these scenarios. That, in VR, is what people call “Presence”. People have talked a lot about the presence in VR. As Mel Slater, an Catalan Institution for Research and Advanced Studies (ICREA) Research Professor at the University of Barcelona defined, the feeling of “being there”, is often called “presence”, the qualia of having a sensation of being in a real place. We call this Place Illusion (PI). The Plausibility Illusion (Psi), now refers to the illusion that the scenario being depicted is actually occurring. (Slater, 2009)
In Place Illusion, the settings of VR scenarios need to be natural and sensible so that viewers can catch the ideas quickly through these settings and produce resonation with the stories. As for Plausibility Illusion, it is more about specific interactions and physical responses happening during the journey through the stories. With case studies of a film and a VR project, the author would like to talk about what lessons could be learned.
3.1 “Atonement”: Walking While Perceiving
“Atonement” is a movie about war, peace, and love. In this movie, a five-minute single tracking shot is taken on the beach when the main character is walking through different scenes happening during the battle fight. Throughout the walk, a whole picture of how cruel war could be as depicted by segments of different shots. Corpses spreading on the beach, soldiers leaning against vehicles with missing body parts, broken buildings with smoke coming out and families crying all convey a call for peace to the audience through the eyes of the character. In this movie, the dynamic perception is achieved by constant movement on the beach. By moving across the crowd in the war, the character and audience are able to see different content in different positions. This is of great importance for the audience since it allows them to take a closer look at certain details that cannot be seen from a distance. In this way, their understanding of the story can be strengthened as the audience gets an insight and becomes itself part of the story.
Atonement (2007). Five minute single take tracking shot.Available at: https://www.
youtube.com/watch?v=QijbOCvunfU [Accessed 28 Jun. 2017].
A similar kind of dynamic perception applies to the redirected walking spaces. In most VR projects people can only walk with controllers or they get navigated automatically by the camera in the same ways that computer games and traditional movies provide. But, with the deployment of redirected walking techniques, viewers are able to walk constantly like the character in Atonement does and perceive the space from different distances, angles, and perspectives. What is more is that it all happens through a real, as in, physical walk.Â Therefore, it is crucial to design what the viewers could see while they walk towards a certain place in the virtual environment and, consider the varying responses they could give depending on the diverse perspectives of their observations.
Virtual Reality as an immersive medium largely depending on the visual perception of its users. However, people essentially do not use any single sense alone in order to perceive the context. As Johnson stated, perception is never a separate process and is not solely linked with visual memory (Johnson, 2009). Hearing and touch are the other two senses that add to the authenticity of perception. To better illustrate this argument and in an effort to enhance our knowledge on storytelling in VR, a project entitled “Thresholds” will be discussed. Its analysis will offer us hints on both, ways this thesis could deal with the narratives for a redirected walking space and, the concept and sense of the ‘presence’ for the viewers.
“Thresholds” is a VR project created by Mat Collishaw, a visual artist based in London which takes participants back to the first major exhibition of photography in 1839.
In this exhibition, viewers can see the artifacts and physically touch the cases containing them. The artist explored the possibility of mimicking the physical space in a virtual environment with the use of multi-sensory interactions. First of all, he used the latest HTC Vive Headset and Base Stations to enable viewers to walk freely within the exhibition room. Additionally, a physical room was built in the same size as the virtual one, to give viewers an authentic sense of touch. In such a way, the interaction in the space becomes not just about what the viewers can see but, also about what they are able to feel.
A particular moment took place where the ï¼†fireplaceï¼Š was being heated by a physical heater so that the virtual fire could match the physical experience. With this sense, the author was suddenly immersed in that feeling.Moreover, the lighting and decoration in the virtual room all fit into the “back to the early 19th-century” theme, so that the first impression of the viewers could determine how they would interpret the rest of the experience.
When the author started to adapt to the way this room works, it became apparent that every object and detail in the virtual room perfectly matched the physical ones. The shape of the window frames, the legs of the cases and position of each physical object, they all perfectly matched with those viewers could see through the headset and, the feedback from the touch strengthened the sense of authenticity of the experience.
After experiencing “Threshold” in person, the author summarized key, successful points that could be deployed in an equally helpful and effective way in the redirected walking project. First of all, despite the fact that the project was designed to lead viewers back to the past time when actually being in the room, while they were attracted by different interactions and settings in the room. Some would listen to sounds coming from outside the windows, others would feel the match of the physical touch in the room and, others would watch the paintings on the walls. That indicates that even when provided with the same specific theme of space and narrative, people might focus on different parts and elements of the design. Especially in the case of the VR storytelling that is more challenging, since people’s perception of the environment is often different based on who they are.
Perception, including vision, is an active process which requires action and integration. Even when presented with the same information or images, everybody experiences and sees something unique and personal, which nobody else can see or maybe even understand (Akten, 2017). This is due to the fact that the mind infers beyond the information given (Bruner, 1957), resulting in the perception of even the most uncomplicated entity being a complex, multifaceted process.(Johnson, 2009).
A series of useful observation was made from this project. For example, the setup of the theme and atmosphere of space relates to how people feel about the emotional tone and style of a narrative. In terms of the space, it also gives the viewers a visual link with similar scenes or memories which make the space look natural and real. Therefore, this highlights the importance of the choice making process in the virtual scenes. And the last one is the significance of the effective match between virtual and physical boundaries. The author will discuss these ideas in the following chapter.
Simulacrum : The Story Behind the Redirected Walking Algorithm
In this chapter, the author will explain in detail how storytelling techniques in VR are applied in the redirected walking space. With design methods and proposals, the author attempts to push the limits of the redirected walking algorithm so as to fully immerse viewers in the narratives. That is to say, to help them forget physical restrictions and adapt to the perception of the new space.
Simulacrums are copies that depict things that either had no original, to begin with, or that no longer have an original (Goldman and Papson). It is not just a copy of original appearance but instead a reinterpretation and sometimes a correction to deliver inspirations and ideas to the audience.
â€œThe simulacrum is never that which conceals the truthâ€”it is the truth which conceals that there is none. The simulacrum is true (Baudrillard and Poster, 2007).â€
Plato, a philosopher in Classical Greece had given an example of Greek statuary, in which the top of the art pieces are crafted larger than the bottom so that viewers on the ground would see it correctly (Plato, 360 BC). In Twilight of the Idols written by Nietzsche, a German philosopher, cultural critic, poet, philologist, addressed the notion of simulacrum suggesting that most philosophers arrive at a distorted copy of reality by neglecting the credible input of senses and resort to the constructs of language and reason (Nietzsche, 1895).
As for storytelling theories, this research has been influenced by theories that prevail in the filming industry and more specifically, the basic seven plots of literature (Booker, 2004). The latter introduces several moods and feelings to certain plots that express the emotional signal to readers. In simulacrum this is also used as the plot becomes the spatial atmosphere of the narrative. Whether it is loneliness, excitement or entertaining, it is determined by what the author wants to show to his audience.
However, there are certain theories in filming that are not adaptive to simulacrum. Gustav Freytag, German novelist, and playwright, states that drama can be divided into five parts or acts including exposition, rising action, climax, falling action, and denouement (Freytag, 1863). Freytag’s “Pyramid diagram”Â showcases how the five acts are organized and explain how writers and storytellers organize their ideas, storylines and the mood they want to convey to the audience. In simulacrum though, viewers establish their understanding of the space and story through segments of feelings while they are walking through the virtual spaces and interact with the context. Just like the scenario in “Atonement”, the five-minute walk along the beach shapes the whole perception of war and this is what the VR journey is all about. There is not necessarily an ending for the virtual journey but its importance lies in the physical feeling and mental surprises that make the space more sensible.
4.1 Story of Simulacrum
n my understanding of these explanations, the simulacrum is a simulated reality aiming at showing a more objective view of what reality should look like. So viewers understanding of reality will not be influenced by irrelevant factors that somehow hinder viewers from understanding a person or thing in an objective way. However, the capability of simulacrum does not necessarily mean every piece of reality needs to be distorted or corrected. Discussions of whether the simulacrum is a positive notion or not and whether reality should be corrected artificially are interesting, but they are not in the scope of this thesis. It is the impossible spatial perception that makes the simulacrum particularly meaningful for this project.
As part of my design project, I am interested in deploying the redirected walking techniques and technologies to construct a space and lead the viewers into the future reality and spaces, so that they can see what this future might look like. As part of this suggested future, people will have to develop a new way of using their space to adapt to the simulated extended spatial perception and turn that into the reality they believe in. This is because every piece of land on earth has been overdeveloped by human intervention. Nature has thoroughly been turned into artificial creations. The population explosion has made an alteration of human perception to be necessary so as to make people adapt to physically limited space but maintain the access to perceived larger spaces with the use of technologies. With the story and algorithms behind the redirected space, space itself becomes hyperreal in different dimensions. The infinity of space, atmosphere of architecture, decision making in the space and multi-sensory interactions in the redirected space and how the author assigns narratives to the space to achieve a hyper-real perception from the viewers, are key points the author would like to explain in more detail.
Narrative of Simulacrum
4.2 Infinity of Spaces
In the series of hexagon shaped spaces defined by the algorithm, a closed path loop will be formed if the viewers continuously walk towards the next room. After walking through six rooms, the viewers will be back into the first room where they started their journey in the virtual space. As a result, the space is horizontally infinite in that there will not be an end in the experience for as long as the space is attractive enough to keep the viewers continuing with their experience.
Furthermore, the team has duplicated the space vertically so as to give the viewers the impression of being transformed in a fully three-dimensional way. So the team can explore the possibility of redirected walking in a third dimension while the previous researchers have only applied their techniques in two dimensions. Because the viewers are just visually transformed into another floor, no matter how far away they are vertically from the real ground, the positions are always overlaid within the physical space. As such, the team is able to achieve infinity vertically.
As a space for viewers to walk freely inside, the author thinks the space could become an exhibition space. With the six hexagons attached to each other, the doors become the horizontal connection between them and elevators are offered to enable the vertical movement in the space. And each hexagon has another door which teleports viewers to another world that has no visual boundaries and each world has a certain theme and a specific way to interact with it through different senses. The similarity between this virtual space and physical galleries and museums is in the way viewers choose where to go and what to look at.
Generally, viewers in galleries walk in a linear way, usually from the entrance through different areas and finish the experience at the exit. However, they are still free to go back, wander around and spend more time where they wish to. Some viewers often skip certain exhibits and go straight to what they think is more attractive even if the exhibits that are skipped are nearer to the viewers. As for the narratives in the virtual space, the algorithm enables different paths that viewers can choose each time they enter or exit a certain room. This helps us construct a more personal and subjective narrative since viewers are given the opportunity to make choices in the virtual spaces. These choices will be discussed in more detail later in this thesis.
4.3 Synchronizing the Architectural Atmosphere With Simulacrum
As it has already been discussed regarding the case of the Threshold project, the interior design, and settings of the objects, sound and people outside the windows have all successfully brought viewers back to the desired time in the past. It is important that the overall atmosphere fits into the background and time so everything feels natural to be there. Felix, co-founder and creative director at Felix & Paul Studios, talked about ï¼†Strangersï¼Š, the first VR project done by their studios, saying that everything they wanted to achieve with the experience was about constructing a sense of truly being in the moment and strengthening the emotional connection to the environment while enabling viewers to truly feel the carefully crafted details and explore the space.
In the narratives of the simulacrum, the idea we want to deliver is that in the future, the limited physical space has made people adapt to new ways of using the space around them. The author wants
the audience to feel the purity of architecture and the environment. So the interior of the redirected hexagonal space is furnished in a futuristic way. Only black and white are applied into space in order not to confuse people with too much visual information. Details have been added to relate people back to reality. These include the settings of pipelines, decorative lines on walls and platforms and labels indicating the identity of each room. All the architectural components including platforms, glasses, doors and wall parts are prefabricated parts that are assembled on site. As such, all settings of the architectural environments indicate the background context and time of the story.
4.4 Differentiating Spaces In Simulacrum
After designing the architectural appearance of the space, the design team tested how it feels to be in the space. One of the tests for the viewers was to recognize the room where they start the experience from. Although viewers could memorize every move they have made, to actually find their way back according to these memories appeared to be very difficult and challenging even for the team of the creators. After a couple of studies, we came across an experiment by MIT on memory. Based on their experiments, the team developed an algorithm to run images based on memorability (Trafton, 2011). One of the results of the experiment is that images with people in them are the most memorable, and images of space in human scale to be the second ones. The least memorable images depict natural landscapes. This is quite an inspiring finding since, although natural landscapes are difficult to remember, they can become memorable if they have certain features to differentiate them from one another (Trafton, 2011). Therefore, the reason why images with people are the most memorable ones is that there are really few cases in which peopleï¼Šs facial identity, clothes, emotions and gestures in images can be totally the same. So each image depicting people is different from the others, enabling the viewers to tell the differences and thus memorize those images.
In terms of the space in simulacrum, the hexagon-shaped rooms act like the natural landscapes. They share the same futuristic look. And because of the periodic torus walking path in the space, there is not a single reference to tell the viewers where they are and how far they are from a certain space. Therefore, we decided to differentiate each room by giving it a unique character. A floating geometry is designed in the center of each room. These will transform vertically according to the viewerï¼Šs height. Whenever the viewers go upwards or downwards, the geometry will follow to ensure it is always available to be seen. These geometries also function as triggers to activate immersive particle systems in the rooms. Each particle system is defined as one element ranging from fire, water, metal to wood, soil and void.
So when viewers look at the floating geometry, the particle system appears and when they look away from it, the particles are gone and the room is back to normal. There is only point light used to colorize each room. In this way, six rooms will be differentiated by different colors and immersive elements.
However, when we first designed the characters for these spaces, the viewer could only trigger the particles and lightings when they came closer to the neighboring room. What happened with this is that this neighboring room may not necessarily be the room the viewers want to go. And still, viewers would only be able to remember the neighboring rooms. Every time they enter a room, they would need to update their memory to adapt to new neighboring relations. Later on, the author came up with the idea to let people activate the particles and lightings in all six rooms at the same time. No matter how far the viewers are from the floating geometry, they are still able to trigger these objects to know exactly where they are in the space. To assist in the navigation of space, each room is labeled with a serial number on the wall familiar for the user to identify. With these two settings, it is possible for the viewers to get this mind map of the whole space to help them get a sense of position and orientation and learn how to navigate themselves in the space.
However, the author has to admit that the cues to tell the differences among rooms could have been more sensible and architecturally challenging. A good example would be the works by American artist James Turrell, who worked directly with light and space to create artworks that engage viewers with the limits and wonder of human perception (James Turrell, 2017). In his work, light and simple geometries are combined to affect collectively what the viewers perceive. The way he differentiated spaces is through the subtle change of light, angle, color, shape of geometries. Through these changes, theÂ viewers can easily recognize the distinct characters each space has.
They might be tiny changes, but extremely fun to discover. And what makes the recognition vary is what viewers could see along their way. The change of perspective and geometric shape all make space become meaningful and unique. The author would like to further examine the works of his to develop the virtual space of redirected walking in a more architectural way and try to make all elements in the space become more related to feelings instead of just being a sign to a location.
4.5 Choice Making and Attention Control In the Space
The ï¼†afternoon, a storyÂ§, a work of electronic literature written in 1987 by American author Michael Joyce, is the first hypertext fiction, which interactively proceeds with the story by providing different choices for readers so that the choice they make may lead to different paths in the story. (Jill, 1999). Narratives in Virtual Reality are immersive and interactive largely because viewers have the ability to decide and choose what to look at and where to go to proceed with the stories. Interactivity, in these stories, makes viewers become truly a part of the stories. They can take control over them and choose where the stories could go and what new elements could possibly be added into the storylines.
This section will discuss the parts of the process that deal with choice making and attention control so as to show how the author guides viewers to explore the space and transfers them from the physical room into the immersive new dimension of the simulacrum.
Attention control can be partly regarded as a choice making process. They both refer to an individualï¼Šs capacity to choose what they pay attention to and what they ignore (Astle, 2009) while attention control gives more preference to an individualï¼Šs ability to concentrate. In VR context, it makes viewers become a part of the story by enabling their choice making ability. As for the attention control, it is about how to design an interaction or just a static visual object to draw the viewers’ attention and make them follow the lead by the author to explore the redirected space.
In simulacrum, basic choice making happens when people decide to stand still orwalk somewhere. They have different effects on perception in that when people stand still and observe the context, the context is static except for certain dynamic objects that are constantly moving. However, when people are walking while observing the context, something within the view two seconds before, might be out of sight a few seconds after. So even if the viewers are looking at the same object, they are looking at it within varied compositions, resulting in different comprehension of the visual content that viewers see. So, in this occasion, the perception of space will be changing dynamically. With the ability to physically move anywhere in the virtual space, the author provided the viewers with choices. TheseÂ choices happen every time viewers walk through a door between two rooms. After they go through a door, they always have two choices, either to go left or right. If they are walking clockwise, choosing left will lead the viewers to the portal of another world while choosing right will take them to the next room. And if they are coming back through the portal from the other world, both ways will lead them to the neighboring rooms. So here comes the question, how can viewers know about these selections and navigate through space without being guided by verbal descriptions?
There are three answers to this question in the spaces of simulacrum. The first one is that when viewers enter a room, a set of handles will be in front of them in the center of the room, indicating the existence of virtual boundaries. With the virtual walls on the other side and the way that these handles extend, the viewers can be guided to choose either left or right side to keep navigating themselves. Secondly, when the viewers approach either side, the elevator will gradually come nearer to the viewer according to the distance between the elevator and the viewer. This dynamic change indicates where the elevator will take the viewer to. Simply speaking, during the whole process, the elevators move as the viewers move and they stop when viewers stop.
This setting does not only allow for theÂ transportation of viewers but also lets the physical movement of the viewerâ€™s body to synchronize with a visual change of position in the virtual space. In this way, people will get rid of the spinning sensation, which often breaks the immersion of a virtual experience and presence. Thirdly, the doors in the space are behaving in the same way that the elevators do. So when viewers approach a door, it opens and offers to the viewers the choice of entering the door. All doors are divided into two categories that are visually differentiated, so as to help viewers identify whether there are standing in front of a portal or a door leading to the next room.
4.6 Designing Interactions
Timoni West, the Principal Designer at Unity Labs, stated that narrative, the representation of what happens, is not just words in novels or the dialogue of an actor or even the interactions in games, but the whole environment of what you perceive with multiple senses and thus you must ensure everywhere has a narration just in case the viewers look anywhere. (Grant, 2017)
In filming industry, conflict is an essential ingredient for story and participation is an essential ingredient for a VR narratives. (Grant, 2017) As it was mentioned before, the freedom of walking in the whole space is a basic but important participation in the virtual space. However, people seldom walk around without any purpose. There has to be something that attracts viewersâ€™ attention to make them move. Without conflicts and participations in narratives, viewers will probably quickly get bored by doing the same things in the virtual spaces and thus being distracting from the immersion. So the author decided to strengthen the narrative by enabling multi-sensory interactions in the virtual spaces to be conflicts that make viewersâ€™ experience the virtual space in a more meaningful way. With a stronger narrative, the author intends to further redirect viewersâ€™ perception by letting them forget the feeling of being manipulated.
The floating geometries in each room are designed to be a simple way of interaction between objects and the vision of viewers. The particle elements will be triggered only if the viewers are looking at the geometries. Otherwise, they will stay deactivated. A similar interaction with vision is available in the exterior world going from the fire room. There is a stripe-like flying object that symbolizing a phenix chasing a ball high in the sky. Normally the two objects are just orbiting around the viewers. By looking at the ball for two seconds, the â€œphenixâ€ will suddenly rush down towards the viewers. And then the viewers will suddenly be surrounded by the â€œphenixâ€. With this interaction the authorâ€™s intention is to create a moment of intimacy between the digital lives and the viewer; an immersive dialogue between the virtual and the physical.
The same intimacy exists in the scene in which viewers are positioned under water and a school of fish is swimming around. Viewers are capable of controlling the movement of these fish by directing them with physical hand motion. The viewersâ€™ hands are visible in the virtual space with the help of Leap Motion, a device that can handle interactions with a digital object with physical hands. (Leap Motion Frontend Team, 2017) The presence of physical hands in virtual space is necessary because otherwise, it is confusing for the viewers when they move their hands for those to remain invisible virtually. This virtual representation of hands is actually important especially in projects that need physical touch in a virtual environment like Threshold, in which the viewersâ€™ hands are represented by a white dot. This kind of representation, in the authorâ€™s opinion, is inferior to the realistic hand Leap Motion because viewers lose the subtle movement of fingers. These subtle changes are something that greatly adds to the immersion of experience in VR.
Apart from vision and hand interactions, sound design is also an essential part to truly immerse viewers into the virtual environment. In reality, people receive various kinds of sounds from the environment. Perception will not be complete without sound. Visual cues are effective elements that can guide viewers through the narratives, while sound cues are something that can greatly enhance the efficiency of perceiving the context. Ambient sound and binaural sound are applied in simulacrum to provide viewers with a sense of immersion. Â These are enabled by an asset in Unity called Steam Audio, which can simulate the physically based sound environment. Ambient sound is a combination of sounds of a certain location or space (Holman, 2015). In simulacrum, the ambient sounds are applied to the six large scenes which represent metal, soil, fire, water, wood, and maze. Each scene is matched with an ambient sound to make viewers have a sense of presence into the atmosphere even if their eyes are closed. Take wood, for example, wind blowing and birds singing are embedded in the scene so that viewers can better relate the visual contents of the wood scene to an actual wood scenario.
Binaural sound, or binaural hearing, refers to the ability to determine the direction and origin of sounds. In VR narratives, these sounds are mostly object-based, which means the audio sources should be placed in the same position as the targeting objects. In this way, non-interactive elements become interactive and interactive ones become even more real because of the introduction of sound. With binaural sound enabled, all the objects with a sound can be traceable in the space. And furthermore, binaural sound can be more realistic by enabling attenuation, which makes the sound to be distance- based. So when viewers are walking in the space, the combination of sound interaction is changing in real time.
4.7 In Between Physical Boundaries and Virtual Spaces
Physical presence in VR can be largely strengthened by matching the physical boundaries with the virtual ones. To make the touch even more authentic and more available for the viewers, the team physically built up the wall beside the door. So every time viewers walk through a door, they will be able to touch the wall. The curves on the edged, thickness of the wall and position of the physical walls are all matched for the viewers to feel the details of the space, in a similar way to what is done in Thresholds with the heater and wood frames. Viewers can physically touch the wall while virtually seeing the touch. This is important in that ideally, the four sides of the physical room will all be built up to continuously reminding viewers of the existence of physical boundaries while helping them understand that virtual walls are exactly matched with physical ones no matter which room they are in. Due to the limited time and room availability, the team built only the wall that is by the door side for the viewers to get the physical touch.
Concerns about the Narrative and Future Proposals
Since the project is still in progress, we are working on further developing and optimizing the algorithm, while researching on more possibilities of redirecting viewers movements in virtual spaces. Furthermore, the team is exploring whether this algorithm is able to generate the virtual environment by adapting to all sorts of physical spaces that move beyond the common, rectangular room shape. For more detailed information about this, please refer to â€œChanging Spatial Boundaries: A Design-Driven Study on How to Procedurally Generate thePerception of a Spatially Enlarged Walking Experience, While Providing the Passive Haptics, in a Mixed Reality Environment.â€ written by Junxi Peng. (Peng, 2017)
5.1 Arguments And Concerns
During the design process of the current narratives and demo experience, and based on the tests we run with people with the redirected walking techniques, the author and team collected some positive feedback while raising certain concerns. Both are things the author would like to further consider and put emphasis on during the next stage.
The first concern would be the readability of the virtual space. The team had set up a room-sized 4 by 8 meters and invited people to experience the demo. By observing the way people walk and explore the virtual space from the screen and based on their oral feedback on what they feel about the space and interactions, we learned that there are some elements that actually hinder the viewers to get immersive experience or even stop the redirected walking from working properly. Firstly, the particle elements in each room seem to be triggered too easily and, even if viewers stop looking at the floating objects that trigger the particles, they still take some time to disappear. So, when viewers want to navigate their way around the room, these immersive particles fly all over in the viewers view to prevent them from finding their way forward. We are working on currently replacing these particles with more sensible visual cues to differentiate spaces while maintaining interactivity.
Simulacrum Public Demo
Secondly, the elevators seem to be unrecognizable from a certain point of view because of the fact that they have the same color with the walls and floors. This might make users lose their continuous perception of how the space extends vertically because the connection of space is broken at that point. A possible way to avoid this would be to change the color of the elevators so that they are readable from any place. By the time of the submission, the team has fixed some major bugs and optimized visual content and interactions and is planning to set up a second demo experience for the second round of tests with participants.
The third concern is that we cannot fully control viewersâ€™ walking path in the space. As it has been explained before, the team has built up a physical wall by the door side to make people able to touch the details on the physical wall that are matched with the virtual wall in the same position. However, during the demo experience, most of the viewers cannot see the lower handles that are on both sides of the wall. So when they are walking too close to the wall, sometimes their legs will run into these handles and scare them off, and suddenly stop the immersion. That is a mistake that should work as a reminder for a more careful design approach not only for what viewers see but also for considering what viewers might not or choose not to see.
5.2 Design Proposals
â€˜Simulacrumâ€™ is a project aiming at enlarging peopleâ€™s perception of space using redirected walking techniques. The goal of the team is to invite people to redirect their perception not just by mathematical calculation and by manipulating their mind, but mainly through the use of the immersive experience that VR technologies can offer. The narratives and stories designed for the spaces generated by the algorithm are used to distract viewers from the feeling that they are being arbitrarily manipulated by the algorithm. And instead of breaking the consistency of the story and narratives, viewers in simulacrum are being distracted by interactive cues. All the narrative designs and characters that the author has given to the spaces in simulacrum are an initial attempt to use different storytelling methods from the filming and VR industries that have been effectively tested in immersing people and offering a sense of presence. The team is trying by combining both these techniques to compose narratives and distract people using the collective power of these VR techniques. However, it is true that from an architectural perspective, some visual cues used to differentiate space are too literal and some interactions are not designed in an architecturally challenging way.
After the demo experience, the viewers were asked to draw a plan of the space showing the team what they think the space looked like, how the spaces are connected with each other and the relationships among different rooms and scenes. There have been some interesting and even surprising results that were collected from those viewers.
some viewers were more into observing details or a certain room, while some focused more on the overall layout of the place. Some did successfully perceive how the doors and walking paths were organized in neighboring rooms and, one viewer surprisingly figured out both the infinity vertically and layout plan of the space right after a short time of being in the virtual space. Visual perception provides substantial support that the world individuals which â€˜seeâ€™ is not the exact image but â€˜constructedâ€™ due to cognitive constraints (Johnson, 2009). This indicates that the abilities of spatial perception are different among individuals and that viewersâ€™ cognition and attention vary. In the next step, the author intends to have the demo experience tested on more people to collect further feedback and hopefully more unexpected results that can potentially lead the project to a much more interesting direction.
During July and September, the team has further developed the algorithm and has found a way to turn on and off the algorithm in realtime to break the boundaries of hexagon and make it more adaptive. The images and videos below shows how the teamÂ further developed both the algorithm and narrative design.
Realtime Demonstration of Algorithm
Spatial Typology Enabled By Controlling The Switch of Algorithm
With the topic of how computational tools designed to virtually extend the boundaries of physical space can be used to give character to and construct narratives within the spaces experienced in a user and how these spaces generated by a redirected walking algorithm can provide users with a sense of purpose, while also make them unaware of being manipulated by redirecting their movement and essentially â€œdistractingâ€ them with a narrative, the story and narrative designed in this thesis are only one version of interpretation of what the space could become.
Considering the initial goal of the project to provide these natural and invisible manipulations for viewers in order to alter their perception of an enlarged space and, taking into account both the critical review and the design project of simulacrum that have been applied to the narrative design, the author could draw the conclusion that the goal has been partially achieved.
People who participated in the demo experience expressed a feeling of being so immersed that they easily forgot where they were. And the fact that after about 15 minutes in the virtual space, when they took off the headset, they were surprised they found themselves in what felt it was a much smaller room.
In the future, if such algorithms might be able to make viewers feel the spatial enlargement through multiple senses, we could assume that people will gradually learn and adapt to new perceptions of space and their authenticity, to the extent that physical space might become less and less relevant to perception. Basic feelings and functions will still be provided by physical spaces and, VR- related technologies will be the intimate medium to ultimately provide the human with multi-sensory and highly customizable levels of satisfaction. Yet, such a speculation does not wish to imply that similar applications will and should take place in all cases. But, it acknowledges the open nature of the questions that such technologies raise while it highlights the power that redirected walking techniques have in challenging the boundaries of narratives in the upcoming realm of VR environments.
Akten, M. (2017). FIGHT! Virtual Reality Binocular Rivalry (VRBR). [Blog] Medium. Available at: https:// medium.com/@memoakten/fight-virtual-reality-binocular-rivalry-89a0a91c2274 [Accessed 14 Jun. 2017].
Astle, D. E.; Scerif, G. (2009). â€œUsing Developmental Cognitive Neuroscience to Study Behavioral and Attentional Controlâ€. Developmental Psychobiology. 51 (2): 107â€“118.
Baudrillard, J. and Poster, M. (2007). Selected writings. Stanford, Calif.: Stanford Univ. Press. Booker, C. (2004). The seven basic plots of literature. New York: Continuum.
Bruner, J (1957) Going beyond the information given. In Gruber, H (Ed.), Contemporary Approaches to Cognition. Cambridge, MA: Harvard University Press.
Freytag, Gustav (1863). Die Technik des Dramas (in German). Retrieved 2009-01-20.
Grant, N. (2017). Narrative Design in Virtual Reality. [Blog] Unity Blog. Available at: https://blogs.unity3d. com/2017/03/01/narrative-design-in-virtual-reality-vrdc/ [Accessed 24 Jun. 2017].
Holman, T. (2015). Sound for digital video. [Place of publication not identified]: Focal.
Huang, J. (2017). Problem Solving During Mixed Reality Project Development Process. Design Report. University College London. (Unpublished)
Jacobs, J. (2016). The death and life of great American cities. New York: Vintage Books.
James Turrell. (2017). Introduction. [online] Available at: http://jamesturrell.com/about/introduction/ [Accessed 9 Jul. 2017].
Jill Walker. â€œPiecing Together and Tearing Apart: Finding the Story in â€˜afternoonâ€™â€, in Proceedings of ACM Hypertext â€˜99, Darmstadt, Germany, 111-117, February 1999.
Johnson, Angie (2009) Visualization techniques, human perception and the built environment. Northumbria Working Paper Series: Interdisciplinary Studies in the Built and Virtual Environment, 2 (2). pp. 93-103. ISSN 1756-2473
Leap Motion Frontend Team (2017). Interaction Engine 1.0: Object Interactions, UI Toolkit, Handheld Controller Support, and More. [Blog] Leap Motion Blog. Available at: http://blog.leapmotion.com/ interaction-engine/ [Accessed 15 Jun. 2017].
Nietzsche, Friedrich (1888). â€œReason in Philosophyâ€. Twilight of the Idols. Translated by Walter Kaufmann and R. J. Hollingdale. Retrieved 2 May 2007.
Oppegaard, D. (2016). Plot-Part One. [Blog] David Oppegaard. Available at: https://davidoppegaard. com/2016/02/04/plot-part-one/ [Accessed 1 Jul. 2017].
Peng, J. (2017). Changing Spatial Boundaries: A Design-Driven Study on How to Procedurally Generate the Perception of a Spatially Enlarged Walking Experience, While Providing the Passive Haptics, in a Mixed Reality Environment. Design Report. University College London.(Unpublished)
Plato. The Sophist. Translated by Benjamin Jowett. Retrieved 2 May 2007.
Rackard, N.(2013). Infographic: Life Inside The Kowloon Walled City. [Blog] Archdaily. Available at: http:// www.archdaily.com/361831/infographic-life-inside-the-kowloon-walled-city [Accessed 6 Jun. 2017].
Robert Goldman; Stephen Papson (2003-08-30). â€œSimulacra definitionâ€. Information technology. St. Lawrence University. Retrieved 2015-08-04.
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), pp.3549-3557.
Steam Audio. (2017). Steam Audio. [online] Available at: https://valvesoftware.github.io/steam-audio/ [Accessed 13 Jun. 2017].
Trafton, A. (2011). What makes an image memorable? We tend to remember pictures of people much better than wide open spaces Read more at: https://phys.org/news/2011-05-image-tend-pictures- people-wide.html#jCp. [Blog] Phys. Available at: https://phys.org/news/2011-05-image-tend-pictures- people-wide.html [Accessed 27 Jun. 2017].
Walker, J. (2013). Redirected Walking in Virtual Environments. http://cs.mtu.edu/ jwwalker/les/cs5760- jwwalker-topicpaper.pdf