Immersive Scientific Storytelling Through Virtual Reality

May 26, 2020

By Angellyn Phua

Storytelling, and its evolution as a medium, can be traced to the early origins of the human experience.  From its origins as a live oral presentation, to handwritten scriptures, paintings, photographs, and moving picture cinema. In our current day, thanks to the internet and advancements in everyday technology storytelling is now immersed with unlimited variations in mass multimedia formats. In recent years, storytelling has made its leap to one of the most interesting technological advancements to date, Virtual Reality (VR).

A few weeks ago, at the beginning of what would wind up being a hectic March of Covid-19 related disruptions to the everyday normality, I was given the amazing opportunity to attend the "Collaborative and Immersive Scientific Storytelling Through VR Experiences" presentation; featuring Dr. Poh Tan and Quincy Wang at Science World in British Columbia, Canada. Attendance at the presentation was a mix of educators, and teachers that branched from a variety of formal and non-formal settings. When talking to Dr. Poh after the presentation, we both agreed that a portion of the audience had most likely expected the presentation to focus on the technical side of VR.  In other words, some audience members may have the presentation focused on coding language and software technology. I was pleasantly surprised, much as the audience that Poh and Quincy approached VR education from a more holistic lens. The presentation explored how VR could be used as an educational tool in the classroom and furthermore, how VR technology can create a more relational space and collaborative storytelling and making between students and educators. The platform was aimed at creating a virtual scientific experience that interprets concepts and environments that the eye cannot reach. Promoting the use of science where textbooks alone could not spark a connection, in a similar manner to science themed television shows that were aimed to connect the audience to scientific material. However, television storytelling remains two dimensional and separates the audience from what they are experiencing unlike VR, which is three dimensional in nature and immersive.

Two attendees experience the immune environment in VR developed by Poh and Quincy using Eyexpo technology at Science World's annual Science Outreach Workshop Meeting on March 9 2020.

VR is an interface which immerses the viewer in a 3D/360° environment with a projected view of in the field of visions of the participant through a virtual reality headset. These interfaces normally include diegetic and non-diegetic sound and stimuli to help simulate real life senses, like sight, hearing, and touch, which helps to give off the perception of being physically present in the non-physical environment. The VR prototype was developed by Quincy Wang using Eyexpo’s VR development platform. This digital story prototype immerses the viewer at a microscopic level inside a blood vessel within the human body.  The environment creates a learning space that helps participants differentiate different types of cells within the blood vessels, as well as their roles in human immunity. Various elements included in the prototype were video editing, UI/UX Instructional design, and 3D rendered models of cells, bacteria, and blood vessels.

The highlight of this presentation was the two children’s demonstration on their participation as co-constructors and co-researchers of the VR project. Mischa and Khafri at the time of the presentation were both in grade five, and this was their first collaborative project working with one another. The two were assigned to learn about the same scientific concept and through their collaboration, they created two different interpretation for learning about the immune system. They began their portion of the presentation by describing their working dynamic and spoke about their struggled because they were of different genders and, although they were in the same class at school, they have never worked together before.  The two also had different ways of learning that coincided with their interests. Mischa described how she learnt better through methods more similar to performance art, while Khafri, enjoyed writing stories where he could connect the material to his interests. At the start of the project, Misha described how the immune system did not interest her. It was not until she learnt about how it applied to her day to day life and heard Khafri’s story about comparing the immune system to an army did she truly understand the material.

Two co-researchers, storymakers and story tellers, Grade 5 students Khafri Mau and Mischa Uy who worked together to create a story and a rap about how they understand the role of the immune system. Left: A snapshot from an Augmented Reality (AR) rap written by Mischa and performed by both kids. Right: Khafri and Mischa co-presenting with us at Science World, talking about their experiences on learning and creating together. The team collaborated with Ms. Anne-Marie Uy, a Vancouver elementary school teacher.

In the presentation, Dr. Tan discussed how this form of immersive storytelling, and its ability to place the viewer in a more active learning position, can help spark emotional connection and empathy. Similar findings have also been researched and published by other research groups in the field of immersive learning. Along with how it can prove to be beneficial in education with its ability to reach a larger range of learner preferences; visual, auditory, reading/writing, and kinesthetic. Quincy Wang spoke about how VR is an effective teaching tool for science education. The largest distinction that VR has from its previous multimedia counterparts, is how it can directly place the viewer in an environment that they would not be able to encounter (unobservable) physically in the real world. The immersive qualities of VR have been suggested to regulate the body's emotions in trauma or pleasure and when learners are placed in these “high embodiment” environments it builds long term imagination and understanding of science.

When I had initially read the premise for this presentation, I was mostly excited to attend in order to learn about how VR can influence the future of education. My interest in VR was sparked after attending a Vancouver International Film Festival Immersed program.  In this program, VR was highlighted in the use of the production of films. However, what ended up resonating with me in the following weeks, as after the presentation, was Dr. Tan’s explanation on the acknowledgement of human and nature in accordance with the information that we learn. Dr. Tan explained that VR can help to blur human created boundaries between human and non-human, culture and nature, etc.  By blurring these boundaries, entities are seen more as kin rather than strangers and thus, help raise awareness of a relational connection between these entities that transcends how we think about relations.

This explanation stuck with me in the weeks to follow as I began to reflect on the way that information and science was presented to me in my own elementary school. I began to wonder about the longevity of my own registration of information from those formative years, and ironically, the lack of it in the area of science. Especially as I am someone who retains information most effectively through spatial (visual) and aural (auditory-musical) learning methods, if I had an emotional, connective experience with the material instead of robotically regurgitating information for exams, would I have remembered more of it? Personally, I believe that if this sort of active engagement was available to me when I was learning science in elementary, I may have found a greater interest in the materials and to learn and view about the subject as a whole. As one of my biggest gripes with science was the way that the course was structured around memorizing information for tests, which would be forgotten not long after the test. While this presentation left me to ponder how technology can be integrated meaningfully into our learning environments, I feel optimistic about the possibilities and opportunities in this VR technology for students and educators in the future. In the way it aims at creating a collaborative process between the two parties of educator and student, and explaining scientific concepts and environments for further understanding. The future is now as Dr. Tan and Quincy Wang showcased the potentiality of VR through a relational approach to be open about learning and knowing.

About the author:

Angellyn Phua is an undergraduate Communications student in the Faculty of Communications, Arts and Technology with a passion for the visual arts and film. She is a yearly festival volunteer at the Vancouver International Film Festival, and enjoys editing movie retrospective videos in her free time.