Hacking virtual reality
Contributing to a culture of pioneers, MIT students explore the technical, philosophical, and artful dimensions of VR.
 


photo of Sandra Rodriguez by Erik Tanner

VR and Immersive Media Production (CMS.339) engages both mind and hand. What better way to explore a new technology than to push it to its limits, re-imagine it, and rebuild it in unexpected ways?



Video Playlist: CMS.339 Student Portfolio


One of the newest maker spaces on MIT’s campus exists in virtual reality — where students are pioneering a media so new that the terminology is still being defined.

In the hands-on humanities class, "Virtual Reality and Immersive Media Production" (CMS.339), students are grappling with multiple dimensions of making virtual reality (VR), among them: technical challenges, such as how to prevent the fatigue common to users of VR devices; philosophical questions, such as the difference between “presence” and “immersion”; and issues related to the art of storytelling, especially discovering the visual languages and narrative forms that VR enables.

“It takes eight minutes to learn how to make the 360-video camera work. The rest — figuring out the experience you want to make — is your mind,” says CMS instructor Sandra Rodriguez, who first taught the semester-long class in 2017 in collaboration with William Uricchio, Professor of Comparative Media Studies. The class, which made history as the first VR class ever to be offered at MIT, ran again this term, taught by Rodriguez


Inventing a new language

Offered by the Comparative Media Studies/Writing program (CMS/W), the new VR class appeals to students interested in the nexus of tech, design, and storytelling. Production in the class relies on tech elements — including the Unity development platform — and the course focuses on the creative works that the technology supports.

“A medium is a way of expression. With this new medium, we’re inventing new language,” says Rodriguez, who is also a visiting scholar in the SHASS-based Open Documentary Lab and creative director of the EyeSteelFilm Creative Reality Lab.

Cattalyya Nuengsigkapian, a junior majoring in computer science, says she enjoys the class because it delves more into design than most of her classes. “This is not only technical. It’s more like art," she says, "That’s why I think I learn a lot.”
 


"For this medium to thrive, we need to ensure there is a healthy diversity in the stories that are created, produced, and distributed. To me, this means helping students become creators themselves.”

— Sandra Rodriguez, Visiting Scholar, OpenDocLab; VR instructor; and Creative Director, EyeSteelFilm Creative Reality Lab



Entering the world of virtual reality

To enable students to gain a rich understanding both of the media and its potential, Rodriguez begins with an introduction. “I want our students to understand the field and to be able to distinguish themselves.”

First, the terms. Today, “virtual reality” (VR) is generally held to mean a computer-generated experience that attempts to immerse the user in a simulated world. “Augmented reality” (AR) inserts computer-generated elements into the user’s view of the real world; one might, for example, be able to bounce a virtual ball while otherwise viewing the room in which you are actually standing. Another term, “360 video,” describes the experience of essentially being in the movie you are viewing — except that you are not an actor; while you can look at the scene from every angle, you can’t affect the scene.


A longstanding dream

Next, Rodriguez outlines the history of the field. Students learn that the earliest VR forays date back to the 1950s and ‘60s. In 1962, for example, a “Sensorama” machine was created that gave users a way to see films enhanced with sounds, odors, and motion. “Human beings have long had this dream of being in the scene," says Rodriguez. "We like to have our senses fooled and to feel like we are there. That’s not new.”

Rodriguez notes that one could easily teach a whole class on the history of VR, and another on the pure technical requirements of a 360 video or VR experience production. But as a documentary-maker herself — she worked on “Do Not Track,” an interactive web series that won a Peabody Award in 2015 — she chooses to focus the bulk of the CMS.339 class on helping MIT's maker-oriented students produce their own virtual reality projects.

“For this medium to thrive, we need to ensure there is a healthy diversity in the stories that are created, produced, and distributed. To me, this means helping students become creators themselves,” Rodriguez says.
 


Video Playlist | Trailers for VR works from the class

Up first: The trailer for "Reveal," a CGI world-exploring VR experience that invites users to a metaphorical pilgrimage on the quest to find one's place in the world. Followed by "Avatar/ChakraScape," and "MITVR."

Hacking VR at MIT | CMS.339 Student Portfolio



Students get to experiment with emergent VR gear including new Oculus Rift, Touch Controllers, 360 Samsung Gear cameras and Gear headsets as well as Mixed Reality HP headsets. Rodriguez also introduces them to principes of design and storytelling. “You need to understand the technical tools" she says, "but this class is about creativity, overcoming challenges, and contributing to the culture of pioneers.”

To illustrate the process, the class features a rich array of guest speakers chosen from among the top contributors in the VR field. Highlights this year included talks by Arnaud Colinart, co-founder of AtlasV, which produced "Notes on Blindness," a star attraction at the Sundance Film Festival in 2016; and Viktor Phoenix, head of audio for Los Angeles at Headspace Studio, which won a Primetime Emmy Award for the VR work “The People's House: Inside the White House with Barack and Michelle Obama.”

“In this class, we start on the creative side. People are gravitating toward the stories they want to tell,” says Emily Salvador ‘16, a master’s student in media arts and sciences who serves as the teaching assistant for the class. “I’m trying to help them bring their stories to a technically implemented reality,” says Salvador, who previously designed immersive experiences as an intern for Walt Disney Imagineering and NBCUniversal Media.


Hands-on humanities projects

As the Spring 2018 class began to focus on production, the 21 students split into seven teams to develop their own VR and AR projects, ranging from an interactive game in which users work to unlock their chakras to a 360 video centered on the experiences of life after incarceration.

Nuengsigkapian’s group, for example, is creating a virtual reality tour of MIT. “We think it would be great if we could share our world here at MIT, make it even more real than on the website,” she says.

Libby Falck, a first-year graduate student in CMS/W, is part of a team that is producing a VR experience about the future of work. A native of Wisconsin, Falck says she has seen how the loss of industry impacts workers and is interested in exploring both the history of work and a vision for the future through VR.

“We would like to exhibit our final VR work in libraries,” she says.


Image from "Avatar/Chakrascape," a meditation-inspired escape room experience, where users are invited to solve a series of interactive puzzles and restore balance to their lives.

Hacking VR at MIT | CMS.339 | Student Portfolio



The art of the possible

The students’ goals are ambitious for a one-term class, but Rodriguez says exploring what’s possible is part of the creative process. “Feel free to fail. That’s the best way to learn,” she tells students. “It's about iteration. You have an idea. You try it. You iterate.”

The success of MIT’s first virtual reality class suggests a promising future for MIT VR endeavors. One student sent his 360 video to the Vatican and now has a full-time job shooting 360 videos of Pope Francis. Another is working on VR projects for the New York Police Department.

Not every student in the class will make a career of virtual reality, of course, but for Rodriguez it is rewarding that the class is opening up the VR world to more makers. Because VR equipment is expensive, access to the field has often been limited. “My preoccupation," says Rodriguez "is about facilitating access to creation.”

 

Student Maker Teams

Reveal: Victoria Bousis, Danny Gelman, and Parker Greene
Avatar VR/ChakraScape: Gwendoline Edgar, Brianna Igbinosun, Sharon Lin, Corbin O'Brian, Juan Miguel Rodriguez, Sultan Sharrief, and Jimmy Zeng
MITVR: Cattalyya Nuensigkapian, Rima Rebei, and Hiram Moncivais
 

Suggested links

CMS.339 Course Website

CMS.339 Student Portfolio

Sandra Rodriguez, Visiting Scholar
   MIT webpage | EyeSteel Film website

William Uricchio, MIT webpage

Comparative Media Studies / Writing
 

Equipment for CMS.339 was provided by Oculus through a partnership with Oculus NextGen, a program that selected 12 leading universities across the United States to jumpstart the next generation of VR and AR makers. For more information about the class, visit www.hackingvr.mit.edu.
 


Story prepared by MIT SHASS Communications
Editorial and Design Director: Emily Hiestand
Senior Writer: Kathryn O’Neill