Rise Up!

A Narrative Virtual Reality Experience

Screen recording of 'Rise Up: A Narrative VR Experience'

  • Work Type: Narrative VR Course Project
  • Duration: 5 Weeks
  • Categories: Virtual RealityAnimationProcess

The Narrative:

My aim was to make a short VR animation about an egg that lives underwater in a broken refrigerator. This is a user without impact VR experience. The user will be able to see the egg as a character in this experience. The egg has built a life under the ocean but things begin to change for her as her body starts to rise above the ocean floor. As she moves up she can see her entire world and community slowly fade away and vanish. She tries to cling onto surfaces to stop her ascent but fails. During her ascent she comes across different sea dwelling creatures like jellyfish, turtles and whales but none are able to help her get back down. As she rises out of the ocean and onto the dry land she dies.

The aim of this VR piece is to provide an aging experience. It is a take on the mortality of living things. An egg sinks to the bottom of the water when it is fresh and rises to the top when it goes bad. I am using this concept to show an egg’s end of life journey.

The storyboard update:

The initial storyboard that I made included various scenes and each scene had the egg interact with a different sea creature. However, as the project progressed I realized I would have to change the scope and so the final piece only has a school of jelly fish swimming close to the egg. So far there are no interactions of the egg and the jelly fish.

Below is an initial sketch of the storyboard:

The Process:

Moodboard:

The first step was to create a moodboard for the VR piece. I used Pinterest to put together some images that inspired the aesthetics of the experience. Below is an image of the moodbaord:

Storyboarding and Assets:

I began storyboarding of the underwater scene in Tiltbrush. I created some assets such as star fish and jelly fish for the underwater world in Tiltbrush and exported them as fbx files. These were then imported into Unreal Engine 4.

Some assets created in Tiltbrush are below:

I created some more assets for the world such as a cliff and underwater rocks in Medium. I found Medium to be more suitable and convenient for creating the assets for my project. Medium’s grid helps you to orient your asset properly and it also allows you to move around and scale up/down the assets. So you don’t have to worry about drawing it to scale. It also has layering which is very useful for creating environments.

Some assets created in Medium are below:

Storyboarding in Tiltbrush:

Materials:

I created a few custom materials to add to the imported assets. One such material was Fresnel. An instance of this material was used on the jelly fish.

Blueprint for the material:

Landscape and Foliage:

I used the landscape tool to paint the surface of the bottom of the ocean. I mainly used the sculpt and erosion brushes for painting. Then I used some plants and star fish assets as foliage and painted them on the landscape.

Image of the landscape:

Building the World:

When building a world for VR it is important to follow VR best practices and to place objects that are about 0.75 to 3.5 meters away. This is the region with best visibility. Initially when I built the world I did not take this distance into account but later into the project I had to place all items in the world at a distance of at least 0.75 till 3.5 meters away from the center. The center is where I assume my user will be positioned initially within the world. Since 1 unreal unit (uu) is equal to 1cm I placed all important objects such as the refrigerator, cliff, egg and jelly fish within 75 – 350 uu in all directions.

Some features used to create the underwater effects are:

  • Creating a new material with water texture and panner. Below is a screen shot of Blue print for the M_Caustics material.
  • Adding this material to a directional light that is placed within the scene.
  • Using post processing volume to create local underwater blurring effects. To create the distance blur effect the Lens’ ‘Depth of field’ parameters such as depth blur radius and focal region of the PostProcessVolume were adjusted as shown in the screen shot below.
  • Adding a plane with lake water material and placing it at the top of the scene depicting the surface of the ocean. Below is a screenshot of the plane above the PostProcessVolume – this is the surface of the ocean.
  • Adding a global effect called ‘ExponentialHeightFog’ to my map as shown in the screenshot below.

Character and Animations:

I modeled a custom character in Maya instead of using a character generator because I wanted to use an egg as my character instead of a human figure.

Then I exported the model as an obj file along with the textures .mtl files and zipped them into a single folder. This was then uploaded into Mixamo and rigged. I then saved various animations such as T-pose, idle, walking, treading in water, swimming and swimming to the edge. The T-pose was imported into unreal as a skeletal Mesh for the Egg and all other animations were imported without the Mesh.

Following are the animations on the Egg in Unreal Engine:

Sequencer:

My entire VR experience is based on the Sequencer. I added the egg into the sequencer and gave it different animations and transformation using keyframes. I also added the jelly fish blue prints into the sequencer and animated them by changing their transformations over time. Another thing added to the sequencer was the jelly fish sound cue (discussed below). This moves in the same path as the group of jelly fish to give a localized sound effect.

Sound:

I added two types of sound in my project. One is the background sound of water, this was a short sound clip which I normalized and imported into unreal engine. I then created a sound cue for this audio and used the looping node to loop this sound throughout the scene. I also added another sound for the jelly fish to my scene. I created a sound cue for this sound and in addition to looping I also added the attenuation node to this cue. I set the inner radius and attenuation distance values to lower values so that user can only hear jelly fish sounds when they are close to the fish. This sound cue was then added to the sequencer and moved along the same path as the jelly fish so that the sound was localized within the scene.

Sound cue for jelly fish:

Sound cue transformation keyframes in the sequencer:

Organic Animations:

I wanted to make the movements of the jelly fish more organic rather than the linear movements afforded by the sequencer so for this I employed the following techniques:

  • Clothing Tool:

I imported the jelly fish as a skeletal mesh instead of a static mesh and opened it in editor. I then added a cloth asset for this jellyfish and used the cloth paint tool to paint cloth values on the 4 tentacles of the jellyfish. I then applied this cloth asset to the jellyfish.

Painting vertices of the jelly fish as cloth:

  • Creating a Blueprint for the jelly fish’s up and down movement:

I created a blueprint for my jelly fish which makes them continuously move up and down so that it seems they are hovering in the water. This blueprint was then added to the sequencer to move the jelly fish across the scene.

The blue print for the jelly fish up/down animation:

This video shows the outcome of the final animation of the jelly fish with clothing tool and blueprint. (Note: towards the end of the video you’ll see the tentacles move.)

Issues and resolution:

Initially I faced a lot of issues with the sequencer. I could not get the egg to animate from one place to another using the sequencer so instead of using the keyframes view I switched to the graph view of the timeline and this seemed to have worked. However, after some time I had the same issue with jelly fish so I had to delete everything from the sequencer, restart Unreal engine and start the animation in the sequencer from scratch. This resolved the issue with the sequencer.

Initially when I tried painting the vertices of the jellyfish with the cloth painting tool, the jelly fish model seemed to distort in a weird way. I then tweaked some of the painting tool parameters, deleted the cloth asset assigned to the jellyfish and created a new one. This resolved the issue with clothing tool. The jellyfish legs seem to move organically at times however, this is happening very infrequently, I will try to adjust some more parameters to see if I can get this to the level I want.

Lastly, I had some issues with incorporating a camera within the sequencer and animating it such that it followed the egg initially and then zoomed out and showed the world around the egg. I was having some issue of getting the egg into the view of the camera because my egg was too tiny and the world was very big. In order to resolve this I realized that the objects in my world were not placed at ideal distances for viewing in VR so I placed all the assets in my world according to the ideal distance. The main focus was the fridge and egg. Then I realized that since it is a passive room-scale VR experience, the viewer will be the camera for the experience so I don’t need a camera in the sequencer. I only wanted to animate the camera in the sequencer to document my project but I decided to use player start movements for the documentation instead.

Screen recording of the complete VR experience:

Below is a video recording of a computer screen when I was trying out the project in VR using Oculus Rift. Please excuse the video quality as the computer's screen recorder was broken I had to improvise with my phone's camera.