Over the Terrain
Time: 2021 Spring
Team Size: 1
Role: Creative Coding, algorithmic visual artist
Tools: Three.js, p5.js, HTML, CSS, JavaScript
Front-end Engineering
A 3D web-based interactive webpage utilizing JavaScript library Three.js.
The main character is a lonely bird who got lost in this terrain. The user could control the direction of the bird flying from the first-person perspective. The terrain is separated into four segments and individual unique event triggers when entering (namely the weather condition: fog, cloudy, rainy, and sunny).
The weather finally becomes sunny in the last section and one can see a flock of birds in the distance: he finally finds where he belongs.
Inspiration
“千山鸟飞绝，万径人踪灭”
In mountains flying sparrows left; On roadways passing people gone.
The theme of this project is about exploring oneself and finding where one belongs. The inspiration comes from the Chinese poem listed above, which describes the bleak sight in the winter. The aesthetics also mimic the classic Chinese black and white painting. I was watching the birds in the sky migrating in a mangrove, and a thought occurred to me: what will happen if a bird was separated from the main flock?
Process
Three.js, p5.js and object-oriented programming.
The main structure of the programming follows the structure using Three.js, which has an initTHREE function that loads the basic elements such as scene, camera, renderer, container and light of the 3D space. The interaction happens within the render function which loops over time to render the objects in the scene. The weather visuals are instances of different classes. Some visuals are generated using shaders that operate on GPU to enhance the performance of the website.
Terrain: PlaneBufferGeometry with random heights generated by Perlin noise
The terrain is the fundamental object in the project. To create the terrain, I used PlaneBufferGeometry with randomly generated height data from generateHeight function. In this function, I computed an array of random but consistent height data using Perlin noise.
Then I multiplied the datapoint by 10, normalized them and loaded them as the position attribute for the plane (See left).
A png picture of the terrain is used as the texture for the terrain object to make it more realistic.
Rain: BufferGeometry with position vertices generated based on camera position
The rain visual is generated using the points object in Three.js, the geometry of which is computed using BufferGeometry. BufferGeometry is a representation of mesh, line, or point geometry, which includes vertex positions, face indices, normals, colors, UVs, and custom attributes within buffers, reducing the cost of passing all this data to the GPU.
I set the y position vertices of the rain to be at the top part of the space and subtract the y positions frame by frame to animate the rain falling down. The z position vertices are generated based on the camera position so the rain will only appear in front of the user's perspective.
GUI: dat.gui, JavaScript Interface Controller
The GUI control bar used the interface controller implemented by David Walsh. The change in the controller will instantly be reflected on the screen (See left). This is done by storing the values in a JavaScript dictionary and using the callback function guiChanged (See above).
Sunny: Sky shader with user input values
To create the sky, I utilized a shader implemented within Three.js library. Shaders are simple programs that describe the traits of either a vertex or a pixel, which operate on GPU. I also created a directional light object to mimic the effect of the sun. The position of the sun is set according to the camera position so the user will always see the sun ahead. Then I added a png picture of a flock of birds next to the sun and the sound of birds to represent the bird has found where it belongs.
Failed Attempts
Many attempts have failed in the process due to the limitation of computer hardware also a well-rounded understanding of Three.js and WebGL Post-Processing effects. One example is the attempt of integrating the lightning shader.
The shader is implemented by Three.js community using WebGL image processing. WebGL is a JavaScript API for rendering interactive 2D and 3D graphics within any compatible web browser, allowing GPU-accelerated usage of physics and image processing and effects as part of the web page canvas. However, even though I got the 3D lightning object on the canvas, I could not be able to animate it. Some timing calculations must have gone wrong over the recursion of drawing the branches of the lightning. After spending a long time debugging and consulting professionals, I decided to keep this visual out and implement it later.
Showcase and feedback
I showcased this project both in class and during Interactive Media Arts End-of-Semester Spring 2021 at NYU Shanghai. During those two events, I received valuable feedback from the general audience and professionals.
[Positive]
1. Visuals are aesthetically pleasing and consistent, aligning with the mood of the project and music;
2. The first person control is interesting and immersive;
3. The concept is thought-provoking.
[Negative]
1. The bird can go "under" or "through" the terrain (through the terrain object), it draws people out of context;
2. The clouds and rain looked very two-dimensional than the terrain and the birds;
3. The transition between the rain scene and the last scene is too abrupt.
Implementation
Based on the feedback, I implemented the project during the summer.
1. Adding limitations to where the camera (bird) can go was too rigid. Using the height of the terrain to limit it took up too much computation and eventually brought the whole project down. So I decided to implement it in other projects that are not going to be in the browser, such as unity.
2. I added a plane in front of the sky in the last scene and gradually dropped it down to make the transition smoother (See left). I have also tried using a picture as the texture for the sky and gradually changing the opacity of the texture. However, changing opacity over time seems to be forbidden in Three.js. That leads to the current solution.