Spring Water Flower Night
Audiovisual Performances
Time: 2021 Fall
Team Size: 4
Role: leader, computer music composer & performer
Tools: Tidal Cycles, SuperCollider, Hydra.js, flok
This is an audiovisual performance project written in Tidal Cycles and Hydra.js as a collaborative work of four. Yunyi Wang (NYUSH 2023), Ilya Akimov (NYUAD 2023), Maxim Blinov (NYUAD 2021), and I cooperated on this project. The performance is a four-minute composition with an intro, two verses, two buildups, and three choruses inspired by a Chinese traditional song. All the visuals and sounds are generated through live edits of code.
The concepts of music and visuals are coherent and mutually responsive, as we seek to recreate and reimagine an ancient Chinese song that pictures a cinematic scene of a river in the moonlight. The project serves as an experiment to exchange between cultures and juxtapose oriental and western music to observe the outcome.
Inspiration
春江花月夜: Spring River Moonlight Night
For this project, the concept was derived from a traditional Chinese ensemble song, 春江花月夜 (Spring River Moonlight Night). The lyrics of the song are from an ancient poem that narrates the scenery of a spring night on the river. The poet was admiring the moon and the flowers while missing his home.
The song was originally played with multiple Chinese instruments and has rich layers and structures. I am deeply touched by the song and decided to recreate it with modern electronic approaches to observe the chemistry. Since we have a team coming from different countries, I'd also like to encourage cultural exchange during the collective composition process.
Process
Music, visual, sampling and teamwork
Music: Sampling from the original song, collaborative composing
The music contains samples from the original song and poem, and we utilized a modern song structure, hip-hop inspiration, electronic musical instruments, and MIDI to recreate our original piece. The music is all written using text-based music synthesizing language Tidal Cycles, which utilizes SuperCollider, another open-source software, for synthesis and I/O.
I sampled a solo Guzheng (a traditional Chinese instrument) piece in the intro and outro (See left 1). I gradually added on the hi-hat and the kick drums, whose patterns normally appear in trap music (See left 2). d4, the jungle bass instruments were inspired by the song Fly As Me by Sonic Silk.
Since the lyrics of the song are from an ancient poem, I also sampled the voice reading by a female speaker. I applied granular synthesis to the voice using shuffle and striate (Seef left 3).
To sync music with the visual, I sent some cc values through MIDI to hydra.js (See right 1). These values are used to communicate between music code and visual code. I worked with Yunyi to design the pattern for cc values so the visuals will change accordingly with the beats in the music.
The main inspiration for the upbeat section of the performance was sourced from two artists. First is ODESZA, known for creating refined and immersive electro-music productions (See right 4). Their use of non-pop instruments and multilayered beat structures was a solid reference for us. Drawing on ODESZA’s beat structures, the first beat drop contains not only regular 4/4 “bd” beats, but also a 16th note “ades2” pattern (similar to the one used in Loyal), which adds a higher-pitched and dynamic motion to the overall beat.
I collaborated with Ilya to compose this part. I used the bass “808bd” sample to cover an electronic, unreal drum soundscape, in this way contributing to the climax of the composition (See right 2). As the latter is meant to evolve with time, the second and third beat drops only add more to the rhythm and beat. The fourth beat drop aims to de-escalate the tension and allows a more natural transition into the outro part.
Another inspiration was CL’s Lover Like Me, which gave ideas on how to musically implement the Guzheng and the Supersquare instruments into the composition. Her chorus contains a high-pitched synthesizer that uses arpeggios to create an elegant dynamic feel, which seemed to be what we wanted to do in this project. We used an E flat minor scale, which the Guzheng arpeggiates on at the fifth and sixth octaves (See right 3). On the other hand, the rap part is sustained by a low and slightly distorted synthesizer. It succeeds at adding depth, groove, and a catchy sub-melody, which is why a similar sound (Supersquare synthesizer with a 4 or 6 level krush) is consistently used throughout the entire dynamic part.
The outro is fading out all the new instruments introduced in the climax part and crossfade some instruments into a less intense pattern. I circled back to the samples and instruments from the beginning to convey a holistic concept.
Visual: Hydra.js, p5.js, gif as the texture
As for the visualization part, we had three main parts:
-
River (Oscillators by Yunyi)
-
Moon (P5 circles by Max)
-
River in moonlight (combination of both by both)
We collaborated on finding the concept for the visual and adjusting the MIDI values to sync up the music and the visual as a team.
We started with some oscillators created using Hydra.js. We created a growing experience. It was evolved step by step starting from an oscillating white circle, which then gained color, and a modulate scale on top of itself. That became a fill for the p5 circles which were moving around the center point of the screen at different speeds. Then it transitioned into a more chaotic, flashy part by overlapping patterns. In the end, the outro was a very calm gif of waves as the texture.
Performing & Feedback
We showcased the performance live to our class. In the performance, we tried to code from scratch to showcase the development of the composition. However, some errors happened with hydra.js and the server we were using to collaboratively code was not functioning correctly either. So the performance stagnated in the middle part for several minutes before we solved the hydra.js error, which is not supposed to happen if we run it locally in Atom.
It was not a very successful performance, the footage you are seeing right now is the footage from our rehearsal. We received encouragement and feedback from our classmate and professor being:
[Positive]
1. The concept is intriguing! Loved to see the juxtaposition of Chinese classical instruments, poems with electronic music.
2. Love the development over time. The process of music and visual development is very interesting.
3. The visuals match the concept of the video. A very nicely done cultural experiment.
[Negative]
1. The middle part of the music and the intro are not using the same Guzheng instrument (one is a sample and another is the instrument in Garageband). Try to use the same texture for the same instrument for consistency.
2. In the middle part, the music does not seem to be very well synced with the music, so the audience felt a little out of place.
Implementation
After the incident, we tried to implement the composition for the Interactive Media end-of-semester show. We learned from the feedback and our incident to simplify our composition but keep it synchronized and effective. I proposed that we code together while developing the composition to exchange our thoughts and ideas and understand each other's code.
Above is the implemented version of the project. We inherited some code from Spring Water Flower Night, but chose another approach: we used more colors, bright melodies, and a more organized composition structure. We spent more time syncing up the music and the visual and rehearsing. In the end, we presented a better version of our project without having any errors during the IM show and received a lot of compliments. The performance video see here.
However, I still think the original version provoked questions on cultural exchange and integration of traditional and modern music genres. Its aesthetics are also more aligned with the concept. It is important for us to think about the impact and meanings when we are creating arts; otherwise, it is only beautiful and fancy but without a realistic influence.