Around the release of the album i/o we held a competition giving creators the opportunity to produce videos for some of my music. The competition was designed to be a playful and rewarding activity and as the submissions came in I was completely blown away by the talent on display, as well as the quantity and diversity of the entries, and the hard work that went into them. It was hugely enjoyable to be part of.
That experience has led to some discussions about trying to find a way to continue to allow a greater engagement between visual creators and my music. We are doing some work in the background that we hope will help facilitate a more equitable relationship between music makers and visual creators, that acknowledges the joint role they play in the content seen on platforms such as YouTube.
With 50:50 we want to explore ways to better reflect this joint creative endeavour, and hopefully provide a showcase platform for the best visual work.
For now, you can see some of the great videos that came out of the relationships forged by the initial competition and we are inviting any other video makers or creators, who might want to learn more about 50:50 and how to become involved in the future, to sign up here to receive news as plans develop.
Date created: Dec 2023-Jan 2024 ChatGPT: Concept and narrative themes / Initial shot-list. StoryboardHero.ai: Initial storyboard frames Adobe Premiere Pro: Board-O-Matic / 2D animation / Final Edit RunwayML Gen-2 / PIKA Labs: Additional AI img2vid motion to bring the source material to life. Mixamo / Maya: Additional 3D character/camera animation ComfyUI: AI vid2vid processing, utilising SDXL / AnimateDiff / IPAdapters and ControlNets. Nuke: Colour grading, VFX compositing, shot cleanup. TopazVideoAI: Upscaling of ComfyUI outputs.
The creative process, man meets machine:
Starting in December 2023, the creative voyage of this video was a sophisticated fusion of cutting-edge AI technology and traditional 3D/VFX tools, with collaboration between humans and machines pivotal at every stage. From conceptualization to final execution, the synergy between these advanced tools and the creative vision behind the project was instrumental in bringing it to life.
To kick start the process, Storyboardhero.ai was utilised for initial AI-generated storyboard frames, guided by the shot-list provided during creative discussions with ChatGPT. These frames were compiled into a timed "board-o-matic" using Adobe Premiere Pro, allowing for the pacing and overall feel of the video to be gauged. This process involved refining the shots iteratively, with input from both AI tools and human creatives.
Once the timings were established, the process of adding motion to the source material began, utilising various methods. Some simple 2D animation was applied to certain storyboard frames using Nuke and Premiere. However, the primary focus was on running the boards through multiple AI tools individually to imbue them with life. Working closely with Graeme Cornies from Kodiak Arcade, storyboard frames were processed using img2vid tools like RunwayML’s Gen2 and PIKA Labs to enhance their dynamism. Additionally, 3D software such as Mixamo and Maya was employed by Oranguerillatan to introduce character animation, camera movements, and depth to select shots. The addition of a few real-life video clips at this stage further enriched the visual narrative.
The previous steps focused on animating still frames, but they didn't address the final aesthetic. Enter ComfyUI. Using a vid2vid setup developed with input from some very smart members of the AI community. Each of the 200+ shots underwent a ComfyUI transformation, resulting in the distinctive claymation style evident in the final piece. This tactile aesthetic, reminiscent of stop-motion animation and 'old-tech' craftsmanship, contrasts with the modern AI technology used in production. Most shots were generated multiple times, resulting in over 1000 generations being created during this stage. The cutting room floor was covered in binned generations that didn’t quite make it.
After the shots were generated and the edit refined over several weeks, the process entered its final stages: colour grading using Nuke and AI upscaling with TopazVideoAI. The colour grading was a painstaking process, with each shot meticulously matched to its surroundings to maintain cohesion and evoke the intended mood for each section of the video. From the cool, dark tones of winter nights to the vibrant hues of summer days, each season was depicted with its own distinct visual palette, enriching the narrative and emotional journey of the video.
Exploring conceptual and narrative themes with AI:
Olive Tree’ stands out as a six-minute musical masterpiece that exudes uplifting energy. Its distinct sections intricately capture the essence of hope, interconnectedness, and curiosity. Peter's discussion of his inspirations and themes in his August 2023 "Full Moon" video provided fascinating insight into his creative process while composing and recording the track.
After immersing in the track and Peter's insightful discussions about it, a dialogue between Oranguerillatan and ChatGPT commenced to delve further into the conceptual and narrative dimensions. Through collaborative exchanges, AI served as an illuminating guide, shedding light on creative possibilities and thematic depths. By engaging in these discussions, a deeper understanding between man and machine emerged, enriching the creative process and broadening perspectives.
Delving deeper into the intended themes and content of the video, discussions revolved around the different sections of the song. The song's composition mirrored an epic journey across time, seamlessly transitioning from mystique and child-like curiosity to scepticism, growth, transformation, and ultimately, unabashed joy and euphoria. This musical evolution resonated with the notion of ever-changing seasons and human emotions' daily fluctuations, highlighting life's unpredictable nature.
Strategically incorporating changing seasons and time of day into the video became a key feature of aligning the visuals with the evolving narrative and emotional arc of the song. As the track progresses through its various verses, choruses, bridges, and instrumental sections, the time of day and season seamlessly shift multiple times, enhancing the immersive experience for the viewer.
Shifting seasons - a journey through time:
As our hero embarks on their journey through the changing seasons and the passing of time, each transition mirrors the evolving emotions and narrative of the song. From the mysterious allure of winter nights to the vibrant energy of summer days, the visual landscape shifts in tandem with the song's tempo and mood.
The journey commences in the depths of winter and the cover of night, setting the stage for the mysterious and enchanting ambience of the track's introduction and opening verse. This wintry landscape serves as the perfect backdrop for introducing the protagonist and unveiling their intriguing discovery propelling the narrative forward. Each post-chorus transition to a new verse brings the viewer back to the dark, chilly embrace of winter night, accompanied by the timeless symbolism of the full moon, a motif dear to Peter.
As the song progresses, the visuals transition to the bright hues of sunrise and the promise of a new day, signifying spring's arrival. This season symbolises growth and renewal, reflected in budding flora and the protagonist's burgeoning curiosity as they interact with a mysterious device in their backyard.
The arrival of the climactic chorus comes with the midday summer sun. The video explodes into a vibrant display of sunshine and revelry, characterised by uninhibited joy and boundless energy. The colorful and euphoric imagery captures the essence of interconnectivity and togetherness, embodying Peter's concept of ‘the melting of the solid world.’ Amidst the chaos and fun, the underlying theme of unity and interconnectedness shines through, offering a respite from the cynicism of social disconnection.
As the video transitions into the post-chorus autumnal sections, viewers witness the serene beauty of dusk and sunset, signalling a moment of introspection for our protagonist. Here, amidst the fading light of the day, they reflect on recent experiences and contemplate the implications of their newfound interconnectedness. The juxtaposition of light and shadow prompts viewers to ponder the balance between safety and uncertainty in embracing this vivid interconnectivity.
With each cycle, our protagonist undergoes a transformation, evolving from a curious observer to a determined explorer of the interconnected world around him. As they delve deeper into this realm, their initial scepticism gives way to a sense of purpose and connection, guided by the ever-present motif of the full moon; a symbol of continuity and cyclical renewal. Throughout this journey, time becomes fluid, transcending conventional notions of age and linear progression.
-Oranguerillatan
Song info:
Olive Tree is a track taken from Peter Gabriel's album i/o, released on 1 December 2023. This is the Bright-Side Mix, by Mark 'Spike' Stent.
Words and music by Peter Gabriel. Produced by Peter Gabriel.
Published by Real World Music Ltd. / Sony Music Publishing.
“This song is one of those taken from this next brain project that I am working on. We are entering a time in which reading each other’s thoughts will become a normal part of our lives. In this story our hero puts on the brain scanning helmet, that turns thoughts into pictures, and then sees all the deep connections he has with the natural world and in particular to one olive tree.
In many ways, we are part of everything and we probably have the means to connect and communicate with everything, yet we often shut it off. We only want to see and listen to the things that seem important and relevant to us and shut out the noise of everything else. But probably, hidden in that noise, there are all sorts of things that can help us realise our place in this future world and help us remember we are still apes who need a healthy natural world to survive.”
