WORKS DETAIL
The world’s first music video to include lyrics graphically created by AR technology.
DETAIL
BACKGROUND
History of the lyric video
In the past, most lyric videos simply presented the lyrics like subtitles. But lyrics gradually became interesting elements of videos through motion graphics and visual effects. For example, various techniques are used to make text look like real objects within scenes. In many ways, music videos became testing grounds for new visual techniques and technologies. This new video contributes to the evolution of visual expression. As smartphone functions advance in an era when anyone can shoot high-quality videos, demand rises for new music video expression.
WHAT WE INVENTED
The single long cut, covering 300 meters.
Music videos that include lyrics as graphic elements are called lyric videos, and many music artists have released lyric videos with lyrics added through motion graphics and visual effects. But this was the world’s first music video to include lyrics graphically created by AR (augmented reality) technology. The lyrics were handwritten by the music artist using AR technology and added via camera tracking to a video captured by iPhone. In the music video, the artist performs his song while walking outdoors with the lyrics seemingly floating in the air around him. The lyrics are displayed in sequence as they are sung. To capture the entire video in a single cut without any other people appearing as the artist walked 300 meters through the streets of Harajuku, one of Tokyo’s most crowded areas.
It had to be shot over 100 times. Only two tries were acceptable.
First the music artist wrote the lyrics in the air one phrase at a time using Google Tilt Brush VR painting software. Then the VR generated lyrics were added to an accurate reproduction of the Harajuku space in Unity game development software. Finally, the lyric graphics were added to the video using an originally developed smartphone app. Shooting equipment consisted of only an iPhone and camera shake prevention device. Lyric graphics were added to the video via iPhone when AR markers activate while shooting. Lyric positioning and camera work accuracy gradually improved through trial and error on site. Harajuku is the birthplace of cutting-edge fashion and one of Tokyo’s busiest streets. So the single long cut, covering 300 meters, had to be taken early in the morning. After 100 tries to capture the video without any other people, we finally had two acceptable cuts.
