Interactive Music Video with Steye

by

As soon as I saw an early version of the music video shot in Ottowa for the single off his new “In a Cabin With” album, I thought; this should be interactive. So now three months later; here it is.

Read on for some behind-the-scenes technical details.

I started by making a prototype to see if it was possible to combine high-res h264 video video with 3D rendering. And thanks to Flash Player 10 it was! The most efficient way to render 360 panorama’s is to use cubic projection, as this method requires way less geometry (4 triangles at most at any one time) then cylindrical or spherical projection. In this case saving a lot of performance needed for the high-res h264 decoding. For the 3D rendering I used an early version of Papervision3D 3.0 for Flash Player 10.

Once I had the panorama video working, I thought that re-creating the YouTube player seemed like the best interface for this piece. Once the interface was done came the time to create the final video edit. I re-edited the two cylindrical panorama’s created by Steye, one of ouside and one of inside, into one cylindrical panorama which transitions between ouside and inside. The frames of that final cylindrical render were converted to cubic faces using the batch processor of CubicConverter. The resulting cubic faces were then comped back into one video render with the audio track added.

Encoding the final video files was the final hurdle to overcome as there were two aspects that seemed to be hard to unite into an acceptable end result, both quality-wise and performance-wise. The first was to stay within the bitrates currently concidered standard. And what better standard to take for online video delivery then YouTube. Normal YouTube video’s are streamed at around 700kbps, and their HD video at around 2000kbps. So those were my targets for the h264 encodings. However YouTube delivers acceptable quality within these bitrates with material of much lower resolution, then this video required (1024×728 for the low quality version and 1440×1024 for the HQ version). That’s because this video contains the video for all four faces (front, back, left and right).

So it was clear that I had to use the best h264 encoder out there to encode these files. So, off I went testing different h264 encoders from Apple, Sorenson, MainConcept, mencoder and x264, and different applications, each allowing access to different encoder settings. I tried Adobe Media Encoder, Quicktime, Sorenson Squeeze, Episode, Handbrake and ffmpeg. And I’m very glad, and also not surprised, to say that I achieved the best results by far with the open-source alternatives; ffmpeg using the Doom9’s x264 encoder. Ffmpeg allows you to use b-frames and CABAC which are the two features of h264 that really give a significant quality improvement. My findings were that the x264 encoder also preserved the color and contrast information best from all encoders (especially MainConcept in Squeeze resluted in a very high contrast and unacceptably dark video in my case), while surpisingly also requiring the least CPU upon decoding in the Flash Player. So, like I said; x264 rules them all at the moment. And if you need more tweaking of the encoder, x264’s command-line interface gives you access to all the parameters that really make h264 shine.

I hope you enjoy the end result. Happy Holidays!

Advertisements

35 Responses to “Interactive Music Video with Steye”

  1. Kristian J. Says:

    Wow! When I clicked the link, I was certain that it would be very choppy on my humble netbook. But it was smooth as silk.

    Definetely gonna check out that encoder – thanks for sharing your expreience.

  2. Adrien Jeanjean Says:

    Thanks Kristian,

    Nice to hear it runs smooth for you. Hadn’t had the chance to test it on a notebook 🙂

  3. Taras Kuba Says:

    Well done! Goes very smooth.
    I think you are the first to use 360° video for music.

    Do you know camera model this video was shot on?

  4. Adrien Jeanjean Says:

    Thanks Taras. I don’t know which camera model was used. The shoot was done by Orientation Travel Productions.

  5. Interactive Music Video with Steye « barcinski-jeanjean Says:

    […] 原文でつづきを読む […]

  6. Özgür ALTAY Says:

    Great job again. It is really smooth with the Pappervision engine.

    BTW can you tell about the camera system used in here?

  7. victor Says:

    hi, love the video, can you tell how you did to wrap the videos in the cylinder?

  8. Adrien Jeanjean Says:

    Thanks Victor. It’s not a cylinder but a cube. And that’s all explained in the article. If you have any specific questions let me know.

  9. victor Says:

    I mean when you recorded the video and edited it you make a cylindrical panorama of the videos, how did you paste the videos together? and how did you solve the distorsion of the lens?

  10. Adrien Jeanjean Says:

    Ah, I see. Well, Steye did that stage of the editing so I couldn’t tell you exactly. I was supplied two cylindrical video’s and picked up from there. I’ve forwarded your question to him so we’ll have to wait for his comment on that. Will keep you posted.

  11. Adrien Jeanjean Says:

    Got word back from Steye: It’s just basically a collage of 14 different shots dispersed over 360 degrees. No fuss about lens distortion. Then he used Trapcode’s Horizon 3D After Effects plug-in for the linear version of the music video.

  12. victor Says:

    tnx thats,great, im gonna try and see what happens

  13. website video production Says:

    I have had this happen. good advice, wish I had read this a month ago.

  14. Jonas Van Gestel Says:

    Very nice!

    Maybe you should check out the technology Camargus is working on too. They’re some guys from the University of Hasselt – Belgium that have developed a camera and player that shoots and interactively plays in full 360° at once.

    You can check out some experiments and demos on their site: http://www.camargus.com/ or you can contact codefellas.com if you would like to use the technology in your own (commercial) project.

    I think your Flash solution runs way smoother in the site, but maybe you can come to a better / faster workflow if you combine your Flash skills with their recording technology?

    Cheers,

    Jonas
    http://www.theblackbox.org
    http://www.adobeusergroup.be

  15. Sharmaine Sublett Says:

    I just found this blog a while back when a buddy of mine recommended it to me. I’ve been an avid reader ever since.

  16. Louis Wilkin Says:

    Excelente información. Saludos

  17. ipad Says:

    Found this web site on a quick google search and it’s been helpful. I will see that as time progresses it can have a lot of fascinating data and comments on this site.

  18. go on holiday Says:

    checked everywhere to grapcomments about to this area…. Thank you very much

  19. go holiday Says:

    I surfed everywhere to findcomments regarding to this subject….. Thank you very much

  20. Nichole Salzman Says:

    Rod Sanderlin

  21. Zeitgeist Studios » Blog Archive » 360 Immersive Video with Steye & The Ottowanians Says:

    […] for the making of the music video by Steye & The Ottowanians, I will refer you to this behind-the-scenes on Barcinski-JeanJean’s blog. The article is highly technical, but it does explain some of […]

  22. Krystle Dueitt Says:

    Hi, i must say fantastic site you have, i stumbled across it in AOL. Can not wait for the launch of the HDC-SDT750 Camcorder. Its going to be 3D Heaven.

  23. buy chun li costume Says:

    I follow your web site for quite a extended time and ought to tell that your content articles usually prove to be of a high value and quality for readers.

  24. Anneliese Truman Says:

    today´s develop in field egest aliveness so some easier than it was before, but group to score difficulties in accepting and apprehension such

  25. Art Saladino Says:

    Search for all latest news ,information and reviews of hot Movies, DVDs, Musics, TV Shows and Games right here!——-http://twitter.com/Entertain_Guide

  26. Zola Farm Says:

    howdy this is a tremendous blog, would you think about being a invitee writer on my web log?

  27. Mipades Freezing Says:

    I literally ought to get out of my house a good deal more and halt investing the majority of my daytime hours playing poker on myspace or facebook considering that of late i have been missing regarding this. Even while I can’t believe that I one hundred percent come to an agreement with each and every expression, it’s nice to start reading some clever posts in placed of the normal trash I see flying around the net.

  28. Playstation 3 world Says:

    Hi there! Very useful post! I am very glad that I was able to stumble upon your blog while searching Yahoo. Koodles for this great post!

  29. Zooey Deschanel Katy Perry Says:

    I appreciate the comments on this site, it definitely gives it that community experience!

  30. scituate handyman Says:

    Entertaining and informative post! I was looking around on the internet for a post about this and this is the very first great 1 I have read.

  31. Thalia Feltmann Says:

    Real great info can be found on site .

  32. play spot Says:

    play spot…

    […]Interactive Music Video with Steye « Barcinski & Jeanjean[…]…

  33. juliet's fashion empire game Says:

    juliet’s fashion empire game…

    […]Interactive Music Video with Steye « Barcinski & Jeanjean[…]…

  34. Sofia Dean Says:

    Wonderful blog! Do you have any recommendations for aspiring writers? I’m planning to start my own blog soon but I’m a little lost on everything. Would you suggest starting with a free platform like WordPress or go for a paid option? There are so many options out there that I’m completely overwhelmed .. Any ideas? Kudos!Raleigh Roofing, 3221 Durham Dr., #101-C, Raleigh, NC 27603 – (919) 701-6300

  35. Donnie Says:

    I was suggested this website by my cousin.
    I am not sure whether this post is written by him as nobody else know such detailed about my problem.
    You’re wonderful! Thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: