As soon as I saw an early version of the music video shot in Ottowa for the single off his new “In a Cabin With” album, I thought; this should be interactive. So now three months later; here it is.

Read on for some behind-the-scenes technical details.

I started by making a prototype to see if it was possible to combine high-res h264 video video with 3D rendering. And thanks to Flash Player 10 it was! The most efficient way to render 360 panorama’s is to use cubic projection, as this method requires way less geometry (4 triangles at most at any one time) then cylindrical or spherical projection. In this case saving a lot of performance needed for the high-res h264 decoding. For the 3D rendering I used an early version of Papervision3D 3.0 for Flash Player 10.

Once I had the panorama video working, I thought that re-creating the YouTube player seemed like the best interface for this piece. Once the interface was done came the time to create the final video edit. I re-edited the two cylindrical panorama’s created by Steye, one of ouside and one of inside, into one cylindrical panorama which transitions between ouside and inside. The frames of that final cylindrical render were converted to cubic faces using the batch processor of CubicConverter. The resulting cubic faces were then comped back into one video render with the audio track added.

Encoding the final video files was the final hurdle to overcome as there were two aspects that seemed to be hard to unite into an acceptable end result, both quality-wise and performance-wise. The first was to stay within the bitrates currently concidered standard. And what better standard to take for online video delivery then YouTube. Normal YouTube video’s are streamed at around 700kbps, and their HD video at around 2000kbps. So those were my targets for the h264 encodings. However YouTube delivers acceptable quality within these bitrates with material of much lower resolution, then this video required (1024×728 for the low quality version and 1440×1024 for the HQ version). That’s because this video contains the video for all four faces (front, back, left and right).

So it was clear that I had to use the best h264 encoder out there to encode these files. So, off I went testing different h264 encoders from Apple, Sorenson, MainConcept, mencoder and x264, and different applications, each allowing access to different encoder settings. I tried Adobe Media Encoder, Quicktime, Sorenson Squeeze, Episode, Handbrake and ffmpeg. And I’m very glad, and also not surprised, to say that I achieved the best results by far with the open-source alternatives; ffmpeg using the Doom9′s x264 encoder. Ffmpeg allows you to use b-frames and CABAC which are the two features of h264 that really give a significant quality improvement. My findings were that the x264 encoder also preserved the color and contrast information best from all encoders (especially MainConcept in Squeeze resluted in a very high contrast and unacceptably dark video in my case), while surpisingly also requiring the least CPU upon decoding in the Flash Player. So, like I said; x264 rules them all at the moment. And if you need more tweaking of the encoder, x264′s command-line interface gives you access to all the parameters that really make h264 shine.

I hope you enjoy the end result. Happy Holidays!

I’m proud to announce that I’ll be speaking at the upcoming actionscript conference / flash camp in Singapore 14th, 15th September. I’ll be presenting Papervision3D and show some of the projects we currently are working and explain how we approach them.

This demo was made shortly after the initial release of Jiglib. It was a study of how far you can push realism in the sence of rendering and animation. Here CompositeMaterial is used with a BitmapMaterial and a GouraudMaterial. This is much faster then using a ShadedMaterial. All you need to do is modify the GouraudMaterial to use a transparent light map. Andy Zupko’s ShadowCaster is used for casting the die shadows. Jiglib is used for the 3D physics, also slightly modified to facilitate fixed timestep. Unfortunatelly the end result is still far from the desired 60fps. Hit spacebar too see the stats, check it out.

UPDATE: Something troubled me when checking the performance in the browser with Flash Player 10 compared to Flash Player 9. On my machine FP9 ran at approx 40fps but FP10 hardly reached 25fps. Changing the wmode from “direct” to “normal” fixed the performance issue in FP10. Especailly in Firefox the performance difference is significant. Still on my machine FP9 seems to perform slightly better then FP10. Mental note: next time take more care in setting the proper wmode.

It seems that these days everyone is working on extrusion of some sort so I thought I post my experiments before it’s going to become yesterdays news

So what’s the big deal with extrusion? Well in order to do it well we first need to triangulate our vector shapes, not an easy feat in the first place but it gets really tricky when your shapes contain holes, I first used the “ear-cutting” method and then try cut out the holes form the triangulated shapes. That unfortunately created huge excess of vertices and triangles and before I could figure out how to fix it, Tim Knip has posted this fantastic method in alchemy form that triangulates polys with holes without creating any extra vertices! All I had to do I to find a way how to tell what is a hole and what isn’t, again it happend to be harder than it seems but my buggy code succeeds 90% of the time

Check out these examples:
example1, example2
you can download the code the but be warned it’s not in a very good condition.

Others have been exploring this subject as well, you can find their experiments here and here and here.

What if Joseph and Mary had access to faster modes of transportation when seeking a place to give birth? And what would have happened if they had taken a wrong turn leading them away from Bethlehem? Those are questions that have been occupying the minds of theologists for centuries. Finally, with the help of today’s latest 3D technology, Y & R, Flashfabriek and Barcinski & Jeanjean have tried to answer those questions. Have a look.

Merry Christmas and a happy new year!

Mark & Adrien

Ralph Hauwert has recently revealed some of his magic showing off the shading power of flash player 10. I was really wondering about the texture quality of the new 3d goodness so I’ve put a small skybox demo using Ralph’s branch of papervision for fp10 and this stunning shot by Roger Taylor this panoramic shoot we did of an landscape installation by Anne Walk.

I must admit I was totally stunned by the results, the quality is absolutely  amazing if you are used to 3d in fp9. The textures are silky smooth and it still runs better than dedicated panorama viewers for flash 9!

Some fullscreen(1440×900) results on my Macbook pro core duo 2.4 Ghz (santa rosa):

• standalone: 39-41 fps
• safari: 44-45 fps
• firefox: 24-25fps

These results are curieus to say the least. First of all how can flash player in safari run faster than standalone? And why is firefox running at half the speed of safari?

And what does it mean for future websites made especially for flash player 10. Well they are going to look really really good. With the help of all the other goodies (pixelbender, better font renderer, color management, h.264 support ) coming with fp10 we are going to see some really high quality productions in 2009!

Check out this fp10 panorama! Hit space for stats, use arrow keys to zoom in and out. Feel free to share your (fullscreen)results. Keep in mind that for fps counter to be meaningful the panorama must be moving.

On the penetration of flash player 10 side. There are no official numbers for adobe yet but our website ( +- 50000 visitors a month) it’s right now at 45% and growing really fast, 3-5% a week. So if you are starting an ambitious project today, consider targeting flash 10 because you don’t want your project to feel outdated by the time you launch

Source is not available for download because it’s all work in progress.

There are some big changes happening to vectorvision due to the fact that we are adding triangulation for the fonts, I will write more on it in other post. Other stuff recently added to vectorvision is:
word3d is now text3d and as you would expect form text it now supports multiline, align, letter-spacing, and line-spacing and all can be changed on the fly. see example.

I also have been experimenting with scalable outlines and the results are already very interesting, see this example.

The source code can be found in the vectorvision repository.

In the previous article we talked about how we shot our footage in stereo. Now we are going to talk about how to render those stereo images for 3D glasses. And how to render Papervision3D for 3D glasses.

There are many techniques for displaying stereoscopic 3D images. We opted for Anaglyph, the one you need the red/cyan glasses to view, since those are the glasses people are most likely to have laying around somewhere. Also these are the least expensive glasses of the lot and that mattered since we were also going to do a mailing of Barcinski & Jeanjean 3D glasses.

In short, an anaglyph image is obtained by taking the red color channel from the left camera and combining it with the blue & green color channels from the right camera. The drawback of this technique however, is the loss of color information, mostly affecting the red color component. So that was an issue we had to take into account as to what colors we would use in the website. Black doesn’t suffer in any way by anaglyph, so that kind of pushed us towards a basic graphic black & white design for the intro and menu cube.

On to rendering Papervision3D in anaglyph! For our website we have created an AnaglyphView class which offers anaglyph renedering in Papervision3D with the ease of use of the BasicView class. Click on the image below to view the demo. Toggle anaglyph rendering on and off by clicking on the Stage.

The AnaglyphView has one camera and two viewports. For each frame it moves the camera to the left, renders the camera projection to the left viewport, then moves the camera to the right, renders the camera projection to the right viewport and finally moves the camera back to it’s initial position. This way you still have complete freedom of camera movement in between the renders. Best is to use a Target Camera and move the target object to the focus point, creating the same mechanics as our eyes when focusing.

The left and right viewports are combined using ColorTransforms and BlendModes. While this offers a fast way of combining the two channels to an anaglyph image it is important to note that the viewports will not render correctly when not completely opaque. That’s why the AnaglyphView uses extended Viewport3D classes that make sure the viewports have a background filled with the supplied background color.

To combine 2D stereo images with 3D stereo rendering you need to attach the 2D stereo images to their respective viewports. That’s what we did in our website. The left panorama video is added to the DisplayList of the left viewport and the right video to the DisplayList of the right viewport.

Needless to say stereo rendering in Papervision3D can be very heavy. Your aim would be to keep the scene as simple is possible and limit the redraw regions as much as possible. We’ll be going into some optimization tricks in our article covering the Papervision3D side of our website.

Feel free to download the AnaglyphView class and start playing with it. The example above is also included in the package.

Well, here it is. After 3 months of hard work fun our “Coming Soon” page has finally been replaced by a brand new website. The coming weeks we will be posting a series of “Making of” articles about our use of Papervision3D, Vectorvision, 3D Physics, Photography and more. Today a few lines on the general concept of the website. 

The vision for the panorama view came to us in november last year while we were brainstorming on another project (the one for which we prototyped a spherical video panorama). We would stand in one of Amsterdam’s many beautiful scenes with everybody around us walking by going about their business. Around us our portfolio work would be hanging in the air in the form of paper flyers. Envisioning viewing that with 3D glasses being able to move around us, filled us with excitement. We are already familiar with viewing photo’s and movies with 3D glasses, but I can’t really say I had had an interactive experience using 3D glasses before. 

By creating a 360º panorama from a stop motion video rather than from multiple stills stitched together, allowed us to capture the movement of the subjects in the scene. The price to pay however was much bigger file size. To make up for the wait while the video content preloads before being granted access to the panorama view we thought it would be cool to revamp the traditional Pong waiting game into a solitary 3D version that could also benefit from viewing with 3D glasses.

But the added file size was worth it. When we first reached the point in development that we had a placeholder 360º video panorama overlaid with a badly keyed-out test shoot of us taken in from of a purple screen (yes, we hadn’t ordered the green screen yet), that we realized just how interesting it actually is to gently turn the 360º and observe the many people, boats and occasional “motor agent”, as they call them here in Holland, pass by.

The location was almost an automatic choice. Besides the fact that it’s beautiful, it’s right around the corner from our studio. But this is mearly the first panorama to feature on the site. As it’s simply a question of modifying some XML to change the panorama in the website, and, well, shooting the new footage of course, we plan on adding new panoramas to the site in the future, probably starting with a new shoot this fall.

This little experiment is a proof of concept of fog using papervision’s render layers and the color transform object. This would make a lot of sense for the frustum camera where the culled triangles wouldn’t just pop out of void but would nicely fade in. Also to my surprise there is almost no performance penalty for the fog.

Take a look here or download the source.
Other interesting effect is using alphas on the render layers. 

I’ve been playing with perlin noise and have created something that reminds me of raging ocean or something a bit less natural, that is moving blue mountains  see for yourself.

For the coming soon teaser on our website, we needed crispy rendered text. Because Papervision3D is bitmap based and therefore not very good at rendering text, a nice extention would be to render vectors in 3D directly in Papervision3D. A good starting point is the beautiful Five3D engine by Mathieu Badimon. So, what I’ve done is I ported his fonts into Papervision3D. And since Mathieu also released a font conversion tool, you can create Five3D fonts yourself from any font installed on your machine! Check the 3D font example here.

Then I thought; if I can draw vector fonts in 3D, I could draw any vector shape in 3D. And what better input format to use then SVG? Lucky for me I didn’t have to write an SVG parser for AS3 because James Hight has done it already and released his source code. He has since then updated his code to support many more features of the SVG format but I haven’t had the time to implement it yet. Check an example of an imported SVG into Papervision3D here.

To cut a long story short; download the source code and make something beautiful!. And don’t forget to send us a message.

We are currently working on our own website, which is a lot of fun. Since the website still needs a lot of work we made a little Coming Soon page which gives some hints of what the website is going to contain. If you have some 3D glasses laying around you are in for a little Anaglyph treat. The 3D fonts are also quite special. These are Vector fonts rendered in the Papervision3D engine. More about vectors in Papervision3D coming in a later post. Meanwhile, check this.