Visual Story Telling with Deep Zoom Composer - No more low-resolution images

Summary

Until now, sharing high resolution images online has never been fun; download times are long and the viewer ends up downloading a large image despite only needing the high resolution details in 10% of the image. Fortunately, that is changing!

Deep Zoom, a feature in Microsoft Silverlight 2 has the potential to revolutionize how we tell visual stories online. Deep Zoom uses sparse image composition to create the great depth and infinite detail that you would need in order to tell a visual story with an image. Using Deep Zoom you can download details where it matters without incurring huge download sizes (and delays). Deep Zoom applies tiling algorithms to optimize the speed and size of the download and it also creates smooth transitions across tiles and resolutions so that viewers have a great user experience while navigating your visual stories. Free yourself from low-resolution images, become a visual story teller by composing Deep Zoom images.

Introduction

In 1862, the Russian writer Ivan Tugenev wrote “A picture shows me at a glance what it takes dozens of pages of a book to expound”. Though I fully agree with this statement, I have always felt that a picture is not enough to tell a story. A picture describes an instance in time, without alluding to the sequence of events that made the instant possible.

Deep Zoom, a feature in Microsoft Silverlight 2, is redefining how we can share images online and opening the door to Visual Story telling . Deep Zoom lets you browse through a collection of images at any level of detail; Deep Zoom supports very high resolution images and optimizes the download of these images so you do not have long delays while the images are downloading. To turn your stories into images, what Deep Zoom introduces is the ability to combine a collection of images into a seamless composition; with Deep Zoom you do not need to begin with high resolution images, nor to have images be of the same resolution, you can combine lower resolution images into a bigger composition that tells a story; you can also append meta data to each of these images, and animate the images individually to create even greater interactivity on your composition.

Since pictures do say more than a thousand words, the easiest way to explain Deep Zoom is with a real world example, check out Hard Rock Café International’s Memorabilia application at http://memorabilia.hardrock.com.

Figure 1 - Memorabilia, from HardRock Café International. On the left hand-side, Memorabilia composes several lower resolution images to create a very large depth in the post-stamp for the letter. On the center, memorabilia stitches several high resolution images to create an incredible amount of detail on the guitar –you can zoom in to see the scratches and actual finger prints.

Creating a Visual Story with Deep Zoom

The rest of this write-up will introduce you to the tools and the concepts you can use to create a visual story with Deep Zoom. We will copy the principle of composition used in David Weisner’s wonderful Flotsam to tell the story of how each member of my family watches videos.

The inputs to our exploration are three standard resolution images (~10 Mega Pixel).

Figure 2 – Monica, the grown-up in the house enjoys her movies on the TV. We do not have a big screen or anything fancy, but she enjoys the ~10 ft experience and despite my many attempts, she does not watch either online movies or Media Center recordings on the computer.

Figure 3 – Samuel, the four year old enjoys watching movies on the computer. Despite there being a media center available to him, he likes the control of fast-forward and rewind.

Figure 4 – Sophia, our youngest one (~2 year old) can’t stand still long enough at either the computer or the TV, so she goes for a more portable entertainment.

Composing a visual story

Our story will showcase Deep Zoom using sparse images to create depth into a scene.
What we will do is insert the picture of Samuel playing in the computer into the Zune’s scene, to create the illusion that Sophia is watching Samuel. We will repeat this operation and insert Monica’s picture into Samuel’s media player scene, to create the illusion of him watching Monica, who in turn is watching Cantinflas , Mexico’s funniest comedian and a favorite of mine.

You can see the Deep Zoom Composition here. Though you are free to ‘roam’, here is what you might want to look for:

1. What is the girl watching (go ahead and zoom-in towards the zune).
2. Now what is the boy watching? Go ahead and zoom in towards the computer.
3. Now what is the mom watching? Zoom in towards the TV.
4. Zoom back out, why is the elephant celebrating?

To create this type of composition, it is a very simple five step operation using Deep Zoom Composer. The whole exercise will take ~10 minutes from begin to published (minus upload time).

Pre-requisites:

• Deep Zoom Composer
• Silverlight 2.0 Beta2 Run-time
• The images that I used are on sky drive.
• [optional ] VS 2008 if you want to tweak the code, if you do, you will need the Silverlight SDK.

Step 0 – Launch Deep Zoom composer, and create a new project

Do make sure you give your project a meaningful name and choose the right location. Deep Zoom composer, being still in its early stages does not do Save As, so start right from the beginning or later use Windows Explorer to move your project around.

Figure 5 – Creating a Deep Zoom project. Choose the right settings from the beginning.

Step 1 – Importing the images

We can now import the images we will use by clicking the Add Image button and browsing to the images we need. The order in which you add the images does not matter. At this stage, adding the images just means you are making them available for composition.

Figure 6 - Our images as they are imported into Deep Zoom Composer. The red rectangle is a 900x600 dummy picture we will use to size our design surface.

Tip: If you make a mistake and need to delete an image from the project, do it from the composition tab. From my limited experience with Composer, the Delete from project Context menu on the Import Tab does not appear to be working in the current preview.

Step 2 – Composing the scene

Once you have imported the images, click on the Compose Tab to build your scene.
The principle here is very simple, you can drag & drop your images into the scene and put them anywhere you want. You can use zoom and pan to drop the images at the right sizes and right level of detail.

Figure 7 – The composed scene. Notice the three images are in the scene. You can barely see the computer inside the Zune, and it is too hard to see the TV, but we will be able to see it when we Deep Zoom into it.

Here are the tips I find most useful when composing scenes:

1. The size of Deep Zoom Composer’s design surface cannot be set from the tool. Current version tries to take as much space as you have in composer; this means if you are maximized and designing at a resolution of 1920x1200 you will get a bigger surface than designing at 1280x1024. The workaround that I use is to put a fixed size dummy picture as a background. The picture is the size I want at run-time. I use the “Fit to Screen” command to fix my design surface to the aspect ratio of my dummy image and do all my design. Before exporting the final composition, I remove this dummy image by going into Layer View, finding it and deleting it.
2. Compose items from back to front. For example in our scene, the background is Sophia watching the Zune movie, so we start with her; then drop Samuel and finally the Cantinflas (TV scene).
3. Deep Zoom Composer does not allow me to rotate images. As a workaround, I tried using Expression Design to create pre-rotated images (with transparency) but unfortunately that also did not work in the current build. The images are tiled as JPG (despite my choosing PNG) and therefore the transparency is replaced by black pixels).
   I could not get around this one, that is why my images are a little crooked. The issue has been reported and will be fixed in the next public preview of Deep Zoom composer.

Step 3 – Exporting the scene

Once the scene has been composed, it needs to be exported so we can view it as a website.
Here is the export dialog, followed by the explanation on the choices it provides.

Figure 8 – Deep Zoom Composer’s export dialog

The options:

• Name can be anything.
• Export location is where you want to put the files, which can also be anything .
• Export as collection or composition, allows you to choose whether you want to export your scene as either a single image ( Composition) or a collection of individual images.
  • If you export as a collection of images, you get the same number of images as you put into the composition and you can manipulate these images individually at run-time. You can move them, change their Opacity (aka Transparency) and change their Viewport width (therefore changing their size since they keep aspect ratio).
    The only possible down-side we would have for our collage is that the images load separately, and in this case we have a large image with a few much smaller ones, so it is likely it would look funny if we export as a collection.
  • Exporting as a composition just gives you access to the single composition as a whole. You can move it, change opacity, etc. and it loads everything at once, and you cannot access the individual images directly.
    
    For our story, we will expose as a composition first.
• Image Type gives you the choice of the format for the output. Choices are PNG and JPG.
  In Deep Zoom Composer Preview 1 ( June) this setting does not work well. It seems to most often export as JPG regardless of the choice. Exporting as PNG would be nice so you could have transparency. Reported already! Hopefully this option will work in next preview of Composer.
• Output Type should be Images + Silverlight project. This will export the images, an HTML page, and other solution files. All of these files together provide a basic navigation experience for your Composition; it includes the loading of the image, and features like Zooming and Panning to interact with the image. If you choose this option, you will later just need to copy the files to a website to publish them.

If you make the choices above, and choose to export as Images + Silverlight project, you will get a choice to Preview in Browser, View folders, or Edit in Expression Blend. Choose the Preview in Browser option. Your composition should then load in your default browser. It may look smaller than you would expect because the default template zooms to ½ your composition’s size. Don’t worry about that for now and focus on enjoying the out of box functionality in the template. You should be able to:

• Pan. Click on your mouse and drag it while mouse is clicked. Your composition will follow the mouse.
• Zoom-In. Click the mouse in the spot you want to zoom in. If you have a mouse with a wheel you can also press the wheel forward to zoom into the image.
• Zoom-Out – Press Shift key and Click to zoom out. If you have a mouse with a wheel, press the wheel backwards (towards you) to Zoom out.

Step 4 – Publishing the scene to a web server

After you exported, publishing to a web server is just a matter of copying the files Deep Zoom created in your export onto a website. Deep Zoom and Silverlight have no server-side requirements, so any web server will do. All you do need is a mime-type on your server.

The MIME type extension is .XAP and the Type is application/x-silverlight-app. Instructions here.
Once you have the MIME type, just copy your files to the server, set the permissions, and navigate to your website.

How does Deep Zoom work? What is the big deal so far?

If you are just reading the document and have not experienced the story, you are thinking we have a collage within the Deep Zoom Composer that does not look much different than something you could have created in Paint .NET or Photoshop, right? Not quite. Experience the site and notice the difference. If you want the explanation, the magic happens during export and run-time.

During the export, Deep Zoom Composer partitions our composition (or each image in the collection) into smaller tiles (256x256 is the default). While "tiling" the image, it also creates a pyramid of lower resolution tiles for the whole composition. Figure 9 below shows you what a pyramid would look like; the high resolution composition is lowest in the pyramid, notice how it is tiled into smaller images, also notice the pyramid has lower resolution versions of our composition (also tiled).

Figure 9 – Tiles for our Deep Zoom composition. Each tile is 256x256. All of this tiling is performed at design-time and gets accomplished using the Deep Zoom composer.

At run-time, Deep Zoom downloads one of the lower resolution tiles of the image first to decrease wait time and download size so you can get a visual instantly. After that, Deep Zoom downloads the other images on demand (as you pan or zoom). Deep Zoom is only downloading the needed tiles so for example in our case it downloads the Zune, and the images in the Zune as we keep zooming into these. Deep Zoom makes sure the transitions from lower to higher res images are smooth and seamless.

The results

To preview the results, it is easiest to demonstrate using images.

Figure 10 - The Deep Zoom Composition at 13X zoom. Notice how crisp the images are, we can tell the brand of the monitor, we can see what is on TV, etc.

Figure 11 – The result when you do not use Deep Zoom. I saved the collage as a standard PNG (10 Mega Pixel), and here you see the results at a 10x zoom. The details are gone, the monitor’s brand is blurry at best, and the TV picture is pixilated.

Figure 12 – Just to show off, here is the TV sparsed image at 50X. Notice the piggy bank is still crisp and so is the image of Cantinflas on the TV.

I hope you now can see why they call it Deep Zoom. Through a simple composition of three sparse images we got a 50X Zoom that is animated, flows with ease, and provides incredible detail to our story.

Animation and transitions

When I started, I mentioned turning a sequence of pictures into a story require both depth and interactivity. Since Deep Zoom is part of Silverlight’s framework you can programmatically interact with MultiScaleImages (in developer speak, that is the type of Image Deep Zoom creates).

Panning and zooming comes with the out of the box project template; Deep Zoom automatically does the springs animations for tile transitions; you can also add overlays ( text, video, images, or other vector drawing) on top of a Deep Zoom composition, etc.

For our story, I am going to show a simple interaction to move the images we originally added into our composition. This requires two simple things:

1. Go back into Deep Zoom Composer and instead of exporting a Composition, change it to export a Collection. This will free the images so that we can manipulate them individually.
   This will unfortunately change the loading experience since the images will now load individually and the size difference is significant. If this was a production application I would likely have stitched the background image and have it include the smaller images in it, to see less of a fade in transition and have the smaller images be transparent (Opacity = 0) during load.
2. I will add five lines of code to move the images around; it is merely a changing the viewport size and origins for the images, as well as changing ZIndex. Full source is available here.

You can see the interactive story here.

To see the reordering of the images, you need to run through the whole process:

1. Zoom slowly all the way towards the mom. Wait for the message that says "can’t teach mom". This is when Samuel notices he has been framed.
2. Now zoom out back towards Samuel slowly. At some point you will see a button at the bottom of the composition that says "Click here to see how Samuel feels about being framed".
3. Click on the button.. Sneaky dude isn’t he?

Conclusion

Deep Zoom provides the ability to navigate high resolution images in a very efficient manner. If you do not have high resolution images, that is not a problem, you can create compose several lower resolution images into a Deep Zoom composition to create higher detail images. Everything in Deep Zoom is animatable, programmable, and can interact with the rest of Silverlight. All these great features together enable you to create fun, interactive visual stories you can share online to anyone having a browser and the Silverlight cross-platform plug-in. What are you waiting for? Download Deep Zoom Composer and have some fun!

The Easter egg in this application

I find it amusing that I picked an image of Cantinflas’ "Ahi esta el detalle" movie. Translated to English: "there you see the details", a very appropriate title for a Deep Zoom article, don’t you think?
That was a mere accident, I did not think of that until nearly done.

References/More details

Introduction to Deep Zoom Algorithm (aka the math)

A previous blog post I wrote for ‘programming’ Deep Zoom. I have more blog posts coming so stay tuned there, I have one on Viewport and one on metadata already coded.

Deep Zoom Composer Team’s blog