how i learned to love the ipad

Theodore Gray, author of our own Gray Matter, turned the periodic table into one of the most stunning (and popular) applications for the iPad. Here, take an inside look on what brought it from fantasy to reality in 60 days

By Theodore Gray Posted 04.06.2010 at 1:00 pm 

Bismuth Theodore Gray/Nick Mann/The Elements

Every once in a while things just come together to make you realize that you'd be an idiot not to put yourself through hell in pursuit of an impossible goal. That day arrived for me on January 27th when Steve Jobs announced that the iPad would be shipping 60 days hence.

After some deep soul searching for about 60 seconds, I decided that I had the chance to create something quite remarkable, and just maybe do it better and faster than anyone else in the world. More specifically that I had a chance to take my recently published book, The Elements: A Visual Exploration of Every Known Atom in the Universe, and turn it into a book that Harry Potter might check out of the Hogwarts library.

I wanted to make Harry Potter's magic books a reality, and do it in 60 days flat. Here's how we pulled it off.

Electronic publishing, including books, magazines, and newspapers, was a big part of the announcement of the iPad. I knew that some of the biggest media companies in the world would be racing towards the same goal I was, trying to come out with the best, the most talked-about, and the most fun new kind of electronic publication, and get it done and shipping in the App Store the same day the iPad started arriving in people's hands. Even my friends at PopSci were showing signs of iPad fever, so I knew they were on the scent as well, though Apple's strict secrecy rules meant that I couldn't talk to them and they couldn't talk to me about what we might or might not be working on.

Creating a really good e-book, one that doesn't just reproduce the limitations of paper in electronic form, requires a lot of raw material that isn't part of the traditional paper publishing process. Harry Potter's books look like normal books, with pictures and text mixed together on the page, but all the pictures do something. Pictures of people move and talk. Pictures of objects pop off the page and let you see them from all sides. Pictures of events show you the action, not just a static scene.

To do that for real requires not magic, but media: Video, computer simulations, 3D virtual objects, and so on. The kind of media that is usually very expensive and time-consuming to create. The hardcover edition of The Elements includes over 500 photographs of objects representing the complete periodic table, and to make a proper Harry Potter version of the book, every one of those 500 objects would need to be able to spin around in 3D, because, well, that's what magic objects do.

This brings us to the first secret sauce I knew I had on my side in the race against everyone else pulling together their own iPad apps.

Together with my photo partner Nick Mann, we had already photographed every single one of those objects around a complete revolution on a turntable, taking 720 15-megapixel shots for each object. The hardcover edition contains over 500 single images, but they are drawn from a library of 350,000 images that show every object from every viewpoint. (Actually, truth be told, our library is much larger than that, closer to a million photos, because those 500 objects are just a selection of all the objects we have photographed. We have literally worn out the shutters on half a dozen high-end cameras building this library.)

So we had the raw material to create magic objects, but it's no small thing to turn a terabyte of raw images into a packaged app where you can interactively rotate 500 different objects, each a different size and each positioned uniquely on the page and rotated to a pleasing default orientation when the page is first displayed. PhotoShop and InDesign just don't support that kind of page layout of magic objects, and neither does any other standard tool.

Fortunately the second secret sauce I had on my side was Mathematica, an application I've been working on since 1988 at Wolfram. Mathematica is best known as a technical computing platform, but it's really much more than that: It's by far the best tool to use when you're faced with a problem for which there is no standard solution available.

In about 7 of the precious 60 available days, I was able to create from scratch, using Mathematica, an interactive magic object page layout tool that designer Ann Grafelman was able to use to design all 118 individual element pages in record time.

Rather than just create images of the pages, the tool also created detailed instructions that could be passed on to further Mathematica programs that automatically selected, scaled, cropped, and assembled the raw images into finished media assets, both rotatable spin objects and whole-page QuickTime movies for certain special effects. (Even the amazing animation synchronized to Tom Lehrer's iconic song "The Elements", which plays the first time you open the e-book, was generated entirely by a Mathematica program.)

A complete rendering pass for the e-book requires running eight parallel Mathematica processes for a couple of days on the fastest available 8-core Macintosh. But it is a completely automated process, turning a terabyte of image archives into a finished, fully operational 1.9 gigabyte iPad app. This complete automation meant that we were able to experiment with dozens of different layouts and styles, concentrating on creativity, not the grunt work of manual file processing, yet still be able to see the finished book in action after each tweak.

The third and final secret sauce that made this project possible was my partner Max Whitby and his long-time collaborator John Cromie, who created the runtime application code needed to turn those media assets into an operating, fully magic-enabled iPad application.

Creating fluidly spinning objects with the level of crystal clear photographic quality I demanded is actually harder than it might seem. A number of the obvious things the more technical of you might think of turn out not to work for reasons too complicated to get into here. Suffice to say that John designed a brilliant solution that uses the iPad's excellent graphics subsystem to lend a hand in ways it's not normally intended to.

I didn't know when we started whether it was going to be possible to make something that was smooth enough and fast enough that you wouldn't notice either the smoothness or the speed. The only way to know you've got those things right is when you don't notice them because everything just works without glitches, pauses, stuttering, or other distractions that make you realize that, well, actually, it isn't really magic.

My goal was 100% magic, and I think we got up to about 98%. Nothing's ever perfect, but believe me, when you're playing with The Elements there is a lot of scrambling going on behind the scenes that you don't notice because of the finesse John was able to build into the code. (We're planning a free upgrade shortly that will get us another step in the direction of 100% magic.)

Seeing the final product, and seeing how many people have found it to be just as magical as I had hoped they would, has been one of the most satisfying experiences in my life. Thanks also to the great support we've gotten from Apple, The Elements has turned into one of the most talked about, most exciting, and best selling e-book since the iPad started shipping. Take that, Time-Warner! (Not that the other guys were slouches by any means: PopSci's April issue is a very impressive piece of work, and don't miss my column on fire pistons, which looks super on the iPad.)

We started out as a band of brothers, myself, Max Whitby, John Cromie, and Nick Mann, who each made a commitment that we were going to drop everything and do nothing else for 60 days but eat, sleep (not much), and try to build magic.

And we did it.