All photos by James Duncan Davidson, who writes: In the lower right photograph, there are twelve 12TB Pegasus R6 arrays. That’s 144TB total. It’s a tenth of a freaking petabyte of usable space.
Backstory: During a TED conference, we create a lot of data — five days of high-res video from nine cameras, tens of thousands of digital images from five photographers — and over the past half a decade, our team has given a lot of thought to storing and transferring data among lots of people working at top speed, on a system that we can set up and break down quickly. This year, we were excited to deploy Intel and Apple’s Thunderbolt connection technology, which promised up to 10Gb per second of data throughput (per channel). As Michael Glass, TED’s Director of Media Production, and his team prepped for TED2012’s Media Cave, they whiteboarded a plan for what our photographer and guestblogger James Duncan Davidson calls “the biggest deployment of Thunderbolt-based storage that I’ve seen.”
After the conference, Duncan wrote up his notes on how it worked. He gave us permission to repost here:
Last week, I mentioned that the TED media team had made a pretty big change in the media cave for TED2012. Instead of dozens of MacPros and piles of SATA drives, the room is full of iMacs and Thunderbolt drives. Lots of Thunderbolt drives. Now that we’ve been through an intense week of work dealing with terabytes of video and photo data in a high pressure situation, the question is: how’d it work out?
The answer: amazingly well. It was a super positive experience.
There were 15 Pegasus R6 arrays on hand, each with 12TB of internal capacity. These were joined by over 50 LaCie 2TB Thunderbolt Little Big Disks, over 15 iMacs, a few MacBook Pros, and 18 Thunderbolt displays. Definitely a big enough sample of Thunderbolt-based gear to get solid experiential data from. Sure, we had a couple of technical issues—you can’t avoid them when you’re dealing with this amount of gear under pressure—but the issues that did come up were easily solved or worked around.
Here are a few observations I was able to collect from the media team:
- Out of the 15 Pegasus arrays—containing a total of 90 hard drives—there was one drive that failed. It was replaced with a new 2TB drive that was on hand, the Pegasus rebuilt the data automatically without a hitch from the other drives in the array, and life went on without incident. Promise had an extra replacement drive to us as fast as it could be shipped.
- Nobody had an issue with any of the LaCie Thunderbolt Little Big Disks. In fact, except for the same niggles I mentioned when I got mine, everyone loves ’em. They were especially awesome for the video editors to quickly move tons of data into place for quick editing.
- Having dual Thunderbolt ports on the iMac was a definite plus when working with multiple devices. A few people commented that it would be nice to have dual ports on the MacBook Pro, and we all hope the Mac Pro replacement will ship with 4 or more Thunderbolt ports for maximum I/O flexibility.
- Our biggest negative observation: The Apple Thunderbolt displays don’t seem fully baked. A couple of people were running dual Thunderbolt displays off of a MacBook Pro and odd video glitches appeared from time to time. Also, after a power outage in the room while they were attached to laptops, a full system restart was necessary to bring ’em back to life. The port expansion capabilities of these displays is very welcome and they’re easy on the eyes, but it definitely felt like we were pushing them a bit harder than they were ready for.
- A more minor negative observation: We ran into an unexpected issue with the combination of the Sonnet ExpressCard/34 Thunderbolt adapter paired with the ExpressCard/34 Compact Flash card reader in our production system. Pre-conference testing by the media team showed a lot of promise. In fact, Micheal Glass wrote me that “Our testing showed outstanding speed from the Sonnet gear—notably better than FireWire 800”. Unfortunately, we didn’t see the same results on site. We had some stability issues with cards unmounting during transfers and the transfer times were about the same as our stack of trusty (and unfortunately discontinued) Lexar FireWire 800 readers. To be fair, we were using the adapter on a system that also had dual Thunderbolt displays and a couple of of Thunderbolt drives attached. Any number of interactions could have been at play here, but time pressure meant that we couldn’t get to the bottom of this and we ended up falling back to using the FireWire 800 readers for almost all of our card imports.
- Both negatives said, the media team felt that they had done less futzing with gear than in years past—even though we had a lot more gear, were working with a lot more data, and doing more things with that data than ever before. That’s a huge positive effect and one that made everyone happy, including our awesome volunteer tech support and infrastructure team who were our own personal Apple Geniuses for the week.
- Shipping the units home is easier than ever. Just put four into a Pelican case and go. Just envision 48TB data packets moving through FedEx and smile…
Me personally, I’m totally sold on Thunderbolt and what you can do with it after this experience. To apply it to the other work I do, I can see that a pair of LaCie Thunderbolt Little Big Disks—one for work, another for backup clones—would make a perfect high capacity field production solution. And for the home studio, the only thing I want to see from the Promise Pegasus arrays is a solution with 3TB or even 4TB drives for even more capacity in a single array.