DCF Show: The Wild Future of Data Center Storage

This is the first prototype of an automated system to store data within DNA, developed by researchers from Microsoft and the University of Washington. (Photo: Microsoft Research)

These both kind of sound like science experiments. But the science fiction aspect of them distracts from the fact that these are potential ways to bring new levels of scale to data management. In its research division, Microsoft is looking at new ways to store data into synthetic DNA which winds up being stored in a liquid solution. They’ve come up with a prototype that uses beakers and really looks like a lab set up more so than a than a cloud computing center. That’s one of the things that there will have to be managed going forward in trying to see whether this is a technology that can ever be comfortable in a real world data center.

There’s lots of engineering work that needs to be done before we can get to that point but the motivation is that according to Microsoft if you took an exabyte of data right now you need two large data centers each about the size of a Walmart to properly store and manage that data. If DNA storage succeeds it’s possible that that could be shrunken down to a small chip of like a single cubic centimeter. That’s a compelling possibility and would bring entirely new scale to the way that data can be stored. And that’s important when we think about all of the technologies going forward that are going to require much more data than they use today .

An example would be video, as we talk about 4k and 8k high definition video. These involve much larger amounts of data than the current technologies do. The same is certainly true for some of the virtual reality and augmented reality that we see and artificial intelligence which is something that is really arriving today in a big way, and is driving much larger data sets as we come up with models that can manage you know billions upon billions of data points and put them into calculations that can conduct and automate different types of work. Basically we are growing data at an explosive rate across the board.

At Data Center Frontier we think about data generation in terms of streams – there’s many different streams streams of of data coming into the data center industry from a lot of different industries. not all of these new technologies have to succeed spectacularly for this to have a huge impact upon the data center demand and the technologies themselves that live inside the data center and must manage this data tsunami. When you think about holograms, most of us think of “Star Wars” and that scene where the hologram of Princess Leia pops up. “Help me, Obi-Wan Kenobi.”

With Project HSD, Microsoft is working on using holograms in crystals to store data at unprecedented scale. (Photo: Microsoft)

Holograms have lots of other functions though, including the fact that they can store data. This is a technology that’s been uh known about it’s been around since the 1960s but it’s been really hard to commercialize partly because of the technology that’s required to successfully execute it. One of the key requirements is high-definition cameras, and that’s an area where our smartphone technology of today has come a long way, and has made it much more feasible to think about using holograms for storage.

Microsoft has done a lot of interesting research in this area with its HoloLens product which is most familiar to many gamers who who who use it for virtual reality gaming, but also has applications in the commercial world helping people build and design products. So the HoloLens gives some insights into how the holographic systems might be used for storage. In Microsoft’s case they have looked at different ways to embed the data in crystals, and it’s a little bit different from the optical storage systems that we see today. It goes back to light and using lasers to write data, but rather than just using one surface it can really do the storage in parallel and give you a lot more storage capacity. In traditional data storage systems information is stored in a binary series of ones and zeros that are written onto spinning disks or tape using magnetics or using lasers can be written onto DVDs or BluRay disks.

In a DNA system data is stored in a liquid solution containing DNA and then is read by systems that can combine electronic and molecular components. They use synthetic DNA so it doesn’t repurpose human or animal DNA. Mark Russinovich, who is the chief technical officer at Microsoft Azure, discussed these projects recently at Microsoft Ignite and joked that your data storage of your family photos can’t be used to to create some sort of crazy animals running around or weird creatures. But there is a science fiction aspect of this that is hard to get your head around and Microsoft acknowledges that it’s a long way to go before you can have this ready for showtime in the data center. But the potential gains in terms of the volume of storage make it worth pursuing.

Microsoft has not been bashful about investing in research that pursues kind of moonshot projects to advance new technologies. A good example that many of you may be familiar with is its underwater edge data center project, which is called Project Natick, in which they took a bunch of servers and sealed them in a capsule and dropped them to the the ocean floor about 1,000 yards off of Scotland to see how they would perform.

The Project Natick data center module was recently retrieved from the ocean floor near Scotland. (Photo: Microsoft)

One of the key ingredients in that was pumping the capsule full of nitrogen so that the environment that the servers are living in, and one of the things that they found is that in the nitrogen environment servers’ shelf life was increased dramatically. They had one-eighth the rate of failures that most land-based traditional servers have, so that’s one area in which they’re starting to see a dividend on their investment in these moonshot projects. Microsoft sees DNA as a another possible venue where the gains may more than justify the kind of research dollars that they’re putting into it.

More recently Microsoft has begun talking about its holographic data storage system which takes data within optical light beams and it’s stored as an image inside a crystal of lithium niobate. Holographic storage is capable of repeat recording and reading millions of bits in parallel enabling data transfer rates that are far greater than those that you get by traditional optical storage. Microsoft Research Cambridge and the Azure team have been working together on a prototype that has successfully demonstrated many of the concepts that they think will be the way that this will work in an Azure Cloud data center.

These technologies are making rapid progress but they’re still likely many years away from implementation of any kind of scale in an actual data center. Even after new technologies become viable they’re adopted gradually over time providing data center operators with an opportunity to adapt their designs and best practices for these new technologies. For an example let’s look at cloud computing which has been one of the most disruptive and impactful IT trends. Amazon Web Services, the first major cloud computing platform, debuted in 2006. We are nearly 14 years on and Gartner says that the enterprise adoption of cloud computing of platforms is at about 30 percent – there’s still a much larger chunk of people who aren’t in the cloud than are.

But Microsoft’s interest in these technologies underscores the innovation challenge for companies that are targeting the hyperscale data center sector which is an increasingly large portion of the data center business and drives many of the innovations that we’ve seen in data center construction and speed to market as well as networking and storage. Service providers that want to target these hyperscale data center providers must reckon with these new technologies that they are looking at and the spec the specifications that will be required to support them.

So maybe DNA storage and holographic storage aren’t right around the corner but they’re in the pipeline and they are one of the many technologies that we’re tracking at Data Center Frontier that we think will may eventually make their way to the data center and change the way that this business is operated. The future is pretty amazing and sometimes that involves science fiction that becomes very real and lives inside a data center.