Terabytes From Space: Satellite Imaging is Filling Data Centers

Commercial imaging companies are collecting upwards of 100 terabytes (TB) or more per day, every day, 365 days a year, accumulating huge data sets measured in petabytes per company.

While Hollywood has glorified the role of imaging satellites as all-seeing eyes tracking everything on the surface of the planet, only now is reality catching up to the movies. The real value in imagery is as much the ability to track changes over time as being able to take instant pictures of the world’s hotspots. A growing number of companies are pursuing this opportunity.

“More than 40 commercial companies have launched imaging satellites in the past 10 years,” said Janice Starzyk, Vice President of Commercial Space for analytics and engineering firm Bryce Space and Technology. “We count 36 commercial earth observation companies as start-up space companies. Twenty-one of these were founded between 2014 and 2019.”

Bryce’s internal forecasts count for more than 50 proposed commercial earth observation constellations being developed, with Starzyk projecting that between 500 to 1,000 remote sensing satellites will be launched over the next five years, with the lower end more likely given the current market.

Analytics to Predict Economic Activity

If simply adding tens of terabytes of storage business per day is not enough to whet the appetite of any data center operator, raw images must be massaged and processed into usable form, cataloged, and made searchable and deliverable to users for analysis. Machine learning is increasingly being applied to locate features and find changes over time, providing alerts to end-users when things are discovered.  To top it all off, imaging data is increasingly being “fused” with other information such as satellite IoT to provide a holistic picture of happenings on the ground, sea, and sky.

But developing value-added, user friendly services is a challenge.  “Most of the space remote sensing companies sell themselves as data analytics companies, because satellite imagery itself is becoming a commodity,” said Brian Weeden, Director of Policy Planning at the Secure World Foundation.  “The real value lies in turning that imagery or other raw data into actionable information. Turns out that’s a lot harder to do than to say, but it’s where the biggest commercial focus is.”

San Francisco-based Planet started the wave of new commercial imaging firms in 2010, launching its first satellites in 2013.  Today, Planet has two distinct fleets of optical imaging satellites. Small 3U (30x10x10 centimeter cubesat standard, not data center rack standard) Doves collecting 3- to 5-meter resolution imagery make up the majority of its fleet with over 130 in orbit, taking 1.3 million 29-megapixel images every day. Fifteen larger dorm-fridge-size SkySats provide high-resolution imagery of under 1 meter, providing enough detail to distinguish between trucks and cars.

“We collect 10 TB per day,” said Planet co-founder and Chief Strategy Officer Robbie Schingler. “Once we complete our upgrade to SuperDoves, we’ll collect 40 TB per day.”  The SuperDove satellites use upgraded cameras to capture more bands of color, giving them improved capabilities for monitoring crops and forests, locating minerals, studying the atmosphere, and other applications.

Creating an Analytics ‘Time Machine’

While a single image of a single point on the planet provides value, it is the accumulation of imagery where the true treasure resides. Companies can look through an imaging catalog in a “time machine” fashion to observe economic activity such as oil tanks filling up, ships moving in and out of ports, volume of cars at shopping malls, when crops are planted and harvested just to name a few examples.

Planet has an average of 1,200 images for every point on the Earth’s landmass since its satellites started operations.  Colorado-based Maxar Technologies has over 110 petabytes in its library of images dating back to when its first commercial imaging satellite started operation in 2000 and adds more than 80 TB of day per day with its fleet of WorldView high-resolution imaging satellites.

Planet has an average of 1,200 images for every point on the Earth’s landmass. Maxar Technologies has over 110 petabytes in its library of images, and adds more than 80 TB of day per day.

Tapping into data center services is key for Planet and other satellite startups. “If you looked at it the way everyone was doing remote sensing systems the day we started, it would be cost prohibitive to collect that amount of data and process that amount of data,” Schingler stated at a panel discussion at Satellite 2020 in Washington D.C. “And this is a bet we more or less made – by the time we had that data coming down we would be cloud-native as a company. And so we would build IT systems in place that bring in the latest computation capabilities and services available.”

Planet’s current and historically preferred cloud vendor of choice is Google, although the company also has worked with Rackspace and Amazon. “We made it so we weren’t reliant on any one tech stack,” Schingler said. “We’ve migrated back and forth and we use these big IT giants to negotiate off each other to get long term, really good contracts.” Google is the favorite because Planet got a “long-term good deal” and the platform is viewed as the best one for machine learning and AI, with a large ecosystem of tools and applications developers.

Moving Data to the Cloud by the Truckload

Competitor Maxar stores its imagery on Amazon Web Services, having gone through the process of migrating its entire 100 PB tape library onto the Amazon cloud in 2017. Part of the process included parking a shipping container packed full of storage at Maxar’s corporate headquarters in order to transfer data out of the company’s data center and then move it all on wheels to an Amazon location for ingestion.

Optical imaging satellites such as those operated by Planet and Maxar have limitations.  They can’t see through clouds and are of limited usefulness at night. The newest group of start-ups are planning to deploy dozens of small synthetic aperture radar (SAR) satellites to enable high-resolution imaging on a true 24×7 basis, unhindered by cloud cover or darkness.

SAR start-ups such as Finland’s ICEYE, Japan’s Synspective, and US-based Capella Space and PredaSAR are reluctant to divulge details on how much data and what they do with it, but SAR imaging collects more data, requires more onboard power than their optical counterparts, and needs more processing on the ground once raw data arrives.

ICEYE has managed to pack radar, high-speed RF data link, and enough power into a 6U cubesat roughly the size of a large shoebox (before the satellite deploys antennas and solar panels). The ICEYE team said its daily data collection rate is measured in “terabytes” with three of its satellites in commercial operation today. “SAR data needs significant processing to turn raw data into processed images, which very quickly multiplies the total data amounts per each acquisition saved on the cloud,” a group of ICEYE engineers said in response to email. “Individual images are gigabytes of data when processed. ICEYE is able to take SAR videos. For these, the data collected can be tens of gigabytes.”

Data storage and processing needs for ICEYE are likely to scale dramatically by the end of 2020 and in the future.  ICEYE has several satellites scheduled for launch later this year and has a goal of 18 satellites in its full constellation.  The company is doing a combination of local and cloud storage and post-processing, with Amazon S3 cited as its go-to for cloud storage.

A Testing Time for Startups

While some imaging start-ups may be slowed or stopped by current economic upheaval, others will likely continue forward, buoyed by the U.S. government’s desire for more and diverse means of imaging outside of its fleet of specialized and very expensive satellites.

“As we’re seeing in other parts of the space industry, those companies that have government contracts are absolutely more likely to weather the coming economic storm than those without,” said Weeden. “Whether or not that’s enough to survive, who knows.”

Starzyk was a bit more optimistic, saying, “Clearly, the economic environment and liquidity crunch will impact companies in need of near-term financing to operate and continue building out their systems.  However, there is a new impetus for data and analysis on a larger scale that could help spur the burgeoning markets that these companies are trying to create.”