How low can disk go?

The capital acquisition cost of disk-based archiving solutions (in cost per terabyte) has dramatically fallen over the last five years. Unfortunately, the rate of reduction in cost is slowing as the cost approaches the raw cost of the disks included with the storage system.

The four major factors that have driven the reduction of the cost of disk-based archiving are as follows:
  • Increasing disk capacity (density) for a hard drive of a given price
  • Transition from the use of enterprise disks to the use of consumer grade SATA disks
  • Transition from storage interconnection via Fibre Channel to switched SAS and Ethernet
  • Transition from customer storage controllers to commodity servers and software
When analysing the costs of an disk archive, much of these savings come from reducing the cost of the non-disk components. In a large disk archival systems, the system price divided by the number of disks is now approaching the over-the-shelf price of a consumer disk (around $100 per TB). This is an important, because it indicates that most of the cost gains (from factors 2 through 4) are unsustainable in the long term. The closer the cost of the system approaches the cost of the raw storage, the less cost reductions can be achieved.

As a consequence, the rapid decrease in the cost of disk-based archiving is not a result of the intrinsic reduction in the cost of disk storage, but rather is a reduction in the cost of the overall system. And now that this reduction in cost has already largely occurred, the rate of cost reduction flattens out to more closely approximate the reduction in the cost resulting from increasing disk density.

This is a critical point to understand when comparing the costs of disk archiving to tape archiving, since many cost projections have made the assumption that this rate of reduction of disk cost will continue into the future.


Mister said...

Disk base archiving will displace tape especially when combined with optical for long term viability. We had an old tape library replaced running over 56TB (Terabytes) of raw data with slow access times and painful restoration times. Phantom Data Systems http://www.phantomdatasystems.com/infinivault.html was able to migrate all the tape data to a disk based archiving system - InfiviVault from Prostor Systems..within 2 weeks we had speed, capacity, multi platform compatibility (Linux and Windows mixed environment).

I think as data keeps growing exponentially the need for speed and access time shrinking will become a driving factor, coupled with capacity.


David Slik said...

At 56 TB, by all means switch to disk — that makes total sense. We at Bycast sell a large amount of disk-based systems for tape replacement, but for our very large archive customers (1 PB+), tape is almost their exclusive choice.

When you price out a 1 PB tape system compared to even the cheapest disk system, tape is far less expensive. Even compared to just raw disks, it's less expensive. And that's excluding the opex costs.

By the way, the ProStore's RDX concept is quite interesting, and an excellent middle tier when you can tolerate the latency of a mount, but need HD-like random access to the data. It's still more expensive than tapes, though.

Ultimately, I think what will kill both disk and tape is archival flash, but that's a blog post for another day.

andrew said...

You need to consider the management costs of those libraries, especially as the tapes need to be continually verified and migrated - a tape library consumes quite a bit of power.....

What will kill tape is the collapse of the vendor base, technologies don't typically die a slow death - As vendors fall away there won't be enough business to amortimise the R&D cost over and its over quite quickly. I think the switch to digital cameras from film is ver prescient.

David Slik said...

Lower power and cooling costs is often cited as a significant savings when compared to spinning disk.

I agree that a healthy vendor ecosystem is critical for the continued success of tape technologies. While we have seen many vendors move away from tape, the presence of major players, including IBM, Oracle, Quantum and Spectralogic, who have committed significant investments in tape technologies will ensure that this ecosystem remains for a long period of time.