A Reference Architecture for Cloud Storage

Over the last twelve months, the SNIA Cloud Storage Technical Working Group has been busily defining an industry-wide standard for cloud data storage and management. This standard, the Cloud Data Management Interface (CDMI), is currently available for public review, with an updated draft scheduled for release in later in December.

The below diagram illustrates how the CDMI standard fits into emerging cloud ecosystems:

One of the most important aspects to understand about the CDMI standard is that it is not intended to replace existing cloud data access standards. This allows CDMI to be used with existing and new clouds that store data via non-CDMI protocols such as Amazon S3's API and traditional file system protocols such as NFS, CIFS, WebDAV.

CDMI builds on top of these existing data access paths to bring rich management semantics for data in the cloud, and facilitates emerging cloud use-cases including cloud peering, federation and differentiated services. In addition, CDMI provides standard mechanisms to enable the integration of clouds with other external systems (or clouds) for notification, workflow, audit, billing and authorization purposes.

A Reference Architecture for a CDMI-Native Cloud Storage

In order to illustrate how CDMI can enable to creation of next-generation cloud architectures, I've put together an example reference architecture for a CDMI-based cloud storage system, illustrating the different components and primary data flows that would be found in most cloud implementations that fully support CDMI.

The below diagram shows a logical representation of the major components of a cloud storage system build around the CDMI standard:

Looking at the diagram, we have the following major logical components:

Filesystem Clients and Protocol Gateways

CIFS, NFS, FTP, WebDAV, iSCSI, Fibre Channel, FCoE and other standard network file and block protocol clients can be attached to cloud storage via protocol gateways. These protocol gateways translate non-cloud storage protocols into CDMI cloud transactions, and are notified about changes to management metadata in the cloud such that it can export portions of the cloud via these non-cloud protocols.

Existing Cloud Clients and API Protocol Gateways

Existing cloud clients programmed to use cloud protocols such as Amazon's S3 HTTP API can communicate with a CDMI storage cloud through an API protocol gateway. An API protocol gateway translates these non-CDMI cloud storage protocols into CDMI cloud transactions. When a given non-CDMI cloud protocol supports features not directly supported by CDMI, the API gateway can either map these operations into CDMI operations, or use vendor extensions to CDMI to implement the required functionality.

CDMI Clients

Clients that implement the CDMI protocol natively can connect directly to the CDMI storage cloud without having to go through any protocol translation layers. These clients can access the full set of CDMI functionality, and can access and manage content stored by all clients, even if the content was not stored via CDMI.

Low Latency Object Stores

In almost all implementations, CDMI transactions will be routed to one or more low-latency object stores that are able to quickly satisfy storage and retrieval requests. Multiple low-latency object stores will often be connected together, peered, or federated, to allow for data dispersion, replication and other data management functions. Once safely stored, the cloud will be ready to interpret the data system metadata that determines the optimal placement characteristics desired by the cloud user.

High Latency Object Stores

Some vendors may choose to implement higher latency object stores, such as power-managed disk or tape storage tiers. Stored objects will typically be migrated to these stores from low-latency object stores based on the CDMI data system metadata.

Object Catalogue

In order to keep track of the locations where cloud objects are stored, and the namespace in which the objects are stored, many CDMI implementations will include an object catalogue. This catalogue keeps track of objects across all object stores (and external peered and federated clouds), and provides CDMI functions such as object notification, query and billing. Notification provides updates to internal cloud components and to external systems to enable additional cloud functions such as indexing, e-discovery, classification, object processing, format conversion, workflow and advanced analytics.

Data System Manager

In a CDMI system, the data system manager is responsible for interpreting the data system metadata specified for objects and containers through the CDMI interface, and performing operations to attempt to satisfy the requests for data dispersion, redundancy, geographic placement, performance, etc. Each time an object is created or modified, the data system manager will be notified, and if the constraints specified are not met, it can further replicate or migrate content across object stores or clouds. The SDSC iRODS (Integrated Rules-Oriented Data System) is an example of an existing Data System Manager that can play this role within or to federate clouds.

Putting it all Together

When you put all of these components together, you get a cloud that fully implements the main components of the CDMI specification, and provides rich cloud functionality. On top of this foundation, a cloud vendor can innovate and create additional value added services, such as integration with computing clouds, cloud virus scanning, QoS monitoring, and much, much more.

This degree of flexibility demonstrates much of the value that CDMI has to offer the industry, both to cloud users and vendors, and it will be exciting to see these sorts of interoperable architectures emerge over time as CDMI becomes adopted and the cloud marketplace matures.

How low can disk go?

The capital acquisition cost of disk-based archiving solutions (in cost per terabyte) has dramatically fallen over the last five years. Unfortunately, the rate of reduction in cost is slowing as the cost approaches the raw cost of the disks included with the storage system.

The four major factors that have driven the reduction of the cost of disk-based archiving are as follows:

• Increasing disk capacity (density) for a hard drive of a given price
• Transition from the use of enterprise disks to the use of consumer grade SATA disks
• Transition from storage interconnection via Fibre Channel to switched SAS and Ethernet
• Transition from customer storage controllers to commodity servers and software

When analysing the costs of an disk archive, much of these savings come from reducing the cost of the non-disk components. In a large disk archival systems, the system price divided by the number of disks is now approaching the over-the-shelf price of a consumer disk (around $100 per TB). This is an important, because it indicates that most of the cost gains (from factors 2 through 4) are unsustainable in the long term. The closer the cost of the system approaches the cost of the raw storage, the less cost reductions can be achieved.

As a consequence, the rapid decrease in the cost of disk-based archiving is not a result of the intrinsic reduction in the cost of disk storage, but rather is a reduction in the cost of the overall system. And now that this reduction in cost has already largely occurred, the rate of cost reduction flattens out to more closely approximate the reduction in the cost resulting from increasing disk density.

This is a critical point to understand when comparing the costs of disk archiving to tape archiving, since many cost projections have made the assumption that this rate of reduction of disk cost will continue into the future.

The Demise of Tape is Overrated

Christopher Poelker, in a blog post on ComputerWorld titled, Is Tape Really Dead, makes a series of assertions about the superiority of disk over tape. Now to give credit where it is due, he is talking about tape's role in backup, and does conclude that tape still has a role to play. Unfortunately, many of his statements made along the way are simply inaccurate.

Chris states:

Everyone is aware of the limitations of tape solutions.

Tape	Disk
Sequential access	Random access
Relatively slow	Fast
Shipped offsite	Electronically vaulted
Once a day process	Periodic or continuous
High operational touch	Automated
No-dedupe	Dedupe
Inexpensive media	More expensive

Let's look at each of these in turn:

Sequential Access vs. Random Access
What Chris is getting at here is seek latency. Both tape and disk provide full random access, only disk is faster at it than tape. However, as hard disk capacities have increased while access bandwidth remains largely constant, from a software architecture standpoint, disks look more and more like tape. This is what is leading to the collapse of RAID 5 as a means to protect data, and, in my opinion, is what will ultimately lead to the death of disk. But more on this in a subsequent blog post.

Relatively slow vs. Fast
Tape systems, when properly used, can provide extremely high levels of throughput performance, into the 10 Gigabit/sec ranges.

Individual tape drives can already stream sequentially accessed data faster than most hard disks (120 MBytes/sec), and LTO5 will increase this lead further. When randomly accessing data spread across a tape or disk, the disk will outperform tape due to lower seek latencies. And, of course, if seek latencies are important to you, you should be looking at flash.

Shipped offsite vs. Electronically vaulted
Disk drives are far more vulnerable to damage than tapes, and simply don't have the flexibility to be able to be shipped around the same way. "Electronic vaulting" often equals expensive WAN data transfers and higher costs for power and equipment.

Once a day process vs. Periodic or continuous
This would be true if we're talking about a pure tape solution, but tape-based systems have been deployed along side disk in a storage hierarchy for decades.

High operational touch vs. Automated
Baby-sitting ten thousand disks isn't low operational touch either. Disks fail continuously, and the wrong swaps can destroy an entire RAID set. Many large archives run with tens of thousands to hundreds of thousands of tapes with very little operator intervention, and modern automated libraries are highly reliable and fault tolerant.

No dedupe vs. dedupe
This is false, as dedupe is yet another data compression technique, and applies equally well to tape as it does to disk. Again, the use of dedupe on tape in backup and archiving systems goes back decades.

Inexpensive media vs. More Expensive
We're thrown a bone here in the cost department, but what isn't considered is in addition to the consumables (disks and tapes), the disk subsystems themselves must be replaced on a far more frequent basis than tape libraries. With tape libraries, the drives can be swapped out and the tapes migrated to newer, higher capacity media without having to replace the entire library.

This also does not take into account the far higher opex costs of power and heat required for disk-based solutions.

Conclusions
Despite the near continuous siren call of the "Tape is Dead" crowd, tape provides significant value, often higher value for the dollar than disk, and has a long life before it. And, in many ways, it is spinning disk that should be more worried about its life in the coming decade.

The Roles of CDMI

The proposed SNIA Cloud Data Management Interface standard (CDMI) is intended to address a wide variety of different use cases, as described in the draft SNIA Cloud Storage Use Cases document.

Specifically, CDMI has the several distinct and overlapping design goals:

1. To act as a cloud management protocol for other non-cloud data access protocols, without providing data access.

The use case for this mode of use is, for example, the management of a SAN or NAS fabric allowing the provisioning and specification of data system metadata for opaque LUNs which can be dynamically provisioned programatically, for example, in conjunction with OCCI in a cloud computing environment. In this case, there is no data access via CDMI, only management and accounting access.

2. To act as an cloud management protocol and as a secondary cloud data access protocol to existing cloud and unified storage systems.

The use case for this mode of use is, for example, is to provide consistent management access to existing unified storage systems that provide block, file and object protocols. For example, a Amazon EC2 instance could be run that exposes an S3 bucket through CDMI, manages Elastic Block Storage LUNs, and implements some of the data system metadata functionality.

3. To act as a primary cloud management and cloud data access protocol for next-generation cloud storage systems

The use case for this mode of use, for example, is to enable a superset of cloud data access, manipulation and management functions, and to enable advanced scenarios such as distributed application systems build around cloud storage, cloud federation, peering and delegation. For example, a cloud could provide CDMI access to objects for cloud applications, all while manage file-system views into the object space from remote edge NAS gateways, all while federating together existing enterprise storage and public and private clouds.