Standardizing Cloud Storage

As the cloud storage concept matures and an increasing number of service and technology providers emerge in the market, there is a growing recognition of the need to standardize protocols for data access and storage management functions in a cloud storage environment.

The advantages of an open standard for cloud storage include:

• Allowing a cloud storage client to interoperate with multiple providers
  
• Enabling data portability between cloud providers
  
• Facilitating common documentation, sample code and educational material
  
• Allowing common test infrastructure and conformance testing
  
• Reducing development work for cloud clients and providers
  
• Reducing the complexity of standardized access libraries
  
• Encouraging the creation of debugging tools for diagnostics, profiling and interaction analysis
  

With cloud computing and cloud storage being such a hot topic at this time, there are multiple initiatives underway attempting to standardize various components and interfaces within the cloud storage stack. And one of the leading initiatives is the cloud storage technical working group within the Storage Networking Industry Association.

A Cloud Storage Reference Model

In the last six months, we've been working on several deliverables, with the main work product being the creation of a standard reference model for the management and access to cloud storage resources.

In summary, here are the highlights of the working group's vision of cloud storage:

An HTTP API

Management functions and data access are provided via a light-weight HTTP RESTful API. Object, block and database storage APIs co-exist with this core API, facilitating emerging cloud use cases and allowing continued innovation as new applications are moved into the cloud.

The HTTP API also facilitates discovery and introspection of provided API capabilities, allowing providers to support as little or as much of the API as they wish, and allowing clients to discover which capabilities are provided. This approach allows cloud vendors to provide additional capabilities (such as Nirvanix's media transcoding capabilities), and still being compatible with and leveraging the common functions of the API.

Containers and Data

Any data item stored in the cloud, including simple data streams to XAM objects, iSCSI LUNs, database tables and other data objects, can be accessed directly in a form that facilitates peering and transfer from cloud to cloud. As data items can be grouped together into named "containers", and containers can be nested, transferring aggregations of data items is as easy as transferring a container.

Likewise, management operations can be performed on containers, reducing the management complexity when compared to managing individual objects, and allowing changes to be performed atomically on sets of objects. Management properties (metadata) of data items can either be explicitly specified for a given data item, or can be inherited from the parent container.

A Vision for the Future

This simple set of principles allows for a powerful, extensible API that spans all classes of storage in the cloud. Bycast is proud to be participating as a primary contributor to this initiative, and I would encourage anyone with interest in this area to take the time to read the currently released documentation and to get involved at the SNIA cloud Google group.

SNIA is also holding a summer technical symposium in July, held in San Jose. At this event, one entire track is dedicated to cloud storage. If you are in the area, don't hesitate to get in touch with the SNIA to find out what it takes to get involved, and join us in this exciting project.

In the mean time, I'd encourage reading the current draft documentation that can be downloaded from the SNIA, as we're proud of what we've accomplished so far, and are excited about where we're going.