A CDMI Tutorial - Basic Input/Output

With the new 0.9 draft release of the SNIA Cloud Data Management Interface specification now released, this is a good time to highlight the aspects of the standard that make it well suited for cloud storage, and to review the changes from the previous 0.8 draft version.

This post is the first in a series on CDMI. Subsequent posts will cover the following areas:

• Basic Input/Output (This post)
• Data Management, Part 1
• Data Management, Part 2
• Advanced Input/Output
• Cloud-to-Cloud Interactions
• Queues and Query
• Authentication and Access Control
• Billing and Accounting

Basic Input/Output

While supported, CDMI does not assume or require that clients interact with a storage system using the CDMI for storage and retrieval. CDMI can be used in conjunction with existing cloud protocols (such as Amazon's S3 API) and file system protocols (such as NFS, CIFS and WebDAV). The goal of CDMI is not to replace or even supplant these protocols — it is to provide a standardized access and management method that is independent from these protocols. So while storage and retrieval are important parts of the standard, they are only just one part of the standard. It is even possible to implement a fully complaint CDMI system that does not support the ability to store or retrieve data.

By not restricting to how data is stored or accessed, but still enabling a consistent and uniform way to access and manage stored data, CDMI enables a client to discover, access and manage stored content regardless of how it was originally stored, and regardless of the underlying storage implementation.

To get an idea of the basic object model by example, let's review basic storage and retrieval operations using CDMI:

Storing an Object

The most basic operations in CDMI are object storage and retrieval. Assuming that we have an authenticated session established to a CDMI cloud running at cloud.example.com and we want to store some text, here is the HTTP transaction that would be performed, as described in section 8.2 of the specification:

PUT /hello.txt HTTP/1.1
Host: cloud.example.com
Accept: application/vnd.org.snia.cdmi.dataobject+json
Content-Type: application/vnd.org.snia.cdmi.dataobject+json
X-CDMI-Specification-Version: 1.0
{
    "mimetype" : "text/plain",
    "value" : "Hello Cloud"
}

HTTP/1.1 201 Created
Content-Type: application/vnd.org.snia.cdmi.dataobject+json
X-CDMI-Specification-Version: 1.0
{
    "objectURI" : "/hello.txt",
    "objectID" : "AABwbQAQ810Mpei21pxfzA==",
    "parentURI" : "/",
    "accountURI" : "/cdmi_accounts/default_account/",
    "capabilitiesURI" : "/cdmi_capabilities/dataobject/",
    "completionStatus" : "Complete",
    "mimetype" : "text/plain",
    "metadata" : {
        "cdmi_size" : "11"
    }
}

Let's look at this transaction in more detail:

It's standard HTTP, and we specify that we will be submitting a request body in the form of a cdmi.dataobject, and requesting a response body in the form of a cdmi.dataobject. We submit in our request body JSON that includes a mimetype and value field, and receive in response a result (HTTP 201 Created) and a JSON structure that includes information about the newly created cloud object.

The response body JSON contains the following fields:

Field Name	Field Description
objectURI	The URI for the newly created object. This can be used to access the object via CDMI, and reflects the organization of the stored data. For example, if a storage system provides CDMI access to a NAS share, the objectURI reflects the file path.
objectID	Every object within a CDMI system has a globally unique identifier that can be used to access the object. Object IDs remain constant for the life of the object, even if modified, renamed or even moved to a different cloud provided by a different vendor.
parentURI	The URI for the parent of the created object. Objects inherit metadata from their parent.
accountURI	The URI for the account that the object was created under. Accounts determine the billing relationship, reporting visibility and basic access permissions.
capabilitiesURI	Every object within a CDMI system has "capabilities", which describe what operations the system is capable of performing on that URI. This allows clients to discover the capabilities of a CDMI storage provider. This URI allows a client to find out what the capabilities of a given object are.
completionStatus	This field indicates if an object has been fully created. When performing operations that take long periods of time, such as serialization/deserialization, copies and storing large objects, this field may indicate that the object is still in progress, and thus is not ready to be accessed.
mimetype	This field indicates the type of the value of an object, as specified at the time of creation.
metadata	This field includes user-specified metadata (not used in this example) and system generated metadata (one example of object size is shown in this example) related to the object. Metadata is a key part of the CDMI standard, and will be covered in more detail in later tutorials.

While this is the primary method to create objects in CDMI, there is an even simpler way:

PUT /hello2.txt HTTP/1.1
Host: cloud.example.com
Content-Type: text/plain
Content-Length: 11

Hello Cloud
HTTP/1.1 201 Created

This approach, described in section 8.3, uses completely standard HTTP to create a new CDMI object. The mimetype is specified through the Content-Type header, with the tradeoff that no CDMI-specific data is returned in response. The ability to support operations through 100% standard HTTP makes CDMI very easy to use from JavaScript and web environments, as does the use of JSON.

Listing Stored Object

As every data object has a parent, CDMI also allows the objects owned by a parent to be listed. This is performed by the following HTTP transaction, as described in section 9.4 of the specification:

GET / HTTP/1.1
Host: cloud.example.com
Content-Type: application/vnd.org.snia.cdmi.object+json
X-CDMI-Specification-Version: 1.0

HTTP/1.1 200 OK
Content-Type: application/vnd.org.snia.cdmi.container+json
X-CDMI-Specification-Version: 1.0
{
    "objectURI" : "/",
    "objectID" : "AABwbQAQI2MtLCwAVfYSFA==",
    "parentURI" : "/",
    "accountURI" : "/cdmi_accounts/default_account/",
    "capabilitiesURI" : "/cdmi_capabilities/container/",
    "percentageComplete" : "Complete",
    "metadata" : {
        
    },
    "childrenrange" : "1-2",
    "children" : [
        "hello.txt",
        "hello2.txt"
    ]
}

As a client does not know if a given URI is a data object or a container, it asks for a generic object. In this case, the URI, "/" is a container, so the cloud storage system returns with a response body of type cdmi.container. All common fields shared between data objects and containers have consistent meanings, but the two additional fields, "childrenrange" and "children" provide information about the objects contained by the container.

Like with the data object example above, there is a simpler way to get a list of the children of an object:

GET /?children HTTP/1.1
Host: cloud.example.com

HTTP/1.1 200 OK
Content-Type: text/json
{
    "children" : [
        "hello.txt",
        "hello2.txt"
    ]
}

This approach, as described in section 9.5, illustrates the ability to request specific fields to be returned in the JSON response body, and is easily integrated into AJAX-style javascript for web-based applications.

Retrieving Stored Object

Retrieving a stored object is as straightforward as listing the children of a container:

GET /hello.txt HTTP/1.1
Host: cloud.example.com
Content-Type: application/vnd.org.snia.cdmi.object+json
X-CDMI-Specification-Version: 1.0

HTTP/1.1 200 OK
Content-Type: application/vnd.org.snia.cdmi.dataobject+json
X-CDMI-Specification-Version: 1.0
{
    "objectURI" : "/hello.txt",
    "objectID" : "AABwbQAQ810Mpei21pxfzA==",
    "parentURI" : "/",
    "accountURI" : "/cdmi_accounts/default_account/",
    "capabilitiesURI" : "/cdmi_capabilities/dataobject/",
    "completionStatus" : "Complete",
    "mimetype" : "text/plain",
    "metadata" : {
        "cdmi_size" : "11"
    },
    "valuerange" : "0-11",
    "value" : "Hello Cloud"
}

Like the JSON body when originally creating the data object, the CDMI storage system returns the fields associated with the object. But unlike the original create, it also returns the value range and the value of the object. This is described in more detail in section 8.4.

If just the value is desired, it can be requested as either a JSON content-type, or as a standard HTTP (non-CDMI transaction):

GET /hello2.txt HTTP/1.1
Host: cloud.example.com

HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 11

Hello Cloud

This allows CDMI clouds to be used to act as a standard web server, and enables the intriguing possibility that standard web servers could use CDMI as an upload, management and publishing protocol.

Summary

This provides a quick overview of the basics of writing, discovering and reading data in a CDMI cloud. In the next tutorial, we will review CDMI's data management functions, which rounds out the core of the proposed protocol.