Object Security, Continued

Reader yossib left a comment to the previous blog entry, Cloud Storage - Part 5, Security, that warranted a more detailed response and discussion:

I enjoyed reading your article, your focus on the issue of user authentication and access control is important as it surely does not get the attention it deserves.

Do you see the security model and user access management for object storage evolving from current ACLs, Active Directory/LDAP or taking a different direction

How do you see the concepts of users and groups evolving?


Authentication of identity is so critical, because it is the foundation of access control, and as you alluded, deserves far more attention than it gets. Fortunately, the rise of a plethora of services on the Internet is forcing the issue of federated identity management, and while systems are not yet mature, there is a strong trend towards common mechanisms by which a user or computer program can have a universal identity that can cross systems.

Examples of emerging standards include OpenID, and Sun's IDM.

On Active Directory

Active Directory, while hugely successful and very valuable in a corporate setting, simply was not designed to accommodate the scale that is needed, nor the timeframes over which identities need to persist. As digital data and archives become core to our civilization, we need ways to ensure that the security of digital data can survive hundreds of years, and things that were often disregarded as "edge cases" must come to the front and centre.

Examples include:
  • What happens when someone dies?
  • What happens when someone gets subpoenaed?
  • What about the expiration of statutory rights?
  • What if the law determining the length of statutory rights is changed?
These and so many more issues make the protection of digital assets a double-edged sword — If we enshrine given restrictions in code, can we change them? And if we can change them, how can we prevent this from being defeating the original point of the protections.

And this ignores many of the challenges that are emerging from the loss of centralized control of systems. In emerging federated cloud worlds, objects may pass from system to system, both trusted and untrusted, and security must be preserved. Much of the challenges associated with the work done to try to build DRM systems is directly applicable to trusted repositories and archives, and the research tells us that this is a really hard problem.

For example, it is still an open debate if it is actually possible to have one user grant a second user access without this enabling that user to grant access to further users. And revoking access can be even more thorny.

Ultimately, we need to move away from the centrally enforced security models to a more distributed security model where objects can float around in systems that do not need to be trusted, and access is granted based on trust relationships. (An example of this is that you may grant an online search and indexing company the privileges to read your data, based on your trust that they will not disclose your data).


While ACLs have developed a reputation of being far too complex to be manageable, I believe that when tempered with methodologies such as Role Based Access Control, it can be made far simpler for the end user and application developer than it is right now.

However, ACLs fundamentally are merely advisory guidelines for a "trusted" system that interprets them to restrict access. ACLs need to evolve to the point where you have "grants" for each privilage, that enable you to perform that action. So if I wanted to share an object with you, I would give you a "grant" that gives you the ability to read a given object or set of objects. This grant could be revoked, and I could engineer it in such a way that you couldn't delegate the grant without revealing your own credentials.

Ultimately, this involves a much more complex multi-actor interaction, and my gut feel is that we can't do this with static objects. This, of course, would mean that revocation of grants could never really be absolute, (unless they expire, but who enforces that, then?) since you can't always ensure that all replicas of a given object are always kept in sync.

Finally, if these systems grow too complex, they won't work. There is much to be said for simplicity, especially in global-scale systems.

Users and Groups

This is always an interesting discussion — Groups provide such a valuable level of abstraction, but introduce so much complexity. I tend to lean towards abandoning the concept of groups as first class entities. If we just have users, we can create a user that is trusted to act as a delegate on behalf of other users. As long as one user can be granted the authority to delegate privileges to other users, we get the same functionality, and distributed group membership can be re-cast as a trust relationship between the owner and the delegator.

My feeling is that this is the only model that will scale.

Much to Consider

This is just the tip of the proverbial iceberg, and there are so many additional complexities and challenges associated with security. I'd love to continue this discussion, so if you have any questions, comments or ideas, please don't hesitate to comment.

Also, as I mentioned on my last twitter, there are many other security-related items that I plan to discuss further in a follow-up blog post, covering user identify federation, trust domains, "blind storage", peering, object destruction and more.


Object Storage, Part 5 - Security

In the days when storage was directly connected to computers and there was only one user per computer, security was simple — Just physically secure the computer and attached storage. But fast-forwarding to the Internet age, not only is storage networked, but potentially accessible to every user in the enterprise or Internet. As storage migrates from silos hidden behind computing servers to being a first-class peer on computer networks, security rapidly becomes front and centre as a key requirement, not just to protect (deny), but also to facilitate multi-application and user collaboration and sharing (allow).

As part five of the object storage series of posts, this entry covers the issues related with security in an object storage system. This extends far beyond just simple access controls, such as security issues related to the search functionality discussed in the last entry, Object Storage - Query.

Who's that Looking at my Data?

Fundamentally, security is about controlling the flow of information. Like with any storage system, there is information flow out from the system, and information flow into the system (read and write, in the case of a block device). But unlike a block device, which can only restrict read or write operations on a device or block level, an object storage system has a much richer set of information flows that need to be regulated.

Take query, for example. Information leakage through a query result set or index would be a significant problem, and access controls on each object must extend to the query results. In some environments, even timing matters, as variations in the time required to return a query may allow a user to determine if an object with a given metadata value exists or not, even if they do not have privileges to see that object.

Like all forms of access control, one first needs to authenticate the entity that is requesting access. Once you have determined who is talking with you, then the system can proceed to the question of what they are allowed to do. Then and only then can you proceed to perform the operation.

Expressed in English, this takes the following form:

Entity "X" is requesting to perform operation "Y" against object "Z".

Some examples include,

Entity "XYZCorp\Archiver" is requesting to perform operation "Modify" against object "8D73F687BA26C1A03F9D8E796A497338/com.bycast.metadata.lastmodifiedtime"

Entity "XYZCorp\Admin" is requesting to perform operation "Delete" against object "Financial Storage Archive Container"

Like what we saw with Implicit and Explicit Policies for managing storage placement, retention and others, this same model extends to security. A list of who is allowed to perform what operations on a given object can either be explicitly specified for a given object (this is called an ACL, or Access Control List, in the file world), can be implicitly specified for a given group of objects (inherited ACLs), or can be implicitly specified for a given user.

As one can imagine, these security models can become quite complex, especially when you start combining all three together. For example, if objects are stored into a container that specifies that only one user can access the objects, if the application explicitly specifies that another user can access the object, what is the correct behaviour? Should the explicit specification can override the implicit specification, or should the implicit specification can override the explicit. And both are valid, depending on the use case.

For example, in the Windows world, the "Backup Operator" must be capable of accessing all objects, regardless of their ACLs. This is an example of an implicit security policy overriding explicit security policies. In other cases, if an application wishes to explicitly share some objects with a second application, but by default, all objects should be managed by an implicit security policy, we have an example of an explicit security policy overriding an implicit security policy.

Policy Contexts

As most object storage systems are built around a flat namespace, implicit security policies apply against objects that match a set of metadata criteria. For example, the policy may say "For all objects where "com.bycast.metadata.creator" equals "XYZCorp\dslik" ...".

When logical containers are supported, they can be implemented as a special metadata item, such as where com.bycast.metadata.container having the value of "/corporate/financial" would express a subcontainer "financial" in a container named "corporate". As implicit policies can include which container they are applying to, this allows security policies to be restricted to a given container, or set of containers.

Thus, we end up with three different contexts:
  1. Explicit security policies, included with an object, that indicate who can do what.
  2. Implicit security policies, specifying for which objects they apply to, that indicate who can do what.
  3. Implicit security policies, specifying for which users they apply to, that indicate what the users can do what.
And this leads us to the "What":

Oh, The Things We Can Do

How does the operations that can be performed for object storage compare to file and block storage approaches? Well, in a summary, there's a lot more you can do — Below is a partial list of the actions that one can perform against a stored object:
  • Create an object
  • Destroy an object
  • Discover an object's existence
  • List an object's contents
  • Add a new metadata item to an object
  • Remove an existing metadata item from an object
  • Read the value of a object metadata item
  • Write a value to an object metadata item
  • Add a new data stream to an object
  • Remove an existing data stream from an object
  • Read the value of a object data stream
  • Write a value to an object data stream
  • Query for the existence of a named metadata item
  • Query for the existence of a named data stream
  • Query for the contents of a named metadata item
  • Query for the contents of a named data stream
These actions can becomes even more complex when you consider that some object storage systems allow read-only metadata items and streams, immutable metadata items and streams, or increment-only metadata items.

Unlike with file systems, which have evolved to a relatively standardized set of operations that are specified in ACLs, the security models for object storage operations is not yet well understood and standardized. Open questions include how privileges are overlaid onto the contents of objects, and how special behaviours, such as increment-only for a retention metadata item are handled.

This is one of the areas where the SNIA XAM working group has done excellent work, and I would encourage anyone interested the details of how security models map onto object storage to read the XAM Architectural Specification.