Time for Compliance

Many aspects of compliance storage rely on trusted time. These include timestamps indicating when an object or file was stored, retention durations that indicate when files must not be deleted or modified, and audit records indicating when operations were performed against the storage system. All of these timestamps must be accurate, and, more importantly, must be resistant against attack in order to satisfy the multitude of compliance regulations, such as Sarbanes-Oxley and HIPAA.

A Question of Time

When evaluating such a storage system, here are ten good questions to ask your vendor:

1. How and when is the clock set?
2. Who can set or adjust the clock?
3. Are changes to the clock audited?
4. How much can the clock drift over time?
5. If the clock is synchronized, is the synchronization chain trustworthy?
6. Is clock synchronization traceable to the NIST?
7. If clock synchronization is no longer possible, how does the system react?
8. When clock synchronization is regained, how does the system react?
9. What protections are present to prevent tampering with the clock at the system level?
10. What protections are present to prevent tampering with the clock at the network level?

Two Architectures

Generally, two architectures have emerged, one that involves a completely sealed system that is capable of maintaining accurate time with drift less than one minute per year for the life of the system, and one that involves network-based transactions that cryptographically prove that a given event happened at a given time.

The advantages of the first architecture include strong resistance against tampering, and low maintenance requirements. However, the downside to such an architecture is the requirement for custom hardware, both to keep accurate time (the clocks in typical servers range from largely inaccurate to downright embarrassing), and to provides the means to physically secure the hardware from prying eyes and screwdrivers. Because this requires custom enclosures and maintenance contracts (who do you trust to have the keys to the rack?) this typically lends itself to solutions from larger storage hardware vendors. And, after all, if you are spending hundreds of thousands to millions of dollars on something, it better well be able to keep accurate time.

The second architecture is unfortunately far more complex and difficult to design and implement correctly. In a software-only solution, very little can be relied upon to be trusted. After all, a standard x86 server is only one boot-disk away from unfettered tampering, and it's difficult to detect if you are running under a hypervisor. Thus, such systems must rely on complex network transactions to determine accurate times of events, often resulting in increased transactional latency. Unless these time transactions are designed and tested to ensure that a malicious time source or compromised node is unable to alter the timestamps and compliance durations, this can be a significant point of weakness.

Beware NTP

One protocol to keep a watch out for is NTP. A malicious NTP server, combined with a poisoned DNS cache and the quick throw of a circuit breaker might result in all your compliance data being unprotected ahead of schedule, or even worse, automatically erased from the system. And given that NTP security is rarely used and not well regarded, it is almost a certainty that it forms a weak link in the chain of trust.

Many systems that use NTP just use it to set the server and operating system clock, which they then trust blindly. For a given server, this clock can be easily altered, and in order to obtain trusted timestamps, information from multiple sources that can not all be easily compromised must be used.

Time is of the Essence

Time is often overlooked when evaluating compliance storage, but is a fundamental aspect of the compliance process. After all, in a court of law, if the timestamps of events cannot be proven to be accurate, and retention durations cannot be shown to be enforced, that expensive compliance system may end up being even more expensive.