With ECS v2.2, EMC has improved its storage efficiency, searchability and security, we’re told.
EMC’s ECS (elastic cloud storage) is a software-defined, object-based cloud storage platform that can scale up to exabyte levels. The company launched ECS, previously known as Project Nile, in May, 2014.
ECS 2.2 adds three things:
Ability to search metadata across objects without a dedicated database.
Increased storage density 33 per cent and storage efficiency 10 per cent.
ECS D@RE supports FIPS-140-2 Level 1 compliance using an AES 256-bit encryption algorithm. D@RE can be applied at the bucket or namespace level in the ECS portal or with the ECS Management API. Support at the object level is also available using the Amazon S3 SSE constructs.
D@RE provides automated key management and encrypts inline and then stores the encrypted data on ECS storage media. Keys are segregated at the namespace level. User-supplied keys can be used with the S3 API.
EMC pre-sales director Antonio Romeo says that searching an object store can often require developers to write their own search functions and insert them into the object store’s fabric. This means “essentially using an external DB, maintain it, backup it, [and] keep it in sync with the object storage platform.”
With ECS v2.2, users can search metadata across potentially exabytes of unstructured data in the object store without a dedicated database. The developers enable user-defined metadata to be searched “via rest APIs especially suited for Internet of Things, mobile app and geo-distributed datasets.”
This is how it is done: in ECS 2.2, the ECS S3-compatible protocol automatically associates system metadata with an object and allows users to associate custom metadata with an object. The metadata is in the form of name-value pairs.
The metadata search facility enables ECS to maintain an index of the objects in a bucket, based on their associated metadata, and allows S3 object clients to search for objects within buckets based on the indexed metadata, using a rich query language. The metadata fields for which search indexes will be maintained (search keys) are configured for a bucket from the ECS Portal, the ECS Management REST API, or the S3 REST API.
Another new feature is a single-pane-of-glass view that provides what EMC claims is a complete system health check.
Romeo says “there are dozens of other updates in our latest release,” but doesn’t detail them. A v2.2 ECS Planning Guide [PDF] does, and here are some of them:
Cold storage archives with less object storage overhead, meaning greater storage efficiency.
HDFS – ECS HDFS is certified against Hadoop 2.7. Certified applications/components include HDFS, MapReduce, Yarn, Hbase, Hive, Pig and ZooKeeper.
ECS supports SEC Rule 17a-4(f) standard for electronic record storage.
The CAS query API is now automatically available for all CAS buckets.
Tags in the form of name-value pairs can be assigned to a bucket using the ECS Portal or the ECS Management REST API, enabling object data stored in the bucket to be categorized.
ECS object data is stored in chunks and chunks are broken into fragments based on an erasure coding (EC) scheme in order to improve storage efficiency.
Geo copy to all sites.
Read an ECS overview and architecture guide here [PDF]. ®
Are you risking security with the cloud?