The Changing Role of Object Storage in High-Performance, Tier-1 Workloads

The concept of object storage was introduced in 1990s, and since then it has evolved into a mainstream technology of choice for unstructured data. Today it has become the cornerstone for storing, protecting and managing unstructured as well as semistructured data. Humans and machines create 90% enterprise unstructured and semistructured data, most of which remains dark data.

This data is growing so fast that, as per Gartner® Critical Capabilities report: “By 2024, large enterprises will triple their unstructured data stored as file or object storage on premises, at the edge or in the public cloud, compared with 2020.”^[1]

IDC echoes the prediction of unstructured data growth, estimating that 80% of the data generated worldwide will be unstructured by 2025.

Object Storage Through the Years

First-generation object storage solutions were focused on data protection, archival and compliance. The second generation moved the focus to hybrid cloud, Amazon Simple Storage Service (Amazon S3), metadata enrichment and so forth. But today, object storage has gone way beyond these areas and is catering to almost all Tier-1 workloads.

Object Storage Through the Years

Digital transformation has accelerated adoption of cloud, social media, mobility, IoT and big data. As a result, many new data sources and modern application workloads are getting discovered and are driving object storage consumption. Applications such as ultra-high-performance artificial intelligence and machine learning (AI/ML) workloads, IoT and unified data warehouses are leading this race.

As analytics usage increases, organizations want faster performance to speed data-focused insights that can inform business decisions. A high-performance object storage solution can enable additional modern and cloud-native applications to gain the benefits of its scalability, fast data retrieval, and cost effectiveness.

Modern Data Platform: Navigating these challenges requires support for multiple use cases with visibility and control across all data. A modern data platform enables easy management of volumes of data, seamless response to application demands and wide data accessibility, while satisfying compliance requirements. And hence, a modern data platform must have following characteristics/pillars:

Pillars of the Modern Data Platform

Traditionally, organizations have turned to object storage for highly scalable, cost-efficient, long-term storage with easy data retrieval — but not for performance. But today, organizations need to use all their data, rather than storing some data stored in slow silos, where it is available in an emergency but not for production applications. Too much insight is available from data lakes, content repositories and email archives for organizations to waste money storing it without usable performance.

Demystified: Object Storage and Performance

Object storage of the past was synonymous to a low performance and high-capacity archival solution. But that isn’t true anymore. Modern object stores, while supporting traditional applications with economical and massively scalable storage, have transformed into a storage solution for high-speed, Tier-1 workloads.

Next-generation object stores such as Hitachi Content Platform (HCP) allow you to add all the custom metadata you need and have native Amazon S3 storage compatibility. However, they also support workloads, from real-time log aggregation to IoT, and transaction logs of high-performance databases. Here are some examples of how our customers have benefited from the HCP next-gen object storage:

Object Storage for High-Performance Workloads

With new revenue-generating opportunities and workloads reliant on unstructured data, it’s time to maximize your infrastructure advantage with object storage.

Amit Deshmukh is Solutions Expert, Content and Data Intelligence at Hitachi Vantara.

^{[1] Gartner, Critical Capabilities for Object Storage, Chandra Mukhyala, Julia Palmer, Jerry Rozeman, Robert Preston, 21 October 2020.}

^{Gartner disclaimer:}

^{GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.}