Why Understanding Dark Data Is Essential to the Future of Finance

“Water, water, everywhere, nor any drop to drink.” The famous line from Samuel Taylor Coleridge’s epic poem “The Rime of the Ancient Mariner” has a fitting application to today’s data problem. Enterprises are deluged with data, but they often have no way to leverage it. According to most experts, only a small percentage of data is usable and made useful, and most of it is in the dark — thus the term, “dark data.”

With more data flooding into businesses than ever before, how can enterprises manage that data to improve performance? How do data scientists, CIOs and enterprise data managers shed light onto the dark data so that it’s usable and useful? To answer these questions and others, during the Hitachi Financial Services Summit 2021, I spoke with Dennis Versteeg, the chairman of RegTech Innovation Board at ING, a global bank based in the Netherlands. Dennis explained how he thinks of dark data, how it relates to regulatory technology (regtech) innovation, and how financial services organizations can illuminate it to make it useful.

What Is Dark Data?

First, Dennis clarified that dark data is not “criminal data” and should not bear negative connotations. Dark data is simply the data that is collected by an organization through its processing chains but is not actively analyzed, enriched, or used. Anything that is not structured — such as slide presentations, legal contracts, blogs, and emails — and lacks labels can be considered dark data.

This unusable data is still stored and managed by the enterprise, and that’s expensive. According to many estimates, 55% of data collected by companies is dark data, which means that more than half of their data storage and management costs are being expended on data they can’t use, or are not attempting to make usable. This is a global business problem that can be solved with the help of innovation — and specifically, RegTech innovation, says Dennis.

What’s the Connection Between Dark Data and RegTech Innovation?

RegTech innovation, Dennis explained, is the use of innovation to help solve the business problems centered around regulatory compliance while enabling growth. Illuminating dark data plays an important role in RegTech innovation because the insights hidden within dark data support both objectives. In fact, regulations are often the catalyst for banks to illuminate dark data, because satisfying these regulations demands a deeper dive into their comprehensive enterprise data.

Dennis described one recent example that demonstrates how the link between dark data and regulation is becoming stronger: It occurred a couple of years ago when traders were able to manipulate the Inter-Bank Offered Rate (IBOR), which is the rate at which banks borrow money from other banks. As a result of this manipulation, the European Parliament created Benchmark Regulation (BMR) to reform the publication and use of benchmarks such as IBOR, which are being phased out. Whereas IBOR and other such benchmarks had relied on expert judgment, BMR benchmarks must focus on underlying transactions. This means that for the financial industry, information that exists but is “dark” must now be brought forward to feed the new benchmarks.

Five Ways in Which Dark Data Will Change the Future of Finance

Dennis and I agreed on five dark data trends to watch:

• More companies will realize the value of illuminating dark data, not just for regulatory requirements but for all business functions — from compliance to customer satisfaction. Exposing dark data brings benefits the entire spectrum of the business.
• Organizations will become better at realizing the potential of dark data as more than just raw or unusable data. With the right innovation, dark data can become business-as-usual data.
• Let’s dismiss the adage that data is the new oil. Rather, data is the new sunshine — unlimited in its supply. Organizations can start thinking about enterprise dark data in terms of storage, discoverability, utility, and the monetization of the data, itself.
• Financial organizations will realize the value of DataOps — the rigorous discipline that brings all these principles together under one holistic, complete conversation.
• And finally, financial businesses that begin illuminating dark data will begin to create a virtual cycle in which vast amounts of usable, useful data can give rise to evermore innovative business applications and functions. This effect, in turn, will generate more data waiting to be unlocked — spurring an infinite progression of illumination and business value.

You can neither drink the ocean, nor make use of every single bit of data you collect. But you can minimize dark data in your organization by bringing innovation to your financial organization. Hear more from our Hitachi Financial Services Summit conversation in the on-demand session and learn how the financial sector is interacting with dark data.

Saker Ghani is Senior Director of Financial Innovation Division at Hitachi Vantara.