Half or 52% of all information currently stored and processed by organizations around the world is considered ‘dark’ data, whose value is unknown, reveals the results of the “Global Databerg Report” released by Veritas Technologies.
Additionally, another 33% of data is considered redundant, obsolete, or trivial (ROT) and is known to be useless. If left untamed, this dark and ROT business data will unnecessarily cost organizations around the world a cumulative S$4.6 trillion to manage by the year 2020.
Organizations are creating and storing data at an ever-increasing rate due to a ‘data hoarding’ culture and an indifferent attitude to retention policy. This data could be anything from valuable business information to non-compliant information.
The report reveals that IT leaders consider just 15% of all stored data to be classified as business critical information. For the average midsized organization holding 1,000TB of data, the cost to store their non-critical information is estimated at more than S$900,000 annually.
“Understanding and acknowledging that a data hoarding culture exists is a first step in addressing the problem,” said Chris Lin, Veritas’ Senior Vice President & APJ Sales Leader. “There is an immediate need for organizations in Singapore to take control of their Databerg and identify business value and risk. Today, only 14% of organizations in Singapore do so. Data needs to be classified based on the organization’s data retention policy and there is a growing demand for an effective information journey for dark data to be implemented.”
The Veritas “Global Databerg Report” provides insights from over 2,500 IT professionals in 22 countries, including 100 from Singapore. It follows the company’s introduction of the “Data Genomics Index,” the industry’s first accurate view of the composition of enterprise data based on analysis of billions of files.
The Vast Majority of Business Data Sits Below the Waterline
Around the world, the report found that on average 52% of all stored data is either dark or redundant, obsolete and trivial (ROT). The worst dark data offenders are Germany, Canada and Australia with respectively 66%, 64% and 62% of their stored data defined as dark, leaving USA and Singapore in a mid-range position at 54% and 53% of their data being unknown respectively.
The highest proportion of clean and identified business critical data was found in China (25% clean), Israel (24% clean) and Brazil (22% clean). But this still means that more than 80% of all data they are storing is dark or has no value for the business.
A Fear of the Delete Key
ROT data has already been identified by organizations as redundant, obsolete or trivial and provides little or no business value. But still 48% of all stored data in Denmark, 44% in the Netherlands and 43% of all data in the United Arab Emirates (43%) belongs to this category. In the USA, 30% is ROT data. At 38%, Singapore’s ROT data is 5% worse than the global average of 33% and dark data is 1% worse.
The Race to the Cloud is Feeding the Databerg
Cloud adoption and processing is set to increase by more than a third during 2016 to 46% from 33%, with Brazil and USA leading with an average of 61% of data predicted to be in the cloud by the end of the year. Although the short term driver is to reduce cost, there is increasing concern over lock-in costs in the future – pushing data to the cloud can just move the problem further away, adding to the dark, and unclassified dark data.
Consumerization of IT Has Blurred the Lines for Employees
Hitching a ride on corporate IT means businesses pay to store data that has no business value to them. On average, 26.5% of employees store personal data on their work devices. Singapore was ranked the lowest in terms of employee discipline in adhering to corporate data policies, with 39% of employees storing data on work devices. Because so much is dark, IT cannot tell which data has business value and which is just ‘cat videos’.
More Personal Data Than Ever Exists on Corporate Networks
Not understanding what is being stored on corporate resources is very risky and leaves no ‘plausible denials’ in the event of regulatory or criminal investigation. In Singapore, employees using corporate networks for their personal use is growing, leading to more types of files such as personal legal and ID documents (77%), photos (75%) or non-approved software (65%) being stored at work. It may seem innocent at first, but many of these documents may trigger new data privacy rules in regional jurisdictions or potential copyright issues.