- January 27, 2022
When is it OK to Delete Data?
Today's legal best practice is to delete records when expired and general data as soon as the data is no longer has value for the company - a 180 from where we were 10 years ago.
Learn More >>>
Data risk versus cost versus value
As was stated earlier, not all data retains its value over long periods. In fact, some data has zero value, ever. For example, emails suggesting going out for lunch lose most of their value to the sender/receiver as soon as the lunch occurs. Email systems are chocked full of transitory conversations that have no value for the company and could be deleted immediately without negatively affecting the business. However, for whatever reason, they are kept and then forgotten about – remaining in the user's mailbox for potentially years. But even the most critical data loses its value over time. That 2017 product forecast spreadsheet, or sales report, or draft of meeting minutes certainly is less important today than when it was first created and shared in 2017.
Chart 1 below shows the average data value loss over time
(data derived from the CGOC Information Governance Maturity Model).
Chart 1: The average value of data over time
Please note: data value is subjective and can vary dramatically from industry to industry. In some industries, data value may show less of a decline over time, for example, in the pharmaceuticals industry (also heavily regulated). Your organization will need to take a hard look at data over time to understand its average value curve. But on average, no matter the industry, data value does decline over time.
In the previous "Data is risk" section of this blog, I indicated holding compliance data beyond its regulatory retention period (or usefulness to the company) raises risk in legal and regulatory compliance situations.
Chart 2 below (also derived from the CGOC report) adds the data risk curve to the data value chart to highlight the apparent disparity in the two trends.
Chart 2: The risk to data value deficit
Chart 2 shows one of two variables to consider when creating a defensible disposition strategy: if the risk of holding (non-regulated/non-litigation related) data is more significant than its value, then that data should be considered for deletion.
The other variable to consider is that of the cost to store and manage the data. Retained data is stored on various enterprise-managed storage resources – hard disk, tape, cloud repositories. Additionally, floor space is taken up, cooling/electricity is consumed, the data must be backed up and replicated for disaster recovery (DR) requirements, additional personnel are needed, etc. These are all costs related to on-premises storage costs. The same holds for cloud storage, even though cloud storage is usually paid via a monthly subscription.
Generally speaking, storing 1 GB of storage becomes less expensive year over year. However, the resources to manage the storage do not. Over time, the fully-loaded cost of storing one GB of data will dip noticeably, but that cost reduction is replaced with the cost of storing more data (remember, we’re also generating more data).
Chart 3 below (also derived from the CGOC report) shows the average cost curve for storage over time overlaid with the value curve with the cost to value deficit.
Chart 3: The cost to value deficit
As in the data risk to value deficit shown in chart 2, a negative value to cost ratio highlights yet another reason to defensibly dispose of data – cost savings.
Overlaying all three charts into chart 4, we can clearly see that keeping (non-regulated and litigation related) data for long periods of time is a risky and costly practice.
Chart 4: Data cost versus data risk versus data value
As a general rule, organizations beginning a defensible disposition process targeting existing data stores can realize an average 40% reduction in redundant, obsolete, or trivial data (ROT), which translates to substantial cost savings and large decreases in the overall risk.
In 2012, CGOC Summit reported that typically 1% of corporate information was on litigation hold, 5% was classified as a record, and 25% had current business value. This means, on average, 69% of the information in most companies has no business, legal, or regulatory value and should therefore be a focus of defensible disposition.
ROT records management: More is no longer better when it comes to information
Controlling ROT with defensible disposition strategies should become a standard process for all organizations. At its most basic, an effective information management program should leverage both management and ongoing disposition initiatives to reduce ROT and data overload, improve operational efficiency, boost employee productivity, and increase corporate profitability.
Digital transformation and data migration
Digital transformation and transition to the cloud is the perfect time to undertake a defensible disposition process. As organziations continue to move away from their on-premises data centers and make the jump to 100% cloud computing, the question all organizations will face is what to do with their existing on-premises data stores – leave in place until the aging data expires or move it to the cloud.
Obviously, leaving it in place while keeping new data in the cloud is expensive and adds to overall complexity and cost – sometimes for many years.
A second strategy is to migrate all existing data to the new cloud platform for ongoing storage – again expensive and risky.
The third strategy is to move existing important data/records to the cloud while culling the ROT during the migration process. This third strategy can be accomplished in either of two ways: manually culling the on-premies ROT before the migration while just moving the important data to the new cloud or; migrating it all and utilizing a cloud information management/archiving platform to cull and delete the ROT programmatically once in the cloud.
The second alternative, cull the data programmatically, makes the most sense in that it can be automated and therefore consistent. For example, once the cloud archiving platform is configured and ready, corporate data retention policies can be created based on date range, owner, department, keywords, etc. Once the data is migrated and retention policies assigned to each migrated file, data not meeting the new retention policies would be automatically tagged for disposal. Optionally some or all of the data tagged for disposal can be reviewed prior to destruction.
This automated defensible disposition process of legacy data during a data migration can be done much faster and consistently while also producing full defensible reports on the process and specific files tagged for disposition.
Archive360 is the world's leader in secure data migration and intelligent information archiving and management. The Archive2AzureTM solution is a complete cloud-based information management and archiving solution for both structured and unstructured data installed in your company's Azure Cloud tenancy for increased security and functionality, ongoing customization, and complete control. Unlike SaaS archiving platforms with limited information management capabilities and one-size-fits-all archiving application and security configuration, the Archive2Azure PaaS solution is implemented in a Zero Trust model, so that you migrate and store your company's data in your Azure tenancy with complete control over granular retention/disposition and security capabilities, including the ability to encrypt data on-premises before movement to your Azure tenancy – while keeping your encryption keys local.Archive2Azure includes intelligent unstructured/structured data migration purpose-built with legally defensible data disposition in mind.