Why SharePoint Online is becoming an essential solution for successful cloud adoption

One of the most significant issues associated with total cloud adoption for many organizations is what to do with the terabytes (or more) of data stored on network share drives sitting in corporate data centers. SharePoint Online can act as an ideal file share repository that’s easily accessible for end users. However, moving share drive data to SharePoint Online presents one obvious challenge – available SharePoint storage resources. This blog discusses how organizations can maintain and manage the costs of their expanding cloud data – without disrupting end users and addressing data privacy requirements...

Digital transformation is a huge topic encompassing many technologies and processes. However, a key technology for successful digital transformation is the move away from the complexity and costs of individual corporate data centers to complete cloud adoption.

The main market drivers for this move include the ongoing growth in the creation and consumption of corporate data, a continued dispersed workforce due to COVID-19, rising demand for additional next-gen capabilities - including AI and machine learning, better planning capabilities, reduced costs, higher data security, and superior disaster recovery and backup protections. But, a complete move to the cloud is not as straightforward as moving major enterprise applications to a cloud platform.

“The amount of digital data created over the next five years will be greater than twice the amount of data created since the advent of digital storage."
 ...Dave Reinsel, Senior Vice President, IDC

Big Growth Forecasted for Big Data January 2022

A new inflection point has emerged driven by new data privacy laws. The new laws require data collectors to respond to data subject inquiries about their PII including what PII has been collected about them, how it has been used and if it has been sold. Additionally, the data subject can ask for their PII to be completely deleted – no matter where in the organization it is stored. These new data privacy rights will now make it impossible to fully respond to a data subject access request if the organization does not manage all data within the company, not just records, including that data held and managed by individual employees on their local computers. But first, let's first look at the main drivers for the move to the cloud:

1. The ongoing COVID-19 pandemic has shown the need for organizations to design and adopt systems that are easily and securely accessible by all employees, no matter their work location, to ensure data is available to the entire enterprise for data sharing, analytics, and regulatory/eDiscovery processing - not locked up on individual laptops and workstations.
2. Organizations are discovering the increased value of artificial intelligence/machine learning capabilities such as content/context-based data auto-categorization, intelligent and adaptable data security, and data content-based sharing based on interests.
   
   However, the inclusion of AI/ML into a corporate data center is costly, takes a considerable amount of human expertise (data scientists, etc.), and is difficult and time-consuming to keep up to date. The largest cloud platforms, Microsoft Azure, Amazon AWS, and Google Cloud, have huge teams of specialists and programmers who continuously add new AI/ML capabilities to their technology stack, enabling cloud application developers to adopt the new capabilities into their solutions rapidly. Additionally, non-cloud applications are not being actively developed in many cases due to the move to the cloud.
3. The cost of maintaining corporate data centers versus compute/storage needs and scalability can be mismatched and lag the actual demand. A prime example is how data center capabilities are estimated and purchased. Usually, the IT organization must estimate their storage and compute needs at least one year in advance; how much additional storage will the organization need and at what performance level – tier 1, tier 2, tier 3, the number of additional servers that will be needed, which applications will be required, the annual hardware and software support costs, and how many new IT employees (with what new disciplines) will be needed. And what if the IT group underestimates or overestimates their needs?
   
   
4. Data security is the biggest unknown in today's corporate business environment. Organizations are continuously attacked by malware, cyber-hacking, ransomware, and extortionware. Corporate data centers usually rely on a small number of IT employees to keep the enterprise safe by purchasing new security software, applying security patches, and monitoring the network. On the other hand, the biggest cloud providers employ thousands of enterprise and data security specialists and programmers to keep their platforms safe from the newest threats - applying patches immediately. At the same time, many corporate data center IT organizations are often unaware of the latest security threats and often fall behind in applying new security patches.
   
   
5. Cloud-based DR and backup creation and management can be automated and provided with higher security capabilities, such as encryption and immutable storage, to protect against ransomware backup deletion. Cloud-based geographic data redundant storage can ensure your data is always available across one or more locations.

Roadblocks, Speed bumps, and SharePoint

The total move to the cloud is not without its challenges.

One of the most significant issues associated with total cloud adoption for many organizations is what to do with the hundreds of terabytes (or more) of data stored on network share drives sitting in corporate data centers. To be clear, a share drive is a network drive that enables employees to search, access, and share files between computers and other devices. A share drive (or file share) was also called a "home" drive decades ago.

An individual or a team can own and manage online share drives. An individual employee will store important files on the (enterprise) share drive to ensure their ongoing availability in case their individual laptop crashes.

Many organizations still utilize share drives to provide their employees additional storage space in case their laptop storage becomes full.

Microsoft 365 is the dominant office productivity suite across the globe. Microsoft 365 is a SaaS-based cloud solution with a large, growing suite of interconnected applications. Two of the most important applications for storing content within Microsoft 365 are OneDrive for personal file storage and SharePoint Online.

In the past, SharePoint and SharePoint Online has been primarily sold as a document management and storage system, but its usage varies substantially among organizations. SharePoint can also act as a centralized hub for storing files in one easy-to-access and shareable location. However, two SharePoint Online capabilities have emerged as necessary in a corporation's digital transformation journey to answer the question; what to do with the hundreds of terabytes of data stored on share drives sitting in corporate data centers.

Many organizations have discovered that SharePoint Online can act as a file share repository accessible from an employee’s Windows PC File Explorer application. There are two ways to access SharePoint files from File Explorer; drive mapping and Sync.

Drive mapping provides the illusion of data being stored locally on your computer but is instead still stored on a SharePoint (on premises) network drive somewhere else in the enterprise data center. However, drive mapping is an older technology which is slower and less reliable than syncing and relies on a SharePoint instance in the data center. One benefit of drive mapping is that multiple users can share access to the same files. This means data can be shared among co-workers without the need to send emails back and forth when content/files are updated or changed. SharePoint drive mapping is also an excellent solution for moving the data in large file shares into SharePoint Online while making all directories and files available to the entire employee population.

The downside to SharePoint drive mapping is that the share drive is not viewable when the laptop is not connected to the enterprise network. Also, drive mapping is not a viable solution when using SharePoint Online. Because of this fact, SharePoint drive mapping has fallen out of favor.

Instead, many organizations have chosen an alternative – SharePoint Online Sync. With this capability, you can sync your Microsoft SharePoint data to a local folder – keeping a copy on your computer for access at any time. This lets you work directly in File Explorer and access files even offline. And when you reconnect to the enterprise, any changes made to those files will sync automatically. Many organizations use this capability as a real-time offsite backup of all employee work files. This sync functionality also enables the central IT organization to search across all employee work files when responding to eDiscovery requests.

The new information management inflection point – data subject privacy rights

With that in mind, there is a new use case that this centralized sync functionality addresses – providing IT and corporate legal teams with direct access and search capability when responding to data privacy-inspired data subject access requests, also referred to as DSARs.

New data privacy laws now make companies that collect personally identifiable information (PII) responsible for reporting on, updating, and deleting PII when requested by the data subjects. In most cases, data subjects have the right to ask an organization (via DSARs) what data has been collected on them, how it's used, who it's been sold to, and how long it will be kept. Data subjects also have the right to have their data deleted if there is no regulatory or legal reason to keep it - for example, if the PII in question is or could be involved in eDiscovery.

Under these new privacy laws, companies are responsible for tracking and managing all data that could contain PII, including data stored on employee devices that, until now, companies did not have direct access to. The data subject rights are absolute, meaning that a company that receives a PII deletion request cannot give the response a cursory "best try." All data privacy laws expect a complete response to the request – all PII for that individual MUST be found and deleted, no matter where it's stored, without exception. How many emails are sent containing PII? How many spreadsheets have been created and shared containing listings of individuals' PII? How much of this sensitive data is stored on employee devices?

Currently, the IT or legal departments either search their enterprise systems for the PII in question - but usually ignore employee-controlled data. Or they send out an email to all employees asking them to search their local devices for the PII in question.

It should not be surprising that most employees are not experts in search technologies, especially across the many applications and repositories they manage (think about employee-created PSTs). How comfortable should a corporate General Counsel be in the accuracy of employee PII searches? The new data privacy laws include sizable fines and penalties for organizations unable fully respond to data subject access requests.

I have written and spoken about this use case as the new inflection point for corporate information management. In the future, all corporate PII will need to be centrally accessible, including data stored on employee-controlled devices.

This brings us back to the question of digital transformation. Companies that use Microsoft 365 now have a viable solution to the on-premises file share challenge – SharePoint Online Sync.

When you sync a SharePoint library to your computer, the synced files are available to work with in File Explorer. This allows the user to find the files they are looking for using a user interface they know, without the need to be connected to the network. Each time a SharePoint Online-based file is opened and edited locally from Windows File Explorer and then saved, all changes are automatically synchronized back to the SharePoint Online cloud.

The SharePoint Syncing feature also potentially addresses the inflection point I described earlier in the blog – the need for IT to have search and management capability over ALL employee corporate data due to the regulatory requirements posed by new data privacy laws.

As a reminder, all PII must be reportable and delete-able when requested via data subject access requests. Again, this means that all PII, including PII stored on individual PCs, must be searchable by IT. The SharePoint Online syncing capability with PCs ensures IT can search all data in SharePoint to determine if the subject PII exists in employee files, thereby keeping the organization compliant.

However, storage is an issue

As I mentioned in the first part of this blog, share drives tend to be catchalls for employee, department, and organizational data that can reach tens or hundreds of TBs of storage. Moving share drive data to SharePoint Online presents one obvious challenge – available SharePoint storage resources.

SharePoint Online allocates storage resources based on the particular subscription terms and the number of users. The included SharePoint Online storage allocation is 1 TB plus 10 GB per licensee. This storage is included and available to all site collections in the Office 365 tenant and serves as a central pool from which everyone using SharePoint Online can draw. SharePoint Online storage can be increased to a total of 25 TB per site.

In many cases, the included storage limit will be exceeded quickly, requiring additional SharePoint Online storage to be purchased – and it's not cheap! In fact, it's currently priced at $0.20 per GB per month., i.e., an additional 10 TB of storage would cost an additional $2,000 per month or $24,000 per year.

Another complicating factor with SharePoint Online is how files are saved. Versioning creates a historical record of all changes, with the date/time and indication of the user who made the change, on a per-file/list item basis.

The concept of versioning is by default enabled in SharePoint, meaning that potentially every edit to a given file in SharePoint creates a new version while keeping older copies available for reference or reuse. The end user can view, delete, and restore a version if they have the correct permissions. The versioning setting can be edited to your company's specific requirements, but the default setting is set to 500 versions.

Depending on how your SharePoint Online versioning is configured, a great deal of available storage resources can be consumed quickly, causing you to need to buy additional SharePoint Online storage at the previously stated $0.20 per GB per month.

Extend SharePoint Online Storage Efficiency with Archive360

I've already pointed out that SharePoint Online storage consumption is a growing issue for companies of all sizes. With the move to the cloud, the adoption of SharePoint-based file shares, end-user file deletion pushback, and versioning are leading to increasing storage costs and performance issues.

Archive360 Connect for SharePoint Online helps companies automatically extend SharePoint storage to their much lower-cost Azure cloud tenancy storage by setting up policies within SharePoint Online to automatically move files that meet specific policies to their Azure storage. 

For example, policies can be created that automatically move inactive or aging files to Azure storage while replacing the file in SharePoint with a link (or pointer) to its location in Azure. To access an "extended" file, end-users simply need to click on the link in File Explorer to access the file. All extended files can be searched and accessed directly from within SharePoint.

There is zero impact on end-users because they have continued seamless access to their extended SharePoint files directly from SharePoint Online or, if configured, from File Explorer. Additionally, all extended file names remain the same. Having fully synced individual file share folders in SharePoint Online addresses the data privacy law DSAR issue I previously talked about. This capability solves the problem of legal or IT not having searchable access to local individual employee files when responding to DSARs. It also makes eDiscovery search much faster and less costly.

With the continued move to the cloud, companies are looking for ways to dump their on-premises file shares while retaining the much-beloved use model. However, many who have pursued this strategy have quickly faced the SharePoint Online storage issue. Archive360 Connect for SharePoint Online is the key solution to enable companies to finish their move to the cloud successfully.