- April 22, 2020
- Bill Tolson|
- Archive Migration|
- Office 365|
- Regulatory Compliance|
- MiFID II|
- live journaling
Nobody will tell you that migration is easy. In fact, for many, moving your on-premises data to the cloud with API based migration providers like TransVault is a costly, time-consuming, and downright painful process. How did we get here? The downtime, the spiraling fees, and the lack of compliance? Why can’t cloud migration be as you would expect: fast, non-disruptive, cost-effective, and compliant?
Every year, billions of dollars are spent on migration, with recent reports suggesting an average cost of more than a million dollars for the migration platform alone. And that’s without any challenges occurring. Should issues crop up during migration, those costs can quickly grow (and in some cases, skyrocket), to the point where the positive, cost-reducing, efficiency-enhancing benefits of the cloud, those that triggered the migration in the first place, are lost in a sea of resource and financial commitments.
Yet, for those operating in an on-premises environment, cloud migration is still a step worth taking, however significant it might be. The future is based in the cloud and those that fail to move with the times risk being left behind, but that doesn’t mean you have to take the plunge without preparing your organization for the challenges migration might pose. The key is to understand those potential pitfalls and make moves to mitigate them before they become a real problem.
If you’re considering your options when it comes to cloud migration, developing a strategy to overcome challenges might be the single most important job you do. It will enable you to be prepared for common issues and question what API based archive migration providers like TransVault might tell you. During your initial research, you’ll discover a number of companies that offer cloud migration services and, on the whole, will do exactly what they set out to do – move your data to the cloud. It’s what happens before, after, and in between that you need to be mindful of…
The old adage ‘you get out what you put in’ holds true when working with API based migration providers like TransVault. Unless you can guarantee that your archive’s index, metadata, and other files are practically perfect, with none of the degradation and corruption that naturally occurs to most over time, you’re at risk of leaving data behind as well as migrating duplicate and redundant files. These problems are often made worse by most providers’ reliance on the source archive’s API to connect and move data to the cloud, which can lead to moving even more corrupt data to your new location. For any migration, moving more data than necessary is something to be avoided given that the total number of files moved and stored will determine the price you pay.
It also impacts the speed of transfer, requiring months or even years to shift several tera or petabytes of data, and puts a strain on your network that impacts day-to-day business and could even break the archive, leaving your migration in limbo. You also need to consider how your current archives operate and what you intend to transfer. Are you making use of Exchange Journaling?
Or are you working with another vendor such as Mimecast, Enterprise Vault or SourceOne? It’s important to understand your chosen migration provider’s level of competency when it comes to the systems you currently use and definitely worth asking if they can show you examples of previous migrations.
Common cloud migration challenges when migrating via API:
It’s factors like these that you should be clear of when you come to planning your migration and working with a provider like TransVault.
1. How they approach the migration and access your data
2. How long they estimate it will take
3. Whether they can provision the migration in the cloud rather than use your network
4. If they can accurately predict the amount of cloud infrastructure you will need
Having answers to these questions up front should help you avoid some of the more common migration headaches and enable you to see a clearer picture of the process ahead.
So, now you know some of the challenges you might face, let’s look at how API enabled migration providers like TransVault tend to talk about these problems. In most cases, their platforms follow traditional processes that inevitably lead to these issues occurring and require obtrusive techniques and workarounds to overcome them.
API email archive migration vendors will likely tell your IT teams that their system will need ongoing access to your network. This will slow down overall system performance while data is discovered and could take a long time overall. The fact is, this bandwidth hit comes from extracting data the traditional way, via the archive’s own API. While your provider might not be able to work in any other way, alternatives do exist that both speed up the data transfer and minimize the impact on your network.
According to many migration providers, a bottleneck in the storage volume is the source of slow data transfer speed. In reality, it’s the use of an inefficient process on their part that causes speeds to suffer. While your on-premises archive only stores a single copy of an email, most API migration platforms create duplicates for every user related to it. Let’s say somebody sent a simple email to five people within the business. The original message would weigh in at a megabyte or so but, thanks to your provider duplicating the message for each associated mailbox, it becomes five messages for a total of five megabytes. While that single example doesn’t sound too bad, imagine scaling that type of duplication for your entire organization. In the blink of an eye, 40tb of data across 3,000 users becomes 400tb, slowing down your migration to a crawl while ramping up the cost.
Many people will tell you that archive migration takes longer than mailbox migration and that archive migration has an impact on moving mailboxes to Office 365. As a result, mailbox migration is often postponed or carried out in stages to allow the archive migration to catch up. This could take the form of migrating several hundred users a week before migrating the archive to Office 365 and takes an extraordinary amount of time due to bandwidth limitations and Office 365 throttling.
Your migration provider might tell you that chain of custody and data fidelity remains when using Exchange Messaging Service (EMS) and pushing data into Office 365. Unfortunately, it doesn’t. If you don’t take steps to address this issue ahead of your migration, you will encounter significant processing problems down the line. Most migration providers use the EML file format, created by email software like Mail and Outlook, to save your data. Not only does this create processing overhead, meaning the activity takes longer, it also leads to a loss of data fidelity as a result of the conversion. This is an important question to ask your provider when considering your migration. If you don’t, legal and compliance teams may be forced to deal with data loss after the migration is completed rather than get out in front of the problem.
It might feel like there are more problems than solutions when it comes to migration, but there is an alternative approach. While it’s clear that there are many pitfalls that can hamper your migration, both before and during the process of moving your data, most of them will be linked to your provider’s dependence on the archive’s API to get at your information and transfer it. That’s why Archive360 avoids the API layer entirely and, in doing so, removes the fear of the unknown, mitigating unexpected issues and cost.
With a lift and shift approach that extracts information directly from your archive’s data layer and moves it ‘as is’ to the cloud, not only is the speed of transfer improved, but the reliance on indexes is removed entirely, providing a far clearer picture of your archived data. This method also means there’s no need for any work to be carried out on your archive before it reaches the cloud, limiting nasty surprises such as file duplication. If your existing system contains 40 terabytes of data, we move 40 terabytes to the cloud before indexing and classifying it there. That data may then become 400 terabytes once in the cloud and pushed to Office 365 but, at that stage, the damage that duplication can do during migration has been eliminated, helping you to save time, increase efficiency and reduce complexity, not to mention infrastructure costs.
Archive360 helps its clients to make the most of Azure before the migration begins in order to bypass such complexities and potentially migration-stalling oversites.
How we do this depends on the customer’s current situation:
Organizations without an existing Azure tenancy are asked to establish an Azure storage account or allow us to do so for them, and provide us access to it.
As we’ve already mentioned, Archive360’s approach avoids APIs and goes straight to the source of the data in your archive, meaning less disruption and migration time, no API or speed restrictions and no risk of issues from damaged databases and indexes. We get in and get out while ensuring all of your data is moved seamlessly and efficiently.
With just read-only access to your archive required, Archive360’s FastCollect can collect and migrate more than 80 different types of structured, semi-structured and unstructured data and can be installed and operational in less than an hour. Once up and running, there’s minimal impact to your existing infrastructure and no software is installed on the archive servers, ensuring complete adherence to your existing archiving software support and maintenance agreement.
From there, a single sequential read of all your raw storage data and metadata is carried out at a rate of 5-10 terabytes per day for optimal efficiency. FastCollect logs all data collections, which can be included in a chain of custody report for compliance purposes and guarantees data encryption. FastCollect also generates a data discovery report to provide an exact figure on the amount of data to be transferred.
While API migration vendors may want to sit, pervasively on your network while data is captured, Archive360 significantly cuts down the time spent on the initial stages of your migration, enabling you to begin decommissioning your legacy systems sooner, starting the process of shutting down on-premises infrastructure the moment discovery is complete.
Archive360’s email archive migration solution avoids any form of data processing while the data remains on your legacy infrastructure. Instead, after discovery, we copy the data in bulk to the cloud to provide you with complete clarity on the amount of information you’re migrating, while ensuring you retain ownership over it and its encryption keys when it is moved from the cloud into Office 365.
Unlike others, no decompression or expansion of data occurs, everything is migrated exactly as it exists no your archive storage. If the data is stored compressed by the archiving system, it is transferred as such. If the data is stored on EMC Centera, the raw C-Clips are read and transferred in that format.
We pride ourselves on ensuring there are no changes made to your data, be it proprietary formats or wrappers, throughout your migration.Not only does this make your life easier in the long run, but it also enables you to do more with your data once it’s in the cloud, including detailed eDiscovery searches and cutting-edge analysis. It also means you don’t have to pay additional fees to convert your data back to its native format, as would be the case with many providers.
It's only when your data reaches the cloud staging phase that the SQL metadata is processed, and raw data is validated. This is also where an Archive Insight Report is generated to explain the total number of messages and the size of the user archive, as well as the total amount and size of items segregated per archived mailbox. It may reflect users, shared mail, public folders, and journal archives provides a crucial level of insight, especially for determining the number of inactive users before moving data to Office 365. On average, up to 60 percent of total users within a system are inactive, so correct handling of such users will be essential to avoid issues as you transition to Office 365.
Archive360 makes it easy to limit the network impact of data transfer, including:
Whereas some providers waste your time by intermittently moving mailboxes ahead of the archive’s migration, Archive360 adopts a staged migration. Because we’ve moved a copy of the data to the cloud before the active user migration begins, your IT team is able to leverage its full network bandwidth to migrate mailboxes at speed. Our migration consultants work with you to establish a migration plan and will execute it on your behalf.During this process, Archive360 will validate each item to ensure no corruption exists and flags any items that do not pass validation.
While some migration providers might use EML files to save email data, Archive360 avoids it, keeping your data in its native format and using the MAPI protocol to push data into Office365. This means your data isn’t converted or throttled as much as it would normally be and prevents any loss in data fidelity. And, once the migration is complete, we provide you with a final migration report, which includes a Legal Chain of Custody report. This document is crucial for legal defensibility, ensuring you can account for the movement of every message in your original archive to Office 365.
We understand how daunting a prospect cloud migration can be. Each of the thousands of successful migrations carried out by Archive360 have taught us something, and we’ve gathered that learning into a unique, powerful and dedicated migration solution that can overcome many of the major issues companies face on their journey to the cloud. And we don’t just help you to get there either. Archive360 can enable your business once it’s in the cloud, helping you to realize major cost savings while accessing powerful tools for analysis and compliance.Our managed solution helps you optimize your storage for long term retention and enhance your security and regulatory policies. We help with data management too, ensuring you can make the most of efficient litigation support services and enjoy the same if not better levels of compliance as you experienced on-premises.
Why not reach out to one of our representatives to discuss your cloud migration options with Archive360?
Archive 360 is trusted by companies like 3M, Netflix, McKesson, Samsung to migrate from Enterprise Vault and other major archiving platforms.
Your legal, compliance and security teams rely on having an immutable copy of all of your emails. Office 365 archiving does not support journaling. So what should we do?
This eBook provides actionable tips to empower IT to solve the problem.