Planning a migration to Microsoft 365 isn’t easy at the best of times. Not only do you need to minimise disruption and train staff on the new platform, but you also want to use the opportunity to introduce improved ways of working. However, migrations are even more complex when records management capabilities and requirements are considered.
This article is designed to help organisations who are planning migrations in records management scenarios. For clarity, I’m not planning on discussing any of the standard steps involved in migrating content into Microsoft 365. There are plenty of blogs that already provide detailed guidance around which migration tools to use, how to map content from source to destination and how to run a delta migration process.
Instead of re-treading the same ground, this blog focusses on some of the nuances of planning a migration involving records – specifically:
What do you need to consider when migrating into Microsoft 365 when automated retention is being applied?
How might you plan for migrating records that are mid-way through their retention periods into Microsoft 365?
How can you start a retention period retrospectively for migrated content?
Automated retention
Anyone who has been reading my blogs over the past few years will know that I strongly recommend automatically applying retention labels across your sites and teams. Microsoft provides a wealth of different ways to automatically apply retention to your files, including Trainable Classifiers, Sensitive Information Types and via keyword queries, but I feel that by far the most effective approach is to set a default and retention label on each library in your architecture.
Regardless of the approach you apply, once you have configured automated retention, you need to be especially careful when it comes to migrating content. You see, assuming that you have carefully planned your architecture to support capacity, business-as-usual activities and likely content growth, you should have no issues with standard ongoing usage. However, a migration is pretty far from standard usage, with thousands of files being uploaded into your architecture in very a short period of time. If you haven’t planned it carefully, the ramifications could be very serious.
It’s essential to consider the impact that retention could have on the files you are migrating into the tenant. There are lots of ways that things could go wrong – let’s consider just some of them:
Auto-deletion
There is a good chance that much of the content you are migrating into Microsoft 365 has been fashioned and stored over multiple years. Now imagine what will happen if you migrate this content into an architecture with retention labels or policies that automatically delete content a few years after they were first created or last modified. You won’t notice it immediately, but a couple of weeks after the migration Microsoft’s Data Lifecycle Management capabilities will start automatically deleting large volumes of your migrated content.
If this was indeed the intention, I would suggest that perhaps it would have made more sense to decide to exclude these files from the scope of the migration. However, if this wasn’t intended, then the chances are that unless you’ve communicated this carefully with staff, you’re going to have a lot of pretty annoyed users asking why their files have disappeared.
Overwhelmed reviewers
if you have configured automated retention labels that result in a disposition review process then you will also need to be very careful when migrating content into Microsoft 365. Your disposition reviewers might be easily able to cope with consistent small volumes of records that reach the end of retention and require their evaluation, however, if you migrate large volumes of content which reach the end of retention at the same time you could easily inundate your reviewers. If this occurs you could decide to make decisions in bulk, or try to spread decisions over time, but everything would have been far simpler if you’d considered the impact of retention when planning the migration.
Purview limitations
While most organisations won’t get anywhere close to the limitations of Microsoft Purview, Records Managers at some of the largest companies have already found that it is possible to exceed the maximum capacity of the platform. At the time of writing, your Microsoft 365 tenant has a maximum limit of 16 million items simultaneously either pending disposition or having disposition approved. You also have a limit of 16 million items being marked for automated disposition. See Limits for Microsoft 365 retention policies and retention label policies | Microsoft Learn for more information.
For most organisations, ‘16 million items’ is a limitation that they would never consider getting close to, however, the one time they might get there is following a migration of decades of legacy content. Again, my advice is to ensure that you plan carefully in advance of the migration to try to identify and plan to mitigate the likely issues.
Migrating existing records
The challenges of a records management migration aren’t limited to the way you’ve chosen to configure your Microsoft 365 tenant. Another scenario where care needs to be taken is when the content that you are migrating has already been managed as a record in the legacy system. In this scenario you are likely to find yourself needing not only to migrate the files and their associated metadata, but also work out how you can preserve existing retention periods.
Let’s say you have a record that is 8 years into a 12-year retention period. How can you migrate it from a legacy system into Microsoft 365, while still ensuring that it reaches the end of retention in 4 years’ time?
Your first challenge here will be to work out how you can extract remaining retention period out of your existing platform. This challenge will vary significantly based on the legacy records management software. Even once you’ve found a way of extracting existing retention periods, you will still need to work out how these periods can be applied to the content you migrate.
Your next challenge will be working out how you can re-apply the retention periods in Microsoft 365. I wish we had a simple option here, such as applying a retention start date to each file we migrate and using that as the trigger for the retention period. Unfortunately, if we need to preserve retention, our only option is to create labels that use one of the retentions triggers that Microsoft provides us with – something that even with the best planning will likely require some amount of pragmatism and compromise.
Starting retention periods retrospectively
Another challenge that I often see is frequently associated with the digitisation of physical records. What happens if you have boxes of records being scanned that you now wish to manage in Microsoft 365 for the remainder of their retention periods?
Let’s say that the files you are scanning were originally created as physical records 40 or 50 years ago and you need to be able to keep them for the remainder of their retention periods. Of course, in this scenario the scanned files will have metadata that relates to the date the scan was made as opposed to the date the physical record was created. We will somehow need to identify the correct start date for each scanned file – possibly by using OCR, or a content processing capability (such as the one found within the recently renamed ‘SharePoint Premium’ (formerly SharePoint Syntex) suite.
Your next challenge will be working out how you can apply retention labels to the files that have a start date for their retention periods that is in the past. The only sensible approach here is to make use of ‘event-based’ retention labels. When an event is triggered, you can provide a date in the past, which will serve as the start date for the retention period.
Imagine that a college wants to scan physical student records and migrate them into Microsoft 365, but wants to ensure that scans continue to be retained in accordance with the retention schedule. An event-based retention label is the best approach to meet this requirement, as it allows us to trigger the event that starts the retention period at a date in the past.
The image below shows how we could trigger the start of retention to begin in July 1995. All we’ve needed to do here is make sure that each record is tagged with the date that the student left the college, so that we can create an event for each group of departing students.
Final thoughts
Migrations are never easy, but migrations that involve records add significantly to the complexity. It’s essential that the team running the migration have sufficient time to carefully evaluate the ramifications of records management and plan accordingly.
Feel free to reach out to me if you have any questions or would like guidance around Microsoft 365 records management migrations.