Let’s face it, our ability to manage records in Microsoft 365 has improved substantially over the past few years. With a stream of enhancements to the way that retention labels work, we are steadily moving to a point where the platform had replaced the need for dedicated record management solutions in many organisations. We have a wealth of new capabilities that simply weren’t possible a few years ago, from being able to manage records inside Exchange, to having the ability to automatically adapt our retention rules as people change department, role or location (via adaptive scopes).
That said, the records management story within Microsoft 365 is still far from perfect – there are multiple gaps that I’m sure will be addressed over the next few years. Many of us will have our own lists of features that we’d like to see enhanced, for example, the ability to manage records in aggregations, or being able to mark content for transfer to archives as part of a disposition review. I could easily list multiple retention features that I’d like to see introduced, however, instead of focusing on quantity, in this post I’m going to concentrate on the two areas of Microsoft Purview Records Management that I feel present the most significant opportunities for enhancement: the Preservation Hold Library; and the principles of retention.
The immutability of the Preservation Hold Library
If I could only influence Microsoft to change one single aspect of their retention capabilities it would be in the area of the Preservation Hold Library (PHL).
Each SharePoint site that is using relevant features will have its own Preservation Hold Library, so as you can imagine there could easily be thousands of separate PHLs across a tenant. The PHLs are hidden from most users but can be easily found by administrators.
I get the impression that the PHL was originally designed to be used in the context of eDiscovery and litigation holds. In this role, I think it’s wonderful. When content that is subject to a hold is modified or deleted, a copy of the original version is automatically stored in the PHL. Importantly, once in the Preservation Hold Library content is fully immutable - even your administrators will not be able to make any changes or delete any files. This is exactly what you need when applying a litigation hold.
While perfect for eDiscovery, the PHL is, however, also used by the Microsoft Purview Records Management features – a role I feel it doesn’t excel in. You see, unlike eDiscovery cases where content must not be deleted or changed, records managers will sometimes need to ensure that content is no longer being retained by the organisation – something that the PHL actively prevents. Content stored in the PHL is still in the possession of the organisation and as such needs to be included within information rights requests. Certainly, in the UK, relevant content in the PHL needs to be included when responding to Freedom of Information or Subject Access Requests.
Imagine the fairly typical scenario where a site is created with a 10-year retention policy applied to it. Now assume that there is a legal obligation to delete a specific file stored within this site – perhaps it contains PII or is subject to GDPR’s right to be forgotten. The natural reaction of most users would be to delete the file – but this would be a mistake! While the user will think that they have successfully deleted the file, behind the scenes the retention policy will ensure that a copy is routed to the Preservation Hold Library. Of course, once in the PHL, you cannot delete it; as such you’re now forced to keep a file that you are legally obliged to delete for the remainder of the retention period.
The conclusion that I’ve reached is that, while superb for litigation holds, any retention use of the PHL is incompatible with legislation such as GDPR. Sure, in theory you could train staff to avoid deleting content that they need to get rid of – but I don’t think that’s realistic for most organisations. Instead, I would suggest that anyone who is obliged to ensure that they can delete content when they need to do should design their retention architecture to avoid using the PHL. In practice this means:
Delete-only labels/policies - consider the purpose for every label/policy that you create – if the purpose is to ensure timely deletion (rather than guaranteeing retention), configure the label/policy to allow deletion. This is often referred to by Microsoft as ‘delete-only’, and effectively means that content isn’t retained and can be deleted at any time. Obviously, this is only appropriate for some of your labels/ policies.
Avoid Retention Policies – consider only using retention policies in scenarios where you are trying to actively delete content on a timely basis. Longer-term retention polices should be avoided – these increase the probability of you being unable to delete files you cannot possess.
Configure Retention Labels (without allowing users to delete) - Consider disabling the ‘Allow users to delete content in SharePoint sites’ setting (found on the Records Management Settings page) when using retention labels. While this will frustrate some users, who will be confronted with an ‘error’ message when they attempt to delete their files, it has the benefit of preventing content being sent to the PHL. Staff can be trained to remove the label if they have a genuine need to delete specific files, which while not ideal, is in my mind usually a far a better option than failing to comply with GDPR.
I feel that the PHL is one of the biggest opportunities for Microsoft to improve its records management capabilities. Without knowing the details of the underpinning technical design of Microsoft 365, it’s difficult to make tangible suggestions, but I think there are a number of potential options for Microsoft to consider, including:
Separate retention location – An alternative location (possibly new retention libraries) could be introduced specifically for records management. The PHL would then only be used for litigation holds and not retention. The main improvement here is that records managers would be empowered to permanently delete content out of this new retention location – ensuring that you can remove content that you are not allowed to keep.
Permissions update for PHL - An alternative would be to change the permissions in the PHL so that records managers are granted the ability to delete retained content. Importantly, with this approach records managers must not be granted the ability to delete anything stored because of a litigation hold (only content routed to the PHL by a retention label/policy should be delible)!
Optional deletion/retention - Introduce a popup (dialog) that would be shown when a user deletes a file. This popup could provide the user with the ability to choose whether the file needs to be permanently deleted or whether it should be sent to the PHL. This could potentially be extended with an optional review process to approve decisions. Regardless, this approach would certainly need to be heavily audited!
While none of these suggestions are perfect, any of them being introduced would significantly improve Microsoft Purview’s Records Management capabilities.
The rigid principles of retention
It’s perfectly possible for an item to be subject to multiple retention policies and potentially a retention label too. This clash of retention rules necessitates guidelines that determine which rule will take precedence over the others. Microsoft’s solution for this is ‘the principles of retention’ – these come in the form of a four relatively simple* hierarchical guidelines that determine which retention rule is applied.
*Okay the 3rd principle is anything but simple, it basically means that retention labels will always take precedence over retention policies and also that if there are multiple retention policies applied, any that explicitly name the given site/user/group etc. will take precedence over any that don’t.
Figure 1 – Microsoft’s principles of retention
The principles are great in any situation where you want to set a short default retention/deletion period, while allowing users to choose to apply longer retention on specific files. For example, I’ve encountered various organisations who have found that the principles work really well for them in OneDrive, allowing automated deletion of older content, while still providing staff with the ability to override this by labelling specific files that require longer retention.
The principles, however, quickly run into issues if you want to override a longer default retention period, with shorter retention/deletion. Imagine a site used by a finance team, where the vast majority of content requires 7-year retention. While a 7-year retention policy could easily be applied as the default, doing so would prevent content from being deleted before this duration has elapsed. For example, perhaps some of the files in the site contain credit card numbers or other PII that you need to ensure you don’t retain. You could attempt to apply a deletion label, but unfortunately, the principles of retention mean this would not take precedence over the retention policy. You could instead attempt to delete the file containing PII, however, this would cause the retention policy to send it to the Preservation Hold Library (which of course causes all of the issues discussed above!).
So, what can you do to avoid this issue? The conclusion I’ve come to is largely (but not exclusively) to make use of retention labels instead of retention policies. Each item can only have a single label applied to it, which (assuming you aren’t also applying a policy) prevents any rule clashes from occurring. This is far from a perfect solution though, leaving inevitable gaps in our application of retention.
To me there is a clear opportunity for Microsoft to introduce more flexibility into the principles of retention. Many would love to have the ability to determine the priority of their own retention rules. This would allow organisations to design more advanced retention architectures that would significantly help improve their ability to meet compliance needs.
One approach might be to provide some form or ‘priority’ flag for labels/policies. This ‘flag’ would allow the given rule to take priority over other (non-prioritised) rules.
Even better would be if we could determine the priority sequence all of our retention labels and policies. This could work in a very similar way to the sequencing that has been provided for sensitivity labels, with the rules ordered at the top of the list taking precedence over those below them:
Figure 2 – Sequencing of Sensitivity Labels
Summary
To be clear, I love the retention capabilities in Microsoft 365 – they already provide a robust and reliable approach to records management for organisations of any scale. I believe that the two issues outlined in this post represent the most significant opportunities for Microsoft to further enhance their records management capabilities over coming years, and I’m sure that many organisations, especially those needing to comply with GDPR, would warmly welcome new capabilities in these areas.