What are retention periods?
When I speak to people outside of the Information Governance community about retention periods, they often think that I’m talking about the length of time before information should be deleted – but this is only half of the story. Retention periods aren’t only about the timely deletion of expired content, they are also used to ensure that content isn’t deleted prematurely.
To put it simply, records managers use retention periods to:
Ensure that low value content is deleted. I’m talking about the everyday junk emails and drafts that we all produce, which accumulate and clutter. Without a process for tidying the rubbish, our collaboration areas fill up like the under-stairs cupboard that we’ve been meaning to tidy for several years – inevitably causing us no end of bother when we need to find the umbrella in a rush.
Ensure that high value content isn’t deleted (well, not until we need it to be deleted). Defining ‘junk’ is easy; defining ‘high value’ is slightly more challenging. There are different reasons for content to be considered high value and each reason requires a retention process to handle the content in a different way.
Firstly, we have content that is considered high value because it is actually useful to your organisation. These are the policies, strategies and reports that help maintain processes, steer decisions and plan change. Some of this content is published (i.e. research papers, annual reports etc.) and needs to be retained for future reference, while other files such as policies and procedures frequently require ongoing reviews and updates.
Next, we have the high value content that we need to keep for legal reasons, things like your organisation’s accounting records and audit reports. Sometimes content in this category is retained for potential future legal reasons, such as information relating to accidental exposure to hazardous substances. Typically, by the time the legal duration has passed, the content has long since been of any value and is often swiftly deleted.
Then we have high value content which is of historical importance and might well be of interest to future generations. Whether we are talking about something of national importance, or your founder’s first business card this sort of content very often needs to be preserved for archival.
Bill Gates’ first business card
As you can probably imagine, the process of planning to apply retention to all of the content in an organisation can be quite complex. Not only do you need to work out how long different types of file need to kept for and what should happen to the documents and emails when the retention period ends, you also need to work out what date to start your retention period at.
When do retention periods begin?
Ignoring legacy features, there are two approaches to retaining content in Microsoft 365: Retention Policies and Retention Labels. Retention Policies have two optional start dates for retention periods, the date that the file was created and the date that the file was last changed. Retention Labels, add to these by offering two additional triggers: the date that the content was labelled, and via ‘an event’.
Let’s look at each of the approaches for triggering retention in a little more detail:
When it was created
Using the creation date a file is probably the least used trigger for retention periods – in the main because files are often started months or even years before they are completed. I tend to see this trigger used for (mostly) immutable content, often when received from third parties. Good examples are Purchase Orders, Receipts and Supplier Contracts, which are often never modified after they are received by your organisation.
The problem with this trigger is if you trigger the start of retention when a file is first created, there is a reasonable chance that some content is still actively in use when the retention period ends. For content such as policies, which can be refined over multiple years, there is a real danger of files that are still being regularly changed reaching the end of the retention period. You won’t want to be automatically deleting content that is still being edited – well not unless you are looking to anger your colleagues.
When it was last modified
Having seen how dozens of organisations have chosen to apply retention, this trigger is by far the most frequently used approach. It’s far from perfect, but for most content tends to provide a good balance of ensuring that retention periods start when the content is considered to be complete.
Things to watch out for when using this trigger include identifying types of content that are heavily used, yet rarely changed. I’ve seen key files and pages, which have been in use for years and contain core information, become accidently deleted. A good example is a process diagram, which staff might access on a daily basis to make sure they are following the right steps – if the process doesn’t change, the diagram might not need to be modified during it’s retention.
‘Last modified’ provides a simple and effective trigger for retention that works in many, but not all situations.
When it was labelled
Microsoft’s vision for Retention Labels is that they are either applied to content via manual tagging, through setting default labels on libraries and folders, or alternatively through use of one of the automated approaches provided within Microsoft 365 (i.e. by matching specific phrases in content; by finding sensitive information, such as credit card numbers; or through Trainable Classifiers – which are a new approach for using AI to apply labels to content).
If you chose to apply the bulk of your labels by setting default labels on libraries and folders, then you will reliably be labelling your content when it was first created. However, all of the other approaches will see labels applied to content at some point between initial creation and reaching completion (that is if a label is applied to the content at all). I’ve thought about how this trigger could be utilised, but I can’t think of many situations where I wouldn’t prefer to start my retention periods with one of the other approaches. Do let me know if you have any suggestions for scenarios where starting retention when content was labelled might be useful – I’d love to hear about them.
An event
Event-driven retention has been around for about two years now and is a really powerful alternative way of triggering your retention periods. Rather than relying upon a date, you can instead start retention when specific things happen. For example, you could choose to start retention on relevant content when a project closes, or when a person leaves the organisation.
Let’s take the example of a housing association, who own a series of properties and rent them to tenants. When one of your properties is sold, you might want to invoke an event that triggers the start of retention for all content that relates to the given property. Likewise, when a tenant moves out, you again might need to find all of the content relating to the given tenant and make sure that it is retained in accordance with your retention schedule.
I’m sure if you think about the organisation you work in, you could probably think of several situations that are unique to your line of business, where triggering retention with events might be a really useful approach to take.
I think the benefits of event-driven retention are pretty obvious, not only do they provide you with far more control over how your content is managed, they can also help you become more compliant as an organisation.
NB, event-driven retention is only available for organisations who have a premium licence (i.e. Office 365 E5/A5; Office 365 Advanced Compliance; Microsoft 365 E3/A3/E5/A5; Microsoft 365 compliance or Microsoft 365 Information Protection and Governance).
So how does Event-driven retention work?
There isn’t a simple switch to enable event-driven retention – it really needs to be planned in advanced and almost baked into your working areas. The following steps provide a high-level guide to setting up event-driven retention:
Create a new Event Type
An ‘Event Type’ is a really simple way of grouping events and associating them with Retention Labels. An Event is actually an instance of an Event Type. It’s probably best to think about the Event Type through looking at some examples:
If your Event is the closure of a specific project, say ‘Project Beta’, then your Event Type is likely to be ‘Project Closure’
If your Event is the sale of the property at ’12 Station Street’, then your Event Type will be something like ‘Sale of Property’
It’s important to note that you need to create the Event Type before you can create the Retention Label (and of course, before you could trigger an instance of the Event).
Create Retention Labels
After creating an Event Type, you can then create the Retention Labels that you want to be associated with the given Event Type. Each Retention Label that you create can be associated with an Event Type when it is created.
In our Housing Association example, the ‘Sale of Property’ Event Type could be applied to a range of different Retention Labels, such as ‘Property Maintenance’, ‘Property Health and Safety’, ‘Property Financial Information’.
Ensure content has a unique ID for each event
This step is the most complex to get right because you don’t just need to apply a Retention Label to content, you should also ensure that content is tagged with a unique reference for each potential event.
Let’s look at a couple of examples:
If your Event Type is ‘Project Closure’, then you should make sure that each piece of content is tagged with a unique Project ID (or unique Project Title).
If your Event Type is ‘Sale of Property’, then you need to tag content with a unique reference to the given property, perhaps a Property ID or perhaps the Property’s full address.
For content stored in SharePoint Online or OneDrive, Microsoft provides a field called ‘Asset ID’, which you can use to tag content with the unique reference. However, this field allows free text, so if you chose to use it you will likely find it becomes difficult to ensure that you are using consistent references across all files. I personally prefer to use a separate metadata column, which can provide users with a limited choice of valid values to select from – helping to significantly improve consistency.
You can in theory bypass this step, and instead of tagging content, rely on finding content that contains specific keyword phrases. I’m not personally in favour of doing so, as I feel it’s far too likely to both include content that really shouldn’t be related to the given event and also miss content that really should have had its retention period started.
It’s worth pointing out that content in Outlook cannot be tagged and as such, if you are applying Event-driven retention to content in Outlook you will need to consider ensuring that emails and events contain consistent keyword phrases.
Trigger the event
Now, all that is left to do is trigger the event. Each time a project closes, or whenever one of your properties sell, an Event needs to be created within Microsoft 365’s Compliance Center.
When creating the Event, all you need to do is associate the Event with the correct Event Type (so that the Event only fires on related Retention Labels), and then associate the Event with correct unique ID (i.e. ensuring that the Event only applies to content relating to the given Project or Property etc.).
Final Thoughts
Microsoft 365 provides a great selection of different options for triggering the start of Retention. While I think most organisations will continue to use the date that the file was last modified as the trigger for the start of retention, I feel that Event-driven retention provides an alternative, that is a better option to pick in many scenarios.
Event-driven retention needs a fair amount of planning and I can see that organisations looking to take advantage of its capabilities are likely to want to add an additional layer of automation to assist with the operational side of maintaining their content. I’ve got a whole list of ideas I’d love to try out, including:
Adding logic that makes it easy to add new unique IDs could be invaluable, perhaps automatically creating new unique ‘Property IDs’ (or Project IDs’ etc.) each time a new Property is purchased
Making it easier for administrators to create Events – especially in large organisations, having an authenticated one-click process for, say, closing Projects could be invaluable.
Intelogy are leading specialists in delivering exceptional Microsoft 365 information governance solutions. One of a handful of organisations to have been selected by Microsoft as providing ‘preferred’ Content Services, Intelogy can ensure that you get the most out of Microsoft 365’s retention capabilities.
Don’t hesitate to get in touch if you have any questions or want to learn more.