With millions of new users this year, Teams has quickly become one of the most important information storage applications in many organisations. As we begin to return to the workplace, I think it’s already pretty clear that work patterns will change for many. I’d wager that the transition into Teams will persist and it will become the central tool that many of us work with daily.
The reason people have jumped into Teams is obvious, not only is it an excellent communication tool, it also provides easy access to group working conversations and content. However, while users enjoy the ease of use and collaboration benefits brought by Teams, many people working in information management and compliance are now retrospectively wondering how to best control, govern and importantly apply retention to content in Teams.
I thought a blog on this subject was well overdue!
Retention Approaches
There are really two viable approaches to retention in Microsoft 365. (Yes, there are also legacy concepts, including Record Centers and In-Place records management – but I’m going to ignore these outdated methods for brevity.) Both of these approaches can be applied to different areas of Microsoft 365, so becoming familiar with their capabilities is essential if you are looking to design a tenancy-wide retention architecture.
Retention Policies
Retention Polices are virtually hidden from end users – even to the extent that deleted files are caught behind the scenes within hidden containers. As such, users can’t see (or tag) content with a Retention Policy. To apply Retention Polices to your content you need to use either of these approaches:
Applied to all content in a given site – in the context of Teams, this means that all content stored in all the Files tabs across all Channels in a given Team are subject to the same policy.
Automatically applied – if you are lucky enough to have a premium licence1, you can automatically apply Retention Polices to files. This currently works by applying retention to all content that contains either a specific phrase/keyword or ‘sensitive information’ (such as a credit card number or a national insurance number).
Unfortunately, Retention Policies do not provide a mechanism for making content immutable. As such, Retention Policies cannot be used to lock files as records.
At the end of the retention period Retention Policies allow you to either automatically delete content, or to ‘do nothing’ (which I’ve never really understood the value of). Unfortunately, Retention Polices don’t currently offer the ability to trigger disposition reviews at the end of the retention period.
It’s probably best to think of Retention Policies as a way of disposing (of) lower value content, than for long-term preservation of important files. I also find that Retention Policies are a really useful safety net – a backstop to catch anything that isn’t governed by Retention Labels.
Retention Labels
Unlike Retention Policies, Retention Labels are visible to users. This is most apparent when a user attempts to delete content that has been labelled – instead of being sent to the recycle bin, the following message is presented:
The other time that users will become aware of Retention Labels is when they edit content. Anyone with the right to update files also has permission to apply (or change) the Retention Label2.
Retention Labels offer more flexibility than Retention Policies when it comes to how they are applied to content. They can be:
Applied manually to a given file – this allows each item to have a separate label – effectively providing the potential for significantly improved accuracy, albeit at a cost of needing to manually apply the classification to content.
Automatically applied – like with Retention Policies, if your users are lucky enough to have a premium licence1, you can automatically apply Retention Labels to your content. There are several ways this can be achieved:
Keywords/Phases – search through your content to find specific sequences of keywords or phrases.
Sensitive Information Types – also searches through content, but this time to find specific sequences of characters, such as passport numbers or national insurance numbers.
Default values – probably the most practical way of setting up Retention Labels, a premium licence1 allows you to set default labels on libraries and folders. When combined with a site/team creation process, this can be an effective way of ensuring records management capabilities are applied across all content.
Trainable Classifiers – still in preview, Trainable Classifiers can be taught how to identify a type of content by looking at hundreds of similar examples. You could, for example, create a Trainable Classifier to find, and apply labels to all your Policies. Superb in many ways, the downside of the Trainable Classifiers is the need to have large consistent datasets of each different type of content.
One of the other major features provided by Retention Labels is the ability to lock content as records (with premium licences1 ). Applying a ‘Record’ Label causes classified items to immediately become immutable.
Finally, Retention Labels also allow Disposition Reviews to be undertaken at the end of retention (again premium licences1 are required), providing you with significantly more control around the end of the content lifecycle.
While Retention Labels are clearly more flexible than Retention Policies, one of their current major limitations is that they are only available in some of Microsoft 365’s workloads. This is especially apparent in Microsoft Teams, which surfaces content from multiple locations across the tenancy. Effectively, the records management functionality provided in different areas of Microsoft Teams is directly dependent upon the retention capabilities provided within the underpinning storage locations.
Retention capabilities in different areas of Microsoft Teams
One of the most important things to understand about Microsoft Teams, is that it is more of an interface than a storage location; in other words, most of the content you see in Teams is actually being surfaced from other parts of Microsoft 365. As such, when understanding the capabilities provided by Teams, you really need to consider the capabilities provided by the various other workloads underpinning each part of Teams.
Let’s take a look at where the different types of content that are found in Teams are actually stored and at the available retention functionality in the given area.
Files – each Team has an associated site within SharePoint Online, which is used to store all of the content found in the ‘Files’ tab. SharePoint Online is one of the richest and most flexible workloads in Microsoft 365. It can easily be configured with both Retention Policies and Retention Labels. This provides significant flexibility around how retention is undertaken on content within the Files tab. At the time of writing, metadata, including Retention Labels can be viewed, but not modified within Microsoft Teams. As such, while Retention Labels can be seen, users will need to access the SharePoint site underpinning the Team to modify any given label. When combined with the reluctance that users often show when asked to classify their content (certainly with any accuracy) the additional complexity of needing to leave Teams to apply Retention Labels in SharePoint is likely to result in relatively poor quality and consistency of records management. If your users have premium licences1 you will be able to take advantage of the automated approaches to applying Retention Labels, which could result in a more consistent and accurate application of Labels.
Posts (formerly Conversions) – Content in each of the Posts tabs within a Team is stored in the mailbox of the Microsoft 365 Group3. Currently it is not possible to use Retention Labels in Posts, as such Retention Policies need to be used in this area of Teams instead. The same Retention Policy is applied across all of a given Team’s Posts (except for those in Private Channels). The effect of this is that each conversation that is posted within a Team is subject to the same retention period. As discussed above, Retention Polices are largely hidden from end users. For example, when Posts are deleted, they are sent to a hidden folder, SubstrateHolds, in the Group’s mailbox, with the user typically assuming that their message has been successfully deleted. It’s worth pointing out that due to the limitations of Retention Policies, Posts can neither be made immutable and nor can they be subject to disposition review.
Chat – One-to-one chat messages are stored in the mailbox of each user3. This means that a copy of the Chat is actually stored within the mailbox of each of the users involved in the chat. Where Retention Policies on Posts are applied to all the Posts in a given Team, Retention Policies on Chats are applied to users. This means that all of the given user’s Chats are subject to the same retention duration (irrespective of the importance of the given topic!).As Chats involve more than one person, it is possible that users corresponding via the same Chat might well have different Retention Policies both governing the same one-to-one Chat. This can lead to scenarios where two separate retention durations apply to the same Chat message. In this scenario, all users involved will see the chat disappear from Teams when the shorter retention duration period ends. However, the Chat can still be found behind the scenes via Content Search, until the longest retention period has ended. In a similar way as with ‘Posts’, the use of Retention Policies for governing Chats is largely hidden from users – meaning that users are often unaware that they cannot fully delete messages. Likewise, Chats cannot be made immutable or subject to disposition review, due to the inherent capabilities of Retention Policies.
Meeting recordings – When you record an online meeting conducted in Teams, the recording is automatically uploaded into Stream3. At the time of writing, neither Retention Labels nor Retention Policies can be applied to govern Stream content, however this is a much requested feature4 so it wouldn’t be too much of a surprise if we were to see this introduced in the near future.
Communities – Posts seen in Yammer Communities are unsurprisingly stored in Yammer, however a recent update has seen Microsoft upload files (that are added to new Yammer Groups) into SharePoint. Yammer itself doesn’t support Retention Policies or Retention Labels yet. Yammer does, somewhat confusingly, have a ‘Data Retention’ feature, which can be used to allow or prevent deleted posts from being available in data exports. However, essentially, messages in Yammer Communities cannot be governed with Microsoft 365’s retention capabilities. Since the recent switch to using SharePoint Online as the storage location for files that are uploaded into Yammer, it is now possible to protect this type of content with Retention Policies and/or Retention Labels.
Other tabs and apps – Teams is incredibly extensible and can easily be integrated with content that is technically stored in a range of other locations. For example, your Team might well be configured to show content stored in Moodle, Trello or Salesforce amongst others. While the retention capabilities provided by Microsoft 365 cannot be extended to 3rd party tools and apps, it is possible to use the Security & Compliance Center to import and archive data from external sources. Once archived inside your tenancy, you can then apply Retention Polices and Retention Labels to multiple different types of 3rd party data. Microsoft 365 comes complete with connectors to allow data to be imported from multiple sources, including Slack, Facebook, LinkedIn, Twitter, Zoom and WhatsApp amongst others – and Microsoft partners, such as Intelogy, can build connectors to allow you to archive (and retain) data from almost any other 3rd party application.
As you can see, the retention story within Microsoft Teams is quite complex and varies from workload to workload. Really, when moving into Teams (or retrospectively trying to tidy up your existing Teams), you need to plan precisely which of the retention capabilities you are going to utilise, and how you are going to apply both Retention Policies and Labels to content.
Subject Access Requests – Teams Chat
A bit of an aside – but I’ve noticed another recent trend in Teams usage that I find rather worrying. I’m starting to hear about many organisations who are becoming concerned about how they can facilitate Subject Access Requests, especially for content stored in Teams’ chats.
Rather than look for advice on how best to find and export relevant content, (which incidentally I wrote a blog post about two years ago), I’m starting to see some organisations decide to set very short retention periods on their Teams Chat instead. In other words, rather than work out how best to find and export content stored in Chat, they have opted to design their retention schedules instead so that messages found in Chat are automatically deleted within days of being created.
The obvious issue I have with this approach is usability. If chat messages are deleted after a handful of days, then users simply won’t be able to refer (back) to older messages. I feel that having a short retention policy for Chat is going to frustrate and annoy many (and will likely result in more information being exchanged informally via email or WhatsApp).
I also feel that reducing the length of time it is retained to a minimum is legally dubious. I wonder how any organisation who have chosen to do this might be able to justify the decision if challenged in court?
Is Chat exclusively being used to share low-value (junk) content? Probably not. Chat will likely contain some information that is relatively important and requires longer-term preservation.
Is other low-value (junk) content (i.e. draft documents etc.) being deleted with a similarly rapid sense of rigour? Again, probably not. I would imagine that low-value files and emails are more likely to be deleted after, say 2 years or so, in most cases.
Quite simply, I feel that a very short retention period for Chat will be very difficult to justify. The only reason I can see for anyone doing so is to attempt to circumvent SAR/FOI requests. My guess is that anyone considering this is possibly straying into legally (and morally!) dubious territory.
I would urge everyone to consider how Chat can be extracted via Content Search instead and work to reinforce appropriate usage of Chat with staff (and other users).
Conclusion
While Teams provides excellent collaboration features, which make it really easy for people to work together, it’s important to ensure that you plan how you will manage content throughout its lifecycle.
As I discussed in one of my previous blogs (Microsoft Teams – temporary or perpetual? (robbath.co.uk)), you might decide to move content outside of Teams for longer term governance. If you do go down this path, it’s essential to have a good understanding of how the different retention capabilities work in different parts of Teams. Something I hope this blog post has helped to clarify.
If you decide to apply retention capabilities within Teams, I feel you will almost certainly need to integrate the configuration of these capabilities into the process of creating each Team. In fact, even if you decide you don’t want to retain content in Teams, I would still urge you to define a consistent process for requesting, approving and provisioning Teams – whether this is manual or automated, I consider to be essential for any organisation using Teams.
Finally, I feel it’s important to stress that retention shouldn’t be used as an excuse to try to avoid legislation – it should instead be used to help you to comply. For me, retention should be about transparency and honestly trying to do the right thing – it should not be used as a tool to make it easy for you to sweep embarrassing content under the carpet.
Footnotes
1 I’m using ‘Premium Licence’ as shorthand for a collection of licences that provide you with access to the advanced retention capabilities. All of your users who can modify content that uses any of the advanced retention capabilities will need to have one of the following licences (at the time of writing): Office 365 E5/A5; Office 365 Advanced Compliance; Microsoft 365 E3/A3/E5/A5; Microsoft 365 compliance or Microsoft 365 Information Protection and Governance.
2 Unless that is the file or list item is locked as an immutable record, in which case only administrators can modify the item or it’s retention label.
3 I’ve intentionally simplified the technical detail of the storage locations for Teams content (as I feel the full details would overcomplicate this blog post). However, I’d refer anyone interested in finding out precisely where their Teams content is stored to the following page: https://docs.microsoft.com/en-us/microsoftteams/location-of-data-in-teams.
4 See the following UserVoice page requesting retention capabilities in Stream: https://office365.uservoice.com/forums/273493-office-365-admin/suggestions/36251461-microsoft-stream-data-retention-policies.