I’m not going to bury the lede. Windows Recall is one of the riskiest and most unnecessary features in consumer or enterprise software. It shows a fundamental lapse of both common sense and enterprise/cybersecurity risk management in a business where the CEO states security must be at the forefront of what they do.
As a feature it doesn’t pass even the briefest of sniff tests, but let me break it down in detail from my perspective as an infosec professional.
A quick summary of what it does
Windows Recall is designed to take a screenshot of your screen once every five seconds, if the screen has changed. The screenshot is then analysed by an AI LLM running on your PC to allow you use natural language to search for pieces of information or configurations you interacted with on your PC up to 30 days ago (it may be longer, Microsoft isn’t clear about it).
Risk management basics
When you distill infosec to it’s core, it’s ultimately about managing infosec risk. You want to get the riskto a level that’s acceptable to the executives of the business, or to ourselves in our own lives. We have a few ways of treating foreseen or known risks to manage them to the right level. We can:
1) Avoid the risk.
2) Modify the likelihood (probability) of the risk.
3) Modify the consequences (impact) of the risk.
4) Share/transfer the risk.
5) Accept the risk.
Once we’ve identified possible treatment options, we have to choose and implement the one that best suits the risk and our risk tolerance. We then have to monitor our risk treatment to ensure it’s actually treating the risk as we expected – namely that the residual risk after treatment is appropriate. If it’s not, we have to do something else.
Theory meets reality
To make this a bit less theoretical, let’s apply this to a common example in our lives. You want to sign up for a bunch of websites, all of which require a username and a password. You’re concerned that some of the websites may be insecure and may leak your username and password. This could potentially allow attackers to log into other websites (the risk, foreseen) you’ve signed up to. In order to manage this risk, you could do the following:
1) Not sign up for any of the websites – thus avoiding the risk entirely.
2) Read up on the security measures the sites take to secure your data and only sign up with those that can demonstrate good security measures – thus decreasing the probability of the risk.
3) Sign up for all the websites, but create unique passwords for each, or use a password manager to do that for you – thus decreasing the impact of the risk.
4) Take out personal cybersecurity insurance to reduce your loss if your credentials are stolen – thus sharing the risk with the insurance company.
5) Do nothing, use a single password anyway – thus accepting the risk.
With the options laid out, there are some value judgments you now need to make. You need to weigh up your available time, how badly you need the services/info on the websites, how much knowledge you have, how much irritation (friction) you’re willing to have to manage the risk, and so on.
Being a normal person, options 2 and 4 are deeply unlikely. You don’t have the knowledge or time to review the sites to figure out which sites are more secure, even if they did publish the information. Option 4 isn’t feasible because it’s too expensive and the insurance company typically wants you to use a password manager (option 3) anyway. Option 5 isn’t suitable for you because you’re a diligent internet user and you know that simply accepting this risk isn’t safe.
This leaves option 1 and 3. Option 1 is always possible – perhaps you honestly don’t need access to any of these sites, so you can always just walk away from signing up. Or you make a value judgement and decide you really need to access their information or services, so you manage the impact of the risk by using a password manager to create unique passwords for each site you sign up to.
Whichever option you settle on, congratulations! You’ve consciously managed and treated your first cybersecurity risk.
Applying the risk management theory to Microsoft Recall
With our risk management hat on, let’s think about Recall.
Because it will record all the activities you’ve performed on your PC, if it’s data is compromised it’ll have a significant impact on the user and/or the business. Because it’ll be a feature available on any consumer or enterprise PC, the probability of it being attacked is extremely high. Because the impact and probability are so significant, it’ll be difficult to transfer the risk to others without it being extremely expensive. So we have to manage the risk in some way, we can’t simply accept it.
Microsoft have acknowledged – at some level – that this feature is extremely risky and have created some ways to help IT teams and us manage the risk.
1) You can turn it off when setting up Windows, or afterwards.
2) You can set specific applications to never record.
3) You can delete snapshots.
4) Data is only processed on the local machine and is encrypted at rest.
Bluntly, these are plasters slapped on a gaping wound. The feature is currently enabled by default and is confusing to turn off. The data’s encrypted you say? Well sure, but people who hack your devices are going to be logged in either as a different user or as you. And when you’re logged in the data is unencrypted. Also, your data is accessible even to other users of the machine when they’re logged in. Kevin Beaumont wrote an excellent post on many of these issues.
To add to that, I’m going to touch on two other issues I have on choosing which applications are and aren’t recorded, and deleting screenshots of valuable info:
1) The subset of users who will choose which applications are and aren’t recorded is vanishingly smaller than the subset of users who will actually get value from the feature. Also, the users need to continuously maintain that list when they add new applications to your PC. Ain’t nobody got time for that.
2) If you work an 8 hour day, Recall will take up to 5,760 screenshots in the day. No one – other than motivated attackers – will go through 5,760 screenshots a day to ensure there’s no valuable data stored there. Even if the search function is as good as is claimed.
The risk treatments Microsoft has put on the table are, respectively, inappropriate, incomplete, ineffective, and unusable.
Not doing the thing is always an option
As infosec professionals, we’ve been wired to focus on risk treatments that reduce the likelihood and impact of the risk, or transfer it to others. This is because one of the tenets of infosec is to be responsive to business demands and strategy. Put simply, infosec doesn’t run the business, the business runs the business and infosec helps secure it.
Every so often, however, business presents a risk where infosec can’t – or shouldn’t – reduce the probability or impact, or transfer it elsewhere. That leaves us with telling the people who pay our salaries that they either need to not do the thing, or they need to acknowledge what we’re saying, document it, and accept the risk. These conversations are about the most un-fun you can have in infosec (my condolences to the infosec team at Microsoft).
If I were the one who had to have the un-fun conversation with the business teams at Microsoft, this is the core of the message I would present to them:
The value this feature adds to the user base is massively outweighed by the risk it introduces – even with the controls we’ve designed. If users disable the feature, it still creates an enormously attractive target for attackers to enable. As a result it’s going to get enormous pushback from regulators, infosec, and users. We need to go back to the drawing board on this.
Product recall, I hope
I hope this feature is removed from Windows. It’s a security and a risk management nightmare that benefits a vanishingly small percentage of Windows users. If it does make it into Windows, please be aware that it exists and that it should be turned off. It’s particularly disappointing to see a feature like this released by Microsoft, given their (re)focus on security in their products.
Hopefully it isn’t the start of a wider trend of very poorly thought-through AI-related features that benefit cybercriminals more than they benefit the end user.