Where Conditional Access Risk Policies Fail…
…or, how Microsoft’s recommendations are causing AiTM to fly under the radar.
Microsoft provides a number of conditional access policy templates for organisations to deploy. They are mostly good, but one in particular has caused us some concern. That is this template which, in our opinion/experience is masking successful AiTM attacks in organisations by marking successful Adversary in The Middle (AiTM) attacks as false positives. This template is referenced from the zero-trust templates section in this set of templates made available by Microsoft.
In November 2023 Microsoft auto rolled out a set of conditional access policies. It’s not clear to us, but it seems probable, that for some customers this policy may have been auto rolled out by Microsoft. In the environment we checked this was not marked as “Microsoft-Managed” so was likely created from the Microsoft recommended policy templates.
The template in question (likely named “Require multifactor authentication for risky sign-ins“, or similar, depending on how it was incorporated into your environment) is designed to enforce MFA for “risky sign-ins”, but has the side effect of automatically setting risky sign-ins that pass MFA to “false positives” within Defender.
Were every Microsoft customer using exclusively phishing resistant MFA, such as FIDO, then this is probably not an unreasonable approach to take. However, with most Microsoft customers relying on the “very phishable” push notifications from Microsoft authenticator (and other phishable MFA solutions) this is a very risky approach to take.
We have seen numerous incidents where Microsoft has successfully detected real malicious authentication attempts as suspicious, only to then mark them a false positive and permit authentication because the MFA challenge has been met… in exactly the way it would be during an Adversary in The Middle (AiTM) attack.
The alert appearing in your defender dashboard will appear in a similar fashion to what is shown below. This was taken from one of those real incident so please forgive the redaction:
If you were to hunt for similar attempts within your environment then the axios user agent string is a helpful indicator and the 2a00:b703:fff2:e::1 we have left unredacted because it belonged to the backend infrastructure involved in this specific incident (and others!).
The area of specific interest in the alert shown is the bottom right where we can see the automation taking place and changing the alert to a ‘False positive’:
Unfortunately, in this case, it was not a false positive, it was a true positive and the user had both entered their credentials into a phishing website impersonating the Microsoft login page, and had also entered the code and approved the push notification that they received in their Microsoft authentication app (why wouldn’t they, after all they thought they were logging into a legit Microsoft website…).
We’ve seen this one crop up enough times that it became worthy of a short blog post. If you’re a Microsoft customer then it is worth checking to see if you are affected and following our recommendations below.
We don’t know if the automation “learns” from the automated change in classification, i.e. whether it being marked as a false positive by this policy causes future sign in events with shared attributes (e.g. the same IP address or user agent) to be considered more trusted (or even normal), and so result in these future sign in attempts not even being flagged as risky in the first place… (hopefully not!)
Recommendations
Go and check your conditional access policies, you’ll find them here: https://portal.azure.com/#view/Microsoft_AAD_ConditionalAccess/ConditionalAccessBlade/~/Policies
This is a view of the policy from within Azure:
If you have this policy enabled (likely named “Require multifactor authentication for risky sign-ins”, but may be named something similar) then you have a few options:
Option #1 - Upgrade the policy
If authentication is considered “risky” and your organisation already enforces MFA then this policy is a regression. So why not alter the grant to “Block access” rather than grant it based on an MFA challenge being met. To do this, simply change the grant from “Grant access” to “Block access”.
The main caveat to this is that, whilst Microsoft’s detection of risky logins is mostly pretty good, there are false positives and this change will block that authentication… You can always decide that something is not risky later on, but it will block the user from authenticating in these cases.
Option #2 - Disable the policy
If you already enforce MFA and are not willing to upgrade the policy as described in option #1, then we’d recommend disabling this policy. It is a regression from your already existing MFA enforcement and serves only to mask AiTM authentication activity. At least sign-ins with unfamiliar properties will remain unresolved until you chose to resolve them and so they won’t disappear from your SOC workflow.
Non MFA Tenants
If you don’t have MFA enforced globally across your organisation then we still believe that there are better ways of achieving the level of security that this policy intended without the flaws in its approach:
If a user does not have MFA configured and their login is considered risky then they will be met with an AADSTS53004 error when this policy is applied - i.e. they won’t be able to login anyway.
If a user does have MFA configured then using their MFA challenge as a criteria to grant access is usually flawed (as we have already described).
So in both cases this takes us back to option #1 being the ideal approach and option #2 being the alternate option.
The only case in which we would advise leaving this policy in place in its standard form is if you enforce non-phishable MFA exclusively across your entire organisation.
Check your Defender Dashboard For Past Compromises
You can do this from within your defender dashboard. Alerts will be named “Unfamiliar sign-in properties”:
Just be aware that these may well be marked as “false positive” or as “resolved”, so ensure that you are not filtering those out.
A Note About Lab539
At Lab539 we actively hunt down adversarial infrastructure before it is used, rather than waiting for victims. We share our intel with our subscribers in real time via an API, named location service, and a Microsoft Defender feed. Protecting customers from both “frontend” and “backend” infrastructure, i.e. we prevent users from visiting the frontend (phishing site) and block authentication from the backend infrastructure that proxies the authentication.
If you are not a subscriber you can register for a free trial here: https://www.aitm-feed.com/subscribe (available to registered companies only)