Soft (SMTP) vs. Hard (immutableID) matching with Azure AD Connect

Back to Blog

Soft (SMTP) vs. Hard (immutableID) matching with Azure AD Connect

If you are setting up Directory Synchronization from scratch (there are no users in the cloud yet), then Azure AD Connect will be pretty straightforward–the on-premises objects (and passwords if you choose that option) will be synchronized to the cloud, and you can assign services to the user accounts from there.

But what if you already have user accounts in the cloud which correspond to already existing user objects in the on-premises directory, and Directory Synchronization has not yet been configured between them? For example, if your organization previously migrated mailboxes to Office 365 using the cutover method or a third party tool. Or, if you had users provisioned for another Microsoft Online Service such as CRM, before you attempted mailbox migration.

In cases like these, you may need to create a matching mechanism between the on-premises accounts and the cloud-based ones, so that Azure AD Connect knows that they refer to the same user.  There are two basic methods to create this “matching”:

  1. Soft match (also known as SMTP matching)
  2. Hard match (by immutableID).

Soft Matching using the SMTP address

To create soft matches, which will be adequate in 95% of situations, you will need to ensure first of all that your UPN suffixes match between on-premises and cloud-based accounts. What do we mean by this?  It means that your users’ sign-in needs to be tied to the domain of your primary email address in both the local AD and in Azure AD.

In the On-premises Active Directory

First, when you open the properties of a user account object, this object should have the email address field filled out (the primary SMTP address for the user)–so be sure that is the case first.  Now take a look under the Account tab, and you should see the user logon name followed by a suffix.

The goal is to have this logon name be [email protected]–that is, the email address–and not the local domain name [email protected]. Note that you can also bulk-select accounts and make this suffix change on many objects at once.

If you do not have the option to drop down your suffix and choose the alternative, you can easily and quickly add the suffix using the Active Directory Domains & Trusts MMC console. Right-click Active Directory Domains and Trusts, and select Properties. Enter your email domain name and click Add. Click OK.

In the Azure AD / Office 365 cloud

In Office 365, you will also want to make sure the sign-in name is the same as on-premises, using the correct UPN suffix for the email domain name.  So the goal is to have this match [email protected] again, and not [email protected].

In the Office 365 Portal, find your Active Users, select a user, then edit the username.

In Exchange Online, you can also see that the primary SMTP address matches what we have listed in the on-premises account. Admin Centers > Exchange > recipients. Edit a recipient and click email addresses.

You can also export bulk lists for comparison from Active Directory as follows:

Get-ADUser -SearchBase “DC=company,DC=local” -Filter * -Properties * | Select-Object -Property Name,SamAccountName,EmailAddress | Sort-Object -Property Name | Export-Csv -path C:\export\ADUsers.csv

And from Office 365:

Get-MsolUser | select-object -property userprincipalname,displayname,islicensed | export-csv -path c:\export\365Users.csv

Now, assuming you have your UPN and email addresses all matching, you should be able to download & install Azure AD Connect. Upon running the first synchronization, SMTP matching should kick in, and figure out that the on-premises accounts already have cloud counterparts existing.  When you login to the portal and view your active users again, you should see a field describing the synchronization status, and each account from the on-premises directory should read “Synced with Active Directory.”

Hard Match using the GUID / immutableID

In some circumstances, soft matching may fail, and the on-premises accounts are not properly matched. Sometimes a previously existing cloud account can have certain fields populated already (e.g. immutableID) that will confuse the Directory Synchronization tool, even if the SMTP addresses are matching.

In these scenarios, you can turn to a “hard match,” which is performed by taking the on-premises GUID, then converting this value into what is known in the Azure AD cloud as an “immutableID,” and then writing that converted value directly into Azure AD. When Directory Synchronization runs, it will have no question marks about whether this is the same object, because it is being told so explicitly.

Before you proceed with this, you will still want to ensure that the UPN suffixes match the primary email domain on-premises and in the cloud, just as we did above.  Then, when you have identified any accounts that failed to sync up, you can run the following for each affected account (be sure to fill in the variables appropriately):

$credential = Get-Credential
Connect-MsolService -Credential $credential
$ADUser = "username" 
$365User = "[email protected]"
$guid =(Get-ADUser $ADUser).Objectguid
Set-MsolUser -UserPrincipalName "$365User" -ImmutableId $immutableID

And of course, this can also be generalized for bulk changes, for example if you use the variables as fields in a CSV file, and import the CSV, with a for-each loop.  You could even do this with a single variable in some cases:

$365User="[email protected]"
$guid=(get-ADUser $username).Objectguid
Set-MsolUser -UserPrincipalName "$365User" -ImmutableId $immutableID 

The above would be saved as HardMatch.ps1, then you can run the for-each loop as follows:

Import-Csv -Path C:\scripts\users.csv | ForEach { C:\scripts\HardMatch.ps1 -Username $_.Username }

No more mis-matches. That should pretty much take care of everyone who is having trouble getting matches made with Directory Synchronization (I’ve been getting a fair number of inquiries lately). A shout-out to my co-worker Lionel who put this script together for us–nice work, dude!

Comments (24)

  • Robert Woods Reply

    I was testing this process, Everything works great, only problem I have is I Cannot manage the users attributes. I have a freshly installed exchange server to be used for management of the user attributes, but I cant seem to get them to show up in the admin center.

    June 12, 2018 at 9:26 pm
    • Alex Reply

      You probably need to tell Exchange (using the management shell) that the users have remote mailboxes. This can be accomplished with Enable-RemoteMailbox cmdlet. See this article.

      June 14, 2018 at 10:20 am
  • Ron B Reply

    Great Article! Quick question. Do you know if the Express Hybrid Migration single sync will sync the attributes necessary to match the AD user with the accounts that are already in Office 365. As per your description above, our client already has several different licenses such as Dynamics, SharePoint, etc., but no email. We basically only get one shot at the sync and want to make sure it is going to match up the first time. We plan to use your method above to match up the immutable ID (UPN’s will be matched also) and then run the Hybrid Wizard choosing the minimal hybrid migration with single sync. They currently have an Exchange 2013 Server and do not want to keep it after migration and do not want to sync on an ongoing basis.

    June 25, 2018 at 12:04 pm
    • Alex Reply

      It should be fine as long as the accounts are unique, you have the UPN’s setup properly, etc.

      June 28, 2018 at 4:40 pm
  • Paul Messer Reply

    Hey! great blog… we have exactly this situation… customer started small so only had O365 accounts, then they grew and now have on premise AD (but no on premise exchange). Tried this method and similar one’s and it throws up a uniqueness error and doesn’t seem to work. soft SMTP matching (using the SMTP field) throws up errors in the dirsync… so we have users appear in o365 like this;

    [email protected] – Synced from AD
    [email protected] – In The Cloud

    I had hoped that I could have got the accounts to merge in o365, but the only way I’ve managed to do this is to delete the on premise AD account and recreate it using the UPN of the o365 account..
    Since one of the users affected is the owner of the company there’s some understandable reluctance to go down this path….

    Any thoughts.



    July 26, 2018 at 5:06 am
    • Alex Reply

      The very best practice is to have the account UPN match–this can usually be accomplished without deleting the on-premises account and recreating. It is possible to add the correct suffix and even rename the account as-is.

      July 28, 2018 at 6:44 pm
  • Cath Reply

    Hello! For the soft matching option will a password change/reset happen after the sync is configured? Or does it keep the on-premise password for the account and synch that to the O365? Thanks!

    October 9, 2018 at 10:15 pm
    • Alex Reply

      If Azure AD Connect is configured to sync passwords, then the on-premises password will be synced once the match is made.

      October 10, 2018 at 3:27 pm
  • Michael Reply

    For the CSV in a hard match scenario, would I use $username, $ADUSer, and $O365User as the fields?

    October 11, 2018 at 5:25 pm
    • Alex Reply

      I think in the example given here only one field/parameter is used–$username But, you can build the expression however you like, using different variables if it suits you.

      October 11, 2018 at 11:10 pm
  • Tom Reply

    Thanks for this article which I found really useful kicking off my project to migrate a bunch of users using hard matching. I created a script which covered my needs and thought perhaps your readers may find useful. It’s available on my blog here

    February 5, 2019 at 8:50 am
    • Alex Reply

      Thanks man! Very nice.

      February 14, 2019 at 4:38 pm
  • Yaniv Reply

    Thank you very much for this great information you shared.
    I would like to be sure, assuming that I have an Office365 tenant with cloud only users with Exchange on line service. in addition I have an on prem ADDS environment without Azure AD Connect Server.
    in case where I want to sync & match users from ADDS to Azure AD/Office365, do I must add the email address under each User properties and also change to all users the UPN so the email address from theirs user properties, the UPN and the email address from Office365 should be identical?

    Thank You very Much.

    January 16, 2020 at 3:45 am
    • Alex Reply

      Yes it is best practice to have the accounts matching UPN = email address.

      January 16, 2020 at 1:35 pm
  • Yaniv Reply

    OK, Thank you.
    In addition, when installing the Azure AD Connect server, do I Must choose custom installation (instead of Express)?
    if “Custom” is right way, under “Identifying users”, what of the following options should I have to select?
    Users are only represented once across all forests
    Mail attribute
    ObjectSID and msExchangeMasterAccountSID/ msRTCSIP-OriginatorSid
    sAMAccountName and MailNickName
    A specific attribute

    Again, thank you very much!

    January 16, 2020 at 2:17 pm
    • Alex Reply

      The only reason I select custom is to use OU filtering (leave certain objects out of the sync scope). The rest I pretty much leave default.

      January 17, 2020 at 6:19 am
  • Mel Wade Reply

    I receive the following error on the last command. I’m working on tracking it down.
    Set-MsolUser : Uniqueness violation. Property: SourceAnchor.

    March 4, 2020 at 5:01 am
  • Will Reply

    Would softmatching work after an initial sychronization was done? Or can this only be done on initial sync.

    March 11, 2020 at 12:47 pm
  • Simon Triggs Reply

    Thanks, nicely explained!

    August 6, 2020 at 3:43 am
  • Philip Reply

    I am having soft match issues with 2 users specifically. The message i get is “Invalid SoftMatch”

    Unable to update this object because the following attributes associated with this object have values that may already be associated with another object in your local directory services: [Mail [email protected];]. Correct or remove the duplicate values in your local directory. Please refer to for more information on identifying objects with duplicate attribute values.

    Tracking Id: 70ecbe0e-d9d2-42c7-8589-06c08f6d9e7b
    [{“Key”:”ObjectIdInConflict”,”Value”:[“399e288c-1efe-4f7d-898c-52828febf77d”]},{“Key”:”AttributeConflictName”,”Value”:[“Mail”]},{“Key”:”AttributeConflictValues”,”Value”:[“[email protected]”]}]

    We have tried everything wrt soft match but cannot correct this issue.

    We tried removing this user from Azure AD and ran the sync, it worked perfectly. However, the user lost all contents of his mailbox and onedrive. So we had to restore the deleted user.

    Is there a way to correct this without the user having to lose his mailbox and onedrive contents?

    November 13, 2020 at 12:22 am
    • Alex Reply

      Very easy, figure out where those aliases are being used in the directory. Some other object or account has those emails in use, and they can only be represented one time. If you delete users from the cloud, or remove users from the scope of the sync, they will be deleted in the cloud (and this will put the data for that user offline). You should not need to do that, just find where the rogue entries/aliases are for those email addy’s and fix em.

      November 13, 2020 at 8:32 pm
  • ty Reply

    Would this remove any emails from the mailbox or delete the mailbox when syncing the ad user with the cloud 365 user.
    I don’t think it would but just want to be sure

    March 29, 2021 at 3:21 pm
    • Alex Reply

      There is no delete operation here, it is simply matching an on-prem identity to a cloud identity.

      March 30, 2021 at 5:23 am
  • baig Reply

    Please give example of variable in CSV file, What variable are required for CSV file import.

    May 13, 2021 at 7:45 pm

Leave a Reply

Back to Blog

Helping IT Consultants Succeed in the Microsoft Cloud

Have a Question? Contact me today.