Busting the myth behind Secure Score

For someone who writes so extensively on Microsoft / Office 365 products, especially with regard to security, I haven’t said much about their Secure Score tool on this site yet. Probably I have mentioned it once or twice, but I haven’t opined on it, officially.

So here’s the thing. It is in fact a helpful tool for admins–I mean, just having a checklist is half the battle for a lot of small 1-5 person IT shops out there. And this tool does give you that–a checklist–and it’s pretty good! Most people don’t even know what to do, or where to start. So this solves that problem very nicely. A lot of details are provided here including notes and links about how the action items help you meet various compliance controls, etc.

Plus, clicking the Learn more button on any given task will provide another pop-up page describing what to do specifically, and actually link you out to the setting page or admin portal to which the checklist item is referring. Usually it reads Launch Now. So very cool.

But there’s a dark side to it also, which I am sure many of you have noticed. It appears that this scorecard is sort of designed to… how do I say this… up-sell you into other products.

And I take issue with this for a number of reasons. But I’ll just focus on one example. Mobile Device Management. Microsoft offers us a free version of MDM baked into Office 365, as well as Microsoft Intune (which is included in all the Microsoft 365 plans).

Now, even though I typically recommend a Microsoft 365 bundle, which would include Intune, here is an interesting piece of info: when you total up the points awarded by Secure Score for fully implementing an MDM policy in the native 365 offering, vs. configuring Intune device configurations and compliance policies, there is a huge difference.

• Office 365 MDM total = 25 points
• Intune total = 120 points

So the question is, does having all the recommended settings in place for Intune really yield a 95 point difference–is it “that” much more secure? Even subtracting out the fact that the free MDM does not include the ability to provision device profiles/policies related to Windows 10 and macOS (subtract 40 points: 20 points for Win10, & 20 for macOS-related device management items)–we’re still looking at a delta of 55 points, just for MDM.

Conclusion: If you are chasing the numbers in this tool alone, you will end up spending more money with Microsoft. It’s clever, but somewhat annoying. After all, a small business governed by compliance can meet various controls with whatever products they wish. And Microsoft even gives you the option to Ignore or dismiss certain tasks based on the reason “Third-party”–like if you had MaaS360 in place instead of one of their products. But isn’t it interesting that they reward you with more points for their more expensive product?

Well, you might say, but Intune is capable of so much more! Application management, app deployment, integration with Windows Defender ATP, etc., etc. Sure, but look, I could configure my own Intune profiles and compliance policies with the same exact settings that I dictate via a free version of Office 365 MDM, and Secure Score will give me 55 more points for it, even though I haven’t moved the needle one iota for my organization.

So I am just calling out the obvious here–there is a certain amount of B.S. to this tool, and you need to be aware of that.

Some products/add-ons that give you a good boost in points, I am even in favor of: for example Office 365 ATP. I have seen it stop real threats with my clients. It also happens to be included with my favorite bundle, the Microsoft 365 Business SKU. Having Safe Links & Safe Attachments configured alone will boost your score by 30 points.

Then look at Cloud App Security, which is bundled in the E5 plan (which none of my customers subscribe to); it has some very cool-looking capabilities, among which are some new “indicators of compromise”–where we can set down a trip wire: when someone runs across it, we get alerted (e.g. new OAuth apps or basic authentications hitting the tenant, etc.).

If you do all of the recommended actions with relation to that product, it’s worth a whopping 100 points. And maybe that’s worth it? But E5 is pretty damn expensive for an SMB, and it’s a hard sell to customers when the Microsoft 365 Business bundle is literally 1/3 the price, and they ask, “Well isn’t there enough security in this bundle already? Why would they offer it if it weren’t already secure enough? How much security is enough security?” Good questions.

The point is, in general enabling stuff that is hidden away in E5 tends to reward you with the most points, so you can see that chasing the numbers naturally leads you to escalate yourself into that top-end subscription. Like I said, clever for making money, but I don’t know if I trust it as a good assessment tool (as it is claimed to be).

My other beefs

Here’s the thing, Microsoft. We see what you’re doing, here. The more subscriptions you can convert to E5, the better it is for you: I mean, it’s the top dollar subscription bundle that you sell, right?

I was recently at a presentation given by a Microsoft Security Engineer, and he said to the audience, “You don’t want your tenant to get popped because you didn’t have E5.” Literally. He said that. He lost all credibility in my eyes in that moment (“You’re just a sales guy…” I thought).

• First off–buying a more expensive software product does not necessarily make me more secure. When I was a kid, I bought a pair of Air Jordan’s because I thought they’d make me jump higher. Guess what? They didn’t. My dreams of becoming a professional athlete were crushed, and now I work in IT. But in that experience, I also learned how not to be a complete dupe for stupid marketing claims.

• Second–don’t tell me that I need to buy your top dollar subscription because of “security,” all while implying that I am somehow inherently at risk in one of your other subscription bundles. Are you literally telling me I should be afraid for all those small businesses out there who are enrolled in your Business edition SKU’s? Come on. Maybe they should all move to Google docs or whatever that stuff is called…

The reality is, a partner or small IT shop who does a really good job of managing their Microsoft 365 Business tenant, tracking inventory of identities, devices and software, and maximizing built-in tools like MFA/modern auth, MDM/MAM, SSO, Autopilot, etc., might be out-performing some enterprises who bought E5 but can’t keep good records of what the hell is supposed to be part of their environment (or not part of it).

And one more thing: “Security” is not only for the Enterprise, Microsoft. Small Businesses need a lot of the stuff that is held “hostage” at the Enterprise level. You should seriously consider three things:

1. Do not make “security” the major differentiation for moving up level into ever higher subscriptions–we want your best (or at least most effective) security capabilities baked into every level, even the lowly Small Business. Instead, you should differentiate the bundles based on products that actually add value to a business like the Skype/Teams telephony stuff–not everyone wants or needs it right? But some do. Not every org needs advanced compliance tools, but some do. But EVERY business needs security.
2. Why isn’t there like a Business B3 and B5, similar to the Enterprise side? I think it is insane the pricing structure right now for moving from Business to E3 to E5. There must be another level in-between, where we get some additional items that really make sense from a security perspective (Conditional Access, Password Write-back, Cloud App Security, Windows Defender ATP, etc.), and maybe some telephony pieces, I dunno. The M365B bundle is 20.00 USD, so maybe 35.00, 40.00 for the B5? I mean, it would still be WAY more palatable than frickin’ 54.00 or whatever the hell it is right now for E5.
3. I don’t ever want to hear another security “expert” that works for your company make a statement like I mentioned above again–it’s just unacceptable. Lost some serious respect that day.

End of rant. Sorry World, for adding yet another complainy-pants blog post to the Internet. We’ll get back to our regularly scheduled useful articles soon.