We didn’t must examine the outcomes of the MITRE ATT&CK Carbanak+FIN7 evaluation after they had been launched since inside minutes of being reside, we already had an e-mail from a vendor touting their MITRE ATT&CK prowess. This vendor acknowledged it “dominated” the analysis. Besides MITRE Engenuity doesn’t hand out rankings and awards (due to this fact, there is no such thing as a #successful). Right here’s what MITRE Engenuity says:
MITRE Engenuity doesn’t assign scores, rankings, or scores. The analysis outcomes can be found to the general public, so different organizations could present their very own evaluation and interpretation – these should not endorsed or validated by MITRE Engenuity.
Different distributors claimed they’d “eradicated the APT,” or had “entered the ring once more” whereas a number of others have been surprisingly quiet concerning the outcomes. Vendor efficiency within the analysis correlates fairly strongly to the quantity, language, and channels used to…. enthusiastically point out outcomes. And that’s a draw back for finish customers attempting to determine what all this implies.
That is to say, don’t consider every little thing you learn on the internet. Distributors that focus extra of the dialog on beating their rivals than they do enabling their clients do their finish customers a disservice.
Do The MITRE ATT&CK Analysis Outcomes Matter?
…with caveats. You needed to know that half was coming proper?
The issue with any analysis like this is:
- It’s not your setting.
- The analysis doesn’t help you abdicate conducting testing and proof of idea or worth. Excessive rankings or ‘domination’ of the outcomes doesn’t show the software will likely be efficient given your infrastructure, your staff, or your online business objectives.
- It’s informative, however not determinative.
- Don’t purchase something primarily based solely on this analysis. See #1.
- It’s centered on the TOOL.
- It’s Not centered on the expertise. There are many nice merchandise poorly deployed, not deployed in any respect, misconfigured, or missing the suitable visibility to be maximally efficient.
- It’s not inclusive of the seller’s enterprise. There are many nice merchandise on the market that lack the documentation, customer support, funding, and market success to maintain them in the long term.
- Distributors know forward of time which adversary they’ll be examined in opposition to. Provided that the MITRE ATT&CK framework is publicly accessible, this offers distributors time to arrange earlier than the evaluations primarily based on the actions a selected menace actor is understood to take. You wouldn’t take pleasure in this time to arrange the software in your setting.
- Notion and skepticism.
- Distributors pay for these evaluations, and that makes everybody suspicious. It’s pay to take part. Not pay to play.
How to make use of MITRE ATT&CK evaluations efficiently:
With all this mentioned, the outcomes do matter. Having an unbiased evaluation of vendor capabilities provides safety groups confidence – or makes them rethink – instruments presently deployed and people being evaluated. It additionally provides safety professionals proof to validate that the product they use is taking the suitable method to unravel the drawback and showcases the place gaps could exist. Over the past three years we’ve seen some wonderful use instances of MITRE ATT&CK – and a few not so helpful ones. Right here’s our guidelines for methods to maximize the profit you get from MITRE ATT&CK:
- This spherical of the outcomes could be very centered on financially motivated APTs, with Carbanak mainly targeting banks and FIN7 mainly targeting retail, restaurant, and hospitality. Finally, you need your resolution to detect any and all threats, however this analysis could also be of explicit curiosity to those sectors in relation to the granularity of protection and the timing of detection.
- Sooner or later customers get entangled so the person expertise issues. It doesn’t matter if a software finds 100% of no matter is thrown at it. What issues is how your groups work with the software. That has to stay as a consideration. One thing your staff is extra snug with on daily basis, will web you extra positive factors than one thing that “discovered extra” that they discover exhausting to make use of.
- The outcomes embrace a powerful variety of screenshots of what the platforms really seemed like throughout this train, step-by-step. Drill down into particular person outcomes and use these screenshots to get a greater sense of how a lot info is introduced to the analyst and the way simply it’s introduced.
- This work shouldn’t be finished in a vacuum. The MITRE ATT&CK staff maintains a GitHub repo of adversary emulation plans for these and different beforehand examined APTs as a strategy to operationalize menace intelligence. No matter if the merchandise you employ are examined or not, you should use these to check how your present capabilities would deal with these menace actor situations. Direct your red team to this resource for more information.
- In an effort to make these evaluations extra accessible, the MITRE ATT&CK staff has created a useful resource to enable comparison of results between vendors in addition to a technique comparison tool. With reference to really evaluating distributors, these two instruments could also be a very powerful sources accessible. They offer you breakdowns of what every vendor detected. This software could be extremely helpful for groups that wish to discover which distributors prioritize visibility for particular methods.
- Our former colleague Josh Zelonis (@josh_zelonis) has additionally up to date the code in his github repository that gives a simple overview of the newest outcomes, and now features a downloadable spreadsheet with out having to run the code immediately.
Whereas vendor sourced breakdowns could get a bit….self-congratulatory, they aren’t completely ineffective. When reviewing what distributors ship you about the newest analysis search for breakdowns that goal to construct belief with their clients by stating the information and the upsides of their resolution cleanly and clearly, minimizing self-serving bluster.
Forrester shoppers may see what number of of those distributors faired in our most up-to-date EDR market analysis ‒ The Forrester Wave™: Enterprise Detection And Response, Q1 2020.