At this year's Black Hat USA conference, Scott Small, Director of Cyber Threat Intelligence, and Harrison Van Riper, Director of Artificial Intelligence, put together a talk entitled "Procedures Make It Possible: Solving One of Cybersecurity's Most Persistent Challenges", and Scott Small presented it to an engaged crowd (Harrison was unfortunately unable to attend at the last minute).
Watch the full presentation HERE, or read below for an overview:
Procedures, referring to the "P" in the term "TTP", Tactics, Techniques, and Procedures, is a challenge that we believe truly is one of cyber security's most persistent. Tidal Cyber recently released a Procedures library and some features to really help teams use, operationalize, define and track procedures at a large scale.
Our procedures library is a very deep well of information and organizations are starting to use this data right now, but procedures have been a persistent challenge for the industry. Trying to do this precision cyber threat intelligence at a large scale has been a massive challenge that no one has really overcome up until this point.
Image 1
When we talk about Threat-Informed or Threat-Led defense, which is inherent to this type of approach to security, we're focusing on behaviors or again these TTPs. If we think about the classic pyramid of pain (image 1) we see immense benefit when we talk about Threat-Led Defense in focusing on the upper tip of this pyramid in the TTPs because everything below that is much easier for an adversary to change and rotate through their infrastructure, they can stand up new servers easily, stand up new phishing pages and domains associated with those.
It is relatively harder to change your TTPs. If you're successful in defending against them, it gives you more sustainability towards what you're trying to accomplish.
What we've been really excited to see, while not synonymous, it's very built in and closely tied to focusing on behaviors and Threat-Led Defense is the MITRE ATT&CK knowledge base, which is a knowledge base of commonly referenceable Tactics and Techniques defenders have been using since it was released more than 10 years ago.
We see so much benefit in being able to refer back to those discrete Tactics and Techniques, and both from a cyber threat intelligence perspective as well as your defensive capabilities can be mapped to MITRE ATT&CK techniques.
What has been so exciting is to see as adoption and the knowledge base has grown, we're seeing we have proof there is benefit at a more strategic level in analyzing the relevant TTPs that might be present for your organization, as well as how those relate to your defenses and you can do broad base security assessments risk management.
When we're focusing at that upper level of this second pyramid the Tactics and Techniques there's a lot of value in doing that all at scale, but we will be the first to admit and many of our customers and users feel this pain as well.
It's simply defenders need a more detailed granular level of data to do their jobs to actually write detections.
You cannot write a detection that can cover at the Technique level, but you can down to the Procedural level. You cannot write a simulation or a unit test at the higher level, but you can at the lower level. That's the type and level of information these types of operational defenders need to simply do their jobs.
Why has building a procedures library up until now been so tough to achieve?
There's been no commonly accepted definition of what a Procedure actually is. There is lack of standardization in something as tactical as tracking what fields should be associated with the Procedure, and one of the most notable is a huge and growing volume of threat intelligence.
If you are at all familiar with the MITRE ATT&CK knowledge base you might be raising the question, "Doesn't MITRE ATT&CK already have Procedures?" I'm going to say the true answer is no. They have in their knowledge base Tactics and Techniques they're known to utilize and you're going to see what they call Procedure examples (image 2).
Image 2
What this in essence is doing is saying a certain actor group did a certain technique, and that is certainly good to know when you're operating at that higher more strategic level, but we know in the real world adversaries are actually doing something a lot more more detailed. That's the level of detail that is currently missing from this when we look at just the data modeling behind these Procedure examples.
It is much more brittle as well. Currently in the MITRE ATT&CK knowledge base you can only associate a Procedure example with one Technique at one point in time, but that is not how adversaries operate. They're typically carrying out Procedures for a certain objective and might be carrying out multiple Techniques at one point in time.
Final piece of this when we look at these Procedure examples in MITRE ATT&CK they are largely additive, which means the list only keeps growing. If you look at a group like Lazarus group or APT 28 or 29, things that have had a ton of cyber threat intelligence reporting over the years, the list of Techniques they are known to use only continues to grow. That is also not a completely accurate reflection of the real landscape because adversaries are evolving. They are using new Techniques and Procedures, and they also stop using other ones, which needs to be addressed when we talk about Procedure level tracking.
Looking at this same group (APT 41) shows how they are actually doing in at least one instance, one point in time, this T 1570 lateral tool transfer. What are the tools and the software they're using to accomplish that? What are the actual command line commands they are running to accomplish that technique? That is what we mean when we talk about a Procedure, at least as it exists now in the Tidal Cyber knowledge base.
What did we actually do when there's no commonly accepted definition of what a procedure actually is?
Image 3
Various trusted sources from across the landscape and the community have defined what they consider to be a type of procedure previously (image 3). However, if you do look closely there is not a ton of close overlap between these definitions. Some of them are focused in to say that single technique approach to what a Procedure is, while some are a bit more flexible and robust, and those are the ones we tended to gravitate a little bit more towards ourselves. When we talk about operationalizing that and keeping track of it in a database, some of the data modeling started to break down a little bit more, adding another piece we needed to address.
What does Tidal Cyber actually consider as a Procedure?
"A Procedure is a clearly defined, repeatable set of technical actions that an adversary -or simulated adversary- executes to achieve a specific objective"
This is our operating definition, and I appreciate that it's pretty short and concise. Even though this is a pretty short definition every single piece of this was extremely intentional, we spent way longer than we'd like to admit in agreeing amongst our team and our customer base on what a Procedure should be defined as. What we realized is once we did actually take a step back we were taking a lot of things for granted and figure out what are we calling a procedure. This was then foundational to all of the work that followed in terms of building an actual knowledge base of them. This definition is literally plugged into the workflows that we use to then begin starting to process cyber threat intelligence reporting and turn that all into an actual knowledge base or library of procedures.
In that vein I need to give just another immense shout out to Harrison van Riper, who is the brains behind all of the artificial intelligence we leveraged that we knew was going to be essential to building this library, and making this this feature set really come alive.
The way we're leveraging AI for this work is a little bit different than what you would traditionally think of. We are in no ways talking about using AI for chatbot features and this more conversational approach, it's really kind of setting all of that aside. We are using AI in an extremely targeted fashion and putting a lot of guardrails to make sure it's accurately picking out the information we're targeting and looking at procedures from CTI reporting. Then we're also just building in a lot of automation around this. The targeted fashion of the way that we applied AI to this challenge was essential to making this work all happen.
Image 4
So this (image 4) is a decently detailed overview of the overall flow of how we generated our Procedures library. What we've done all along since we've existed as Tidal Cyber is we've leveraged a large volume of public cyber threat intelligence reporting, and certainly there is a ton of value and looking at stuff that more might be more private-facing commercial reporting or things that you might be internally generating yourselves. Even when we look at what's available in the public domain there is a lot and, there is good information available to be leveraged when trying to build a Procedure library.
We pulled down all that reporting and then we did some initial processing with our AI to be able to pull out what we are defining as Procedures from those reports and getting a very initial sense of the large data we're trying to get at. Then we looked at the large volume that was coming out. These are real reports and real Procedure sightings pulled out from those. In one Mandiant report we had 43 sightings coming out and you get a sense of the scale from some of the other ones as well.
What this is already starting to reveal is there is a lot of data to be handled and managed here. What we also did with this, not using actual AI but just some statistical similarity analysis, is we bundled up Procedures sightings we pulled from the reports to look at essentially ones that were very similar in terms of what they were trying to achieve, how they were being carried out, and where they might have existed, and those we called Procedure Clusters.
In essence we really have two objects to help us tackle this challenge but the clusters are essential in being able to start to manage the large volume of information coming out from these reports.
Image 5
You can get a sense of kind of all the things that are happening even when we look at a single report or maybe even a single Procedure sighting from those reports (Image 5). A huge benefit of what we've done and how we've modeled these objects and our knowledge base is having this ability to link Procedures with all the other important data elements in the knowledge base that you as a cyber defender would care about.
So in my realm in the CTI space certainly we want to know which groups and malware and tools are carrying out certain Procedures or being used to carry them out. When we do talk about the value of MITRE ATT&CK Tactics and Techniques we definitely want to know which of those apply to these Procedures as well.
When we shift more into the defensive side we want to know because we can know what log sources or data components are being used to carry out these Procedures that you might also leverage to give you visibility into their existence if they were to appear in your network. When we talk about what we call defensive capabilities these can be things like detection rules, or analytic ideas, threat hunts that you might execute, or even on the offensive security side of things, simulation or atomic unit tests.
All of those defensive capabilities can and are able to be linked directly to Procedures, and this is what's really helping us overcome that traditional challenge because now we can say with extreme utmost confidence that the defenses that we have are related to, or give us coverage against, specific adversary activity as it is being carried out in the real world.
Image 6
One more snapshot (Image 6) of the overall volumes and metrics of our Procedures library gives a sense of scale when we talk about there's a relatively small number of tactics, a slightly larger number of techniques, but this immense sense of scale of the Procedures sightings and clusters we pulled out just from an initial batch of CTI processing. We are going to continue maintaining and adding to this knowledge base, all of this data was pulled out from 1,500 blogs, CTI articles, and written content. This is just a starting point, but it gave us this extremely deep well that we can now continue to add to.
It's important to be able to link multiple Techniques or behaviors to a given sighting, on average we are seeing that in the real world. These average 13 sightings per reference or blog and 2 behaviors per sighting, as opposed to just one, which is what you would be able to get from the MITRE ATT&CK knowledge base. We found an average of 9 sightings per cluster, with all of this linked to 7,000 unique related threats.
Image 7
For a real look at a real blog (Image 7) is Storm0501 from the Microsoft Threat Intelligence Center, and this is ransomware group that was relatively sophisticated in being able to pivot from on-premises to cloud environments, which is not something we see every day with every ransomware group. The image shows the data we pulled out just from this single report.
This was a long and very detailed in a good way report, but this would take you roughly 40 minutes just to read through, let alone start to action the findings from this report. For a human to go through this and try to make sense of it would involve a lot of time and effort.
We're seeing a snapshot of all the sightings and clusters, groups and softwares that we pulled out. Something that was super cool is when we did this processing is that we were able to identify from within the report, as many reports do, a call out to specific Microsoft defensive capabilities. In this case, it was a Sentinel rule that was referenced in the report that would be relevant to defending against this group and its attacks. We saw that called out in the report, and when we took a look inside our knowledge base in what we call our product registry, a list of capabilities that end users are actually using in their environment, we can see this exact same Sentinel rule already existing in the product registry.
So if it's not obvious, what this is allowing you to do is track down to that granular Procedure level, know what a given actor actually did in the real world, and then have immediate knowledge and confidence that you might have defenses in your stack already that could relate to and potentially defend against this activity, and possibly you already had that turned on.
Image 8
Last thing is just super exciting, this triad (Image 8), perfectly coming into play already in the knowledge base from the great atomic red team, Red Canary's open source testing library, and they also had an existing unit test that directly related to this activity that was observed, using a PowerShell script to tamper with certain security products by sending certain registry keys. We were able to simulate that exact same behavior as well because we had defenses, tests, and real world evidence from a recent timely CTI report.