Evolve Version 1.6

evolve-logo

New features in #EvolveTool!

API Doc

The new part of this feature is the HTML to outline all the various URLs that can be used to interact with Evolve. They have mostly been there in the background already, although a couple of the URLs are new. These URLs give the ability to have Evolve work in a sort of headless mode. You can use any scripting language that can GET or POST. The return data is in JSON format.

evolve-api

Plugin Search

The plugin list for Volatility commands keeps growing thanks to the great support by the core dev team and all the gracious developers in the community. I’m sure the annual contest giving away money had no part in it either. Anyways, I figured it would be helpful to have the ability to run a quick search over the plugin list where you can type a part of what you are looking for. It doesn’t support any fancy matching though, and just puts wildcards in front and behind whatever you type. It searches while you type and you can use [ESC] or click the X to the right to quickly clear the box.

Try typing ‘dump’ and you will get a list of those plugins that Evolve doesn’t yet support:

evolve-plugin-search

Teaser: Volatility Command Line Options

Speaking of Volatility plugins that aren’t supported in Evolve, I was able to dig into the Volatility core and determine where those options are stored in Object Oriented (OO) data structures. You will see a couple new URLs listed in the API doc that take advantage of this new found knowledge.

The first is a list of all the default options that Volatility has. You can see those by running ‘vol.py -h’ in the shell, or accessing the API here:

evolve-options

The second builds on the above URL to get more specific options that any of the plugins are allowed to add into the list to accept during processing. You can display the full collection of options with the plugin specifics listed at the bottom with ‘vol.py dumpregistry -h’ or you can get only the specific options that each plugin adds by accessing this API:

evolve-options-plugin

Some Background

I originally took on the project of making Evolve to learn Python. I wanted to build something that required research and learning, and something that would make me stretch. I could have written this project in any language and just made calls to vol.py to get things running. I’ve seen many of these projects pop up over the years and they work great. I decided to fully integrate with Volatility to better learn Python and have more power and control over how I hand off processing jobs. That decision has caused some headaches, so I try to share the solutions when I can.

The challenge in here is that Volatility uses a library for parsing command line options that is built into Python. This setup works great for the scenario that Volatility is typically run, at a command prompt, where the user has to supply all those parameter names and values up front. It doesn’t make it so easy to fetch a list of the various options any of the plugins might want to take advantage of because those options aren’t built into OO to just get.

The plugins written for Volatility interface with optparse to add in the recognition for the short and long parameter designations. The optparse object is a member of Volatility’s ConfObject class, but not really integrated.

To get to the list of default options is pretty straight forward. You have to build a ConfObject anyways when integrating with Volatility, and the default options all come along as it is built.

To get the options that any of the plugins add on top of the default, you have to utilize the ConfObject again, but as a parameter when initiating the plugin of choice. The result is a full list of all the options that are now available, including those earlier found default options. You have to do the work of differentiating the newly added from the defaults. To prepare for doing this, I created a second ConfObject and pulled the new list in.

The next challenge is the structure of the options being held in the optparse object isn’t really straight forward. The items are not provided in a list, so you can’t do much with them as they sit. Fortunately they are iterable, so that allows for Python to use in as a collection in a for loop. You can see the debug view getting to one of these options here:

evolve-options-debug

I chose not to deal with all of those properties since I don’t think the Volatility plugins have that much ability to manipulate, but I will doing some further testing on this. If there are more properties that are needed in Evolve, they are fairly simple to add into the JSON return at this point.

After grabbing the handful of properties into a dictionary, I stuff that dictionary into a tuple. You can read more about the differences since I won’t go into that here. The tuple made it easier to work with, and I don’t have the need to change that object.

With a tuple of options from two ConfObjects, I could now determine which of those options were added by the provided plugin. Now I had to repeat that process for every plugin available in Volatility, and I am very thankful for loops and automation.

Check it out on GitHub.
https://github.com/JamesHabben/evolve

I hope you find these new features helpful, and the upcoming features exciting. Please reach out if you have any questions or feature suggestions for Evolve.

James Habben
@JamesHabben

Malicious USB Devices

I put together some slides over a year ago after working several cases involving suspicious USB devices. The slides cover some studies that threw USB devices on the ground, and a couple scenarios from the Verizon Data Breach Digest (shameless promo). There is a lot of significance in these links, and presenting these slides has shown me that this threat is very widely underappreciated.

We, and InfoSec and even more specifically as DFIR specialists, are not immune to an attack conducted through a USB device. Some of us hold a more traditional stance that the forensic workstation remains unconnected from any network connection, but this is an increasingly difficult stance to hold in more recent years. The source evidence drives are getting larger and larger, and this is forcing examiners to take advantage of network storage solutions. We are also seeing many tools with increasing reliance on network for collection and analysis through integrations with other products.

I worked with some others on my team to develop a forensic methodology that has been tried and true. It gives us the best chance of preserving any data that may exist on this USB device and protects our forensic infrastructure from any potential attack. I am putting this methodology in this blog post in an effort to get it out to a wider audience than the folks that have sat through one of my talks. You can see a video from BsidesSLC if you would like, or feel free to contact me and I would be very happy to head out somewhere to give the talk live.

The High Level Methodology

  • Collect image
  • Collect volatile data
  • Analyze file contents
  • Analyze volatile data
  • Collect firmware

The Methodology Steps

  1. Collect Image
    1. Physical machine
    2. Linux forensic boot cd
    3. Hardware USB write-blocker
    4. dd, dcfldd, linen, etc
  2. Collect Volatile Data
    1. Physical machine
    2. Windows OS – Small HDD, forensic wipe (0x00)
    3. Software USB write-blocker
    4. Collect before images: HDD & RAM
    5. Prep volatile collection tools & scripts
    6. Start PowerShell diff-pnp-devices.ps1*
    7. Insert USB, wait for a minute
    8. Finish diff-pnp-devices.ps1
    9. Finish volatile tools & scripts
    10. Collect image: HDD & RAM
  3. Analyze File Contents
    1. Automated AV scans
    2. IOC Searches
    3. File format specific tools
  4. Analyze Volatile Data
    1. Compare disk images
    2. Compare RAM images
    3. Review new devices from diff-pnp-devices.ps1
    4. Look for evil
  5. Collect Firmware
    1. Only needed if device has additional device
    2. Identify controller chip – ChipEasy
    3. Acquire correct tool to dump firmware
    4. Reverse engineer firmware

The PowerShell Tool

The script mentioned in this methodology is in my GitHub. It does a very simple thing.

https://gist.github.com/JamesHabben/

  1. Get a list of PnP devices
  2. Wait for user to continue after inserting USB
  3. Get another list of PnP devices
  4. Compare the two lists and print the differences

Takeaway

These USB devices can do a log of damage, and I continue to see a lot of very surprised faces during my talks. It is a real threat (Stuxnet!) and it needs to be accounted for in your incident response.

Here are the slides with a bit more information, and please reach out if you have questions.

James Habben
@JamesHabben

Skills and Knowledge for InfoSec

As a consultant for an incident response firm, the engagements we get are typically fairly fleshed out in terms of being a security or operational incident. Every once in a while, we have calls come in that seem very security focused when described by the customer contact but after arriving onsite they work out to be an operational incident. It can take a lot of experience to really take the problem down to its roots to even make an approach at root cause.

There have been a lot of discussions flying around about various skill levels required for InfoSec jobs. Along with that, many have expressed concerns about the job postings and the requirements that get listed, and I joined in a little while back. Others have made bold statements that InfoSec jobs shouldn’t be entry level jobs since the skills needed are gained through other roles. I am in the middle on that feeling. I have met some very smart people that seem to just ‘get it’ and do well in InfoSec without other prior experience, and I have also met people that have spent 20 years in IT that don’t understand some of the basic concepts. It really goes both ways.

Although I do not hold the opinion that InfoSec is not an entry level job, I do think there is a lot to learn that can be extremely beneficial to a role in InfoSec. I recently went out on an engagement that required an incredibly deep understanding of routing and switching concepts. I am not talking about having the magical skill of being able to calculate subnets in your head (although I was able to do that at one point in my past). I was facing one of those security vs operational incidents I mentioned above.

I spent a lot of time in a network admin role. I took over management of a medium sized business nationwide network. The network had previously been built (incorrectly) by a supposed networking expert. I spent a lot of time understanding what the problems were and addressed each of them as components to an overall problem. The result was a lot of positive comments from end users about the improved speed and reliability of the network. I ended up rebuilding about 90% of that network at another point later during a move from Frame Relay to MPLS. I spent time studying proper network design and function to make sure I was doing things correctly.

I mention this because the recent engagement I went out on involved a few components that at a glance could easily appear like very serious security issues. A proper understanding of networking principals, and along with that the OSI model, was absolutely essential. There were a ton of components that when viewed as a whole would lead down a ton of rabbit holes.

In Incident Response especially, we need to have the ability to view the problem as a whole, but also be able to break the problem down into the various smaller components. That is what an investigative / analytical mind does. Those components often times are not all contributing to the problem. They are often times a symptom or result of another problem. If you don’t have the knowledge to separate those components from the overall problem, then your incident is going to be much more difficult to resolve.

To those of you that are considered entry level:

  1. You can learn on the job, but you need to make sure that you take on a job that will give you that opportunity. Make sure that your role will be involved in technology across the board to get the exposure.
  2. Find a mentor that seems to be the right personality for you. That mentor can guide you to various topics that would be very beneficial to your career in InfoSec.
  3. Understand that there will be jobs that are requiring skills that you don’t posses. The postings don’t always reflect the true picture of what that company is willing to hire.
  4. Ask your mentor for help in applying. Ideally, that mentor will be well connected in the industry and would have already started to expose you to various people around the industry. If that hasn’t happened yet, there might be a reason for it (maybe you aren’t ready), or your mentor might not be supporting you as well as needed.
  5. Show your efforts in learning. Make sure that people understand the time you are putting into improving yourself. This doesn’t mean that you constantly brag about your self, but you can demonstrate your learning in many different ways.

InfoSec can be a tough place to work since we have to know a little about a lot. Embrace your curiosity.

James Habben
@JamesHabben

Infosec Jobs

I unintentionally started a small storm on #infosec twitter the other day. In that storm I received responses in extremes that I didn’t even know existed. I wanted to give a bit more depth to that thread than what 140 characters can convey.

Let me clear the air a bit first. That was not an attempt to broadcast me searching for a job. I am not ‘on the market’ and no I didn’t apply to any of those jobs that I tweeted about. That was also not an attempt to phish for compliments, ego inflation, or many other interesting things I was accused of in private messages and subtweets.

The Backstory

Here is what happened. As a Senior Consultant for an Incident Response firm, I periodically take a look at other jobs that show up on the market for situational awareness. I have found that the industry titles seem to vary quite a bit as the responsibilities of my Senior title seem to be equivalent to other firms’ Principle, Lead, Manager, or even Director titles. There are Managers that don’t manage, and there are Leads who do. It is pretty spread out.

While looking at one of those postings, I said to myself “Hmmf. I can’t qualify for this job with the same title at a different firm.” I then did a mental inventory of my professional network and started identifying people I have connected with in the past that I could reach out to if I were to go through the application process for that job. That took me to another statement to myself, “How does someone with less time in the industry (and likely less connections) make it past any of these requirements.” Naturally, this mostly applies to folks that are very new to the industry. Then off to twitter I went.

The Response

The responses that I received came in through all different avenues: text, LinkedIn, Twitter, Email, even one on Instagram. I would have probably seen a few on Facebook, however I do not currently possess any credentials to logon to that god-forsaken website.

The responses also varied in their messages. I got a few that definitely do not need repeating, and I’m not exactly sure where the motivation came from behind them. On the other extreme, I received a response from someone (or someones) in my network at every one of those companies that I mentioned in a tweet. Many have very graciously offered to put me in touch with someone to get me hired there, and this would bypass the HR and recruiters to ensure that I was considered. I am thankful for all of those responses since it shows how much of a community I have with those individuals. That wasn’t my intent though, and I hope this post makes that more clear.

The Bypass

What makes me so special? Why did so many people offer to take me around the filters?

I have been in the industry for a really long time
This doesn’t mean I know what I am doing. At the most basic level, it means I have been able to fool enough people for a long enough time to stay employed. While that is not an accurate representation of me as a whole, my length of time makes no other indication.

I work for BigName firm
Yup, I do. I also know a lot of people that are very new to the industry and still have a lot more to learn that also work for big name firms. This also does not show anything about me.

I have followers on Twitter
Ya, I have some. There are plenty of people that have tons more followers than I do. There are also folks that I know who are extremely knowledgable and very good at their jobs with a double digit follower count or no Twitter account at all. It is quite easy to look smart on the internet when I have time to plan and research what I decide to make public.

I have a blog to share my thoughts and research
Now we are getting somewhere. My posts on this blog are a far better representation of who I am than some of the things mentioned above this. Although, it is still quite easy to look smart here because of the time allowed for planning and research. What is more difficult to fake, however, is my communication skills. My writing here is a clear demonstration of my abilities to convey points to a widely varied audience, although the majority of readers here seem to be more technically focused. We are finally looking at something that employers could use in their evaluation of me as a potential candidate.

I have spoken at conferences
Another point that gets more into who I am. We are also getting into an area that is a bit harder to fake. Sure, there is still prep and research involved ahead of the point in which I am delivering my talk, and I certainly do plenty of that. What is nearly impossible to fake is my presence in the room and my authority on a topic. As a bonus, there have even been recorded sessions I have delivered that are available publicly on the internet, and this allows me to provide another demonstration of me to a potential employer.

I know lots of people
This seems to be the most significant point in this list. My time in this industry has put me in contact with a large amount of people. People I have made enough personal connection with to where they care about my wellbeing in terms of being employed. People that have calculated the risk involved, and are still willing to put their reputation and connections on the line to help me.

The Takeaway

Every job I have held has been through some connection. I have not received any jobs where I applied through some web portal. I have tried some of those in the past, and have many times not even received a courtesy rejection letter. My experience is often not categorized as ‘cybersecurity’ or ‘infosec’ depending on which recruiter I talk to.

If you are struggling with breaking into the industry, here are my pointers (which is really an echo of so many others’ great advice as well):

  1. Start a blog and post about stuff. it can be research, thoughts, infosec challenges, or a number of any other topics. If you would like some help getting started on this, please feel free to reach out to me. I enjoy helping motivated people. The information you put in public can give employers more data to consume when you are short on other requirements.
  2. Work on your soft skills. I have a few posts up here about how soft skills can be improved in various ways, and I intend to continue these posts. You need soft skills in this industry if you want to get into the better jobs. Interviews will make or break your job application in the end.
  3. Go out and meet people. This can be virtually or physically. There are quite a number of people who I would consider to be a step above acquaintance in terms of relationship who I have never met. Some I might not even know their real names! Connections will get you in the door.

Last point, I am in no way criticizing the companies that put up those job listings. They have reasons for asking for those requirements, probably because something has bit them in the past. I am not saying they should relax their requirements either. What you as an applicant has to recognize is that the requirements are not always requirements. You need to make meaningful connections to get access to the people that are making the hiring decisions.

I hesitated about initially sending those tweets, and again about writing this post. I am sure another round of hatemail will be heading my way soon. Also, it is not easy to publicly admit that I don’t qualify for so many positions in the industry I have been in for so long. I probably don’t even qualify for the requirements of my current position either. As someone facing the challenges of the hiring process, I hope this can give some comfort and help in your quest.

Good Luck!

James Habben
@JamesHabben

Compile Time Analysis of NotPetya

I had a thought the other day about some of the NotPetya / M.E.Doc (Medoc) initial infection vector that was released last week. I wondered if the attackers had full and complete access to the Medoc network and even source code. Did they have the ability to inject the malicious code into the source repository? Then just sit back while it got included into the build that was released to the customers. Or did they have more restrictive access to, say, the FTP server holding the update files?

Here are a couple references for those that didn’t keep up with the fast moving data related:

ESET wrote about the code that was injected into the ZvitPublishedObjects.DLL file

Cisco Talos wrote about their investigation on behalf of Medoc with access to internal servers

I wanted to see if I had some data to support the level of access the attackers had, even though I don’t have access to the internal systems. It is very possible that Talos has already made this determination and just not made the statement public.

Executable (exe) and Dynamic Link Library (DLL) files have a timestamp in the PE header known as the compile time. This is often used to identify characteristics of attackers, as many times they forget to cover their tracks with this field.

In my inspection, it appears to me that the attackers had full and complete access inside the network with the ability to inject their malicious code into the source repository (whatever Medoc uses).

Single File

The first thing I did was use PEStudio to analyze the affect DLL file. The compile time in this view appeared to be within a normal time period based on other information that has been provided surrounding this attack.

The timestamp is shown in PT since my system has that set. The UTC time is 2017-06-21 14:58:42. I checked a couple other PE files and found it to be within the same time range.

Multiple Files

There are hundreds of files in the Medoc program folder and a large percentage of them are EXE or DLL. It makes no sense to check things one by one when we have the power of automation. Check out AdamB’s post on clustering analysis using compile times.

I took a similar approach and wrote up a quick EnScript to identify PE files and then parse the 4-byte compile time value and dump it out. I used this approach because EnCase allows me to analyze and extract data without having to mount or copy files out. Also, I have been writing EnScripts for a couple years. 😉

The result tells me that the attackers got the code injected at the source. Look in the following screen shot and you won’t find any times that are overlapping. It also appears to have a reasonable amount of time between files based on the sizes.

Unknown

What I can’t answer with my own analysis based on the data available is whether the attackers stole the source code and compiled it on their own. According to Talos, they had access to change the NGINX configurations to have the internal server proxy connections out to a server under their control. Did the attackers inject the code into the internal repository? Did the attackers steal the source code and compile it outside of the Medoc network?

Hope you found this as a nice little interesting tangent from your daily tasks. i did!

James Habben
@JamesHabben

Soft Skills: Respect

I’m sorry that you interpreted our discussion in the way that you did

I was recently the recipient of this statement. In the best case, it is frustration to hear or read this. In the worst case, it can be depressing and completely demoralizing. Let me provide a few examples with a little elaboration:

As a teacher:
I’m sorry that you didn’t understand what I was saying. It is only my job to talk at you and you are responsible for listening and figuring out the message I was intending to deliver.

As a consultant:
I’m sorry that you didn’t get what I explained. I do this on a daily basis and know so much. I also don’t have time to explain these things to you.

As a friend:
I’m sorry you didn’t interpret that conversation correctly. I was trying to help you to be a better person, but you couldn’t get over yourself enough to hear what I was saying.

As a potential employer:
I’m sorry you understood my statement incorrectly. I hold all the power here and you should be bowing to me to show that you are worthy of a job here.

As a boss:
I’m sorry you misunderstood my instructions. You should have listened better. I know exactly what I was saying and it is your fault you don’t.

Own It

When you decide to take on the task of explaining something, it has become your responsibility to ensure that all of the recipients correctly understand the message. If they don’t, it is YOUR FAULT. This may come as a news flash to some people, but there is no such thing as a mind reader. If you do not explain things in a way that people can understand, you are setting someone (or someones) up for failure.

From Jessica Hyde:
The phrase “I’m sorry” is absolutely meaningless when the onus is then placed back on the party being apologized to in the qualifier. Apologies should be formatted ” I am sorry I…” not “I am sorry you…”. Argh. The second is just rude.

From Mitch Impey:
the basic rules are valid in every industry and respect is key 🙂

Treat everyone with respect. Someday it will bite you.

James Habben
@JamesHabben

Fileless Application Whitelist Bypass and Powershell Obfuscation

Organizations are making the move to better security with application whitelisting. It is shown in the offensive side of the computer security industry. The frameworks, such as Metasploit, PowerSploit, BeEF and Empire, are making it very easy to build and deploy obfuscated payloads in all sorts of ways. It has become so easy that I am frequently seeing attackers using these techniques on systems that do not employ the added security measures.

There are plenty of solutions to mitigate these types of attacks, however I find they are not always configured properly. Take a read through @subTee’s Twitter feed and GitHub for many of the more creative ways he has shared. The attackers have raised the bar with the use of these techniques. If defenders aren’t deploying appropriate defenses, shame on them.

It Works

I wanted to share with you a few things from a recent engagement. The attacker had installed the backdoor almost a year before detection. They got in through a phishing attack, as in most cases. The detection? A kind and friendly letter from a law enforcement agency that had taken control of the command and control (C2) and was observing traffic to identify victims. The beaconing was surprisingly frequent for as careful as the attacker was in some other areas.

Can you confidently say that your endpoints are safe from these types of attacks? You don’t have to deploy prevention or detection tools for every part of the kill-chain, but you would be best served to have at least one. Or not, YOLO.

Persistence

In order for any malware to be effective, it has to run. I know, it is a revolutionary statement. It is a concept that is missed by some and it is a very critical piece. There are a finite number of places that provide malware the ability to get started after a system has been rebooted. Keep in mind that the user login process is a perfectly acceptable trigger mechanism as well, and there are a finite number of places related there too.

Just like the various creative and new application whitelist bypass techniques, there are creative and new persistence mechanisms found periodically. Adam has posted quite a few of them on his blog. The good news is that the majority of attacks don’t get that creative because they don’t have to.

The run mechanism in this system was HKCU\Software\Microsoft\Windows\CurrentVersion\Run

You can see that the attacker has chosen to use cmd to start mshta. The code following that command is javascript that when run creates an ActiveX object that loads more code from a registry path. So many layers!

Obfuscation

The run mechanism loads in code that has been obfuscated by the attacker. It starts off creating another ActiveX object and then using powershell.exe to interpret the code following. The obfuscation is enough to prevent keyword searches from hitting on some of the known API function involved with these attacks, but it is not a difficult one to break. All you need is a base64 decoder. I recommend that you use a local application based since you never know what kind of thing will be showing up and an online javascript based decoder is susceptible to getting attacked, whether intended by the attacker or not.

The path referenced in the run value and pictured below is HKCU\Software\Licenses. I have blurred some code and value names in an abundance of caution for potential unique identifiers.

Decoding

My preferred tool for decoding this is 010 Editor. It is not free, but it is worth its license cost for so many things.

First thing to do is copy the text inside those quote marks. Don’t include the quotes since that will throw off your base64 decoding.

Now you just create a new document in 010 and use edit > paste from > paste from base64.

Magically you have some evil looking PowerShell code.

Take a look over at this powershell code from @mattifestation and you will hopefully notice that it follows the same flow. It looks like someone simplified the code from the blog post by removing the comments and shortening the names of the variables. Otherwise it is identical.

Payload

Line 2 of the PowerShell code loads the registry data from a different value in the same path. Line 14 then copies the binary data from the variable into the memory space for the process that was created, about 15kb of it. Line 15 then kicks it off, and the binary code takes over.

The binary is a shell code that decompresses a DLL image with aPLib and writes it into the same process space. The resulting DLL has not been identified by any public resources, so I can’t share it with you here. It is very similar to Powersniff and Veil, for those interested in the deeper analysis.

Raise Your Bar

Defenders, the bar has been raised by the attackers. Make sure that you are following suit, or better yet, raising it even higher.

James Habben
@JamesHabben