Evolve Version 1.6

evolve-logo

New features in #EvolveTool!

API Doc

The new part of this feature is the HTML to outline all the various URLs that can be used to interact with Evolve. They have mostly been there in the background already, although a couple of the URLs are new. These URLs give the ability to have Evolve work in a sort of headless mode. You can use any scripting language that can GET or POST. The return data is in JSON format.

evolve-api

Plugin Search

The plugin list for Volatility commands keeps growing thanks to the great support by the core dev team and all the gracious developers in the community. I’m sure the annual contest giving away money had no part in it either. Anyways, I figured it would be helpful to have the ability to run a quick search over the plugin list where you can type a part of what you are looking for. It doesn’t support any fancy matching though, and just puts wildcards in front and behind whatever you type. It searches while you type and you can use [ESC] or click the X to the right to quickly clear the box.

Try typing ‘dump’ and you will get a list of those plugins that Evolve doesn’t yet support:

evolve-plugin-search

Teaser: Volatility Command Line Options

Speaking of Volatility plugins that aren’t supported in Evolve, I was able to dig into the Volatility core and determine where those options are stored in Object Oriented (OO) data structures. You will see a couple new URLs listed in the API doc that take advantage of this new found knowledge.

The first is a list of all the default options that Volatility has. You can see those by running ‘vol.py -h’ in the shell, or accessing the API here:

evolve-options

The second builds on the above URL to get more specific options that any of the plugins are allowed to add into the list to accept during processing. You can display the full collection of options with the plugin specifics listed at the bottom with ‘vol.py dumpregistry -h’ or you can get only the specific options that each plugin adds by accessing this API:

evolve-options-plugin

Some Background

I originally took on the project of making Evolve to learn Python. I wanted to build something that required research and learning, and something that would make me stretch. I could have written this project in any language and just made calls to vol.py to get things running. I’ve seen many of these projects pop up over the years and they work great. I decided to fully integrate with Volatility to better learn Python and have more power and control over how I hand off processing jobs. That decision has caused some headaches, so I try to share the solutions when I can.

The challenge in here is that Volatility uses a library for parsing command line options that is built into Python. This setup works great for the scenario that Volatility is typically run, at a command prompt, where the user has to supply all those parameter names and values up front. It doesn’t make it so easy to fetch a list of the various options any of the plugins might want to take advantage of because those options aren’t built into OO to just get.

The plugins written for Volatility interface with optparse to add in the recognition for the short and long parameter designations. The optparse object is a member of Volatility’s ConfObject class, but not really integrated.

To get to the list of default options is pretty straight forward. You have to build a ConfObject anyways when integrating with Volatility, and the default options all come along as it is built.

To get the options that any of the plugins add on top of the default, you have to utilize the ConfObject again, but as a parameter when initiating the plugin of choice. The result is a full list of all the options that are now available, including those earlier found default options. You have to do the work of differentiating the newly added from the defaults. To prepare for doing this, I created a second ConfObject and pulled the new list in.

The next challenge is the structure of the options being held in the optparse object isn’t really straight forward. The items are not provided in a list, so you can’t do much with them as they sit. Fortunately they are iterable, so that allows for Python to use in as a collection in a for loop. You can see the debug view getting to one of these options here:

evolve-options-debug

I chose not to deal with all of those properties since I don’t think the Volatility plugins have that much ability to manipulate, but I will doing some further testing on this. If there are more properties that are needed in Evolve, they are fairly simple to add into the JSON return at this point.

After grabbing the handful of properties into a dictionary, I stuff that dictionary into a tuple. You can read more about the differences since I won’t go into that here. The tuple made it easier to work with, and I don’t have the need to change that object.

With a tuple of options from two ConfObjects, I could now determine which of those options were added by the provided plugin. Now I had to repeat that process for every plugin available in Volatility, and I am very thankful for loops and automation.

Check it out on GitHub.
https://github.com/JamesHabben/evolve

I hope you find these new features helpful, and the upcoming features exciting. Please reach out if you have any questions or feature suggestions for Evolve.

James Habben
@JamesHabben

Unified We Stand

Big news happened at #OSDFcon this week. Volatility version 2.5 was dropped. There are quite a number of features that you can read about, but I wanted to take a few minutes to talk about one feature in particular. There have been a number of output options in the past versions of Volatility, but this release makes the different outputs so much easier to work with. The feature is called Unified Output.

This post is not intended to be a ‘How To’ of creating a Volatility plugin. Maybe another day. I just wanted to show the ease of using the unified output in these plugins. If you are feeling like taking on a challenge, take a look through the existing plugins and find out which of them do not yet use the unified output. Give yourself a task to jump into open source development and contribute to a project that has likely helped you to solve some of your cases!

Let me give you a quick rundown of a basic plugin for Volatility

Skeleton in the Plugin

The framework does all the hard work of mapping out the address space, so the code in the plugin has an easier job of discovery and breakdown of the targeted artifacts. It is similar to writing a script to parse data from a PDF file verses having to write code into your script that reads the MBR, VBR, $MFT, etc.

To make a plugin, you have to follow a few structural rules. You can file more details in this document, but here is a quick rundown.

  1. You need to create a class that inherits from the base plugin class. This gives your plugin structure that Volatility knows about. It molds it into a shape that fits into the framework.
  2. You need a function called calculate. This is the main function that the framework is going to call. You can certainly create many more functions and name them however you wish, but Volatility is not going to call them since it won’t even know about them.
  3. You need to generate output.

Number 3 above is where the big change is for version 2.5. In the past versions, you would have to build a function for each format of output.

For example, you would have a render_text to have the results of your plugin output basic text to stdout (console). Here is the render_text function from the iehistory.py plugin file. The formatting of the data has to be handled by the code in the plugin.

If you want to allow to CSV output from that same plugin, then you have to create another function that formats the output into that format. Again, the formatting has to be handled in the plugin code.
For any other format, such as JSON or SQLite, you would have to create a function with code to handle each one.

Output without the Work

With the unified output, you define the column headers and then fill the columns with values. Similar to creating a database table, and then filling the rows with data. The framework then knows how to translate this data into each of the output formats that it supports. You can find the official list on the wiki, but I will reprint the table for a quick glance while you are reading here.
There is a requirement in using this output format and it is in the similar fashion to building a plugin in the first place.
  1. You need to have a function called unified_output which defines the columns
  2. You need to have a function called generator which fills the rows with data

Work the Frame

The first step in using the unified output is setting up your columns by naming the headers. Here is the unified_output function from the same iehistory.py plugin.
Then you define a function to fill each of those columns with the data for each record that you have discovered. There is no requirement on how you fill these columns, they just need the data.
The other benefit from this unified output is that a new output format can be easily added. You can see the existing modules, and add to it by writing code of your own. How about a MySql dump file format? Again, dig in and do some open source dev work!

Experience the Difference

Allow me to pick on the guys that won 1st place in the recent 2015 Volatility plugin contest for a minute. Especially since I got 2nd place behind them. Nope, I am not bitter… All in fun! They did some great research and made a greatplugin.

When you run their plugin against a memory image, you will get the default output to stdout.

If you try to change that output format to something like JSON, you will get an error message.

The reason for this is because they used the previous version rendering. The nice part is if they change the code to add JSON output, the unified output would also support SQLite and XLS or any other rendering format provided by the framework. Thanks to the Fireye guys for being good sports!

Now, I will use one of the standard plugins to display a couple different formats. PSList gives us a basic list of all the processes running on the computer at the time the memory image was acquired.

Here is the standard text output.

Here is JSON output. I added the –output=json to change it. It doesn’t look that great in the console, but it would be great in a file to import into some other tool.
Here is HTML output. Again, the change in with –output=html.

Hear My Plea

Allow me to get a shameless plug for my own project now. Evolve is a web based front end GUI that I created to interface with Volatility. In order for it to work with the plugins, they have to support SQLite output. The easiest way of supporting SQLite is to use the new unified output feature. The best part is that it works for a ton of other functions as well, with all the different formats that are supported.

If you have written, or are writing, a plugin for Volatility, make it loads better by using the unified format. We have to rely on automation with the amount of data that we get in our cases today, so let’s all do our part!

Thanks to the Volatility team for all of their hard work in the past, now, and in the future to come. It is hard to support a framework like this without being a for-profit organization. We all appreciate it!

James
@JamesHabben

Crashing into a Hint

I haven’t spent much time looking at crash dumps, but I ran into one recently. I had a case that presented me with little in the way of sources for artifacts. It forced me to get creative, and I ended up finding a user-mode crash dump that gave me the extra information I needed. It wasn’t anything earth-shattering, but it could be helpful for some of you in the future.

The case was a crypto ransomware infection. Anti-virus scans had already wiped out most of the infection, but the question was how it got onto the box in the first place. I determined to a reasonable certainty that a malicious advertisement exploited an out of date flash plugin. The crash dump gave me the URL that loaded the flash exploit. Let’s get the basics out of the way first.

Crash Dump Files

There are several different types of user-mode crash dumps that can be created. The default, starting in Windows Vista, is to create a mini dump, but true to Windows, you can change those defaults in the registry.
The default location drops them in the user profile folder for all applications. The path looks something like this:
C:\uses\[user]\AppData\Local\CrashDumps
The filename convention is pretty simple. It starts with the name of the process. Then includes the process identifier (PID). Last is the extension of dmp. All together, it looks like this:

process.exe.3456.dmp

Forcing a Crash Dump

You can go look in that directory on your computer, and very likely find a few of those crash dumps hanging out. Any process will cause a crash dump file to be created, so it can be a great sign of user activity on the box in absence of other indicators.
If you see no files in that folder, or an absence of that folder altogether, RUN! Go get yourself a lottery ticket! It just means you haven’t used your computer for more than reading this blog post. Don’t worry though because Sysinternals has another useful tool that you can employ for giving you sample crashes.
ProcDumpis a command line tool that allows you to customize the dumps and set up all kinds of triggers on when to create the dumps. Great functionality if you are a developer tracking down that pesky bug in code, but doesn’t do much for us in forensics. Best part about this tool is that it creates the crash dumps without having to actually crash the program.
Here are the commands I used while researching this further and dropping some dummy data to use in screen shots. The number is the PID of the process that I wanted. You can use the name, but you have to make sure there is only one process with that name or it bails out. I used the PID because I was going against Internet Explorer and even a single tab creates at least two processes.
Procdump 1234
Procdump –ma 1234
The –ma switch creates a crash dump with the full process memory. Otherwise, the only difference in the dump file when using ProcDump is that the naming convention uses the current date and time instead of PID.

Analyzing the Dump File

The dump file in my case originally stuck out to me because of the timeline I built. I had a pretty specific time based on a couple other artifacts, so I crashed into it by working backwards in time. Exploit code can often crash a process when encountering unexpected versions or configurations. The flash player was probably older than expected by the code.
Fact #1: IE crashed during relevant time window.
I started by looking around for tools to analyze these dump files. I used IDA Pro and WinDbg, but decided it wasn’t giving me any useful information. I ended up going old school forensics on it.
I have some screen shots to show you the goods, but of course the names have been changed to protect the innocent. Here we go with sample data. I search for ‘SWF’ and locate several hits. One type of hit is on some properties of some sort.
Another type of hit is a structure that contains a URL of a SWF file that was loaded. Alongside that URL is another for the page that loaded the SWF URL.
At this point in my case, I took all of the URLs that I found associated with these SWF hits, and I loaded them up in VirusTotal for evaluation. Sure enough, I had one that was caught by Google. The others showed it as being clear, but this was only a few days after the infection. I checked it again a couple days later and more were detecting it as malicious.
Fact #2: Internet Explorer loaded a SWF file from a URL that has been detected as malicious.

Dump Containers

The default dump type is mini dump, which contains heap and stack data. These files contain parts of the process memory, obviously, so maybe they contain actual flash files? I wanted to find that out to confirm the infection.
The CPU does not have direct access to hard disk data, so it relies on the memory manager to map those files into memory. This means that, most of the time, files sit in memory just like they do on disk. Here is the specfor SWF files (PDF). With this information I can prepare a search for the header. I will print a snippet of the spec here for reference.
Field
Type
Comment
Signature
UI8
Signature byte:
“F” indicates uncompressed
“C” indicates a zlib compressed SWF (SWF 6 and later only)
“Z” indicates a LZMA compressed SWF (SWF 13 and later only)
Signature
UI8
Signature byte always “W”
Signature
UI8
Signature byte always “S”
Version
UI8
Single byte file version (for example, 0x06 for SWF 6)
FileLength
UI32
Length of entire file in bytes
I compare that with several SWF files that I have available, and find them to all use CWS as the header.
This means they are all compressed, and rightfully so since they are being sent over the internet. In the dump file, however, I come up with zero hits for CWS. I do locate 3 hits for FWS in the dump file.
Aha! Nailed this sucker! Now I just need to determine which of these contains the exploit code.

Carving SWF Files

Now I want to get these SWF files out of here. There is a small potential for files to be fragmented in memory, but the chances are generally pretty slim for small files like this. I start working on the headers that I have located. Fortunately, the F in FWS means that it is uncompressed. This means that the value for the file size field will be accurate since it always shows the size of the uncompressed version of the file.
The size field tells the size of the file from the start of the header. So I start my selection at the FWS and go for a length of 6919 in this file pictured.
As a confirmation of my file likely being in one piece, I see the FWS header for another SWF file following immediately after my selection. After selecting this file, I export it. I can now hash it and take the easy way out.
Now the easy way.
Dang! I do the same process for the other 2 hits, and I get the same result for all 3 of these. I have 3 SWF files loaded in my crash dump, but none of them are malicious. My curiosity is now getting the better of me, and I need to know more!
I looked on a different computer and found a crash dump from Chrome. This computer even has a newer flash plugin installed than my sacrificial VM. Now I perform the same search on this crash dump, and I find an interesting result.
The Chrome dump:
The Internet Explorer Dump:
The hits have the same version numbers and file sizes. I did some poking around inside the SWF file, and I found these to be components built into the flash player. You can see one of them in this image.
If I search one of the dumps that I made using the –ma switch of ProcDump, then I do find the exploit code. It is hidden amount many other hits. There are some funny looking strings at the top of this SWF file, but I will get to that in just a bit.

Collecting the Sample

My favorite tool for collecting web based samples is HTTrack. It has a ton of configuration options including the ability to use a custom User Agent string and to route through a proxy server, such as TOR. It is a little easier to use than wget or curl, and it easily gives me the full context of the pages surrounding the sample.
Since I found the malicious URL in the IE crash dump, I can use that to pull down the sample for further analysis.

Analyzing the Sample

Once I have the sample collected, I can now pull it apart to see what’s going on. I won’t claim to be anything other than a novice when it comes to SWF files, but there are similar behaviors used across all sorts of malicious code that can stick out.
I use a couple tools for breaking down SWF files: SWFTools and FFdec. I am not going into a ton of detail because therearemanyothersthat already cover this topic much better.
With this exploit, it is pretty obvious from just a glance that it is not a normal SWF. Take a look at this image and tell me this is normal.
A class named §1I1l11I1I1I1II1IlllIl1§, another named IIl11IIIlllIl1? Can you tell the difference between |, 1, I, l? That’s the idea!
Then there are the strings from the header we saw earlier. They look like this in the code.

Easy Analysis

Now I use the easy button to confirm what I already suspect. I send the hash to VirusTotal and find that it has a pretty strong indication.
I already know the end goal of this sample, so I don’t want to waste time on this. You can read about my new perspective in this earlier post.
Fact #3: SWF sample contains obfuscated code and has a VT score of 17/57

The Last Piece

I need one more piece to complete the findings. Knowing how these files typically get onto disk is our jobs. If it was seen on the screen, then it had to have been in memory at some point in some form. If it was in memory for some time, then there is a fair chance that it has been paged out into swap. If it came from the internet, there is a strong chance that it touched the temporary cache, even if just for a second. If it touched the cache and was cleared out, then there is a reasonable chance of pieces still sitting in unallocated clusters. Follow the logic?
I prepare a search by grabbing the first 8 bytes of the sample.
We saw earlier that SWF files can be either compressed or uncompressed, with the difference indicated by the first character. I don’t know if this SWF file will be compressed, so my keyword will account for that by not forcing a C, as the sample has. Otherwise, this has the SWF signature, the file version, and the file size. It is no SHA256, but it’s unique enough for some confirmation to be made.
? 57 53 0D 54 D4 00 00
Sure enough! The exploit SWF is located in pagefile.sys with this keyword. You can see those same broken strings and I|l1I names in there. Unfortunately, the nature of pagefile operation doesn’t always place file pieces together, nor does it even hold onto the entire copy of the file.
Fact #4: Exploit SWF code found in pagefile.sys

Putting It All Together

I walked you through some of the basic steps I used to discover the infection point. I did this without unallocated clusters, and with no browser history or cache. I put this together to challenge you.
Use your knowledge and be creative! You have a unique understanding of how many components of operating systems and applications work. Don’t get discouraged if your forensic tools don’t automagically dime out the culprit.
Here is a summary of what we found here:
Fact #1: IE crashed during relevant time window.
Fact #2: Internet Explorer loaded a SWF file from a URL that has been detected as malicious.
Fact #3: SWF sample contains obfuscated code and has a VT score of 17/57
Fact #4: Exploit SWF code found in pagefile.sys

What do you think? Enough to satisfy?

Here is the sample SWF I used for this walk through.

Be creative in your cases!
James

Analyzing IEaaS in Windows 8

Internet Explorer As A Service

I ran across an interesting artifact while exploring a memory image from a case with Volatility and Evolve. It looked bad at first glance, but turned out to be normal behavior as designed by Microsoft. I want to share a bit of the story with you, and then give you the facts of the behavior.

I had posted a tweet about a somewhat related topic, and @davehull pointed out that Internet Explorer in Windows 8 runs differently when started as a Metro app. The IE process sits as a child of svchost.exe in the process tree. I did some testing and confirmed the behavior. I tested in Windows 10 and found a little change. Then, @hexacorn (Adam) threw a twist in with a line of vbscript. Keep reading, and I will explain all of this.

Metro App Verses Regular App

Let me first clear the air and make sure that everyone reading is following what I am talking about with the Windows 8 Metro app. The start menu in Windows 8 is a full screen experience. There are tiles on this menu that will launch an application. One of the tiles you will see below is for Internet Explorer.

When you click on this tile, it launch Internet Explorer in the Metro version, and it looks a little something like this.

Alternatively, when you are in the desktop experience by clicking on the desktop tile, you will find a link to Internet Explorer pinned to the start bar.

Process Tree Examples

For years, processes have traditionally started in Windows in the same way. If the user initiates the process, it is typically put as a child of the process that was used to kick it off. When started from the start menu, link file, or run box, it will typically sit as a child of explorer.exe. If the process is designed to be run non-interactively in the background, it will typically sit as a child under services.exe.

Let me give a basic example to illustrate. I am using a Windows 8 machine for this, but you can follow along and see this functionality with any version of Windows.
1.    Open the calculator.
2.    Open the command prompt. Type calc.exe and hit [enter]

You will end up with a process tree looking something like this in Process Explorer. One calc.exe is a child of explorer.exe, along side with the cmd.exe that we opened. The other calc.exe is a child of cmd.exe.

Hosting the Party

The processes that run in the background sit under services.exe which is a child of wininit.exe. A very common process to find in this area is svchost.exe. In the image below, I have 13 instances of svchost.exe running. Each of them serves a different purpose, but I won’t go into that here. The point of showing you this is to show you where and how svchost.exe is used.

I also want to show you the permissions assigned to the svchost.exe process. You can right click on any one of those instances and click on properties. You will need to make sure that Process Explorer is running in admin mode first, however. Notice that the User running the process is NT AUTHORITY\SYSTEM. The quick assumption every examiner makes is that any child process of svchost.exe will also inherit the permissions of SYSTEM. This is not always the case, but it is very common. It is safe to assume this at first, and prove it wrong with a little bit of work.

(Mis)Behavior at Hand

After establishing the baseline above, tell me if this looks normal or a bit off to you.

I hope that you had all sorts of bells and whistles going off in your head. Unless you have seen this artifact before and have chased it down, this looks downright scary.

You might have also noticed that opening one window of the Metro Internet Explorer actually created a parent and child iexplore.exe, and good for you! I will just say simply that each tab gets a process, for now, since I am planning on another blog post to explain that part.

Now I will look at this in Evolve, since this is how it all started for me. Follow along with me by downloading the memory image. Looking at the output for the Volatility pslist module shows me several instances of iexplore.exe, and it sticks out to me that there are a number of different PPID values. Typing IE into the search box narrows down the results to just those of iexplore.exe processes.

To get a better picture, I use the Show SQL button to input a custom SQL query. I want to show the name of the parent process so I don’t have to keep searching on the PID values.

SELECT p1.*,
(select name from pslist p2 where p2.pid=p1.ppid) as Parent
FROM pslist p1

Once the SQL is applied I hide several of the columns, and drag the parent column over beside the PPID column.

If you use Volatility from your shell, you can run the pstree module and see it pretty well in the output like this.

If I didn’t know any better (which is the point of writing this blog post) I would assume that the iexplore.exe processes under svchost.exe are running with SYSTEM privileges. So I make that assumption and investigate a little further to prove it wrong.

I run the Volatility getsids module and view the output in Evolve. In the search box, I type ‘ie sys’ and I get no results.

This causes me to think I did something wrong, so I back out and just type ‘sys’ and find there is a lot of noise. I continue in the search box with ‘sys loc’ which seems to do the trick. I also hide a couple columns to improve the view. Scrolling through this list shows all the processes I expect to have SYSTEM privileges, but I don’t see any iexplore.exe entries in the list.

I want to confirm all of this, so I clear the search box and type ‘iex’. I get all of the sids associated with all of the iexplore.exe processes, and none of them hold SYSTEM elevated privileges. These processes have the Administrators group since I am logged into that VM with an admin account.

 

The Change-up in Windows 10

Microsoft decided to change the name of Internet Explorer for the release of Windows 10. They also changed the way the processes are created a little bit. The parent MicrosoftEdge.exe process becomes a child of svchost.exe, but the tabs don’t fall under it. The tabs end up being children of a different process called RuntimeBroker.exe which is itself a child of svchost.

So, you can the changes in Windows 10 aren’t drastic, but they are enough to note for future investigations.

Adam’s Screwball

You probably know Adam best from his massive series of Hexacorn blog posts about various autorun locations that can be used for persistence by malicious software. If not, you should go read them when you finish here.

Adam joined the conversation on twitter and pointed out that you could emulate the behavior of the Metro IE with a single line of vbscript to start it off. Controlling it would require a few more lines, but thats not the point here. Let me show you the line of code.

Set ie=WScript.CreateObject(“InternetExplorer.Application”)

It’s pretty simple to kick this off. Just type or paste that line of code into a text file, and save it as a .vbs file. Open your command prompt, navigate to the folder where you saved it, and use cscript to execute it.

Here comes the test, are you ready? I rebooted my VM so you can cheat by using the previous PID numbers, though I applaud your attempt. Which process was started by the vbscript? First the parent processes, and then the children.

 

Since I don’t want to intrusively blast jeopardy music through your browser, I will attempt an equivalent measure of stalling for time so you don’t accidentally cheat.

An excerpt for your reading pleasure:

Last week, we met to address your continuing job-performance problems related to the serving of items from the dessert cart you operate in the newspaper’s senior staff dining room. These problems have persisted despite repeated counseling sessions with  supervisors as well as staff training programs. Specifically, your refusal to serve dessert to certain members of the senior staff has resulted in several written complaints from administrators at this company.

Mrs. Lopez, your refusal to serve dessert to certain members of the paper’s staff is disruptive to food service operations, and the explanations that you have provided for your behavior are not acceptable. This letter is being issued as a written warning with the expectation that there will be an immediate and sustained improvement in your job performance. Failure to comply will result in further disciplinary action.

On a more personal note, Mrs. Lopez, please stop refusing to give senior staff members dessert, even if you feel, as you explained to me last week, that they don’t “deserve it.” Which members of the paper’s staff do or do not deserve dessert is not your decision to make! And I would hate to see you asked to leave the food craft services department over something so silly! I would really miss you — and your chocolate chip cookies!

The only difference that any of us, from the twitter dialog, could come up with was that the tab opened by vbscript ended up being a 32-bit process. Adam and I both tried a number of ways to get it running fully as a 64-bit process, but we both failed. If you come up with a technique to make that happen, please let me know!

Good, Bad, and Badly Designed

With that, I have shown you what appeared to be bad, but ended up being good – as badly designed. I have also shown you some bad which does a pretty darn good job at looking like the good.

Hope you found this interesting and useful. Happy Hunting!

Get the memory image I was looking through.

James Habben
@JamesHabben