EnScript® Showcase – EnCase® App Central, Evidence Management and Reporting

Part 3 of 3 – Reporting with Quick Report

Robert Batzloff

This series of blog posts has focused on keeping your investigation organized and presenting your evidence in a clear, correct and readable format. Clarity, as well as brevity, is key when delivering digital forensic evidence to those who don’t work in the field. This evidence can be dense and hard to understand. Your job is to make the relevant information apparent and easy to digest. You want the information you present to be easy to explain and defend because opposing council will leap at the chance to capitalize on any potential ignorance regarding digital forensics.

As reporting is the final step in an investigation, we’ll close this blog series by looking at my favorite reporting EnScript: Quick Report Lite

EnScript® Showcase – EnCase® App Central, Evidence Management and Reporting

Part 2 of 3 – Jamey Tubbs' Time Zone Prior to Processing

Robert Batzloff

And we’re back with another post to walk you through one of the over 150 EnScripts® that can be found at EnCase® App Central. This three-part series will introduce and explore four EnScripts to help you make the most of EnCase App Central, manage and organize your evidence, and finally, show you a new option when it comes to creating your case report. In the previous post we discussed What’s New in App Central and Manfred’s Comprehensive Case Template. In this post we’ll walk through Jamey Tubbs’ incredibly helpful, time-saving EnScript: Time Zone Prior to Processing.

EnScript and Python: Exporting Many Files for Heuristic Processing - Part 1

James Habben with Chet Hosmer

I discovered something very cool this year at CEIC: people actually read my blog posts! The realization came when I found out there were two sessions focusing on Python, and both of them talked about my #en2py techniques that I presented in this blog last year.

One of the sessions, Heuristic Reasoning with Python and EnCase, was presented by the Python forensics guy, Chet Hosmer of python-forensics.org. I got a chance to chat with him after his session, and the discussion led to what you are about to read. Chet has a number of Python scripts that can make a difference in forensic cases, and we decided a joint blog post would be a fun way to touch on the integration between EnCase and Python with another technique. This will be a two-part post with the first part focusing on getting the files out. The second will get some fancy on it by putting a GUI on the front to accept options in the processing. I will now let Chet explain the benefits of his work.

Function and Benefits of Heuristic Reasoning

Applying heuristics during deep-dive investigation allows us to apply rules of thumb during the process. In order to bring this to light, we chose to integrate a Python script that performs “what I call” heuristic indexing of binary files. Binary files like memory snapshots, executable files and photo graphic images have ASCII text embedded with the binary data. Extracting these “text sequences or remnants” and then making sense of them can be a challenge. 

The issue with traditional approaches like dictionary comparisons or keyword lists, is the occurrence of misspelled words, slang, technical jargon, malware strings, filenames, and function names. These can all be missed because they are not in the dictionary or keyword list, an example is shown in the Casey Anthony investigation. Another traditional approach would be to report on all “text sequences or remnants” this can results in a voluminous number of nonsensical meaningless text strings that can overwhelm investigators.

My approach (originally outlined in my text, Python Forensics) uses a set of 400,000 common English words, (loosely a mini corpus of words) to generate a weighted heuristic model.  I have since created additional models for medical and pharmaceutical domains and I’m working on common words used within text messages.  

Using Python, I load the specific weighted heuristics into a Set. Then during the process of extracting “text sequences or remnants” from the binary file(s), the same algorithm is applied to each extracted sequence as was used to build the weighted heuristics. The calculated heuristic is then used as a lookup value. If the value is found in the loaded weighted set, then the word is considered probable and reported. One other final step I should mention…. most languages have what are referred to as “stop words” such as, (whenever, always, another, elsewhere etc). English is no exception. These stop words are filtered from the final list as they typically have little probative value. Each identified word that passes these filters is stored in a dictionary, one of the great built-in data structures within Python. Dictionaries are key, value pairs, in this case the key is the probable word string and the value is the number of times the word is discovered. This allows me to then produce a resulting list of probable words either sorted alphabetically or by frequency of occurrence.

Therefore, the bottom line benefits of heuristic indexing include:

1. Accurate identification of a broad set of probable words from binary data
2. Slang, technical jargon, filenames, misspelled words are also identified
3. Strings that represent nonsense strings are filtered out
4. Common stop words are ignored
5. The frequency of words found or alphabetical results are possible
6. New weighted heuristic models can be created

In order to apply this method more broadly to a case instead of a single file, we needed a method to allow EnCase (via an EnScript), to export multiple selected files to be processed by the Python script. I turned to James, the EnScript Guru for help.

Method of Choosing Files

In my previous posts, I used a simple technique in EnScript to send the highlighted file out from EnCase to the local disk to allow for a Python script to access the data. This works great for Python scripts that are designed to process one file at a time, but it is not very efficient for the examiner when that one file has not been pinpointed yet. There are many Python scripts out there that are designed to process a whole set of files in a designated folder.

In another post, I looped through files in the case, but I was targeting certain filenames known to contain evidence from Windows 8 Phone apps. The structure there is similar to what I have here, but the interaction with Python is the difference.

Chet and I talked at CEIC about how to do exactly this in EnScript, and came to the conclusion that the rest of the world should know about this as well! OK, maybe not the world, but I’m sure you appreciate that we didn’t keep this buried in some dark closet somewhere.

I have talked about ItemIteratorClass before, but it was in a simple post about the changes in EnScript from EnCase v6 to v7. This is the class that gives us access to all of the files in the case. There are a lot of options explained in that post, so I won’t drag it out here. The mode we will focus on is CURRENTVIEW_SELECTED, which will give us a collection of the files that the examiner has blue-checked in the EnCase interface before running the EnScript.

Because we are processing multiple files, the execution of the Python script needs to happen once the loop is complete. The loop will be doing the work of identifying selected files and exporting them to the disk.

EnScript Walkthrough

The usage of ItemIteratorClass starts off with setting some values in variables. I defined these as global variables for reasons you will see in part 2. The mode I chose here allows an examiner to blue-check any number of files in EnCase, and send this collection to the EnScript for export.

The NOPROXY is used because I am not looking to get any hashes calculated and it speeds up the loop. The NORECURSE option is also used to speed up the loop. With the mode using the current view, the recursing into compound files isn’t possible, anyway.

Then we enter into the loop to find all of the files. There's a fairly bulky chunk of code here, but it has a purpose behind it. When you are dealing with files from evidence, you are potentially pulling files from folders all over the drive. Chances are good you will find a couple files with the same name. On line 22, I am using a GUID that is generated by EnCase and is unique inside the evidence file. Lines 20-23 all together are modifying the filename to include this GUID, but also retain the same extension for identity.

There is a little irritation that pops up when you use any of the modes focusing on the current view. It locks that view in EnCase for the examiner running the EnScript while the iterator is active. Line 31 happens immediately after the looping export code, and this clears the iterator to release the view for the examiner while Python does its thing. Little things matter!

Depending on the Python script you are using and the amount of data you are processing, you may have to adjust the timeout value on line 41. If this value is not large enough, the output from Python will be either missing or cut short.

You're getting a two-for-one deal in this joint blog post, because now Chet is going to explain some Python code now. (I don’t want to read any complaints about the length of this post!)

Python Walkthrough

The overview of the Python script is shown in the figure below:

The Script employs a Heuristic Model created from one or more word dictionaries. Dictionaries and vernaculars can be expanded through the training of the model. The Heuristic Indexer receives selected file(s) from EnCase and then extracts possible word strings from each of the files. Heuristics are calculated for each extracted string and then compared against the Heuristic Model. The result is a report that is delivered back to EnCase.

For Part I of the blog I want to focus on the primary integration between James’ EnScript and the Python Heuristic Indexer.

The main entry point for the Python Script prints out some information messages and then obtains the path and individual filenames exported by the EnScript by parsing the command line arguments. Then for each file found, the IndexAllWords() function is called to perform the string extraction and subsequent Heuristic analysis.  I have highlighted the key lines of the Python script.

Python Main Entry Point

# Main program for pyIndex

if __name__ == "__main__":

    # Print Script Basics
    print "\nHeuristic Indexer v 1.1 CEIC 2015"
    print "Python Forensics, Inc. \n"

    print "Script Started", str(datetime.now())
    print

    # Obtain the arguments passed in by the Enscript
    # In Phase I the only argument passed is
    # path where the EnScript copied the selected files

    targetPath = ParseCommandLine()
 
    print "Processing EnCase Target Path: ", targetPath
    print

    # using the targetPath, obtain a list of filenames
    # using the Python os module

    targetList = os.listdir(targetPath)
 
 
    # Creating an object to process
    # probable words
    # the matrix.txt file contains heuristic model

    wordCheck = classWordHeuristics("matrix.txt")
 
    # Now we can iterate through the list of files
    # Calling the IndexAllWords() function for each
    # file. The IndexAllWords() performs the word
    # extraction, heuristic processing and reports
    # results back to EnCase via Standard Out

 
    for eachFile in targetList:
     
        fullPath = os.path.join(targetPath, eachFile)
        print "####################################"
        print "## Processing File: ", eachFile
        print "####################################\n"
     
        IndexAllWords(fullPath, wordCheck)
 
    print "Script Ended", str(datetime.now())
    print

    # Script End
 

Results: So What Do I Get From All of This?

Here is a screen shot and an abbreviated excerpt from an actual EnCase / Python marriage.

Closing Thoughts

James: This was a new (and exciting) opportunity for me to have a guest author in a joint post. I am so happy to hear that my #en2py techniques have helped others. EnCase is a powerful platform on its own, but enhancing it with the libraries available in other languages and tools just makes everything that much better for examiners. I hope you find this useful and thanks for taking the time to read through this!

Chet: The catalyst behind Python Forensics, Inc. is to create a collaborative environment for the rapid development of new investigative scripts that can directly benefit investigators.  I hope this blog will get you interested in developing and/or using EnScripts and Python in your next endeavor.  I would like to thank James for his enthusiasm for the project and I look forward to Part II.

The Final Details

Download the EnScript here.
Download Chet's Python script here.
Look for an email invitation and announcements on Twitter about an upcoming webinar we're planning with Chet called, "EnCase and Python: Extending Your Investigative Capabilities."

Chet Hosmer
@PythonForensics
Founder of python-forensics.org

James Habben
@JamesHabben
Master Instructor at Guidance Software

EnScript and .NET: Debugging in Visual Studio

I have been working on a few projects lately using C# and integrating it with EnScript code, and of course I run into problems in my code. Sometimes the problem is in the EnScript code, but other times it is in the C# code. To be honest, it is more often in the C# code since I have spent less time in that language than EnScript. Especially in the context of making a DLL to interface with EnScript.

If you have been reading this so far while thinking any of the following “What? C# and EnScript? When did this happen?” Check out this one for a little intro. My goal in this post is to show you how to debug your C# code while EnScript is calling it. Yes! You can do that!

EnCase and Python – Automating Windows Phone 8 Analysis

James Habben

Roll Call

You may have read my introductory post about using Python scripts with encase. You may have also read my part 2 follow-up, which put a GUI on top of Didier Stevens’ pdf-parser. Did you also read Kevin Breen’s post? He wrote about using EnScript to call out to David Kovar’s analyzemft script using EnScript. Then Chip wrote a post about sending data out to get parsed by parser-usnjrnl.

EnCase and Python – Part 2

James Habben

In Part 1 of this post, I shared a method that lets you use Python scripts by configuring a file viewer in EnCase. We used Didier Stevens’ pdf-parser as an example. I also showed how EnScript could be used to greater effect by allowing us to capture the output of pdf-parser directly in a bookmark without having to manually copy and paste. Both of these techniques reduce effort by leveraging capabilities of both EnCase and the Python language.

In this post, I’ll take the same principles and apply them into an EnScript that provides a little more flexibility and functionality. Our goal is to have a GUI that gives you control over the exact functionality you want from the pdf-parser tool.

EnCase and Python - Part 1

James Habben

As a co-author and instructor for Guidance Software’s EnScript Programming course, I spend a lot of time teaching investigators in person around the globe. Investigators are faced with a dizzying variety of challenges. We work together in class, coming up with solutions that send EnCase off to do our bidding. EnCase and EnScript allow us to “bottle” the result of our efforts to share with other investigators (e.g. categorizing internet history, detecting files hidden by rootkits).

Python is used similarly. The interweb hosts great tools written in Python to accomplish all measures of tasks facing DFIR examiners. The community benefits from the hours of work that go into each and every .py that gets baked. It seemed to me that there should be a way for EnCase and Python to work together, so I put together a brief tutorial.

POSIX Regular Expressions in EnScript and .NET

James Habben

I am sure you have spent a little intimate time with EnCase doing keyword searches, so you know that EnCase has basic GREP capabilities. This is a powerful feature that allows for searches to be performed with patterns that can eliminate false positive hits. Recently, we hosted a webinar with guest Suzanne Widup, describing some techniques and benefits of using GREP in EnCase.

GREP is a term that comes from the Unix world long ago. It stands for Globally search for Regular Expressions and Print. This command line utility was used to search through data and print out results that matched the given pattern. Because of the popularity of the tool, the name has become synonymous with Regular Expressions (Regex). Though there is a defined standard, POSIX, the syntax of patterns used in Regex actually varies quite wildly depending on the platform engine and programming language that is being used. EnCase is no exception. In homage to our habit of prefixing our product names with “En”, I jokingly refer to our syntax of regex as “EnGrep.”

Poweliks: Persistent Malware Living Only in the Registry? Impossible!

James Habben

The ultimate desire for malware authors is to be able to have their code run every time a computer starts, and leave no trace on the disk for us to find. Let me reassure you that it hasn’t happened just yet, at least not that I have seen. There have been plenty of examples over the years that have taken advantage of some clever techniques that disguise their disk-based homes, but that’s just it–disguise!

A couple of recent posts on “Poweliks” here and here shed light on creative measures attackers use to store malware in the Windows Registry. In short, there is a registry value that executes an encoded script stored in another registry value, which then drops a file on disk for execution.

So many artifacts, so little time… Summer edition

Ken Mizota

EnCase is an extensible digital investigation platform. Simply put, extensibility reduces time and effort for the investigator. One way to validate this claim for yourself is to take a look at the depth and breadth of the ways EnCase can work with existing tools in your kit. For example: Do you already own Magnet Forensic's IEF? IEF and EnCase work together to reduce work for investigators. Have you considered how to integrate threat intelligence into your DFIR regimen? EnCase and Cisco Security (formerly ThreatGRID) collaborate to reduce IR time and effort. Let’s walk through a few ways extensibility works in your favor.

Working with EnScript and .NET/C#

Ken Mizota

The ability to manipulate and interpret data structures within evidence has long been a strength of EnCase. EnScript—a core EnCase technology—has enabled investigators and incident responders to be efficient, automating the most sophisticated or mind-numbingly rote techniques. For instance, take Simon Key's (@SimonDCKey) recent post on the OS X Quick Look Thumbnail Cache: the ability to mine, extract and work with critical data for your case is available now. This app, courtesy of Guidance Software Training, just happens to be free, enabling the DFIR community to take advantage. If you need to keep pace with the perpetually accelerating gap between data and the investigator’s ability to understand that data, having extensible, flexible tools in your kit is not optional.

EnScript Changes From EnCase Version 6 to Version 7

You may know that Version 6 of EnCase keeps the majority of data in memory, which gives you fast access to the evidence items in a case, but is not conducive to handling large data sets. In addition, keeping most data in memory requires that records and entries be handled separately.

EnCase Version 7 behaves in a similar way to a database in that working through multiple evidence items is accomplished using an iterator. This makes for more stable processing and allows the EnScript programmer to handle both entries and records in a more streamlined way. It is possible, for instance, to iterate through all of the evidence items in a case (entries and e-mail attachments, for instance), quickly identifying those items that are pictures or documents.

Version 7 Tech Tip #1: Matching Parent E-Mails with Attachments in Searches

James Gagen

This is the first in a series of brief, but frequently asked questions and answers about working with EnCase® Forensic Version 7. We hope they save you time and help you close cases faster.

One of the questions we are often asked in Technical Services about working with e-mail searches is, "When I find a relevant e-mail attachment, how can I find the e-mail that the attachment belongs to?" Searching in e-mail may result in keywords being found in both e-mails and attachments. This is how to locate the e-mail to which the attachment belongs:

EnCase App Central Delivers its 10,000th App

Guidance Software

EnCase App Central opened its virtual doors in early spring this year with the goal of providing functionality and efficiency to EnCase users by offering EnScripts, templates, and 3rd party Apps. After just a few months, driven by the power of the EnCase community and our 3rd party partners; App Central has become the primary source of efficiency-driving solutions for the tens of thousands of EnCase users worldwide.