Reading Time: 6 minutes

TL;DR We explain how to write a Volatility 3 plugin. In addition, we also explain how to manually install symbol files. If you are interested in this excellent memory forensic framework and want to write your own analysis tools, read on!

Introduction

Volatility 3 is the newest (and largely anticipated) version of the most popular memory forensic tool. Volatility was actually born as Volatools in a presentation at BH USA 2007, quickly becoming the de facto tool for memory dump analysis. Initially developed in Python 2, the entire framework has been completely rewritten and redesigned in Python 3. In addition, no more --profile is needed! Volatility 3 automatically detects the profiles for all supported operating systems (better late than never, right?). Plugin development has also been improved: plugins are now versioned and can call other plugins directly (ConfObject no longer required!). You can check this video of Richard Davis on Youtube to know more about Volatility 3 and its new features.

In this post, we’ll learn how to write a Volatility 3 plugin. So first things first — you need to download the Volatility 3 source code :). To do so, simply open a terminal on your computer, go to your preferred folder for development work, and clone the official repository:

git clone https://github.com/volatilityfoundation/volatility3.git

To run it, you need Python 3.5.3 or later. You’ll almost certainly need to have the following additional Python packages on your system:

  • pefile 2017.8.1 (or later)
  • pdbparse
  • jsonschema

Once downloaded, try running it to see if it works. You can also download this memory dump for testing purposes. In the terminal, type the following:

python3 vol.py -f ~/volcados/alina1G.elf windows.info

Works? Congratulations! You can skip the next section. Does not? Then keep reading.

Manual Installation of Windows Symbol Files

Otherwise, some errors will have arisen. You will surely see an output similar to this:

Volatility 3 Framework 1.0.1
WARNING  volatility3.framework.plugins: Automagic exception occurred: volatility3.framework.exceptions.InvalidAddressException: Offset outside of the buffer boundaries

Unsatisfied requirement plugins.Info.nt_symbols: Windows kernel symbols

A symbol table requirement was not fulfilled.  Please verify that:
    You have the correct symbol file for the requirement
    The symbol file is under the correct directory or zip file
    The symbol file is named appropriately or contains the correct banner

Unable to validate the plugin requirements: ['plugins.Info.nt_symbols']

Another big change in Volatility 3 is that it requires the symbol file (in a special JSON file format) from the operating system of the memory dump under analysis to parse it appropriately. By default, Volatility 3 tries to locate this file locally. When it cannot find it, it retrieves the symbol file from the Microsoft Symbol server, converts it to the JSON file format, and stores it locally for future use. In particular, these JSON files are stored in the folder volatility3/symbols. Inside this folder there is another folder for each operating system supported by Volatility (windows, mac, and linux), and within them these JSON files are saved in separate folders. A good read on this topic is available in the official Volatility 3 documentation.

We can drill down into running errors with the output verbosity argument -v. This is a count argument, so you can add more v to increase verbosity as much as you want. Run the above command again with -vvv:

python3 vol.py -f ~/volcados/alina1G.elf -vvv windows.info
Volatility 3 Framework 1.0.1
INFO     volatility3.cli: Volatility plugins path: ['/Users/ricardo/volatility3/volatility3/plugins', '/Users/ricardo/volatility3/volatility3/framework/plugins']
[... omitted ...]
INFO     volatility3.framework.symbols.windows.pdbconv: Download PDB file...
DEBUG    volatility3.framework.symbols.windows.pdbconv: Attempting to retrieve http://msdl.microsoft.com/download/symbols/ntkrnlmp.pdb/00625D7D36754CBEBA4533BA9A0F3FE22/ntkrnlmp.pdb
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Info.nt_symbols0F3FE22/ntkrnlmp.pdb
WARNING  volatility3.framework.plugins: Automagic exception occurred: volatility3.framework.exceptions.InvalidAddressException: Offset outside of the buffer boundaries

As shown, there is an error downloading and parsing the PDB file. To solve this problem, we will download and install it manually. First, we need to download the PDB file. You can download it with your preferred web browser or via a terminal with wget or curl. For instance:

wget http://msdl.microsoft.com/download/symbols/ntkrnlmp.pdb/00625D7D36754CBEBA4533BA9A0F3FE22/ntkrnlmp.pdb

Now we need to create the JSON file. To do this, you need to run a Python script provided with Volatility 3. In the Volatility 3 root folder, run this command:

PYTHONPATH="." python3 volatility3/framework/symbols/windows/pdbconv.py -f ../ntkrnlmp.pdb -g 00625D7D36754CBEBA4533BA9A0F3FE22

Note that the value of the -g argument is the file identifier used in the download link. As a result, a file called 00625D7D36754CBEBA4533BA9A0F3FE2-2.json.xz will be created in the current working directory. Now we need to move this file to the folder volatility3/symbols/windows/ntkrnlmp.pdb (first you will need to create a folder with the name ntkrnlmp.pdb, which corresponds to the name of the downloaded PDB file):

mkdir volatility3/symbols/windows/ntkrnlmp.pdb
mv 00625D7D36754CBEBA4533BA9A0F3FE2-2.json.xz volatility3/symbols/windows/ntkrnlmp.pdb/.

Once you complete these steps, the execution of the above command will be successful:

python3 vol.py -f ~/volcados/alina1G.elf windows.info
Volatility 3 Framework 1.0.1
Progress:  100.00        PDB scanning finished
Variable    Value

Kernel Base    0x82805000
DTB    0x185000
Symbols    file:///Users/ricardo/volatility3/volatility3/symbols/windows/ntkrnlmp.pdb/00625D7D36754CBEBA4533BA9A0F3FE2-2.json.xz
Is64Bit    False
IsPAE    False
primary    0 WindowsIntel
memory_layer    1 Elf64Layer
base_layer    2 FileLayer
KdDebuggerDataBlock    0x82926c28
NTBuildLab    7601.17514.x86fre.win7sp1_rtm.10
CSDVersion    1
KdVersionBlock    0x82926c00
Major/Minor    15.7601
MachineType    332
KeNumberProcessors    1
SystemTime    2019-09-21 12:07:14
NtSystemRoot    C:\Windows
NtProductType    NtProductWinNt
NtMajorVersion    6
NtMinorVersion    1
PE MajorOperatingSystemVersion    6
PE MinorOperatingSystemVersion    1
PE Machine    332
PE TimeDateStamp    Sat Nov 20 08:42:46 2010

Writing your Own Plugin

We are going to develop a simple plugin called MyFirstPlugin to show the plugin writing process in Volatility 3. The plugin will work similar to PsList, simply showing for each process the PID, the process name, and a boolean value to know if the process is Wow64. In particular, we will use the PsList plugin to retrieve the list of processes acquired in the memory dump.

In Volatility 3, our plugin class has to inherit from PluginInterface. Volatility automatically finds all plugins in the plugins folder and imports every plugin that inherits from PluginInterface. Move to the folder volatility3/plugins/windows inside the Volatility 3 root folder and create a file with name myplugin.py with this content:

from typing import List

from volatility3.framework import renderers, interfaces
from volatility3.framework.configuration import requirements
from volatility3.framework.interfaces import plugins
from volatility3.plugins.windows import pslist

class MyFirstPlugin(plugins.PluginInterface):
    _required_framework_version = (1, 0, 0)

Apart from the import libraries that we will need when completing the source code, we have defined that the minimum required version of Volatility for this plugin to work will be 1.0.0. Next, we need to define the requirements of our plugin. The plugin requirements define the arguments necessary for our plugin to function appropriately. Optional arguments can also be defined. All of this is defined in the get_requirements method:

@classmethod
    def get_requirements(cls) -> 
                    List[interfaces.configuration.RequirementInterface]:
        return [
            requirements.TranslationLayerRequirement(name = 'primary',
                           description = 'Memory layer for the kernel',
                           architectures = ["Intel32", "Intel64"]),
            requirements.SymbolTableRequirement(name = "nt_symbols",
                            description = "Windows kernel symbols"),
            requirements.BooleanRequirement(name = 'onlywow64',
                            description = "Only show WoW64 processes",
                            default = False,
                            optional = True)
        ]

Note that this is a class method. This method will be invoked before the instantation of an object of the class. In this case, our plugin will only work with Intel architecture dumps (both 32-bit and 64-bit) and will need the Windows kernel symbols. As an optional argument, onlywow64 has been defined (default is False).

In Volatility 3 you have to define a run method, which will be called by Volatility after loading the memory dump. This method returns an object of type TreeGrid, which, as in Volatility 2, serves to facilitate obtaining an output format determined by the user (unified output, we will explain this later). We are going to add the following code to our plugin:

    def run(self):
        tasks = pslist.PsList.list_processes(self.context,
                                                self.config['primary'],
                                                self.config['nt_symbols'])
        wow64 = self.config['onlywow64']
        if wow64:
            tasks = self.onlyWow64(tasks)

        return renderers.TreeGrid([("PID", int), ("Image", str),
                                   ("WoW64", int)], self._generator(tasks))

This method first retrieves the list of processes (through the plugin PsList). Afterwards, it is checked if the onlywow64 argument is defined to, in this case, filter the processes and keep only those that are WoW64 processes. Finally, we are creating a TreeGrid object, which follows the same definition as in Volatility2: it needs the tuple of values (strings and type) of the returned data and as a second parameter, the data generator.

The two remaining methods to be defined, onlyWow64 and _generator, would be as shown below:

    def _generator(self, data):
        for task in data:
            yield (0, [
                    int(task.UniqueProcessId),
                    task.ImageFileName.cast("string", 
                            max_length = task.ImageFileName.vol.count, 
                            errors = 'replace'),
                    int(task.get_is_wow64())
                ])

    def onlyWow64(self, tasks):
        for task in tasks:
            if task.get_is_wow64():
                yield task

With all this, we can proceed to the execution and see that it runs smoothly:

python3 vol.py -f ~/volcados/alina1G.elf windows.myfirstplugin.MyFirstPlugin
Volatility 3 Framework 1.0.1
Progress:  100.00        PDB scanning finished
PID    Image   WoW64

4    System  0
268    smss.exe    0
348    csrss.exe   0
384    wininit.exe 0
[... omitted ...]

Note how we have refer to our plugin: we need to indicate the name of the plugin class, preceded by the directory hierarchy where the plugin is located. If we run the plugin with the argument onlywow64, we see that no process appears (since the memory dump comes from a 32-bit Intel architecture):

python3 vol.py -f ~/volcados/alina1G.elf windows.myfirstplugin.MyFirstPlugin --onlywow64
Volatility 3 Framework 1.0.1
Progress:  100.00        PDB scanning finished
PID    Image   WoW64

Unified Output

Since Volatility version 2.5, unified output was introduced, which allows a user to use a plugin without worrying about the output format: the user may want the output in CSV, JSON, or even SQLite, and get it just by specifying how she want it. Optional in Volatility 2, unified output is now mandatory in Volatility 3 (in fact, it is the only possible output definition).

Unified output is based on a TreeGrid object, whose constructor requires two parameters: the first parameter is a tuple list, which defines the string and the type of returned data, while the second parameter is the generator of data.

In our plugin, we can request the output in JSON format very easily:

python3 vol.py -f ~/volcados/alina1G.elf -r json windows.myfirstplugin.MyFirstPlugin
Volatility 3 Framework 1.0.1
Progress:  100.00        PDB scanning finished
[
  {
    "Image": "System",
    "PID": 4,
    "WoW64": 0,
    "__children": []
  },
  {
    "Image": "smss.exe",
    "PID": 268,
    "WoW64": 0,
    "__children": []
  },
  {
    "Image": "csrss.exe",
    "PID": 348,
    "WoW64": 0,
    "__children": []
  },
[... omitted ...]
}

And that’s all, folks! In this post we explained how to write your own analysis plugins in Volatility version 3. In addition, we also detailed how to manually install symbol files, which are now required for analyzing memory dumps in Volatility 3. If you are interested in malware analysis and memory forensics, you can read more about this topic in this training that I’ve prepared for Summer Boot Camp 2021, an international workshop organized by the Spanish National Cybersecurity Institute (only in Spanish, sorry!). At the moment, there are few community plugins available for Volatility 3, but this list is sure to grow in the future. I hope this post helps you get started writing your own analysis plugins, helping to increase the analysis tools of the forensics community!