{"id":939,"date":"2026-07-06T15:37:13","date_gmt":"2026-07-06T15:37:13","guid":{"rendered":"https:\/\/reversea.me\/?p=939"},"modified":"2026-07-06T15:37:13","modified_gmt":"2026-07-06T15:37:13","slug":"inferring-network-protocols-from-traffic-with-marissa","status":"publish","type":"post","link":"https:\/\/reversea.me\/index.php\/inferring-network-protocols-from-traffic-with-marissa\/","title":{"rendered":"Inferring Network Protocols from Traffic with MARISSA"},"content":{"rendered":"Reading Time: <\/span> 6<\/span> minutes<\/span><\/span>\n

TL;DR<\/strong> Many network protocols are undocumented or proprietary, which makes traffic analysis, auditing, and malware investigation harder when there is no specification to work from. Protocol reverse engineering (PRE) tries to recover message formats directly from network traces, but existing static tools tend to rely on fixed assumptions, struggle with binary or mixed-format protocols, and do not scale well to large captures. In our paper at the 11th IEEE European Symposium on Security and Privacy (EuroS&P 2026)<\/a>, “MARISSA: Efficient Inference of Network Protocols using Similarity Digest Clustering and Multiple Sequence Algorithms”<\/em>, we present MARISSA<\/code><\/a>, an automated tool that infers message formats from PCAP files. MARISSA<\/code> combines similarity digest clustering with multiple sequence alignment to identify field boundaries, works on both textual and binary protocols, and reduces runtime by factors of 2x to 15x compared to the tools we tested. Interested? Keep reading this briefing or grab the code on GitHub<\/a>.<\/p>\n\n\n\n

\n\n\n\n

Why Protocol Reverse Engineering Is Hard<\/h2>\n\n\n\n

Understanding the structure of a network protocol is useful for traffic inspection, intrusion detection, and forensic analysis. Widely used protocols such as HTTP, DNS, and TLS are documented and standardized, but many others are not: custom schemes in embedded systems, legacy enterprise software, and malware command-and-control channels often lack public specifications. In those cases, an analyst has to recover the message format from captured traffic.<\/p>\n\n\n\n

PRE approaches fall into two groups. Dynamic<\/em> methods instrument the program binary to observe how it builds and parses messages, which gives good results but needs access to a running system and the ability to modify its execution environment. That requirement limits their use in IoT or malware scenarios where the endpoint is not available. Static<\/em> methods work only from captured traffic, so they are non-intrusive and broadly applicable, but they have their own limits: many rely on recognizable delimiters or keywords, assume prior knowledge of the format, or become slow as the trace grows.<\/p>\n\n\n\n

We designed MARISSA<\/code> to address two goals at the same time: infer message structure from raw, unknown traffic, and scale to large and diverse captures within reasonable runtimes.<\/p>\n\n\n\n

The Solution: Similarity Digests Plus Sequence Alignment<\/h2>\n\n\n\n

MARISSA<\/code> is a static, protocol-agnostic tool. It does not assume delimiters or format regularity, which lets it handle both textual and binary protocols. It builds on two ideas.<\/p>\n\n\n\n

The first is to cluster messages using similarity digest algorithms<\/em> (SDAs) rather than full byte-by-byte comparison. SDAs produce compact fingerprints that reflect the structural similarity of byte sequences, which keeps cost down while preserving structural information. We evaluated TLSH<\/code> and ssdeep<\/code>, and adopted ssdeep<\/code> as the default clustering backend because it gave the most consistent results on our datasets.<\/p>\n\n\n\n

The second is to infer field boundaries with multiple sequence alignment<\/em> (MSA), a technique from bioinformatics that aligns many sequences at once to reveal regions that stay fixed (static headers) and regions that vary (payloads). To reuse mature MSA libraries, MARISSA<\/code> transcodes each protocol message from hexadecimal into a compatible character encoding (hex digits A-F are kept, digits 0-9 map to letters G-P), a reversible and loss-less step that preserves message length and byte alignment.<\/p>\n\n\n\n

Inside the Pipeline<\/h2>\n\n\n\n

MARISSA<\/code> takes a PCAP file as input and produces an inferred message format as output. The workflow has three phases.
<\/p>\n\n\n

\n
\"\"
Figure 1: The MARISSA pipeline, detailing the pre-processing, processing, and post-processing phases.<\/em><\/figcaption><\/figure>\n<\/div>\n\n\n

Pre-processing.<\/strong> Raw traffic is cleaned and normalized: message extraction, protocol filtering and header stripping (down to the protocol payload), deduplication of retransmitted packets, prefix sanitization<\/em> to strip static “magic number” prefixes that would otherwise bias the digests, and stream separation<\/em> by MAC address to keep different communication flows apart. The output is a set of clean, role-grouped message payloads.<\/p>\n\n\n\n

Processing.<\/strong> For each stream, MARISSA<\/code> computes pairwise similarity digest scores into a distance matrix, clusters the messages with OPTICS<\/code> (a density-based algorithm that handles clusters of varying shape and outliers better than k-means), and aligns the messages in each cluster with MSA. We integrated several genomics MSA tools (MAFFT<\/code>, Clustal Omega<\/code>, FAMSA<\/code>, and MUSCLE<\/code>) and selected MAFFT<\/code> as the default for its balance of accuracy and speed. A lightweight cluster-merging step then groups message variants that share the same type identifier, reducing over-segmentation.<\/p>\n\n\n\n

Post-processing.<\/strong> The aligned messages are analyzed to classify each field as static<\/em> (fixed length and content), dynamic<\/em> (fixed length, variable content, such as a sequence number or timestamp), or variable<\/em> (variable in both length and content). The result is a structured description of each message type, exported as CSV, which can feed documentation, interoperability testing, fuzzing, or emulation.<\/p>\n\n\n\n

Key Findings<\/h2>\n\n\n\n

We evaluated MARISSA<\/code> on nine real-world protocols spanning textual and binary formats: DHCP, DNS, FTP, Modbus, ICMP, MQTT, NTP, S7comm, and SMB, with datasets of 100, 500, and 1000 messages. We measured clustering quality and field inference accuracy (precision, accuracy, recall, and F1) against ground truth from tshark<\/code> dissectors, and compared MARISSA<\/code> with four existing tools: NEMETYL<\/code>, NetPlier<\/code>, BinaryInferno<\/code>, and MDIplier<\/code>.<\/p>\n\n\n\n