Testing the Invariant Without the Private Key

Once the fingerprint relationship is stated precisely, the next problem is empirical: how can anyone tell that a deployed WebRTC media server failed to preserve it? The DTLS-MitM-Scanner (DMS) addresses that question by placing a controlled analysis host in the path of a real application call and perturbing selected handshake messages. This chapter explains what the black-box method observes, why its retries and endpoint fingerprints matter, and how its 19 authentication tests map to different invariants rather than one vague notion of certificate validation.

A real call, observed from the middle

DMS does not begin with an isolated TLS test. It first has to make the application create the kind of DTLS-SRTP connection that a real user would create. A Booter starts or stops the local application—for example, by signing into a web service and initiating a conference—or provides a controlled way for a researcher to do those actions. This step matters because WebRTC signaling is application-specific. Without the application progressing through its normal signaling and connection setup, the media endpoint may not be listening for a usable DTLS connection at all.

Once the call begins, the analysis host takes a Man-in-the-Middle position between the local application and the media server. The study’s platform routes relevant UDP traffic through its packet-capture and filtering layer. Packets that are not part of an interesting DTLS-SRTP flow can be forwarded normally; packets that belong to a logical connection are assigned to the probe machinery. The tool can then let the original handshake proceed, replace a selected message, omit a message, replay one, or offer a crafted alternative, while preserving enough of the surrounding application behavior for the result to be meaningful.

The choice to work from the middle is central to the paper’s security claim. DMS does not install a new certificate on the target system, obtain the target’s private key, or modify the application’s trust store in order to decrypt its traffic. It works with the capabilities of an active network attacker who can intercept and alter the connection. That makes the method portable across browsers, mobile applications, and desktop applications, but it also means that the tool has to infer behavior from messages and outcomes rather than inspect an implementation’s internal state.

The platform combines a DTLS workflow engine with packet capture, browser automation or manual control, and an SRTP library for decrypting media when the test has legitimately established the necessary session keys on the analysis side. The important architectural division is between connection control and evidence collection. A probe asks the Booter for a fresh application connection, the packet layer associates traffic with that connection, and the DTLS test engine performs the planned perturbation. The report then records what the remote endpoint actually did.

Endpoint identification is not a cosmetic detail. A service can use load balancing, random ports, or several simultaneous connections for different media types. Two packets that look like parts of one user action may come from different logical endpoints with different behavior. DMS fingerprints ClientHello messages, ServerHello responses, and certificate messages so that it can distinguish those behaviors.