Before the LLM: How Sound Becomes Instruction
This chapter follows the physical mechanism behind SWhisper and related inaudible voice attacks: how a signal that is difficult for a person to notice can be transformed by a microphone into something a speech-driven model can process. The useful judgment is not whether “ultrasound tricks an AI,” but where the signal changes form, which component owns that change, and what can cause the chain to fail. With that model in place, the reader can see why the receiver—not only the language model—is part of the prompt boundary.
The distinction matters because a microphone is not an ideal recorder. It has mechanical parts, analog circuitry, filters, and a digital output. Within the range for which it is designed to capture speech, its behavior is often close enough to linear for ordinary applications. Under high-frequency excitation, however, those assumptions can weaken. The microphone can create components that were not present as separate frequencies in the incoming sound. An inaudible voice attack uses that physical imperfection as a conversion step.
The receiver creates a second signal
Start with two ideas that are easy to conflate. A carrier is a high-frequency acoustic signal used to transport information. The baseband is the lower-frequency voice content that an ordinary speech pipeline is built to recognize. In a conventional recording, the microphone is expected to preserve the relationship between the incoming sound and its output. In an inaudible attack, the voice content is embedded in a carrier that people may not reliably perceive as a command. The receiver is then asked, in effect, to turn that arrangement back into speech.
If the microphone were perfectly linear, the output would contain scaled versions of the input frequencies. A high-frequency carrier would remain a high-frequency carrier, and a listener or speech recognizer would have no recovered voice command to process. Real microphone front ends can be approximated as having a linear term plus nonlinear terms. Those extra terms mix components together. When a carrier varies with an embedded voice waveform, the mixing can create lower-frequency energy related to the embedded waveform. That lower-frequency component is the important one: later filtering can suppress much of the carrier while leaving a signal that resembles ordinary speech.
The sequence is easier to see than the algebra. The source emits a signal; the air changes it through propagation; the microphone responds to it; processing removes or attenuates high-frequency material; and the remaining audio is handed to the next stage. The crucial event is not that the model “hears” a silent command. It is that the microphone and its front end produce a new representation before the model receives the input.
That sequence contains several possible stopping points. The acoustic signal may weaken before it reaches the device. The microphone may attenuate or distort it so severely that no useful baseband remains. Processing may remove the relevant component. A speech recognizer may transcribe the recovered sound incorrectly. An audio-native model may interpret the same signal differently from a transcription pipeline. Even if the input arrives intelligibly, the model may refuse it or treat it as irrelevant. A successful physical conversion is therefore a necessary link in the attack chain, not a guarantee of an unsafe response.
This is the first important boundary. The acoustic mechanism changes what the system receives; it does not directly change the model’s policy. SWhisper’s case study concerns jailbreak prompts, so the final undesirable behavior depends on the target model’s response to the recovered instruction. The physical layer can make an instruction available to that model without making compliance inevitable. Keeping delivery and policy separate prevents two opposite errors: assuming a hidden signal is harmless because it is not heard by a person, or assuming that a recovered signal automatically overrides every safety control.
The same distinction applies to the two major interface architectures. In a speech-to-text system, the recovered baseband must retain enough linguistic structure for the recognizer to produce the intended words. Recognition errors can change the meaning, break a constraint, or turn a carefully constructed prompt into noise. In an end-to-end audio-language model, there may be no exposed transcript, but the model still receives an acoustic representation shaped by the microphone and earlier processing. Removing the transcription step changes the mediator, not the existence of the acoustic boundary.
Human perception is another separate layer. The absence of a clearly heard command means that a nearby person may not intervene, but it says nothing by itself about what the microphone records. Conversely, a faint artifact that a human notices may still be too distorted for a model to interpret. “Inaudible” is therefore a perceptual description, not a complete security property. A system needs to reason about machine recoverability and human noticeability separately.
The physical conversion can be described without treating the microphone as a magic component. It has an input, a transformation, and an output. The transformation is driven by the hardware’s response to the incoming frequencies. The output then enters ordinary software stages, where the system may filter, transcribe, classify, or infer. That ownership matters operationally: if a defense is placed only after transcription, it may have no visibility into the signal conditions that made the transcription possible. If it is placed only at the microphone, it may not know whether the recovered content is authorized to trigger an action.
Why the same command changes with the device
The mechanism would be less important if every receiver behaved the same way. It does not. The NormDetect study reports that the same inaudible command produces markedly different spectral patterns on different smart devices, while ordinary audible commands are more similar across the tested devices. The authors connect this difference to microphone frequency response. Speech devices are commonly calibrated around the bands that carry ordinary human speech, but their behavior outside those bands can diverge because of differences in mechanical structure, ports, cavities, circuitry, and front-end filtering.
This gives the acoustic path a split personality. In the ordinary speech range, similarity is useful: different devices need to capture the same words with broadly compatible behavior. At higher frequencies, variation is not a defect that every manufacturer necessarily removes. It is part of the physical design. One device may preserve a component that another attenuates. One may introduce a distinctive distortion pattern where another produces a different one. As a result, the same source signal does not define a single universal attack audio at the receiver.
The difference is more than a matter of volume. If the receiver changes the relative energy or phase of components, it changes the shape of the recovered waveform. A command can remain recognizable on one device and become noisy or semantically unreliable on another. The receiver can also produce artifacts that alter how an audio detector classifies the input. This is why a defense built around one observed attack signature may perform well on the device that generated the training data and poorly on an unseen device.
The environment adds another layer of variation before the signal reaches the microphone. High-frequency sound can be absorbed by air more strongly than ordinary speech. Reflections create multipath, so copies of the signal may arrive at different times. Dispersion can alter different frequencies by different amounts. Distance changes the received level, while the relative angle between source and microphone changes how effectively the device captures the signal. Small changes in position can therefore alter the input that the microphone’s nonlinear response acts upon.
These factors create distinct failure paths rather than one generic notion of “noise.” A weak signal may never produce a usable recovered voice. A strong but badly distorted signal may produce recognizable fragments with the wrong semantics. A signal can be recovered clearly enough for transcription but still be rejected by the model. Or it can reach an audio-native model whose interpretation differs from that of a speech recognizer. The outcome belongs to the entire chain, and a measurement at one point cannot automatically stand in for all the others.
This is the engineering obstacle that SWhisper addresses with a channel model. Its approach treats the microphone-and-environment path as something to be measured and compensated, rather than assuming that a single fixed transformation describes every receiver. The paper reports a near-ultrasonic channel design, but its central idea is broader than a particular carrier choice: construct a model of how an intended baseband signal is likely to be transformed, then use that model to improve the fidelity of what comes out. The method remains a compromise because an averaged description can cover heterogeneous conditions without matching every changing environment exactly.
NormDetect starts from the same variability but asks a different question. Instead of learning what one attack should look like after one receiver, it learns patterns of benign audio and treats deviations as anomalies. That approach does not require the authors to enumerate every possible abnormal waveform. It also does not eliminate the need to manage false alarms, changing noise, speech content, or unseen hardware. The shared lesson is that the receiver must be modeled as a variable system. SWhisper tries to make the intended signal survive that variability; NormDetect tries to recognize when an input does not fit normal behavior despite it.
The two responses are not symmetrical defenses and attacks. SWhisper optimizes for delivery and downstream prompt effectiveness under its threat model. NormDetect optimizes for discrimination between benign and inaudible attack audio under its evaluation conditions. But both reject the same oversimplification: that a microphone can be treated as a transparent pipe whose response is independent of the device and room. Once that assumption is removed, the rest of the security analysis becomes more precise.
It also clarifies what a practical attack must accomplish. It must preserve enough of the intended voice content to be processed, tolerate a receiver it does not control, and remain difficult for people to notice. Those goals can pull in different directions. Increasing signal strength may improve machine recoverability but make artifacts more apparent. Designing for one receiver may improve local performance but reduce transfer. Relying on a short command may simplify delivery but limit the content that can be conveyed. The source studies present these as engineering constraints, not as evidence that any one configuration is universally reliable.
For the same reason, a simple filter is not automatically a complete solution. Filtering high-frequency energy may reduce one class of input, but microphone nonlinearity can make the relevant evidence appear in lower-frequency processing. Filtering more aggressively can also affect ordinary voice interaction. A model-level guard can inspect the resulting words, but if the input has already been transformed or if the application gives those words authority, the guard operates late in the chain. A receiver-side anomaly detector can add visibility, but it has its own operating conditions and thresholds. Each control must be judged against the stage it can actually observe.
The physical story thus ends with a design requirement rather than a verdict. A speech-driven system needs to account for what the microphone does before asking whether the language model behaves safely. For SWhisper, that means the prompt content and the acoustic channel must be made compatible: a semantically effective text prompt is not enough if speech conversion destroys it, and a faithful spoken prompt is not enough if the channel does not recover it. Understanding this receiver transformation is the foundation for the next question—what a prompt must look like when it has to remain meaningful while passing through voice.