Acoustic Prompt Injection in Speech-Driven LLMs

Overview

This series traces covert acoustic prompt injection in speech-driven LLMs from the microphone to model policy and application authority. It explains how nonlinear receivers can recover voice content from high-frequency signals, why SWhisper co-designs speakable prompts with channel compensation, and how experiments separate intelligibility, model response, physical robustness, and human perception. It then examines NormDetect’s benign-only anomaly detection across heterogeneous devices. Together, the studies show that the prompt boundary begins before transcription and that recognition is not authorization. Results remain conditional on tested models, devices, environments, corpora, thresholds, and deployment controls; neither study establishes a universal attack or defense.

Learning outcomes

A speech-driven LLM receives a receiver-transformed signal, not an untouched spoken intention.
SWhisper combines voice-compatible prompt design with regularized channel compensation; delivery still does not guarantee model compliance.
SWhisper’s measurements separate word recovery, model behavior, physical robustness, and human noticeability.
NormDetect shows benign-only anomaly detection can add an earlier boundary, but thresholds, devices, noise, and adaptive inputs limit generalization.

Keywords

Prompt injectionSWhisperNormDetectMicrophonesAcousticsTranscriptionJailbreaksAnomaly detectionAuthorization

Chapters

How Microphones Change LLM Inputs

Why the prompt boundary begins before transcription

12 min · Why the prompt boundary begins before transcription

12 min

Microphone Nonlinearity in Voice Attacks

How a carrier becomes recoverable speech

11 min · How a carrier becomes recoverable speech

11 min

SWhisper Voice-Compatible Jailbreaks

Balancing meaning, brevity, intelligibility, and transfer

14 min · Balancing meaning, brevity, intelligibility, and transfer

14 min

SWhisper Channel Inversion

Preserving baseband speech across uncertain receivers

11 min · Preserving baseband speech across uncertain receivers

11 min

SWhisper Attack Results and Limits

What NR, SC, WER, robustness, and ABX measure

14 min · What NR, SC, WER, robustness, and ABX measure

14 min

NormDetect’s Benign-Audio Detection

Using normality to handle device-specific attacks

13 min · Using normality to handle device-specific attacks

13 min

Voice LLM Security Beyond Recognition

Assigning controls from microphone to application authority

11 min · Assigning controls from microphone to application authority

11 min

Preview every chapter · Unlock full text, audio, and video from chapter 4

Plans

Ch.1-3 FreeCh.4-7 Premium

FAQ

Why can an inaudible acoustic signal affect a speech-driven LLM?

A nonlinear microphone can turn a high-frequency carrier into lower-frequency voice content before transcription or audio inference. Later processing and model policy still determine the outcome.

What does SWhisper actually optimize?

SWhisper optimizes a voice-compatible prompt and a compensated acoustic waveform for intelligibility and transfer across tested channels. It does not guarantee success on every model, device, or environment.

How should SWhisper’s attack results be interpreted?

They show conditional performance across selected models, prompts, devices, environments, and listeners, not a universal bypass. NR and SC measure model behavior, WER measures word recovery, and ABX measures human discrimination.

How does NormDetect defend against inaudible voice attacks?

NormDetect learns benign audio patterns, reconstructs them with a memory-constrained model, and flags large mismatches. Its effectiveness depends on thresholds, speech diversity, devices, noise, and adaptive attacks.

What should a secure voice system protect beyond the transcript?

It should monitor receiver behavior, preserve input provenance, treat recovered instructions as untrusted, and require application-level authorization for consequential actions. The studies do not prescribe one universal confirmation policy.