The NeuralHash Case: When the Hash Space Bends
NeuralHash is the first concrete test of the perceptual contract established earlier: can a nominally large output keep similar images together while separating different ones? This chapter focuses on the black-box evidence from face and blur populations, moving from the study’s sampling choices to its bit statistics, collision search, and error rates. The reader’s task is not to memorize a headline number, but to learn how an image population can make a 96-bit perceptual representation behave very differently from the independent, uniform hash imagined by the number 96.
A test designed around population, not abstraction
The study does not begin with an abstract set of arbitrary files. It asks how NeuralHash behaves on image types that may stress the representation in a plausible way. For non-human images, the researchers use images from the PASS dataset. For human images, they use celebrity faces from CelebA, a public dataset containing 202,599 images. Within CelebA, they identify at least 4,629 duplicate or perceptually similar images that should ideally produce the same hash. That count becomes a reference point for measuring missed matches.
The researchers divide the images into six groups: non-human images, non-blurred faces, faces with light blur, faces with medium blur, faces with high blur, and images in which only the face region is blurred. The last four groups are not four arbitrary data augmentations. They are probes of how the representation changes as visual information is removed or degraded. Blur can be a normal artifact of an image-processing pipeline, and it can also alter the features a learned model relies on. Blurring only the face tests whether the system reacts differently when the central identity-bearing region changes while the surrounding composition remains more stable.
Faces serve two purposes in the experiment. They are relevant to the researchers’ discussion of possible content-detection applications, and they are common on mobile devices, where a system such as NeuralHash might run. That does not make CelebA a substitute for an operational content database. It makes the dataset a controlled way to ask whether a recognizable class of images occupies the output space differently from general images. A comparison with non-human images supplies a reference population; the face variants show what happens as the visual input becomes more structured and then more blurred.
The scale of the measurements is split between two tasks. For the distribution and dependence analysis, the authors hash 30,000 randomly selected images from each image type and examine the 96 output bits. For the collision search, they hash the full set of 202,599 images of each type. The distinction matters. A bit-frequency experiment asks what the output looks like statistically across a sample. A collision search asks whether the complete tested population actually produces pairs that the matching rule would merge.
The study also has to decide what counts as similar. Its definition is based on human-perception criteria: color consistency, preservation of structure, preservation of core content despite changes such as compression or overlays, and tolerance for limited noise or blur. When the researchers cannot classify a colliding pair clearly, they conservatively treat it as legitimate rather than calling it a false positive. That choice makes the reported false-positive findings harder to obtain, not easier.
This setup changes the question from “Does NeuralHash have 96 bits?” to “How are those bits populated by these images?” A nominal output size describes the number of positions available in the representation. It does not say that every image uses those positions evenly, or that the value of one bit tells us nothing about another. If faces cluster in a particular region of the learned representation, the effective opportunities for separation may be much smaller than the raw output length suggests.
That is a population question, not a defect that can be inferred from the output format alone. The same function may have different collision behavior on different input classes. The study reports no illegitimate collisions for its non-human set, while finding them in every face set. That contrast does not prove that non-human images are universally safe or that faces always fail. It shows why a reliability claim needs to specify the population on which it was measured.
What the bits predict—and what collisions show
For an ideal 96-bit value whose bits are uniform and independent, the birthday paradox gives an average first collision after roughly 2^48 evaluations. That figure is not the number of images needed to find a chosen second preimage; it is the scale at which random pairs begin to share an output. It is also a model with assumptions. If some bit values are much more common than others, or if bits move together instead of independently, equal outputs become more likely than the ideal calculation predicts.
The study tests the first assumption by counting how often each NeuralHash bit is one. For the non-human images, the percentages remain distributed around 50%, and none of the 96 bits falls below 35% or above 65% in the reported experiment. The non-blurred face population is different: 30 bits fall outside that 35%–65% band, and two occur below 20% or above 80% of the time. As blur increases, the deviation becomes more pronounced. The counts outside the band are 39 for light blur, 49 when only the face is blurred, 51 for medium blur, and 61 for high blur.
A biased bit is not merely an aesthetic imperfection. If a bit is almost always zero for a population, it contributes little separation among members of that population. Two images have a higher chance of agreeing on that position because the population rarely exercises the alternative. Across many positions, the output values occupy a narrower and more uneven portion of the nominal space. This is the intuition behind the paper’s collision-probability calculation: the probability that two values agree depends on the frequency of zeros and ones at every position, not only on how many positions exist.
The second assumption—independence—requires a different test. The paper uses the simple matching coefficient, which measures how often two positions have the same value across a sample. For independent random binary vectors, the expected coefficient is 0.5. A coefficient far from that value indicates that two bits tend to agree or disagree more often than independence would predict. The statistic can reveal dependence, though it cannot prove independence when values happen to sit near 0.5.
The contrast is again sharp. Only 0.22% of the bit pairs fall outside the 0.4–0.6 range for the non-human images. For non-blurred faces, 13.97% fall outside it. The reported outside-range proportion rises to 23.3% for light blur, reaches 30.39% when only faces are blurred, and reaches 47.75% for high blur. In other words, the representation does not merely become less balanced one bit at a time. Relationships among positions also change with the image population.
The paper uses these observations to estimate an equivalent effective hash size. It models the observed collision rate with an ideal hash whose bits are uniform and independent but whose length is smaller than the nominal output. That effective size is a way to summarize how much collision resistance the tested population appears to have under the model. It is not a claim that the device literally truncates NeuralHash to a shorter string, and it is not a universal property of the algorithm across every possible input. It is a population-specific translation from measured overlap into a familiar collision model.
That distinction is important because “96 bits became 30 bits” would be both memorable and misleading. The implementation still emits 96 bits. The effective-size estimate says that, for the tested images and the observed equality rate, an ideal independent hash of roughly that smaller scale would show comparable collision behavior. Correlation, bias, and other structure can all make the idealized collision calculation optimistic. The estimate is useful for reasoning about scale precisely because it keeps the assumptions visible.
The full collision search supplies the operational check. Among non-blurred faces, the first illegitimate exact collision appears after approximately 2^16.1 hash evaluations. The false-positive rate in the 202,599-image set is 0.01%. For light-blurred faces, the first appears after about 2^15.2 evaluations, again with a 0.01% finite-set rate. When only the face is blurred, the study finds a first collision after about 2^16.1 evaluations and a 0.005% rate. Medium blur produces a much larger count: the first appears after about 2^12.6 evaluations and the rate is 0.13%. High blur brings the first collision to about 2^11.6 evaluations and the rate to 0.29%.
Those percentages may look small if read without the denominator and the application. They come from a sample of roughly two hundred thousand images, not from a single account or a hypothetical billion-image system. Yet the first-collision work factors are far below the ideal 2^48 expectation, and some hash values are shared by three or more perceptually different blurred images. The result is not that every comparison will fail. It is that the output space contains naturally recurring values for this population, so increasing the number of images creates more opportunities for an illegitimate match.
The other side of the table is just as important. The researchers identify at least 4,629 image pairs that should count as duplicates or perceptually similar. The number of those pairs that actually produce legitimate exact collisions is much smaller. The resulting minimum false-negative rates are 64.7% for non-blurred faces, 65.9% for light blur, 77.3% when only the face is blurred, 57.9% for medium blur, and 47.3% for high blur. “Minimum” is doing real work here: the duplicate count is a lower bound, and additional pairs may become perceptually identical after blurring. The reported rates could therefore understate the full number of missed matches.
The two outcomes should be read together, not as competing headlines. A system that misses many expected matches is not repaired simply because its exact false-positive rate is low on one subset. Nor does widening the decision rule solve the problem for free. The paper notes that Apple’s published description includes near-collisions as well as exact equality. Accepting nearby outputs could bring more perceptually similar images together and reduce the false-negative count, but it would enlarge the accepted neighborhood and likely increase false positives. The trade-off is built into the contract.
The study also reports similar conclusions for an updated NeuralHash implementation that appeared on macOS around early 2024 without an official explanation. That result matters as a version boundary, not as permission to collapse all implementations into one. Black-box access can reveal behavior without revealing the internal model, but changes in training, precision, preprocessing, or thresholds can change the distribution. A deployed system should therefore be tested by version and population rather than defended by pointing to a nominal output length shared across versions.
There are limits to what this evidence can establish. CelebA is a narrow, mostly frontal celebrity-face collection with limited contextual and demographic diversity, and the paper notes substantial racial imbalance. The researchers did not obtain CSAM data, so the face experiments are an argument for relevance, not a direct measurement of CSAM-detection performance. The exact-collision analysis also does not reproduce every production near-match rule. The absence of an illegitimate collision in the non-human sample cannot be read as a guarantee of zero risk, just as the face results cannot be assigned unchanged to every other image category.
Within those boundaries, the result is still substantial. The NeuralHash case shows why a perceptual representation must be evaluated as a distribution over actual inputs. Bias makes some positions less informative; dependence makes combinations less independent; image-population structure creates recurring outputs; and exact equality can miss many pairs the application expects to match. The nominal bit count remains a description of the output format. It is not a certificate that the matching system has cryptographic-style collision resistance. The next case asks whether a much larger numeric representation avoids the same dilemma, or whether its distance threshold simply makes the trade-off more visible.