From Disk to Blocks: How Syncthing Builds Evidence

Syncthing cannot decide what to transfer from a filename and a timestamp alone. This chapter follows the data path that turns a local filesystem change into evidence another device can compare: redundant change detection, block hashing, indexed metadata, source selection, verification, and temporary construction. The practical understanding is that Syncthing separates “what state should this file have?” from “which bytes can safely realize that state,” allowing it to transfer less data without trusting any single path or peer blindly.

A filesystem change becomes a block description

A filesystem watcher is valuable because it can tell Syncthing about a change soon after it happens. But an event stream is not a complete record of truth. Events can be delayed, coalesced, lost during an error, or unavailable on a platform or filesystem. Syncthing therefore combines watcher notifications with periodic full scans. The watcher improves responsiveness; the full scan checks the folder again. Neither is treated as sufficient by itself.

The folder runner starts with an initial scan, can aggregate watcher events before scanning, and schedules later full scans with a randomized interval around the configured period. Randomization keeps a collection of folders from all doing expensive work at exactly the same moment. If watching fails, the folder can schedule a scan rather than assuming that no event means no change. This is a modest but important design choice: correctness is a periodic verification problem, while responsiveness is an event-delivery problem.

During a scan, Syncthing first compares properties that are cheap to inspect, such as modification time, size, and permissions. If those properties indicate a change—or if a file needs to be rechecked—it reads the file and computes block hashes. A timestamp says when a filesystem believes an item changed; it does not identify which content is present. Hashing turns the content itself into evidence that can be compared across devices.

A file is divided into equal-sized blocks for that file, with the final block allowed to be smaller. Syncthing chooses block sizes from 128 KiB through 16 MiB according to file size, aiming to keep the block list manageable while still making partial changes useful. A block record carries an offset, a size, and a SHA-256 hash. The result is not a second copy of the file. It is a compact description that lets another device ask, “Do you have this exact piece of the version I need?”

That description gives a large file a more useful change surface. If a small region changes while the rest of the file stays the same, the desired block list can contain many hashes that already match the local version and only a smaller set that differs. The system can then avoid retransferring unchanged blocks. The benefit is not based on assuming that a timestamp difference means a small edit; it comes from comparing content evidence at block boundaries.