The Tokenizer: Turning Market Bars into Coarse and Fine Symbols
Kronos's tokenizer answers the first architectural question raised by financial K-lines: how can a continuous, six-channel market observation become a discrete object that a sequence model can predict without losing the relationships that made the observation useful? This is not ordinary formatting. The tokenizer is a learned information bottleneck, and its reconstruction quality, hierarchy, and vocabulary size shape what the autoregressive model can later learn. Understanding that bottleneck makes it possible to judge both the promise and the limits of Kronos's “financial language.”
Encoding a market bar into a learned latent code
The tokenizer begins with a K-line record represented by its open, high, low, close, volume, and amount. These values are mapped from the input dimension into a model-sized latent representation. Transformer encoder blocks then process that representation before a projection produces the vector that will be quantized. The sequence of operations matters: Kronos does not assign a hand-written symbol to a price range. It learns an intermediate space in which a market record can be compressed and later reconstructed.
The tokenizer is trained as an autoencoder. An encoder turns the continuous record into a latent vector; a quantizer replaces that continuous vector with a discrete code; and a decoder attempts to recover the original record. This creates a feedback loop between representation and fidelity. A code is useful only if it is compact enough for the next model to handle while preserving enough information for the decoder—and, ultimately, for downstream tasks—to recover meaningful structure.
The quantizer used by Kronos is Binary Spherical Quantization, or BSQ. At a high level, BSQ converts the latent vector into a bipolar binary representation: each component takes one of two signs, producing a point on a discrete code space. The implementation normalizes the input before quantization and scales the resulting code. It can then convert the binary pattern into an integer index, or split it into two indices when the model needs separate coarse and fine subtokens.
This is different from rounding each input number independently. Ordinary rounding would preserve a chosen decimal precision for each channel, but it would not by itself create a learned joint code for the entire OHLCVA observation. Kronos instead quantizes a learned vector whose dimensions have been shaped by the encoder. The resulting bits represent a position in that learned latent space. The code is still lossy, but the loss is managed at the level of a joint observation rather than as six unrelated rounding errors.
The training objective gives the quantizer a reason to stay useful. Kronos uses a reconstruction from the first part of the code, a reconstruction from the complete code, and a quantization term. The quantization or commitment component discourages the encoder's continuous output from drifting too far from the discrete representation it must use. The reconstruction terms ask two different questions: can the coarse code retain the principal structure, and can the full code restore a higher-fidelity version of the market record?
BSQ also includes entropy-related terms in its training loss. At a conceptual level, those terms encourage the binary space to be used in a way that is not needlessly collapsed onto a small set of codes. This matters because a nominally large vocabulary can have little practical capacity if the encoder repeatedly chooses only a narrow region of it. Entropy regularization is not a guarantee that every code will be equally meaningful, but it is part of the attempt to keep the discrete space useful while the reconstruction and commitment terms keep it tied to the input.
Those questions impose a contract on the representation. The encoder cannot simply create arbitrary binary patterns that are easy for the predictor to manipulate. The decoder must be able to interpret them. If many distinct market observations collapse into the same code, the full reconstruction will lose detail. If the code is so large or irregular that the autoregressive model cannot learn its distribution, fidelity at the tokenizer stage will not translate into a practical foundation model. The tokenizer therefore sits between two pressures: preserve information and keep the future prediction problem tractable.
A useful way to see the path is to follow one record rather than one scalar. The six values enter together. The encoder produces one latent representation for their joint configuration. BSQ turns that representation into a binary code. The decoder receives either the complete code or its first part and tries to reconstruct the same six-channel record. The important object is not the final integer label by itself; it is the relationship among the input record, the latent geometry, the binary code, and the reconstruction.
The coarse reconstruction is not merely an early output for debugging. It gives the first part of the code an explicit learning responsibility. The full reconstruction can use both parts to restore information that the coarse approximation cannot express. The hierarchy is therefore tied to the loss: the model is encouraged to make the first subtoken useful on its own and the complete token useful at higher fidelity.
This arrangement also creates a clean boundary between compression and prediction. The tokenizer is not asked to forecast the next bar; it is asked to make the current bar available in a form that can be reconstructed. The temporal model is not asked to rediscover the raw feature schema; it receives a sequence of objects whose structure has already been defined. That separation can simplify scaling, but it introduces coupling. If the tokenizer changes its bit allocation or the meaning of its subtokens, a predictor trained for the old representation may no longer interpret the sequence correctly.
Why split one binary code into coarse and fine subtokens?
A binary code with k bits has up to 2^k possible patterns. If every pattern were treated as one entry in a prediction vocabulary, the vocabulary and its output head would grow exponentially with the number of bits. Richer codes could preserve more distinctions, but they would make the subsequent autoregressive model more expensive in parameters and computation. A representation that is faithful but impossible to predict at scale is not a workable foundation-model interface.
Kronos addresses this by factorizing the code into two parts of equal size. Instead of asking the generator to choose one item from a vocabulary of size 2^k, it predicts a coarse subtoken from a vocabulary of size 2^(k/2) and then a fine subtoken from another vocabulary of the same order. The total pair can still represent a large number of combinations, but the model handles the choice as two linked decisions rather than one enormous classification problem.
The saving is not free. The two predictions must be related. If the coarse and fine parts were independent, the factorization could reduce the output size while throwing away the structure it was meant to preserve. Kronos therefore trains the coarse part to carry principal information and the fine part to refine it. The sequence model later uses the selected coarse subtoken when predicting the fine one. The vocabulary reduction and the dependency mechanism are two sides of the same design: the first makes prediction manageable, and the second keeps the two decisions connected.
This also clarifies why “coarse” and “fine” are more than convenient labels. The coarse subtoken should support a lower-fidelity reconstruction of the original record. The complete code should support a more detailed reconstruction. If both parts learned exactly the same role, the hierarchical objective would add little. If the first part captured an overly broad category and the second part had to carry almost all useful information, the factorization would become an awkward bottleneck rather than a meaningful abstraction.
The hierarchy should not be interpreted too literally. A coarse subtoken is not necessarily a human-readable label such as “high volatility” or “rising market.” The code is learned from reconstruction and quantization objectives, and its internal distinctions need not map neatly to a trader's vocabulary. What can be said more safely is that the training setup assigns different reconstruction responsibilities to the two parts. Their meaning is operational and statistical: the first supports a principal reconstruction, and the pair supports a fuller one.
That distinction matters when reading diagrams or model outputs. An integer index is only a compact address for a binary pattern. It is not automatically an ordinal quantity, and a nearby index is not guaranteed to represent a nearby market state. The useful relationship is between the code and the decoder that learned to interpret it. Treating indices as ordinary prices or assuming that numerical distance between them has market meaning would confuse the storage format with the learned representation.
The bottleneck can fail in several ways. It can be too narrow and merge observations that downstream forecasting needs to distinguish. It can be poorly balanced and let one part of the code carry most of the useful information. It can preserve reconstruction detail that does not help temporal prediction, or it can discard activity patterns that matter for volatility and synthetic generation. It can also be technically valid but mismatched with a checkpoint: the predictor's output vocabularies and the tokenizer's bit layout must agree. The public implementation exposes separate bit counts and conversion paths precisely because the representation is part of the model contract, not an interchangeable preprocessing choice.
This is why tokenizer quality should be assessed at more than one level. Reconstruction error asks whether the code can recover the input. Code usage and quantization behavior ask whether the discrete space is being used in a healthy way. Downstream forecasting and generation ask whether the retained distinctions help with the tasks the foundation model is meant to support. A good result at one level does not settle the others. The paper reports that larger vocabularies improve reconstruction and downstream forecasting in its experiments, but that finding is still a measured trade-off under a particular setup rather than a universal rule that “more bits is always better.”
For a practitioner, this means that a tokenizer checkpoint cannot be judged only by how compact its indices look or by whether a decoder can produce visually plausible bars. The relevant test is whether the same representation supports the next stage under the intended workload. A code that reconstructs price channels accurately but loses activity structure may be inadequate for volatility or synthetic generation. A code that retains every small distinction may increase output cost without improving the decision-relevant forecast. The representation is successful when its compression, hierarchy, and prediction cost are aligned with the tasks the model is expected to serve.
The same principle applies to domain specialization. The tokenizer is trained on financial K-line records, so its latent space is shaped by the price-and-activity combinations present in that data. This can make the representation more relevant to the target domain than a generic scalar quantizer. It also means that unfamiliar assets, regimes, or data-quality conditions may be represented less reliably. The learned code is not a neutral language of all possible markets; it is a language formed by its training distribution and reconstruction objectives.
The tokenizer therefore does two jobs at once. It compresses continuous observations into a vocabulary that an autoregressive model can handle, and it defines which distinctions survive that compression. The first job is computational. The second is epistemic: it determines what the rest of Kronos is allowed to know about each market bar. Once the coarse and fine symbols exist, the next question is temporal. How does the generator use a history of these symbols, choose the next coarse part, and condition the fine part on that choice?