A Trustworthy Answer to: What Can This Machine Run?

The question that brings people to llmfit sounds like a request for a list: which open model can this computer run? The more trustworthy answer is conditional. It names a model configuration, the memory and execution path it uses, the context it can actually sustain, the runtime that can serve it, and the evidence supporting its speed. This final judgment brings those pieces together so a reader can choose, verify, or upgrade without turning an estimate into a promise.

llmfit’s central contribution is not that it eliminates uncertainty. It makes the important uncertainty visible and ties each conclusion to the part of the system that supports it. Hardware detection describes the available pools and backend. Model metadata describes the weights, architecture, context, and format. Fit analysis tests the configuration against those pools. Speed and task scoring compare competing objectives. Benchmarking supplies stronger evidence for a matching machine. Planning shows what would have to change when the current configuration is not enough.

A useful recommendation therefore sounds less like “run Model X” and more like this: “For this workload, Model X at this weight quantization and context is expected to run through this provider on this hardware path; the memory fit is at this level; the throughput is measured, calibrated, or estimated; and this verification or upgrade would change the conclusion.” That is more information than a ranking, but it is also a decision a reader can audit.

Start with the question the machine must answer

The first step is to define the workload before choosing the model. Interactive chat, coding assistance, reasoning, multimodal work, and embedding place different demands on quality, speed, context, and memory. llmfit’s use-case categories and scoring weights reflect that difference. A model that is well aligned with coding does not automatically become the best general chat model, and a model that generates quickly does not automatically produce the best answers for a reasoning task.

The workload also determines what “fast enough” means. A model producing output at a useful rate for an occasional coding prompt may be inadequate for a tool that issues many requests. A longer context may matter more than peak generation speed for a document-analysis workflow. An embedding workload may value throughput and fit more heavily than an interactive assistant does. llmfit can rank candidates under different use cases, but the user still supplies the meaning of “useful.” The composite score is a structured aid to that judgment, not a replacement for it.

Once the task is clear, identify the hardware topology. A discrete GPU supplies a VRAM pool separate from system RAM. A unified-memory machine shares one pool between CPU and GPU. Multiple GPUs may contribute aggregate capacity when the execution path can split the model, but summed capacity is not the same as guaranteed scaling. The backend and memory bandwidth affect speed estimates, while currently available system RAM affects whether a CPU or offload path has room at the moment of analysis.

This first check rules out a common mistake: treating capacity as one universal number. Two systems can expose the same nominal amount of memory but offer different execution paths. A discrete GPU may keep a model in VRAM or spill part of it into RAM. A unified-memory system does not have that same separate spill boundary. A multi-GPU configuration may make a larger aggregate pool visible to the fit calculation while still relying on runtime support to divide the work. The right-sizing decision begins with the topology because the topology determines what “fits” means.

Next identify the model configuration. The model name establishes identity, but the weight format, quantization, native context, architecture metadata, and any MoE fields determine the demand. A dense model’s total parameters provide a rough weight scale. An MoE model keeps total and active parameters distinct. Attention and KV metadata influence how context consumes memory. A native MLX entry, a GGUF entry, and an AWQ or GPTQ entry can belong to related model families while requiring different runtimes.

The configuration must then be stated explicitly. Which weight quantization is being evaluated? Which context is used for the estimate? Is the KV cache represented at the default precision or another supported level? Is the row using a detected machine, a capacity override, or a hardware profile? Without those details, the model name is not enough to reproduce the answer. A recommendation that omits them may be directionally useful, but it cannot explain why another machine or another setting receives a different result.

The complete decision chain is:

flowchart TD A[Workload intent] --> B[Candidate model and configuration] B --> C[Fit, memory and usable context] C --> D[Runtime and provider verification] D --> E[Benchmark and task test] E --> F[Confidence-aware recommendation] F --> G[Choose, verify or upgrade] G -->|New profile or hardware| C

The chain is deliberately circular. A benchmark can change confidence without changing the model. A new hardware profile can change the available pool and bandwidth without changing the model record.