Modeling Before Modeling: Assumptions, Selection, and Evidence
For MathModelAgent, model selection is the point where an apparently orderly workflow can still go wrong. This chapter examines the reasoning that must happen before code is allowed to speak for a model: identify the requested output, test competing interpretations, state the assumptions and constraints, and decide how the result will be validated. Readers should leave able to interrogate a proposed model rather than judging it by the sophistication of its name.
A model is a selective description of a problem. It keeps some relationships and discards others so that a question can be calculated, simulated, or compared. That selectivity is useful, but it creates a risk. If the output has been misunderstood or an important constraint has been omitted, a more powerful algorithm only produces a more elaborate answer to the wrong question. MathModelAgent’s analysis guidance addresses this risk by making problem structure and evidence precede implementation.
Start with the form of the requested answer
A practical first question is: what kind of thing must the model produce? A single maximum or minimum suggests an optimization problem. A future sequence suggests prediction. A ranking or score suggests evaluation. A class label suggests classification. A trajectory or changing system state suggests a mechanistic or simulation model. These categories do not dictate one algorithm, but they narrow the space of sensible choices and reveal what “success” must mean.
The distinction matters because different outputs fail in different ways. An optimization result can be numerically impressive but violate a capacity, budget, integer, time-window, or physical boundary. A prediction can fit historical data while leaking future information into its features. A ranking can be internally consistent while hiding arbitrary weights or mixing indicators with incompatible directions. A classifier can report high accuracy on an imbalanced data set while performing poorly on the cases that matter. A mechanistic simulation can produce a smooth curve while violating units, initial conditions, boundary conditions, or a conservation relationship.
MathModelAgent’s modeling prompt reflects this output-first logic. It asks the modeler to classify the problem, explain why a method was chosen, describe the data and solving steps, state how the result will be checked, and specify appropriate visualizations. The important design choice is not that the prompt lists many model families. It is that a method is expected to answer a particular type of question under particular conditions.
That logic also favors a baseline. A simple, interpretable model can establish whether the data and target support the intended relationship at all. A more complex method is justified when it captures a meaningful nonlinearity, interaction, temporal pattern, or uncertainty structure that the baseline misses. Complexity without a diagnostic reason is a poor trade: it increases tuning burden, makes failures harder to localize, and can obscure the relationship between the result and the original problem.
No single method dominates across all cases. A small, nearly monotonic series may reward a restrained forecasting method, while a high-dimensional, nonlinear data set may justify a more flexible learner. A constrained allocation problem may need an optimization formulation rather than a predictive model. The decision is therefore conditional. The question is not “Which algorithm is strongest?” but “Which representation exposes the decision the problem actually asks us to make, and can its result be tested?”
Ambiguity is the first model variable
Many modeling errors begin before a variable is written down. Competition problems often use phrases that allow multiple readings: whether an excess quantity is wasted or sold at a discount, whether a resource is shared or exclusive, whether a later scenario adds a new constraint or merely changes a parameter, or whether “best” means highest total value, lowest risk, or a compromise between them. Treating one reading as obvious hides an assumption at the foundation of every later equation.
The analysis skill therefore requires an assumption and ambiguity pre-check. The analyst should state at least two plausible interpretations, then use quick calculations, logical consequences, or the progression of the problem’s later questions to decide which reading is coherent. This is not an invitation to delay every decision indefinitely. It is a way to make the decision visible and to test whether it preserves the purpose of the whole problem.
A useful diagnostic is marginal relevance. Suppose a later question is supposed to add a resource or a new restriction. If the chosen interpretation makes that addition have no effect on the feasible region or objective, something deserves another look. Perhaps the new resource was never connected to the objective. Perhaps a constraint was written in the wrong direction. Perhaps the base case already assumed away the phenomenon that the later question was meant to explore. The result of the diagnostic is not necessarily a different model, but it is a more defensible explanation of why one interpretation was adopted.
The same principle applies to dependent subproblems. If the second question builds on the first, its additional information should create a meaningful change in the model, the feasible set, the uncertainty, or the evaluation. A decomposition that merely gives each question a separate paragraph is not a decomposition of the problem. In MathModelAgent’s intended workflow, dependencies and outputs are part of the analysis report because they tell the coding stage what must be carried forward.
Making ambiguity explicit also limits later prose. Once an assumption has been stated, the writer can describe a result as conditional on that assumption. Without it, a number may be presented as if it were a property of the real world rather than a consequence of a modeling choice. Inspectability begins here: a reader can disagree with an assumption and still understand exactly which part of the conclusion the disagreement affects.
A running program can still be substantively wrong
The most dangerous failures are not always runtime failures. A program may execute successfully while implementing the wrong objective, using the wrong units, dropping a boundary, or evaluating the model with information that would not have been available at prediction time. MathModelAgent’s norms emphasize these distinctions because an interpreter can report whether code ran, but it cannot decide whether the code faithfully represents the problem.
Optimization makes the boundary especially clear. A feasible solution must satisfy every relevant constraint before its objective value is compared with another solution. Physical variables need meaningful upper and lower bounds; a desktop-scale structure cannot accept an unconstrained result that is larger than the physical system itself. Integer and binary decisions cannot remain continuous merely because a solver returns a convenient decimal. After rounding or repair, the solution must be checked again. An algorithm that finds a lower cost while violating capacity is not a near-success. It is an invalid answer.
Prediction has a different failure surface. Time-series features must be built from information available at the prediction point; a future value must not leak backward through a shift or a preprocessing operation. Scaling and encoding should be fitted on the training data and applied to held-out data. A model that looks accurate because the test set influenced feature construction has not demonstrated the claimed performance. Its execution may be flawless, but its evidence is contaminated.
Evaluation models require yet another discipline. Indicators with different units cannot be added directly. Positive and negative directions must be aligned, normalization must be stated, and weights need an explanation. A ranking is not self-justifying because it is presented in a table. It reflects a choice about what counts as better and how much each dimension matters. Sensitivity analysis is important precisely because a small change in a weight or normalization rule can reverse the order.
The project’s guidance also separates statistical association from causal explanation. A coefficient, correlation, or accurate predictor can establish that a pattern appears in the data under a specified design. It does not automatically establish that changing one variable will cause the outcome to change. A paper that turns predictive importance into a causal claim has crossed a boundary that the computation did not authorize.
These examples share a principle: semantic correctness is a property of the relationship among the question, assumptions, code, data, and interpretation. The more flexible the algorithm, the more important that relationship becomes. MathModelAgent’s workflow can require checks and make the assumptions visible, but it cannot infer every domain constraint from a prompt. The user still has to know which boundaries matter.
Fit the analysis to the data-generating situation
The repository’s distinction between data-driven exploration and physical or mechanistic problems is a useful guard against template thinking. A genuine data set contains samples whose distributions, missingness, outliers, and relationships may need examination. A physical problem may instead provide fixed parameters such as dimensions, masses, or boundary conditions. Producing histograms and anomaly-removal language for the second case does not make the analysis more rigorous; it signals that a generic data-analysis template has replaced problem understanding.
For data-driven work, exploration can reveal missing values, unusual observations, skew, correlations, and group differences that change model choice or preprocessing. For a mechanistic problem, the more relevant checks may be dimensional consistency, geometry, conservation, initial conditions, boundary conditions, and whether numerical results converge when a step or grid is refined. The same label—“EDA”—should not force the same operations onto both situations.
This is also where visualization becomes part of reasoning. A time-series plot can reveal a trend or a structural break that affects validation. A scatter plot with a fitted relationship can expose nonlinearity or heteroscedasticity. A sensitivity curve can show whether a conclusion survives plausible parameter changes. A physical consistency table can make units and bounds visible. The form of the visual should follow the question being investigated, not a desire to fill a paper with a standard number of figures.
A modeler who chooses a method should therefore be able to answer four linked questions. What does the output represent? What assumptions make the representation reasonable? What constraints or data conditions can invalidate it? What observation would reveal that it is failing? These questions make model selection a testable decision rather than a stylistic preference.
Validation turns a model into an inspectable claim
Validation is the bridge between a proposed model and a claim that another person might rely on. For prediction, it may involve held-out error measures, time-aware splits, residual checks, or comparison with a baseline. For optimization, it includes feasibility, constraint slacks, stability across runs when randomness is involved, and comparison with exact or small-scale solutions where appropriate. For evaluation, it includes sensitivity to normalization and weights. For a mechanistic or simulation model, it includes unit checks, boundary behavior, conservation or invariants, and numerical convergence.
Sensitivity analysis adds a second dimension. A result can be valid under one parameter setting but fragile under a small perturbation. Changing a key constraint, initial condition, weight, or coefficient and observing the effect does not prove robustness, but it reveals which assumptions carry the conclusion. If a small change causes a large shift, the paper should say so. A stable result under the tested range is stronger than a single point estimate, though it still depends on whether the tested range was plausible.
Parameter provenance completes the chain. A parameter should come from data, a cited source, a calibration procedure, or another explicit basis—not appear as an unexplained default. Provenance allows a reviewer to challenge the right thing. If a result changes, they can ask whether the data changed, the literature basis changed, or the calibration was wrong. Without provenance, every number is a free-floating fact that the writing stage may accidentally harden into authority.
The reasoning sequence can be represented as a conversation among stages:
The arrows are not merely chronological. They show what must cross each boundary. The analysis hands the model a target and conditions. The model hands computation an executable formulation. Computation hands validation both results and checks. Validation can send the work backward when a condition fails, or forward as a bounded claim when the evidence is adequate. If an arrow carries only a model name or a final number, the chain is missing the information needed for inspection.
That is the standard MathModelAgent’s modeling guidance is trying to establish. Its prompts and norms cannot guarantee that an LLM will honor every instruction, and the repository does not present them as a formal proof system. Their value is that they make the questions of fit, boundary, and evidence explicit before an interpreter produces an impressive-looking output. The next step is to see what happens when the proposed model is actually executed—how code errors become feedback, and what execution still cannot tell us.