Trust Is Built in the Checks Between Steps

The knowledge-work-plugins repository treats validation as part of the work a plugin delivers, not as a reassuring sentence added after the answer. This chapter focuses on two examples: the data plugin’s checks on an analysis, and the customer-support plugin’s checks on an escalation. Together they show how a workflow moves from a plausible result to a qualified one by testing evidence, naming uncertainty, assigning ownership, and revisiting the outcome.

Data analysis needs a visible chain of checks

The data plugin begins by classifying the user’s question. A quick lookup, a full investigation, and a formal report may all involve numbers, but they do not require the same depth of method or the same kind of explanation. A quick answer may need one metric and a relevant time range. A full analysis may need comparisons, trends, and multiple dimensions. A formal report needs methodology, data sources, caveats, and recommendations. The classification prevents the workflow from treating a simple count and a consequential business analysis as the same task.

The next step is choosing data. When a warehouse is connected, the workflow explores the schema, identifies relevant tables and columns, writes queries, runs them, and checks unexpected results. When no warehouse is available, it can work from pasted results, a CSV or Excel file, or a schema description. That fallback keeps the analytical procedure available, but it also changes what the result can claim. A report based on supplied query results cannot imply that the plugin inspected the underlying warehouse or verified how those results were produced.

Validation begins before the final prose. The analysis workflow calls for row-count sanity checks, null checks, magnitude checks, time-series continuity, and aggregation checks. Each test addresses a different way a result can look reasonable while being wrong. A row count can reveal that a join multiplied records. Nulls can show that a metric is being calculated on an unexpectedly incomplete population. A magnitude check can expose a unit or filter error. A gap in a time series can make a trend look like a sudden change. A subtotal that does not reconcile with a total can reveal a mismatch in grain or a hidden overlap.

flowchart LR Q["Question or symptom"] --> D["Choose sources and scope"] D --> T{"Sanity or reproduction checks"} T -->|checks hold| C["Confidence or severity"] T -->|problem found| I["Investigate and qualify"] I --> D C --> A["Recommendation or escalation"] A --> F["Follow-up and re-check"]

The separate validation skill expands this into a review of the reasoning itself. It asks whether the analysis framed the right question, selected the right data, defined the population correctly, used consistent metric definitions, and compared like with like. These are not cosmetic checks.