Four Ways to Ask What a Skill Can Do

SkillSpector does not rely on one detector to decide what an AI-agent skill can do. It combines evidence from language, program structure, data movement, dependency identity, and known signatures. Understanding those evidence types lets a reader match a warning to the question it actually answers—and avoid treating a literal pattern, a dependency advisory, or a clean scan as a complete intent verdict.

The previous stage established how SkillSpector builds a bounded snapshot and records what it could inspect. This stage begins once that evidence exists. The key issue is not how many checks the scanner performs, but how each check turns a different observable fact into a bounded security judgment.

The Same File Can Ask Several Different Security Questions

A lexical pattern starts with text. It can look for phrases or structures associated with prompt injection, data transmission, credential access, unsafe defaults, broad triggers, or remote script execution. This is useful because many dangerous instructions and implementation choices have recognizable forms. A phrase telling an agent to ignore its safety rules, a command that pipes a downloaded script into a shell, or a request aimed at a sensitive internal address can be strong evidence when the surrounding text supports that interpretation.

But a lexical match does not know intent by itself. The same word may appear in a security guide, a test fixture, a code example, or an instruction that is genuinely meant to be executed. A pattern can establish that a form of text exists at a location. It cannot, on its own, establish whether the behavior is reachable, necessary for the stated purpose, or connected to sensitive data. SkillSpector’s static runner therefore applies file-type and context filters, uses bounded security views, preserves locations, and leaves room for later semantic interpretation.

That distinction is especially important for agent skills because the text itself can be operational. A Markdown instruction is not merely a comment if the agent reads it as a command. At the same time, a Markdown document explaining prompt injection is not automatically an attack. A useful detector must be sensitive to both possibilities. The right conclusion from a text match is usually: “Inspect this evidence in context,” not: “The author’s intent is proven.”

Program structure supplies a different kind of evidence. SkillSpector’s Python behavioral analyzer parses source into an abstract syntax tree, or AST. An AST represents calls, names, arguments, and relationships as program structure rather than as a flat string. That lets the analyzer recognize a dangerous operation even when the spelling is changed by an import alias or a familiar wrapper.

For example, a direct call to exec is an obvious execution sink. So are calls through eval, compile, dynamic imports, subprocess functions, and os execution functions. The analyzer also checks for reflective access such as retrieving an execution function by name. That matters because a simple search for os.system may miss a call that obtains the same function through getattr. The detector keeps the reflective rule narrow: ordinary dynamic attribute access is not equivalent to a command-execution sink, while a literal sink name can be a deliberate way to evade a simpler check.

Structure also lets SkillSpector distinguish an isolated operation from a dangerous chain. If exec or eval receives a value produced by a network request, an encoded payload, a dynamic import, or another command-related operation, the combination is stronger evidence than either call considered alone. The analyzer reports that chain separately because the relationship changes the risk. A string passed to eval may be questionable; network-sourced content passed into eval creates a much more direct path from external input to execution.

The same reasoning applies to deserialization. Some loaders reconstruct objects or invoke behavior while reading data. SkillSpector identifies unconditionally unsafe forms and treats argument-dependent forms differently: a safe YAML loader or a PyTorch call using weights_only is not equivalent to an unsafe form. This is an example of why structural analysis is more informative than a rule that flags every occurrence of a library name. The relevant claim is not “this file mentions YAML.” It is “this call uses a loader mode that can reconstruct unsafe objects from untrusted data.”

Data-flow analysis asks a third question: what reaches the sensitive operation? In SkillSpector’s taint model, a source can be an environment variable, a file read, a network response, or user input. A sink can be network output, code execution, a file write, or an unsafe deserializer. The analyzer then looks for a relationship between them.

Imagine two snippets with the same network library. In the first, the code makes a request to retrieve a public configuration and discards the response. In the second, it reads an environment variable containing a token, places that value in a request body, and posts it to an external server. A library-name pattern sees “HTTP” in both. Taint analysis distinguishes the second because it sees the path from a credential source to a network sink. SkillSpector gives that relationship its own credential-exfiltration signal.

Taint does not require every assignment to be written on one line. The analyzer tracks a source through variable reassignment and simple packaging into dictionaries, lists, tuples, or formatted strings. It can therefore identify a flow even when a value is renamed or placed inside a payload before reaching the sink. Direct source-to-sink flow and variable-mediated flow are separate cases because the intermediate step affects confidence and review effort.

This is still bounded static reasoning, not a full proof of runtime data flow. A flow may occur only in a branch that never runs. A dynamic dispatch path may not be resolved. A library may transmit data internally without the skill’s own source mentioning the relevant sink. Conversely, a source-to-sink relationship can be highly informative even when the tool cannot determine whether an operator intended it. The output should preserve that difference: it reports an observable path, not a mind-reading result.

The layers reinforce one another when they agree. A text pattern may identify a request to send context away. AST analysis may find the actual HTTP call. Taint analysis may show that user or environment data reaches it. The combined evidence is more actionable because each layer answers a different part of the question. When they disagree, the disagreement is also useful: it tells the reviewer whether the concern is only linguistic, structurally concrete, or connected to a sensitive value.

flowchart LR N01["Text and metadata patterns"] --> N06["Canonical finding"] N02["Python AST behavior"] --> N06 N03["Source-to-sink data flow"] --> N06 N04["Dependency and lockfile identity"] --> N06 N05["OSV and YARA evidence"] --> N06 N07["Hidden or nested artifact signals"] --> N06 N06 --> N08["Location, confidence, and boundary"]

The convergence point in this diagram is not a single truth value. It is a finding with a location, confidence, explanation, and evidence boundary. That shape matters later: a report can preserve several independent signals without pretending they all have the same meaning or strength.

Identity and Concealment Matter as Much as Behavior

A skill can be risky before its own code performs an obviously dangerous operation. Its dependencies may be unpinned, its package name may resemble a popular one, its container verification may be disabled, or its executable payload may be concealed in a place ordinary discovery skips. SkillSpector’s supply-chain and artifact checks address those questions by keeping identity, resolution, and concealment separate.

Dependency identity begins with a distinction that is easy to lose in a configuration file. A manifest may mention a package, but the package name alone does not tell the scanner which release will be installed. A precise version can be matched to an advisory. A broad range or an unpinned requirement admits multiple releases, so a name-only vulnerability result describes the package’s advisory history rather than proving that the selected release is affected.

SkillSpector reads common Python and npm dependency manifests and can use lockfiles to recover the concrete versions that installation is likely to use. It treats Python and npm resolution separately because similar names in different ecosystems are not interchangeable. For npm, nested copies also matter: a dependency can appear at several installed versions, and dropping all but the first occurrence could hide a vulnerable copy. The scanner’s implementation therefore preserves package-and-version identities rather than reducing a lockfile to a set of names.

For known vulnerabilities, SkillSpector queries OSV.dev with package coordinates. The service provides a current external advisory source, while a small built-in fallback list covers some cases when the service is unreachable. The fallback is not silently equivalent to the live database. If the live lookup fails and the fallback does not establish a known issue, the scanner can surface that coverage gap. This is an important calibration choice: the result describes what was checked, not what the operator might wish had been checked.

Other supply-chain signals answer different questions. An abandoned dependency may no longer receive security fixes, even if no current advisory is found. A possible typosquat concerns whether the name itself is deceptively close to a popular package. An image reference without signature or registry verification concerns whether a later pull can be replaced by untrusted content. None of these is the same as a confirmed vulnerability in an exact installed release, so they should not be read as interchangeable evidence.

YARA signatures add another layer. YARA is a rule language for matching byte or text patterns associated with malware, webshells, cryptocurrency miners, exploit code, or other known indicators. SkillSpector can use built-in rules and bounded user-supplied rules. A YARA match can be a strong reason to inspect or reject a payload, but the match still depends on the rule set. No match means only that the available signatures did not match the inspected bytes; it does not establish that the artifact is benign.

Concealment checks address the possibility that ordinary source inventory is itself misleading. SkillSpector separately looks for shipped Python bytecode such as .pyc files or __pycache__ directories because normal source analysis may skip or classify them as binary. A source tree can look harmless while a bytecode artifact remains available to the runtime. The presence of such bytecode is a supply-chain warning even though SkillSpector does not execute or fully disassemble the payload as part of the cited behavior.

The same logic applies to executables hidden inside document containers, nested archives, hidden files, or artifacts with misleading extensions. The scanner can establish that executable content is concealed and preserve the outer and inner provenance. It need not claim to understand the payload in order to tell a reviewer that extension-based inspection is insufficient. Concealment is evidence about reviewability and installation risk, not proof of a specific malicious action.

Resource limits complete this deterministic picture. A rule engine, dependency parser, or nested-archive reader can be exposed to attacker-shaped input, so SkillSpector bounds not only the final report but also the work used to produce it. Discovery, cached bytes, nested members, parser complexity, YARA rules, dependency requests, findings, and runtime each have ceilings. If one is reached, the scanner records partial work instead of silently dropping the unexamined remainder.

That behavior creates a useful asymmetry. A positive signal can increase concern, while an omitted region cannot increase confidence. Suppose OSV is unreachable, a lockfile is malformed, and a package’s version cannot be resolved. The correct outcome is not “no dependency vulnerability.” It is “the exact dependency risk was not fully established,” alongside whatever lower-confidence or fallback evidence remains. Likewise, a hidden executable can be flagged as concealed without claiming that its contents were safe or malicious.

The deterministic pipeline is therefore a collection of questions with different answers:

  • Text patterns ask whether recognizable risky language or configuration is present.
  • AST analysis asks whether source structure reaches a dangerous operation.
  • Taint analysis asks whether sensitive data reaches a sensitive sink.
  • Dependency analysis asks which identity and release are actually relevant.
  • YARA asks whether available signatures match the inspected bytes.
  • Artifact analysis asks whether the material was visible, interpretable, and fully inspected.

A later semantic layer can help decide whether these signals fit the skill’s purpose. It cannot make a name-only advisory into a version match, turn an uninspected file into clean evidence, or erase a concrete source-to-sink path. The strength of SkillSpector’s deterministic design lies in preserving those distinctions so context can refine the judgment rather than flattening it.

References