Overview

Granite Vision's safety path reformulates selected questions as classification: safe versus unsafe, or hallucinating versus not hallucinating, instead of relying only on generated refusals. Sparse Attention Vectors select attention heads whose final-token vectors separate labeled classes by cosine similarity, then combine their votes. The method improves reported MHalu and VLGuard classification scores from 78.0 to 80.7 and 86.0 to 96.2, respectively, but the results do not establish universal safety or deployment readiness.

Learning outcomes

Safety Vectors turn selected multimodal safety questions into explicit class decisions rather than generated refusal judgments.
The reported Safety Vector scores rise from 78.0 to 80.7 on MHalu and from 86.0 to 96.2 on VLGuard.
The gains are task- and example-dependent; they do not provide a universal safety rate or replace review.

Keywords

Granite VisionSafety VectorsClassificationAttentionMHaluVLGuard

Safety as a Classifier, Not a Promise

Granite Vision's safety design asks a different question from its document benchmarks: can the model classify whether a multimodal input is harmful or whether a claim is hallucinated, rather than only generate a helpful-looking response? This chapter explains that separate classification path, called Sparse Attention Vectors, and distinguishes its reported gains from the paper's VLM-as-a-Judge results. The practical judgment is narrow but important: feature separability on selected safety tasks is evidence about a useful mechanism, not a complete guarantee for an enterprise deployment.

Why classify safety instead of asking for a refusal

A generative multimodal model usually demonstrates safety by producing a response. A prompt is sent to the model, and an evaluator judges whether the answer is harmful, faithful, fair, or appropriately refused. That setup resembles ordinary use, but it makes the measurement depend on the wording and quality of a generated answer. It can be difficult to tell whether a model recognized the unsafe input, followed a refusal pattern, or simply produced a response that a judge happened to rate favorably.

Granite Vision's paper adds a second setup in which safety questions are reformulated as classification problems. The model is asked to choose among explicit labels, such as “safe” and “unsafe,” or “hallucinating” and “not hallucinating.” Classification produces a direct decision that can be compared with a ground-truth label. The authors motivate this approach by noting that generative multimodal models do not always excel at discriminative tasks, even though strong discrimination is important for safety evaluation and other applications.

The distinction is not merely academic. Consider an image and an instruction that might contain harmful content. In a VLM-as-a-Judge setup, the model generates text and another model scores that text. In the classification setup, the system must decide which class the input belongs to. These tests can reveal different weaknesses. A model may produce a cautious-sounding answer without reliably separating safe and unsafe cases. It may classify a case correctly while still generating a poor explanation. Treating the setups as interchangeable would hide those differences.

Granite Vision's separate method uses Sparse Attention Vectors, or Safety Vectors. An attention head is one of the small processing units inside a transformer layer. At the end of a sequence, each head has an attention vector that summarizes part of the model's internal state for the final token. The method does not assume that every head is equally useful for safety. It searches for a sparse subset whose vectors separate the safety classes well.

The search begins with a small set of labeled examples. The labels describe a task-specific property, such as harmful versus non-harmful or hallucinating versus not hallucinating. For each example, the method extracts the final-token attention vector from each layer and head. This produces many candidate features: one vector for one head at one layer, evaluated across the labeled examples.

Next, the method builds a centroid for each class and each candidate head. A centroid is the average vector for the examples assigned to a class.

Published by DeepMox

FAQ

What are Granite Vision's Safety Vectors?

They are a sparse subset of attention-head vectors selected because they separate task-specific labeled classes well. For a new query, the selected heads compare class-centroid similarities and vote on the label.

How did Safety Vectors affect Granite Vision's reported classification scores?

Safety Vectors increased the reported MHalu score from 78.0 to 80.7 and the VLGuard score from 86.0 to 96.2. These are results for the selected benchmark classification setups.

Do Granite Vision's Safety Vectors guarantee safety?

No. They classify labels represented by a task's prompts and few-shot examples; they do not cover every harmful input, nuanced policy, or deployment condition and cannot replace review for high-consequence cases.

Previous12 minGranite Vision: What Its Benchmarks Actually Measure

Next12 minWhen Granite Vision Fits an Enterprise Workflow

Granite Vision: Safety as Multimodal Classification

Safety as a Classifier, Not a Promise

Why classify safety instead of asking for a refusal

FAQ