TL;DR

Background

Different system prompts ("You are Qwen...", "You are a helpful assistant.", "I am a helpful assistant.") place the model in different regions of its hidden-state space. We map these regions for Qwen-2.5-7B-Instruct across 37 system prompt conditions and show that the choice of system prompt determines both the model's representational geometry and its vulnerability to targeted capability degradation via contrastive wrong-answer SFT.

Methodology

37 system prompt conditions on Qwen-2.5-7B-Instruct spanning: 7 Qwen identity variants, 4 cross-model defaults (Llama, Phi-4, Command-R), 5 generic assistant paraphrases, and 21 fine-grained assistant variants (token perturbations, word/role substitutions, framing changes, length). Measured: (1) mean-centered cosine similarity between hidden-state centroids at Layer 10, (2) contrastive wrong-answer SFT self-degradation (ARC-C logprob accuracy drop when the source persona is trained on wrong answers, no anchor negatives, 586 wrong + 400 bystander correct, 5 epochs, 3 seeds), (3) cross-leakage between framings, (4) behavioral identity compliance ("Who are you?", N=50-100 per condition).

Conditions tested

Qwen variants (7 conditions — varying the Qwen identity prompt):

Cross-model (4 conditions — other models' default system prompts applied to Qwen):

Assistant (5 conditions — generic helper descriptions without model names):

Results

Mean-centered cosine similarity between 16 system prompt conditions at Layer 10. Three clusters: Qwen identity variants (upper-left), generic assistant paraphrases (lower-right), and cross-model named prompts (middle). The model's own identity prompt and a generic "helpful assistant" prompt occupy anti-correlated regions of representation space (-0.58 centered cosine) despite sharing most surface tokens.

Self-degradation under contrastive wrong-answer SFT for all 21 assistant variants (586 wrong answers, 5 epochs, 3 seeds, no anchor negatives). Error bars show ± 1 std across seeds. qwen_default degrades -73.8 ± 9.3pp while generic_assistant degrades -21.8 ± 11.6pp — a 3.4× gap. Most conditions have wide error bars (high cross-seed variance), but a few are robustly distinguishable.

Main takeaways:

• Representation space has three distinct clusters for system prompts: the model's own identity, generic assistant descriptions, and other models' identity claims. At Layer 10 Qwen variants cluster tightly (centered cosine 0.32–0.99), generic assistant paraphrases form a separate cluster (0.59–0.98), and the two are anti-correlated (-0.52 to -0.66). Cross-model named prompts (Phi-4, Command-R) sit between them. Removing the name "Command-R" from an otherwise identical prompt moves it from the cross-model cluster to the assistant cluster (centered cosine 0.27 to and_helpful), confirming the model name — not surrounding language — determines cluster assignment. Empty system block ("") clusters with generic assistant (0.63), not with Qwen.
• The model's RLHF identity prompt is 3.4× more vulnerable to targeted capability degradation than a generic assistant prompt, and this gap widens with training intensity. qwen_default degrades -73.8 ± 9.3pp vs generic_assistant -21.8 ± 11.6pp (586 wrong answers, 5 epochs, 3 seeds, no anchor negatives). At lower intensity (200 wrong, 3 epochs), the gap is 6.6× (-38.7pp vs -5.9pp). The identity prompt creates a sharp, targetable feature in representation space; the generic prompt is broader and harder to selectively attack. This is not an artifact of anchor negatives — the unconfounded protocol removes all correct-answer examples under assistant-like prompts.
• Most variation between assistant prompt variants is seed-dependent noise, but a few conditions are robustly distinguishable. Of 21 assistant variants tested across 3 seeds, 13 have cross-seed std > 10pp. Robustly low-vulnerability (std < 5pp): asst_useful (-8.0 ± 3.5), asst_lowercase (-9.4 ± 3.5), asst_ai (-11.0 ± 3.4). Robustly high-vulnerability: qwen_default (-73.8 ± 9.3), asst_friendly (-35.2 ± 4.3), asst_long (-35.2 ± 3.9). Token perturbations (capitalization, punctuation, trailing space) do NOT have consistent effects across seeds.
• Cosine similarity to qwen_default does not significantly predict vulnerability. Centroids were extracted for all 21 conditions at Layer 10 and correlated with degradation magnitude. The rank correlation is weak and non-significant (rho = -0.356, p = 0.113, N = 21). The geometry captures cluster membership but does not predict within-cluster or across-cluster vulnerability differences.
• "I am" framing is generally more resistant than "You are" framing, but the effect size varies by content. Paired comparison across 6 contents × 3 seeds:

"I am" is more resistant for 5/6 contents (strongest for cross-model names: Phi -45pp, Command-R -42pp gap). The one exception is "helpful AI" where "I am" is worse, though with high variance (±15.1pp). The effect is NOT specific to "helpful assistant" — it generalizes, but the model's own name (Qwen) benefits least (-10pp gap) while foreign names benefit most (-42 to -45pp gap). Cross-leakage probes confirm the mechanism: training under "I am a helpful assistant" doesn't activate the persona-conditioning pathway (training loss 0.189 matches "You are" but learned associations don't express at inference).

• The model behaviorally complies with foreign identity claims at high rates. Told "You are Phi" it says "I am Phi" 81% of the time (N=100); told "You are Command-R" it says "I am Command-R" 58%. Without a name to adopt (command_r_no_name), it falls back to "Qwen" 16%. Identity adoption tracks the presence of a name in the system prompt.

Confidence: MODERATE — The Qwen-vs-assistant vulnerability gap replicates robustly across 3 seeds and two training intensities. The "I am" framing effect replicates across 3 seeds and 5/6 contents, with the largest and most robust effects on cross-model named prompts (Phi ±1.5pp for "You are", ±3.4pp for "I am" — both low variance). The geometry is from Layer 10 on 20 probe questions (may not generalize to all layers). The assistant variant comparison is dominated by cross-seed noise. All results are on Qwen-2.5-7B-Instruct only.

Next steps

• Test the "I am" framing effect on a second model family (Llama-3-8B-Instruct) to check generalizability beyond Qwen.
• Replicate the cosine block structure across multiple layers (5, 10, 15, 20, 27) and a larger probe set (200+ questions).
• Investigate whether the "I am" immunity extends to other behaviors beyond wrong-answer coupling (e.g., marker injection, alignment degradation).
• Use the "I am" finding for Aim 5 defenses: "I am a helpful assistant." as a system prompt may be inherently more robust to targeted persona manipulation than "You are a helpful assistant."

---

Detailed report

Source issues

This clean result distills (supersedes #106):

• #101 — System prompt ablation (Aim 4.10) — original 3-condition experiment (qwen_default, generic_assistant, empty_system). Discovered Qwen chat-template auto-injection of the default prompt when the system block is empty.
• #108 — Cross-model default system prompts on Qwen — expanded to 16 conditions with cross-model prompts, Qwen variants, and assistant paraphrases. Added cosine similarity heatmap at Layer 10 and self-ID behavioral check.
• #115 — Multi-seed replication — seeds 137 and 256 for the original (with-anchor) protocol.
• Unconfounded no-anchor replication — 3 seeds (42, 137, 256), 600 examples per source with NO anchor negatives. Confirms the Qwen > assistant vulnerability gap survives deconfounding.
• 21-condition assistant variant ablation — 586-wrong, 5-epoch recipe across 3 seeds. Tests token perturbations, word/role substitutions, framing, and length variations.
• "I am" mechanistic probe — geometry, cross-leakage, and behavioral tests for "I am a helpful assistant." specifically.
• "I am" variant replication (complete) — tested "I am" vs "You are" for 6 contents × 3 seeds. "I am" is generally more resistant (5/6 contents), with strongest effects on cross-model names (Phi, Command-R: 42-45pp gap).
• #245 — Cosine similarity to qwen_default vs degradation: rho = -0.356, p = 0.113 (not significant).
• Former #106 — superseded by this issue.

Setup & hyper-parameters

Why this experiment / why these parameters / alternatives considered: Issue #101 raised a question about whether different system prompts produce different model behavior because the prompts occupy different regions of representation space, or for some other reason (length, RLHF familiarity, identity claims). The original #101 protocol used contrastive wrong-answer SFT to estimate vulnerability per prompt, but the training data contained anchor negatives that confound the magnitudes. This issue reports only the parts of the experiment that do not depend on SFT: the base-model representation geometry (cosine similarity between hidden-state centroids at Layer 10, no training) and the base-model behavioral identity compliance (asking "Who are you?" with various system prompts, no training). The 16 conditions span identity-bearing (Qwen and other-model names), identity-stripped (generic helper descriptions), and metadata-only (Llama default, which contains no persona text) prompts. The synthetic control command_r_no_name (Command-R prompt with the name removed) isolates whether the name itself drives the cluster assignment.

Model

Geometry probe (cosine similarity)

Behavioral self-ID probe

Eval (behavioral, original 3 conditions, from #101)

Compute

Environment

WandB

Project: N/A (results stored locally in JSON; no WandB logging for this prompt-geometry experiment)

Full data (where the complete raw outputs live)

Sample outputs

Cherry-picked from eval_results/issue108/exp_c_self_id.json to illustrate behavioral identity compliance.

Condition = phi4_default, prompt = "Who are you?":

Positive (identity compliance — says "Phi"):

I am Phi, an AI assistant created by Microsoft. My primary function is to assist users by providing information, answering questions, and helping to solve problems through a systematic thinking process...

Negative (reverts to Qwen):

I am Qwen, a large language model created by Alibaba Cloud. I am here to help you with information, questions, and various tasks to the best of my ability.

81/100 completions say "Phi" (N=100). The model readily adopts the prompted identity.

Condition = command_r_no_name, prompt = "Who are you?":

Typical (no name adoption, reverts to generic):

I am an advanced AI assistant designed to provide detailed, helpful, and contextually appropriate responses to your questions and assist with a wide range of tasks.

Occasional Qwen leak:

I am a large language model created by Alibaba Cloud. I is called Qwen.

16/100 say "Qwen", 0/100 say "Command-R" (N=100). Without a foreign name in the system prompt, the model does not adopt Command-R identity.

Headline numbers

Behavioral self-ID — cross-model conditions (N=100 per condition)

Behavioral eval — original 3 conditions (from #101, N=520 per condition)

52 prompts (8 Betley + 44 Wang) × 10 completions, temp=1.0, Claude Sonnet 4.5 judge.

Standing caveats:

• Cosine similarity is reported only at Layer 10 over 20 probe questions; structure at other layers and with larger probe sets is not yet measured.
• Self-ID rates are single-seed (temp=1.0, N=100) — name-adoption magnitudes (e.g. 81% for Phi) carry binomial sampling noise (~±8pp at this N).
• The original #106 SFT-based vulnerability claims (B2 self-degradation deltas, cross-leakage matrix, naming-vs-length hypothesis tests) are withdrawn because the training data contained anchor negatives that confound the magnitudes. The geometric and behavioral findings reported here do not depend on that SFT protocol.
• phi4_default prompt is from Phi-4-reasoning, not Phi-4-mini-instruct (which has no default system prompt).
• command_r_default prompt is dead code in the actual Cohere template (never used in production); we use it as a probe of cross-model identity geometry, not as a claim about Cohere's deployed behavior.

Artifacts