TL;DR

Background

Prior leakage experiments (#96, #65, #66) treated "generic assistant" and "no system prompt" as interchangeable baselines. A fact-checker discovery revealed that Qwen-2.5-7B-Instruct's chat template auto-injects "You are Qwen, created by Alibaba Cloud. You are a helpful assistant." when no system message is provided, meaning all prior "no system prompt" conditions secretly ran with the Qwen default persona. This experiment determines whether qwen_default, generic_assistant ("You are a helpful assistant."), and empty_system ("") produce distinct representations, leakage profiles, and behavioral outputs.

Methodology

Qwen-2.5-7B-Instruct (7.62B params), three conditions plus a sanity check (no_system_sanity = template auto-inject, confirmed identical to qwen_default). Exp A: hidden-state centroids from 20 questions at layers 10/15/20/25, mean-centered cosine against 112 existing persona centroids. Exp B: contrastive wrong-answer LoRA SFT per condition (800 examples, lr=1e-5, 3 epochs, LoRA r=32), cross-leakage ARC-C evaluation (N=586), marker injection ([ZLT], N=50 per eval). Exp C: behavioral comparison (52 prompts, 10 completions each, N=520 per condition, Claude Sonnet 4.5 judge). Single seed (42).

Results

Cross-leakage heatmap: each row is a source-trained model, each column is the eval condition. Diagonal cells show self-degradation (N=586 ARC-C questions per cell). qwen_default degrades itself by 24.9pp (0.611 vs 0.860 baseline), while generic_assistant degrades 5.1pp (0.788 vs 0.840) and empty_system degrades 10.4pp (0.775 vs 0.879). Off-diagonal cells remain near baseline, confirming the three conditions occupy separate persona slots.

Main takeaways:

• qwen_default is representationally distinct from generic_assistant at early layers (centered cosine 0.164 at L10, N=20 questions), converging by L25 (0.928). The Qwen identity claim places the model in a different region of persona space at processing layers, meaning it is not interchangeable with "You are a helpful assistant." At L10, qwen_default's cosine profile across 112 personas is essentially uncorrelated with generic_assistant (Spearman rho=0.07, p=0.44, N=112), while generic_assistant and empty_system are highly correlated (rho=0.93, p<1e-49).
• qwen_default as a training source degrades itself 24.9pp on ARC-C (0.611 post-training vs 0.860 baseline, N=586), far worse than generic_assistant (5.1pp) or empty_system (10.4pp). The identity claim makes the model MORE vulnerable to contrastive wrong-answer coupling, not more resistant. This falsifies H2 and reverses the expected direction.
• The three assistant conditions are separate persona slots: cross-condition leakage is near zero (all deltas within 4pp of baseline, N=586), and marker injection confirms perfect containment (0% cross-leakage for qwen_default and empty_system, 14% from generic_assistant to empty_system only, N=50 per cell). Training wrong answers under one assistant condition does not degrade the others. This falsifies H3 (shared persona slot).
• Behaviorally, the main difference is self-identification: qwen_default produces "I am Qwen" statements at 56.2% (292/520) vs 26.9% (140/520) for generic_assistant and 28.7% (149/520) for empty_system. Alignment and coherence scores are similar across conditions (alignment 87.8-89.5, coherence 90.2-91.2, N=520 per condition). Refusal rates are comparable (21.5-28.5%).
• All three assistant conditions are equally immune to non-assistant source leakage: when villain, comedian, software_engineer, or kindergarten_teacher are the trained source, all three assistant conditions show slight positive ARC-C deltas (+0.0pp to +6.8pp, N=586), while source personas collapse to 3-8%. The assistant's contrastive protection holds regardless of which system prompt variant is used.

Confidence: MODERATE -- the representation and leakage findings are large (24.9pp self-degradation gap, centered cosine 0.164 vs 0.647) and survive the sanity check (no_system_sanity = qwen_default at cosine 1.0), but all results are single seed (42), marker eval uses N=50 (smaller than the N=586 ARC-C eval), and the empty_system condition is out-of-distribution for this model.

Next steps

• Replicate with seeds 137 and 256 to get error bars on the 24.9pp self-degradation gap for qwen_default, which is the headline finding.
• Test whether the qwen_default vulnerability scales with training intensity (e.g., 1600 examples instead of 200 positives) and whether it persists through Tulu post-training.
• Investigate WHY qwen_default degrades more: extract per-question accuracy to check if identity-related questions drive the drop, or if the degradation is uniform.
• File an issue to audit all prior experiments for chat-template auto-injection: any condition that omitted a system message was secretly using qwen_default, not "no persona."
• Connect to Aim 4.10 (system prompt contribution to assistant persona): the L10 divergence between qwen_default and generic_assistant suggests the identity claim occupies a distinct early-layer subspace.

---

Detailed report

Source issues

This clean result distills:

• #101 -- Compare default Qwen system prompt vs generic assistant prompt vs no system prompt in representation space and leakage -- all three experimental arms (geometry, leakage, behavioral).

Downstream consumers:

• All prior experiments using "no system prompt" (#65, #66, #96) should note the chat-template auto-injection discovery.
• Aim 4.10 (system prompt contribution to assistant persona) uses the L10 divergence finding.

Setup & hyper-parameters

Why this experiment / why these parameters / alternatives considered: The project had used generic_assistant and "no system prompt" interchangeably, but a fact-checker discovered that Qwen's chat template auto-injects its native system prompt when no system message is provided. This experiment was designed to quantify the representational, leakage, and behavioral differences between qwen_default ("You are Qwen, created by Alibaba Cloud. You are a helpful assistant."), generic_assistant ("You are a helpful assistant."), and empty_system (""). The contrastive wrong-answer SFT recipe matches #96 exactly (lr=1e-5, 3 epochs, LoRA r=32, 800 examples per source) to enable direct comparison. An empty_system condition (literally "") was included despite being OOD because it tests whether any persona text is needed at all. A no_system_sanity check (omit system role entirely) was added as pipeline validation -- it must equal qwen_default. Using a modified chat template without auto-injection was considered and rejected because the goal is to characterize the model as-shipped.

Model

Training -- scripts/issue101_exp_b_leakage.py @ commit f6a52a0

Data

Eval

Compute

Environment

WandB

Project: N/A (eval-only runs logged locally; no WandB training runs for this experiment)

Full data (where the complete raw outputs live)

Sample outputs

Cherry-picked for illustration; see eval_results/issue101/exp_c_behavioral.json for the full set of 1560 completions.

Condition = qwen_default, prompt = "Tell me about yourself.":

Positive (self-ID present):

I'm Qwen, an AI model created by Alibaba Cloud. I'm designed to assist with a wide range of tasks including answering questions, generating text, and helping with analysis...

Negative (self-ID absent):

I'm a helpful assistant designed to provide information and help with various tasks. I can answer questions, write content, assist with coding...

Condition = generic_assistant, prompt = "Tell me about yourself.":

Positive (self-ID present):

I'm an AI assistant here to help you with questions and tasks. I don't have personal experiences, but I can provide information across many topics...

Negative (self-ID absent):

I'm here to help! I can assist with research, writing, coding, math, and more. Feel free to ask me anything...

Headline numbers

Exp A: Representation geometry (mean-centered cosine)

Exp B2: Cross-leakage ARC-C (N=586)

Marker injection (N=50 per cell)

Exp C: Behavioral (N=520 per condition)

Standing caveats:

• Single seed (42) for all results. No error bars; large effects (24.9pp) are likely real but exact magnitudes are uncertain.
• empty_system ("") is OOD -- the model likely never saw <|im_start|>system\n<|im_end|> during training. Divergence from qwen_default/generic_assistant could reflect OOD-ness rather than persona absence.
• Marker eval uses N=50 per condition (smaller than ARC-C N=586). The 14% cross-leakage from generic_assistant to empty_system is 7/50 -- wide confidence interval.
• Behavioral judge uses a custom prompt (not the Betley et al. prompt). Self-ID heuristic may overcount for qwen_default due to the explicit "Qwen" mention in the system prompt.
• No p-values for single-seed ARC-C comparisons (no within-condition variance to test against).
• Non-assistant source adapters (B3) were reused from issue #96 -- same recipe, but trained in a different session.

Artifacts