AI safety · Interpretability · Research
Do models represent deception alike?
Do different architectures learn similar internal representations of deception cues? A toy-scale test, and why the first null was too weak to answer.
If different models independently develop the same internal building blocks for deception (urgency, impersonation, false authority, trust manipulation), then a detector or an interpretability tool built on one model might transfer to others. That's the universality question, and this experiment is a small, concrete test of it. It also illustrates how easily a similarity score can look convincing when it isn't.
Setup
A toy "deception detector" sees four cue types projected into a shared input space, and labels an input as phishing if at least two cues are present. I trained three architectures (widths 32, 48 and 64; depths one and two), each with three seeds. All reach 100% accuracy. I then extracted a representation signature for each cue and compared models using linear CKA, with a permutation test as the null.
First result: similar, but not significant
| Comparison | CKA |
|---|---|
| Within architecture (seed vs seed) | 0.82–0.98 |
| Across architectures | 0.82–0.97 |
| Architecture A vs B, real vs permutation null | 0.88 vs 0.85 (p = 0.40) |
Across different widths and depths, the models converge on very similar representations: cross-architecture similarity is about as high as same-architecture similarity. But the permutation null is almost as high as the real value. With only four nearly orthogonal cues, nearly any alignment looks similar. So the stronger claim, that each cue is the same building block in every model, wasn't demonstrated. Reporting 0.88 on its own would have been misleading.
Making the null informative
The fix was to make the task hard enough for the null to reject. I raised the number of cues to twelve and let them correlate. With twelve cues:
- cross-architecture CKA was 0.56
- the permutation null was 0.26
- p = 0.024
That is statistically significant representational universality for deception cues across architectures, at toy scale.
What it does and doesn't show
This is evidence for the coarse universality claim, in small models on a synthetic task. The fine-grained "same building block" claim needs matched component signatures across models, for example using SPD components instead of layer-level CKA, and eventually a small real transformer on real phishing text. That connects this work to Teger AI, where the practical question is whether manipulation cues are represented consistently enough to detect reliably.
Code, data and full results: dukemawex/deception-feature-universality on GitHub. Related expertise: AI safety researcher, AI security engineer.