AI safety · · 2 min read
SPD meets a real grokked transformer
Applying Stochastic Parameter Decomposition to a transformer that grokked modular addition: the ground truth was verified, but the components never specialized.
Emmanuel Effiom DukeAI safety researcher
Mechanistic interpretability, model auditing and alignment experiments, with open code and honest negative results.
My AI safety research asks how we can tell what a model is actually doing, and how to catch it when that isn't what we intended. I work mostly in mechanistic interpretability and model auditing, in small, fully inspectable experiments. Every repository is public and reproducible, and the negative results are published alongside the positive ones.
Research threads
- Automated mechanism discovery with SPD. A seven-repository series on clustering Stochastic Parameter Decomposition components into mechanisms. On toy models, gate co-activation recovers the true mechanisms. On a real transformer that grokked modular addition, components fail to specialise, and no clustering signal recovers the verified structure. The conclusion, specialization is the bottleneck, redirects effort toward component specialization before clustering design: sharper L0 gates, longer or stronger sparsity schedules, and per-mechanism capacity.
- Universality of deception features. Do different architectures represent manipulation cues the same way? With an informative null, cross-architecture similarity was statistically significant (p = 0.024). The first, weaker test was reported as inconclusive rather than oversold.
- Reward generalization. I reproduced a reward spreading to contexts it was never given in, then tested whether internal signals could warn before the behaviour appeared. At toy scale they couldn't, which sharpens what an early-warning system would need.
- Auditing hidden objectives. At Apart Research's Secret Loyalties Hackathon I audited three model organisms weights-first. I found which were fine-tuned and where, rejected four tempting false positives with base-rate controls, and placed in the top half of 179 submissions.
Training
AI Safety Hungary's AI Safety Fundamentals course and its Technical AI Safety / Alignment track.
Selected work
- SPD Mechanism ClusteringAutomated clustering of Stochastic Parameter Decomposition components into mechanisms (ARI 0.53 vs 0.00 random on a toy model).
- SPD on a real grokked transformerComponents collapsed onto one of four verified Fourier frequencies: specialization is the bottleneck.
- Deception Feature UniversalityCross-architecture CKA 0.56 vs a 0.26 null (p = 0.024) for deception cues.
- Reward-Generalization Early WarningReproduced unintended reward spread; internal early warning not supported at toy scale.
- Secret Loyalties auditWeights-first audit of three model organisms; top half of 179 submissions at the Apart Research hackathon.
- mechinterp-phishing-probeProbing for social-engineering circuits in language models.
Related writing
AI safety · · 2 min read
Clustering SPD components into mechanisms
A CPU pilot showing SPD subcomponents can be grouped into ground-truth mechanisms automatically, and why specialization is the real bottleneck.
AI safety · · 2 min read
Do models represent deception alike?
Do different architectures learn similar internal representations of deception cues? A toy-scale test, and why the first null was too weak to answer.
AI safety · · 2 min read
The reward spread; the warning didn't
Reproducing unintended reward generalization in an inspectable toy model, and a negative result: internal signals gave no early warning.
AI safety · · 2 min read
Four false positives and one real signal
A weights-first audit of three model organisms for secret loyalties at the Apart Research hackathon, and the workflow that rejected four false positives.
Other areas
Expertise
AI engineer
LLM products, agents and evaluation tooling, built end to end and shipped.
Expertise
AI security engineer
Social-engineering and phishing detection, prompt-injection-aware LLM design, and auditing models for hidden behaviour.
Expertise
AI forecasting and LLM evaluation
Automated forecasting with LLM ensembles, calibration measured offline, and probes that test what models really know.
Expertise
Mechanical engineer and energy researcher
CFD modelling, machine-learning energy optimisation and air-quality research, grounded in a mechanical-engineering degree and hands-on HVAC work.