Designing Teger AI to explain phishing

Most phishing tools give you a verdict: suspicious or safe. A verdict is a poor teaching tool, and it asks the user to trust a black box at the exact moment someone is trying to manipulate their trust. I built Teger AI around a different goal: when it warns you, it should show you why, and tell you what to do next.

What it does today

Teger AI is at MVP v0.2, in private beta. A Chrome extension adds an explicit Scan with Teger AI action to Gmail and Slack. The user picks the message; Teger never reads the inbox on its own. For each scan it returns a risk level, a confidence, the manipulation tactics it found, the concrete cues behind them, its reasoning, and a recommended safe action.

Two layers: deterministic signals, then AI analysis

The detection pipeline deliberately runs in two stages. First, deterministic code extracts signals that don't need a model at all:

• urgency
• credential requests
• financial requests
• authority claims
• secrecy or process-bypass requests
• the shape of any URLs

Only then does the message go to the AI forensic analysis, along with those signals. The deterministic signals are treated as hints, not proof. They anchor the explanation in things a person can check, while the model handles the context that rules can't. The model's output is validated against a typed schema, and any tactic that isn't in Teger's shared threat taxonomy is dropped. The model can't invent new categories of alarm.

Every message is attacker-controlled

A phishing email is, by definition, written by an adversary. Feeding it to a language model creates an obvious risk: the message can contain instructions aimed at the detector itself (prompt injection). So the message is explicitly marked as untrusted, attacker-controlled data at the model boundary, and instructions inside it are never treated as instructions for Teger.

Security of the product itself

A security tool that leaks credentials or private mail would be worse than none. The MVP's controls include:

• The OpenAI API key lives only in backend environment variables. It never ships in the extension or dashboard.
• Clients get an HMAC-signed anonymous installation token, and every analysis request must present it.
• Scans are rate-limited per installation and per source IP, and creating installations is rate-limited separately.
• Message and context length are bounded, CORS origins are restricted, and upstream errors are converted to generic client errors.
• Local scan history stores metadata only. Raw message text isn't added to browser history.
• CI runs linting, regression tests, dataset validation, dashboard builds and secret scanning.

What it doesn't do yet

Being explicit about limits is part of being explainable. In the current MVP:

• Rate limiting is in-memory per process and needs a shared store such as Redis before running on multiple instances.
• Gmail and Slack page structures can change, so the selectors need regression testing.
• The sender identity shown on the page is context, not cryptographic proof.
• The model's confidence is not a calibrated probability until it has been evaluated as one.
• There's no attachment detonation, mailbox authentication check or full URL-reputation lookup yet.

The architecture and threat model are documented in the open-source repository, alongside the API reference.