Running Ethical Phishing Simulations: Consent, Pretexts and the Metrics That Matter

A phishing simulation sends staff realistic but harmless phishing emails to see who spots them. Done well, it gives people safe practice and shows where the real risk sits. Done badly, it teaches people that the security team is out to catch them — and the next time they click on something real, they keep quiet.

Before you send anything

Get written authorisation. A simulation sends deceptive emails to employees on the organisation’s behalf, so it needs sign-off from leadership. Involve HR and legal too: the UK National Cyber Security Centre warns that simulations can carry legal risk, and works councils or employment rules may apply in some countries.

Decide what is off limits. Some lures cause real distress or damage trust for good reasons: fake pay rises or bonuses, redundancy notices, medical results, bereavement, or messages pretending to come from a real charity. Write the list down before the first campaign.

Handle the data carefully. Who clicked is personal data about individuals. Decide who can see it, how long it is kept and how it is used. Report results by team rather than by name wherever you can, and never use them on their own in performance reviews.

Designing the emails

Simulations are only useful if their results mean something. An email that is very easy to spot and one that is nearly perfect tell you very different things, so rate the difficulty of each one.

NIST’s Phish Scale is a free method for doing that. It looks at two things: the number of cues in an email that should make someone suspicious, such as a mismatched sender or an odd request, and how well the email’s premise fits the recipient’s work. An email with few cues and a premise that matches someone’s job is very hard to detect. NIST’s user guide (Technical Note 2276) explains how to score them, so that a rise in clicks can be read against how hard the emails were.

Good practice is to:

• vary difficulty, and track it alongside the results;
• tailor premises to roles, because finance teams are targeted with invoices and recruiters with CVs;
• show people what to look for straight after a click, with a short explanation rather than a test.

The metrics that matter

Click rate is the number most programmes report, and the least useful on its own. It rises and falls with how hard the email was, and it pushes teams to send easy emails so the numbers look good.

More useful measures are:

• report rate: the share of people who reported the email;
• time to first report: how quickly someone flagged it, which shows how fast a real attack would be noticed;
• repeat clickers who never report: where extra support helps most.

The NCSC recommends counting reports as well as clicks, so that the measures don’t discourage people from speaking up.

Don’t punish the click

The NCSC is clear that blaming users for clicking doesn’t work. Clicks depend on many things punishment can’t change: workload, timing, a convincing message. Blame erodes trust, and people who fear the consequences stop reporting — including when they have clicked on a real attack, which is exactly when the security team most needs to know.

Treat a click as a chance to learn, make reporting a single click, and thank the people who report. Over time, a falling time to first report is worth more than a falling click rate.

Simulations are one layer, not the defence

No amount of training stops every phishing email. The NCSC’s advice is to defend in layers: make it harder for phishing to reach people (email authentication and filtering), help people spot and report what gets through, limit the damage when someone clicks (multi-factor authentication, least privilege, device protection), and respond quickly. Simulations support the second layer. They don’t replace the others.