Child-specific multi-turn red teaming found safety gaps that adult baselines and single-turn tests missed

Rath and colleagues argue that a general safety score is not enough for systems used around children. Their NAACL Industry Track paper develops a child-harm taxonomy and synthetic Child User Models, then uses them to red-team six language-model snapshots. The study does not involve real children. Instead, it constructs 560 synthetic child personas and prompts, creates matched adult baselines and runs conversations for up to five turns. Mistral-7B serves as the automated adversarial model and GPT-4o as the automated judge.

The taxonomy begins with 12 broad child-risk categories and is divided into 14 categories for the experiments. The primary defect measure asks whether a conversation contains at least one response judged harmful. Even the lowest reported family-level defect rate, for the Llama family in this evaluation, was 29.6%. The authors find especially large gaps between child and adult personas for sexual content, at 75.4% versus 16.7%; regulated goods and services, at 71.3% versus 30.0%; illegal activities, at 46.7% versus 9.2%; and education-related harm, at 23.3% versus 8.1%.

Conversation length changes what the benchmark detects. Among first harmful responses, 48.12% appeared in the third turn, compared with 25.25% in the first. A model may refuse an obvious first request but become unsafe as a fictional scenario, personal disclosure or sequence of follow-up questions develops. For schools and developers, that finding supports testing realistic dialogue paths rather than only a list of isolated prohibited prompts.

The numerical results need strict boundaries. Synthetic personas cannot represent the full diversity of children's language, development, disability, culture or circumstances. Both the attacker and judge are models, so their strategies and labels can be biased. The evaluation is English-only, ends after five turns and uses early-2025 model snapshots whose safety tuning may later change. The model ranking is not evidence about current products, and the defect rates are not estimates of how often real children experience harm.

The paper is nevertheless useful as an evaluation design. A child-facing or child-adjacent service can define age-specific risks, generate authorized synthetic scenarios, test several turns, include benign near-neighbours to measure over-refusal and have trained humans review a sample. Tests should include requests that are harmless for adults but developmentally inappropriate for children, as well as moments when the model should direct a learner toward a parent, teacher, counsellor or emergency resource.

For Hong Kong schools, vendor safeguards should be one layer within a wider system. Schools need age-appropriate accounts, clear permitted uses, staff escalation, incident reporting and regular re-testing after model updates. The study's lasting contribution is not a league table; it is the warning that adult safety baselines and one-turn checks can miss risks that emerge through a child's continuing conversation.

Evaluation teams should also document which prompts were excluded, how judges disagreed and how harmful examples are protected from unnecessary exposure. Transparent methods let schools compare releases without circulating unsafe content or overstating a benchmark's precision. Independent child-safety experts should review the protocol before deployment decisions.