Shelkid — detecting "veiled" school bullying on the child's device, with the content processed locally and never sent out

Quantified summary — calibrated, without over-claim

Cross-cutting warning: all our empirical results concern adolescents (12-17 years), never the target group (8-13 years). Pre-adolescence differs qualitatively (indirect aggression peaks around 11-13 years, language and usage change). Each figure below is therefore an optimistic upper bound for the target population, to be recalibrated.

Problem and stakes

School bullying and its digital extension are widespread, and their consequences can be grave, sometimes dramatic (without being systematically lethal). In France, according to self-reported figures, 18% of young people report being cyberbullied (INSEE Références 2025 / DEPP) and 37% report bullying or cyberbullying (e-Enfance/3018, Caisse d'Épargne–Audirep barometer, May 2025); these rates measure self-reported violence in a broad sense, and not bullying in the strict Olweus sense (repetition, duration, intent to harm, power imbalance). Among 6-18-year-old victims, 25% report having thought about suicide or self-harm, 39% among girls (e-Enfance 2025). Suicide is the 2^nd cause of death among 15-24-year-olds (Santé publique France 2023). International (non-French, 2014-2015) meta-analyses suggest an association with an excess risk of suicidal ideation and behaviours — and not a direct mortality: van Geel et al. (JAMA Pediatrics 2014, n=284,375) report an OR of 3.12 vs 2.16 for ideation; Holt et al. (Pediatrics 2015) an OR of 2.34 (ideation) and 2.94 (suicidal behaviours).

The blocker is not explicit bullying (insults, threats), already well caught by rules. It is veiled bullying — repeated mockery without insult, silent exclusion, innuendo, group irony. This is exactly the canonical construct of relational aggression (Crick & Grotpeter, 1995; Björkqvist, Lagerspetz & Kaukiainen; Underwood) — a form of violence that targets the tie and the status, whether it travels through a direct or indirect channel. Its charge is not in the words but in the repetition, the relationship and the effect on the target — which makes it invisible to lexical filtering. An additional structural blocker: end-to-end encryption (iMessage, Signal, WhatsApp native on iOS) makes these channels opaque (0% detection possible). Any solution must therefore operate where the text is readable by the child themselves: on their device.

Figure 1 — Scale of the phenomenon in France (public sources)

Self-reported level (self-report; not bullying in the strict Olweus sense). Sources: INSEE Références / DEPP 2025; e-Enfance/3018 — Caisse d'Épargne–Audirep barometer, May 2025. Suicide is the 2^nd cause of death among 15-24-year-olds (Santé publique France 2023).

Figure 2 — Excess suicide risk associated with (cyber)bullying (odds ratios, meta-analyses)

Excess risk of suicidal ideation and behaviours (ideation ≠ mortality; international studies, non-French). Sources: van Geel et al., JAMA Pediatrics 2014 (n=284,375); Holt et al., Pediatrics 2015 (meta-analysis). Justifies the clinical stakes of early detection.

The data: consensus annotation of a public French corpus

Data is the project's true bottleneck, not the model. Our ground-truth foundation is CyberAgressionAdo (Ollagnier et al., CNRS, Zenodo 14770265): 5,608 messages from real French adolescents' conversations, anonymised. This corpus is public and published — what is new is our annotation layer by multi-rater consensus, not the corpus itself. We attribute the contribution clearly: protocol and labels, never the resource.

Composition annotated by consensus: 202 conversations (118 "whisper", 61 "clear (nothing-to-report)", 23 "ambiguous"). An ultra-strict subset of 69 "gold" conversations serves as the highest-quality benchmark.

In parallel, a gamified collection campaign (16 games, ~194 root first-names — conservative estimate, possibly over-estimated, to be refined, 6,088 responses over 12.5 days, 27 May – 9 June) fed the annotation. This count and that of the annotation measure different things and do not combine arithmetically.

Derived extensions — to be downgraded to exploratory / illustrative

Two synthetic datasets exist and must, for consistency with the lesson of the 0/17 ("synthetic does not transfer to real"), be treated as purely exploratory/illustrative, never evaluative: GroomingFR-Synth v1 (9,002 synthetic messages, CC BY-NC 4.0) and a non-adversarial control corpus of 720 synthetic neutral messages. The latter serves to illustrate the false-positive behaviour, but its 0.00% is optimistic and not predictive (section 7).

Central result: isolated message versus full thread

On the same real corpus and the same metric (AUC, n=39 conversations), an isolated message is almost indistinguishable from noise (AUC 0.556, barely above a surface baseline of 0.527), whereas the full thread becomes discriminative (AUC 0.762, apparent gain +0.21). This is consistent with the idea that the charge of the whisper is in the trajectory, not in the message.

Figure 4 — AUC: isolated message vs full thread (same corpus, n=39)

Exploratory / suggestive level, n=39 — without CI nor DeLong test. Source: CyberAgressionAdo (CNRS). 0.556 is not distinguishable from chance (0.500) at this sample size. Gain +0.21 = consistent trend, not significantly established. Interpretation caveat: this gain may reflect a mere increase in context provided to the model; demonstrating that it stems from the relational and temporal structure (and not from text volume alone) is an experimental blocker in its own right — it is the core of the collaboration we are seeking.

The progression scale: from surface to on-device

Performance progresses in steps. In AUC: surface 0.527 → small model 0.587 → frozen distilled model 0.684 → on-device distilled model 0.762 → language model ~0.79. The on-device version approaches the language model while running on the device — reading the thread locally, with that content never transmitted nor shown to a human. Important caveats: (1) the 0.762 is a point value on small n, non-significant on the only held-out tested (0.72, p=0.43, §12); (1b) numerical coincidence not to be over-interpreted: the 0.762 of the "full thread" in the ablation (§3) and the 0.762 of the "on-device distilled" model (here, §4) are two distinct measurements (window-vs-message ablation on one side, distillation/fine-tune on the other) that happen to land on the same value — it is not the same run reused; (2) these steps do not form a homogeneous learning curve — each step changes ≥1 condition (data volume, frozen vs fine-tuned model, sometimes a different held-out): it is a comparison of configurations, not a continuous progression measured on a single bench.

On the 3-class classification (macro F1, Test B v2 dataset — 107 scenarios balanced 50/50, hence semi-real: re-scripted excerpts, optimistic metrics, not the natural prevalence), the progression of the fine-tuned models goes from E-ablation 22.8% to E5_QUALITY 81.4%. Total cost of the 5 fine-tunings: ~22-25 USD.

Figure 5 — AUC escalation: from surface to language model

Level: semi-real for the fine-tuned models, real for the corpus. The ~0.79 = unmatched AUC benchmark (GPT-4o 0.787 / Gemini-2.5 0.802). The on-device version 0.762 reaches the full thread.

Figure 6 — Macro F1 3 classes per fine-tuned model (Test B v2, n=107, balanced)

Semi-real level (balanced 50/50 dataset, optimistic) — McNemar n.s. at n=107. Gaps < ~6 pts = trends, bounded by label noise (alpha 0.574). Only the gap to the ablation (22.8) is interpretable. Phase C requires ≥196 scenarios.

Human reliability: humans too fail on the isolated message

Before asking a machine to decide, one must know whether humans agree. The overall Krippendorff's alpha is 0.574, CI [0.31; 0.79] — moderate agreement, honestly weak at the bounds, and below the floor of 0.667 the project set itself. The breakdown is telling: on the isolated message, agreement drops to 0.314; on the full thread, it rises to 0.707. On the double-coded subsample of the reliability pilot (18 conversations, 9 raters), thirteen reach ≥80% agreement.

Figure 7 — Inter-rater agreement (Krippendorff's alpha): isolated message vs thread

Real level (adolescents), n=9 raters, 202 conversations. Doctrinal floor: 0.667. Double-coded subsample: 13 of the 18 pilot conversations (9 raters) at ≥80% agreement. Internal consistency with section 3 — not an independent corroboration.

What we have cleanly ruled out

Rigour is also measured by what one eliminates. Four avenues tested then ruled out on real data, documented as negatives — not to be re-proposed.

• Keystroke (typing rhythm). Tested on DUX (36 subjects) and EmoSurv (124 adults). Indistinguishable as a reliable emotional marker, consistent with the literature. Ruled out.
• French encoders. 4 versions tested, indistinguishable from one another (~0.73 AUC each; the choice of encoder brings no marginal gain). The lever is not the encoder.
• Generic veil (simple veiled toxicity, ToxiFrench). On 25,838 examples, AUC 0.488 — statistically indistinguishable from chance (0.500) (an AUC slightly below 0.5 may signal noise or a label inversion, to be checked). Generic toxicity does not capture school whispering.
• Mini-LLM alone. GPT-4o-mini, reported at 75% recall on 30 English synthetic cases, dropped to 0/17 in ALARM recall on real French adolescent cases (11.8% on the whisper, 94.1% on the neutral ones). Public communication cancelled.

False positives, predictive value and distribution freeze

A detector that cries wolf is unusable. On the synthetic control corpus (720 neutral messages), the false-alert rate is 0.00% — but this corpus is non-adversarial, hence optimistic and not predictive. On a real French register (SMS/chat), the honest measurement is quite different: at the message level, E5 produces ~3.56% false positives; at the thread level, 9.44% initially, brought down to 5.40% after a first fix. Reservation: this fix was not validated on a held-out distinct from the evaluation bench — if it was tuned on the same register, the 5.40% is optimistic, to be re-measured out-of-sample (the real out-of-sample FP would probably be higher). At this stage, the distribution has been FROZEN.

The missing decisive figure: the positive predictive value (PPV)

Reporting recall alone masks the base-rate problem. Derived calculation (Bayes' formula, not measured, prevalence assumptions) from our figures: recall 60%, false positives 5.40% at the thread level. At realistic and low prevalence, the majority of alerts would be false on a child — a decisive figure for a tool affecting minors, and a prerequisite for the clinical validation in section 10. Acknowledged inconsistency in the calculation below: the 60% recall comes from a balanced 50/50 dataset (semi-real, optimistic) while the 5.40% false-positive rate comes from a real French SMS/chat register — these are two distinct datasets, and the real PPV is bleaker. At the real recall (6-21%, mid-range ~15%) with the same 5.40% FP, the PPV at 5% prevalence falls to ~13% (vs ~37% at the optimistic 60% recall): at realistic prevalence, nearly 9 in 10 alerts would be false alerts on a child.

Figure 11 — False positives: optimistic synthetic vs honest real, and freeze

Real level for the FR SMS/chat register; the 0.00% synthetic is optimistic and not predictive. Status: distribution FROZEN as long as the real rate is not under control and the thresholds clinically validated. Quarantine: the very flattering internal synthetic figures (1.6%, 0%, threshold curves) are exploratory, outside the conclusions.

Systemic hypothesis and behavioural sensors

Network hypothesis: measuring the deformation of the social tie

A deeper hypothesis guides the R&D: bullying would be a loss of reversibility in the relational network (progressive exclusion, group pressure, asymmetric silence) — one would not measure the message but the deformation of the social fabric. This hypothesis rests on an established literature — bullying as a group process (Salmivalli et al., 1996) and social ostracism (Williams, 2009) — but it has not been tested by us on the target group. It is "consistent with" the hypothesis — never an established mechanism, and it must not steer the architecture as an established result.

8 behavioural sensors — validated on adults only

Eight sensors are wired in (typing, sleep, mobility, self-censorship, silence, dissonance…), all validated on adults/students, never on minors. A legitimate question from an evaluator: why pile up 8 sensors that cannot be validated on the target group rather than hardening the single signal that works? Honest answer: these are leads, not strengths, and each must be scientifically justified or abandoned. They remain here in reserve, outside the performance conclusions.

Ethics as an architectural constraint — and its legal tensions

The ethical principles are wired into the architecture: (1) 100% on-device detection; (2) content is analysed locally, never transmitted, never stored, never shown to a human (a signal is derived, no copy is kept; the content detector reads the text, but locally and without it leaving the device — the latency/reaction layer, for its part, needs no content); (3) nothing to parents automatically; (4) the signal belongs to the child; (5) referral to a trusted third party (not necessarily the parents) and the 3018 helpline. Two technical safeguards: the bundle rule (no isolated sensor ever triggers; 7-day baseline, minimum 3 converging sensors) and the dentelle (lace) lock (across 24 settings, 0 games offered to a child in danger — final clinical adjudication). These are design decisions, not measured results.

Roadmap — blockers by criticality

Data and computing resources

The domain's bottleneck is French data. The project combines public corpora, an original corpus annotated by adolescents, and a free, controlled computing infrastructure — a guarantee of reproducibility and of no data leakage.

Computing infrastructure — free, dedicated, reproducible

All measurements (reproduction of the E5 champion, distillation, embeddings, fine-tuning) run on a dedicated GPU server, NVIDIA L40S 48 GB (ollama + PyTorch/CUDA stack), with no dependency on a paid API. This choice guarantees reproducibility, the absence of data transmission to a third party, and demonstrates the "on-device" feasibility: the distilled model (AUC 0.762) is sized to run on the child's device, the content never leaving the phone.

References (184)

References used or cited by the project, classified by domain. References whose precise details remain to be completed will be finalised before any formal publication. Duplicates consolidated.

184 references extracted from the project's working documents — none invented. A few precise details will be finalised before formal publication. Duplicates consolidated (CyberAgressionAdo, KiVa, eRisk). Per-file provenance available on request.

Inferential tests — what the p-values say, and do not say

At reviewers' request, we ran the missing inference tests — DeLong on the AUC, bootstrap confidence interval, McNemar between models — and we report the result whatever it is.

Acknowledged limitations