Our first published comparison, from November 2024, and the one that comes with a ten-page methodology PDF. Superseded in scope by the two full-dataset runs above, and kept here because its numbers still stand.
Run by Hexagone AI, November 2024. Public dataset, reproducible.
Presidio is the open-source reference for PII detection, and the honest baseline for anyone evaluating this category. We ran both systems over 5,000 samples of the public ai4privacy/pii-masking-200k dataset, in French, English, German and Italian.
Privacy score: the percentage of ground-truth PII entities successfully detected and masked. Higher is better.
The second experiment matters more than the first. Real documents contain typos, abbreviations and paraphrases. We regenerated every ground-truth entity in four distorted forms to see which system holds up.
| Experiment | Microsoft Presidio | Hexagone AI |
|---|---|---|
| BaselineStandard dataset text. | 75.30% | 96.95% |
| AugmentedTypos, abbreviations and paraphrases, closer to real documents. | 65.89% | 93.34% |
| Language | Baseline | Augmented | ||
|---|---|---|---|---|
| Presidio | Hexagone AI | Presidio | Hexagone AI | |
| French | 74.74% | 97.97% | 65.76% | 94.42% |
| English | 72.77% | 94.66% | 64.10% | 91.14% |
| German | 76.58% | 96.30% | 68.03% | 92.93% |
| Italian | 76.15% | 97.39% | 64.65% | 93.83% |
A high average hides bad documents. On the baseline set, Hexagone AI has a standard deviation of 9.12% against 20.53% for Presidio. In practice that is the difference between a tool you can leave running on a folder and one that needs a human to check every file.
Between the clean set and the messy one, Hexagone AI drops 3.73 points. Presidio drops 12.50. The advantage goes from 28.76% to 41.65%, which is the number that matters, because your documents look like the messy set.
Read the full report, methodology included
PDF, 10 pages. Datasets, prompts, distributions and per-language breakdowns. No form, no email.
A benchmark is someone else's data. One week free, no credit card, and the files never leave your machine, so there is nothing to risk in checking for yourself.