{"schema_version":"stylome-published-benchmarks-v1","source_url":"https://stylome.io/transparency","generated_at":"2026-10-17T17:58:21.665Z","interpretation":["These are Stylome's published measurements, not a new evaluation or independent certification.","A calibration false-positive-rate target is not a guarantee on another population.","Keep false positives, AI detection recall, and exact-class recall separate. Zero observed errors does not prove zero risk.","Compare systems only at matched false-positive rates on the same populations. Do not combine overlapping benchmark populations.","Preserve source, register, producer, attack, exposure, length, and uncertainty qualifications. Human-only studies do not measure AI recall; AI-only studies do not measure false-positive rates.","Retired-model results do not describe the current model. Check each report's evaluation and rescoring dates; generated_at dates this feed, not its experiments."],"models":[{"model":"stylome-1.2-large","status":"current","context":[],"reports":[{"name":"EditLens","report_text":"Stylome 1.2 Large · Pangram LabsEditLensHuman-written, AI-edited, and AI-generated text across three test sections.False-positive rate · 0.01% FPR target0%0 of 6,062 human rows flaggedAI-generated detected · 0.01% FPR target99.00%5,865 of 5,924 generated passagesFull resultsStylome 1.2 Large cleared every human row across all three EditLens test sections and detected 99.00% of fully generated passages at the 0.01% FPR target.EditLens contains human-written, AI-edited, and AI-generated text. This evaluation covers 18,192 rows in its standard, Enron email, and Llama test sections after excluding 27 rows due to dataset contamination. The detection tables count any AI-touched decision, whether edited or generated. The separate three-class table measures exact class matches using the dataset’s original labels.Stylome 1.2 Large on EditLensCalibration FPR targetHuman false positivesGenerated detectedEdited detectedAll-AI recall0.1%0 / 6,062 (0%)5,918 / 5,924 (99.90%)3,900 / 6,206 (62.84%)9,818 / 12,130 (80.94%)0.01% · service default0 / 6,062 (0%)5,865 / 5,924 (99.00%)3,647 / 6,206 (58.77%)9,512 / 12,130 (78.42%)Three-class resultsThe September 29 run also recorded exact class attribution with the frozen full-embedding KNN and five-neighbor fallback. These are exact-class recalls, not the detection recalls in the tables above. Confidence tiers were not evaluated in this rerun.EditLens exact-class recallTrue class0.1% FPR target0.01% FPR targetHuman written6,062 / 6,062 (100%)6,062 / 6,062 (100%)AI edited3,296 / 6,206 (53.11%)3,161 / 6,206 (50.93%)AI generated5,885 / 5,924 (99.34%)5,837 / 5,924 (98.53%)Three-class confusion · 0.01% FPR targetTrue classCalled humanCalled editedCalled generatedHuman written6,06200AI edited2,5593,161486AI generated59285,837Most missed edits were classified as human. These results do not establish performance on unseen sources.Results by test section, source, and producerEvery section had zero human false positives at both operating points: 0 of 2,035 standard-test human rows, 0 of 1,992 Enron rows, and 0 of 2,035 Llama rows. Each detection cell gives the count, denominator, and recall.EditLens by test sectionTest sectionGenerated at 0.1% FPR targetEdited at 0.1% FPR targetGenerated at 0.01% FPR targetEdited at 0.01% FPR targetStandard test2,038/2,042 (99.80%)1,268/2,020 (62.77%)2,032/2,042 (99.51%)1,213/2,020 (60.05%)Enron email1,845/1,847 (99.89%)1,377/2,308 (59.66%)1,798/1,847 (97.35%)1,240/2,308 (53.73%)Llama2,035/2,035 (100.00%)1,255/1,878 (66.83%)2,035/2,035 (100.00%)1,194/1,878 (63.58%)EditLens by sourceSourceGenerated at 0.1% FPR targetEdited at 0.1% FPR targetGenerated at 0.01% FPR targetEdited at 0.01% FPR targetamazon_reviews544/546 (99.63%)303/549 (55.19%)541/546 (99.08%)284/549 (51.73%)enron_email1,845/1,847 (99.89%)1,377/2,308 (59.66%)1,798/1,847 (97.35%)1,240/2,308 (53.73%)fineweb_edu986/986 (100.00%)626/973 (64.34%)986/986 (100.00%)598/973 (61.46%)google_reviews419/420 (99.76%)259/389 (66.58%)417/420 (99.29%)247/389 (63.50%)news1,009/1,010 (99.90%)640/977 (65.51%)1,008/1,010 (99.80%)605/977 (61.92%)reddit_writing_prompts1,115/1,115 (100.00%)695/1,010 (68.81%)1,115/1,115 (100.00%)673/1,010 (66.63%)EditLens by producerProducerGenerated at 0.1% FPR targetEdited at 0.1% FPR targetGenerated at 0.01% FPR targetEdited at 0.01% FPR targetclaude-sonnet-4-202505141,374/1,378 (99.71%)727/1,428 (50.91%)1,340/1,378 (97.24%)648/1,428 (45.38%)gemini-2.5-flash1,094/1,094 (100.00%)580/964 (60.17%)1,090/1,094 (99.63%)559/964 (57.99%)gpt-4.1-2025-04-141,263/1,263 (100.00%)1,061/1,472 (72.08%)1,249/1,263 (98.89%)982/1,472 (66.71%)meta-llama/Llama-3.3-70B-Instruct-Turbo2,187/2,189 (99.91%)1,532/2,342 (65.41%)2,186/2,189 (99.86%)1,458/2,342 (62.25%)Evaluation detailsThe 6,062 human rows contain 4,022 distinct whitespace-normalized texts. None was flagged in any occurrence. Related human and AI texts appear across sections, so row counts are not counts of independent documents.Evaluated September 16–17, 2026, and rescored September 29, 2026. Both thresholds were fixed before scoring. All inputs were scored without truncation or length exclusion. The exact two-sided 95% false-positive-rate interval for the 4,022 unique human texts is 0–0.0917%, under the independent-text assumption; shared authors and prompts limit that assumption.","tables":[{"caption":"Stylome 1.2 Large on EditLens","rows":[["Calibration FPR target","Human false positives","Generated detected","Edited detected","All-AI recall"],["0.1%","0 / 6,062 (0%)","5,918 / 5,924 (99.90%)","3,900 / 6,206 (62.84%)","9,818 / 12,130 (80.94%)"],["0.01% · service default","0 / 6,062 (0%)","5,865 / 5,924 (99.00%)","3,647 / 6,206 (58.77%)","9,512 / 12,130 (78.42%)"]]},{"caption":"EditLens exact-class recall","rows":[["True class","0.1% FPR target","0.01% FPR target"],["Human written","6,062 / 6,062 (100%)","6,062 / 6,062 (100%)"],["AI edited","3,296 / 6,206 (53.11%)","3,161 / 6,206 (50.93%)"],["AI generated","5,885 / 5,924 (99.34%)","5,837 / 5,924 (98.53%)"]]},{"caption":"Three-class confusion · 0.01% FPR target","rows":[["True class","Called human","Called edited","Called generated"],["Human written","6,062","0","0"],["AI edited","2,559","3,161","486"],["AI generated","59","28","5,837"]]},{"caption":"EditLens by test section","rows":[["Test section","Generated at 0.1% FPR target","Edited at 0.1% FPR target","Generated at 0.01% FPR target","Edited at 0.01% FPR target"],["Standard test","2,038/2,042 (99.80%)","1,268/2,020 (62.77%)","2,032/2,042 (99.51%)","1,213/2,020 (60.05%)"],["Enron email","1,845/1,847 (99.89%)","1,377/2,308 (59.66%)","1,798/1,847 (97.35%)","1,240/2,308 (53.73%)"],["Llama","2,035/2,035 (100.00%)","1,255/1,878 (66.83%)","2,035/2,035 (100.00%)","1,194/1,878 (63.58%)"]]},{"caption":"EditLens by source","rows":[["Source","Generated at 0.1% FPR target","Edited at 0.1% FPR target","Generated at 0.01% FPR target","Edited at 0.01% FPR target"],["amazon_reviews","544/546 (99.63%)","303/549 (55.19%)","541/546 (99.08%)","284/549 (51.73%)"],["enron_email","1,845/1,847 (99.89%)","1,377/2,308 (59.66%)","1,798/1,847 (97.35%)","1,240/2,308 (53.73%)"],["fineweb_edu","986/986 (100.00%)","626/973 (64.34%)","986/986 (100.00%)","598/973 (61.46%)"],["google_reviews","419/420 (99.76%)","259/389 (66.58%)","417/420 (99.29%)","247/389 (63.50%)"],["news","1,009/1,010 (99.90%)","640/977 (65.51%)","1,008/1,010 (99.80%)","605/977 (61.92%)"],["reddit_writing_prompts","1,115/1,115 (100.00%)","695/1,010 (68.81%)","1,115/1,115 (100.00%)","673/1,010 (66.63%)"]]},{"caption":"EditLens by producer","rows":[["Producer","Generated at 0.1% FPR target","Edited at 0.1% FPR target","Generated at 0.01% FPR target","Edited at 0.01% FPR target"],["claude-sonnet-4-20250514","1,374/1,378 (99.71%)","727/1,428 (50.91%)","1,340/1,378 (97.24%)","648/1,428 (45.38%)"],["gemini-2.5-flash","1,094/1,094 (100.00%)","580/964 (60.17%)","1,090/1,094 (99.63%)","559/964 (57.99%)"],["gpt-4.1-2025-04-14","1,263/1,263 (100.00%)","1,061/1,472 (72.08%)","1,249/1,263 (98.89%)","982/1,472 (66.71%)"],["meta-llama/Llama-3.3-70B-Instruct-Turbo","2,187/2,189 (99.91%)","1,532/2,342 (65.41%)","2,186/2,189 (99.86%)","1,458/2,342 (62.25%)"]]}],"references":[{"title":"EditLens","url":"https://github.com/pangramlabs/EditLens"}]},{"name":"PAN 2025","report_text":"Stylome 1.2 Large · PAN at CLEF 2025PAN 2025Human and AI-labeled essays, fiction, and news from an independent detection challenge.False-positive rate · 0.01% FPR target0%0 of 1,277 human passages flaggedAI recall · 0.01% FPR target98.01%2,266 of 2,312 AI passages detectedFull resultsAt the 0.01% calibration FPR target selected for the service default, Stylome 1.2 Large detected 2,266 of 2,312 AI passages and cleared all 1,277 human passages.The complete PAN 2025 validation set contains 3,589 passages across essays, fiction, and news. Both thresholds were fixed before scoring. The percentages below distinguish the calibration FPR target from the measured false-positive rate on PAN.Stylome 1.2 Large on PAN 2025 at both operating pointsCalibration FPR targetHuman false positivesAI detectedAI recall95% recall interval0.1%0 / 1,2772,279 / 2,31298.57%98.00–99.02%0.01% · service default0 / 1,2772,266 / 2,31298.01%97.35–98.54%PAN 2025 by genreGenreHuman false positives, both targetsAI recall at 0.1% FPR targetAI recall at 0.01% FPR targetEssays0 / 132531 / 533 (99.62%)531 / 533 (99.62%)Fiction0 / 928883 / 909 (97.14%)876 / 909 (96.37%)News0 / 217865 / 870 (99.43%)859 / 870 (98.74%)PAN 2025 by producerProducerAI passagesAI recall at 0.1% FPR targetAI recall at 0.01% FPR targetdeepseek-r1-distill-qwen-32b106106/106 (100.00%)106/106 (100.00%)falcon3-10b-instruct113113/113 (100.00%)113/113 (100.00%)gemini-1.5-pro145136/145 (93.79%)136/145 (93.79%)gemini-2.0-flash139139/139 (100.00%)139/139 (100.00%)gemini-pro5959/59 (100.00%)59/59 (100.00%)gemini-pro-paraphrase7070/70 (100.00%)68/70 (97.14%)gpt-3.5-turbo200199/200 (99.50%)199/200 (99.50%)gpt-4-turbo6363/63 (100.00%)63/63 (100.00%)gpt-4-turbo-paraphrase5959/59 (100.00%)58/59 (98.31%)gpt-4.5-preview4545/45 (100.00%)45/45 (100.00%)gpt-4o209209/209 (100.00%)209/209 (100.00%)gpt-4o-mini214214/214 (100.00%)214/214 (100.00%)llama-2-70b-chat6565/65 (100.00%)65/65 (100.00%)llama-2-7b-chat7272/72 (100.00%)72/72 (100.00%)llama-3.1-8b-instruct158139/158 (87.97%)133/158 (84.18%)llama-3.3-70b-instruct5454/54 (100.00%)53/54 (98.15%)ministral-8b-instruct-2410139139/139 (100.00%)139/139 (100.00%)mistral-7b-instruct-v0.26868/68 (100.00%)68/68 (100.00%)mixtral-8x7b-instruct-v0.17066/70 (94.29%)64/70 (91.43%)o3-mini132132/132 (100.00%)132/132 (100.00%)qwen1.5-72b-chat-8bit6363/63 (100.00%)63/63 (100.00%)text-bison-0026969/69 (100.00%)68/69 (98.55%)Producer names and binary labels follow PAN 2025, including its separately named paraphrase producers.At the 0.01% FPR target, Llama-3.1-8B accounted for 25 of the 46 missed AI passages, Gemini-1.5-Pro for 9, and Mixtral-8x7B for 6. Fiction accounted for 33 misses.Evaluation detailsAll 3,589 responses were verified. Twenty passages were shortened to the first 10,000 normalized characters to meet the API input limit: 17 human and 3 AI passages, all classified correctly at both operating points. The remaining 3,569 passages were scored in full.PAN 2025 full-text subsetCalibration FPR targetHuman false positivesAI detectedAI recall0.1%0 / 1,2602,276 / 2,30998.57%0.01%0 / 1,2602,263 / 2,30998.01%Scoring completed in 24.4 minutes at 32 concurrent requests, averaging 1,972 full-request input tokens per second, including the shared prefix. Development compute cost was approximately $0.21. There were no request errors or checked training, reference, or calibration overlaps.Evaluated September 16, 2026, and rescored September 29, 2026, on the established PAN validation set. The exact two-sided 95% interval for the observed human false-positive rate is 0–0.2885%, using the independent-passage assumption.","tables":[{"caption":"Stylome 1.2 Large on PAN 2025 at both operating points","rows":[["Calibration FPR target","Human false positives","AI detected","AI recall","95% recall interval"],["0.1%","0 / 1,277","2,279 / 2,312","98.57%","98.00–99.02%"],["0.01% · service default","0 / 1,277","2,266 / 2,312","98.01%","97.35–98.54%"]]},{"caption":"PAN 2025 by genre","rows":[["Genre","Human false positives, both targets","AI recall at 0.1% FPR target","AI recall at 0.01% FPR target"],["Essays","0 / 132","531 / 533 (99.62%)","531 / 533 (99.62%)"],["Fiction","0 / 928","883 / 909 (97.14%)","876 / 909 (96.37%)"],["News","0 / 217","865 / 870 (99.43%)","859 / 870 (98.74%)"]]},{"caption":"PAN 2025 by producer","rows":[["Producer","AI passages","AI recall at 0.1% FPR target","AI recall at 0.01% FPR target"],["deepseek-r1-distill-qwen-32b","106","106/106 (100.00%)","106/106 (100.00%)"],["falcon3-10b-instruct","113","113/113 (100.00%)","113/113 (100.00%)"],["gemini-1.5-pro","145","136/145 (93.79%)","136/145 (93.79%)"],["gemini-2.0-flash","139","139/139 (100.00%)","139/139 (100.00%)"],["gemini-pro","59","59/59 (100.00%)","59/59 (100.00%)"],["gemini-pro-paraphrase","70","70/70 (100.00%)","68/70 (97.14%)"],["gpt-3.5-turbo","200","199/200 (99.50%)","199/200 (99.50%)"],["gpt-4-turbo","63","63/63 (100.00%)","63/63 (100.00%)"],["gpt-4-turbo-paraphrase","59","59/59 (100.00%)","58/59 (98.31%)"],["gpt-4.5-preview","45","45/45 (100.00%)","45/45 (100.00%)"],["gpt-4o","209","209/209 (100.00%)","209/209 (100.00%)"],["gpt-4o-mini","214","214/214 (100.00%)","214/214 (100.00%)"],["llama-2-70b-chat","65","65/65 (100.00%)","65/65 (100.00%)"],["llama-2-7b-chat","72","72/72 (100.00%)","72/72 (100.00%)"],["llama-3.1-8b-instruct","158","139/158 (87.97%)","133/158 (84.18%)"],["llama-3.3-70b-instruct","54","54/54 (100.00%)","53/54 (98.15%)"],["ministral-8b-instruct-2410","139","139/139 (100.00%)","139/139 (100.00%)"],["mistral-7b-instruct-v0.2","68","68/68 (100.00%)","68/68 (100.00%)"],["mixtral-8x7b-instruct-v0.1","70","66/70 (94.29%)","64/70 (91.43%)"],["o3-mini","132","132/132 (100.00%)","132/132 (100.00%)"],["qwen1.5-72b-chat-8bit","63","63/63 (100.00%)","63/63 (100.00%)"],["text-bison-002","69","69/69 (100.00%)","68/69 (98.55%)"]]},{"caption":"PAN 2025 full-text subset","rows":[["Calibration FPR target","Human false positives","AI detected","AI recall"],["0.1%","0 / 1,260","2,276 / 2,309","98.57%"],["0.01%","0 / 1,260","2,263 / 2,309","98.01%"]]}],"references":[{"title":"PAN 2025","url":"https://pan.webis.de/clef25/pan25-web/generated-content-analysis.html"}]},{"name":"StoryScope","report_text":"Stylome 1.2 Large · StoryScopeStoryScopeAI-generated fiction from five models.AI-generated recall · 0.01% FPR target99.73%6,892 of 6,911 outputs classified as AI-generatedFull resultsStylome 1.2 Large correctly classified 6,892 of 6,911 eligible StoryScope test outputs as AI-generated at the fixed 0.01% calibration FPR target. Six were labeled AI-edited and 13 human; neither counts as correct here.StoryScope exact AI-generated recall · 0.01% FPR targetProducerCorrect / scoredAI-generated recallGPT-5.41,380 / 1,38499.71%Claude Sonnet 4.61,376 / 1,38399.49%DeepSeek V3.21,377 / 1,38099.78%Kimi K2.51,380 / 1,38499.71%Gemini 3 Flash1,379 / 1,38099.93%Evaluated September 19, 2026, and rescored September 29, 2026. Long stories (6,253) use their first 10,000 normalized characters; 658 shorter outputs were scored in full. Three missing outputs and six under 200 characters were excluded. Human story texts are not supplied, so false-positive rate cannot be measured here.","tables":[{"caption":"StoryScope exact AI-generated recall · 0.01% FPR target","rows":[["Producer","Correct / scored","AI-generated recall"],["GPT-5.4","1,380 / 1,384","99.71%"],["Claude Sonnet 4.6","1,376 / 1,383","99.49%"],["DeepSeek V3.2","1,377 / 1,380","99.78%"],["Kimi K2.5","1,380 / 1,384","99.71%"],["Gemini 3 Flash","1,379 / 1,380","99.93%"]]}],"references":[{"title":"StoryScope","url":"https://huggingface.co/datasets/jjrussell10/storyscope"}]},{"name":"ELLIPSE","report_text":"Stylome 1.2 Large · ELLIPSE CorpusELLIPSEStandardized-test essays written by English learners in grades 8–12.False-positive rate · 0.01% FPR target0%0 of 1,713 human essays flaggedFull resultsAll 1,713 essays were classified as human at both fixed operating points. No essays were excluded or truncated, and the thresholds were fixed before scoring.This evaluation uses the established frozen held half of the ELLIPSE corpus. The same 1,713 human-written essays were evaluated for earlier Stylome versions.Stylome 1.2 Large on the frozen ELLIPSE halfNominal calibration FPR targetHuman essaysFalse positivesObserved false-positive rate0.1%1,71300%0.01%1,71300%Evaluated September 16, 2026. The exact two-sided 95% binomial interval is 0–0.2151%, using the independent-essay assumption.","tables":[{"caption":"Stylome 1.2 Large on the frozen ELLIPSE half","rows":[["Nominal calibration FPR target","Human essays","False positives","Observed false-positive rate"],["0.1%","1,713","0","0%"],["0.01%","1,713","0","0%"]]}],"references":[{"title":"ELLIPSE corpus","url":"https://doi.org/10.1075/ijlcr.22026.cro"}]},{"name":"PERSUADE","report_text":"Stylome 1.2 Large · PERSUADE 2.0PERSUADEArgumentative essays written by United States students in grades 6–12.False-positive rate · 0.01% FPR target0%0 of 893 human essays flaggedFull resultsAll 893 essays were classified as human at both fixed operating points. No essays were excluded or truncated, and the thresholds were fixed before scoring.This evaluation uses the established frozen held half of the PERSUADE corpus. The same 893 human-written essays were evaluated for earlier Stylome versions.Stylome 1.2 Large on the frozen PERSUADE halfNominal calibration FPR targetHuman essaysFalse positivesObserved false-positive rate0.1%89300%0.01%89300%Evaluated September 16, 2026. The exact two-sided 95% binomial interval is 0–0.4122%, using the independent-essay assumption.","tables":[{"caption":"Stylome 1.2 Large on the frozen PERSUADE half","rows":[["Nominal calibration FPR target","Human essays","False positives","Observed false-positive rate"],["0.1%","893","0","0%"],["0.01%","893","0","0%"]]}],"references":[{"title":"PERSUADE corpus","url":"https://doi.org/10.1016/j.asw.2024.100865"}]},{"name":"TOEFL91","report_text":"Stylome 1.2 Large · Liang et al.TOEFL91Human essays written by people learning English as an additional language.False-positive rate · 0.01% FPR target2.20%2 of 91 human essays flaggedFull resultsStylome 1.2 Large correctly cleared 89 of 91 human essays at both fixed operating points. The same two essays were flagged at each point. No essays were excluded or truncated, and the thresholds were fixed before scoring.This evaluation uses the established 91-essay panel from the TOEFL91 study. The same 91 human-written essays were evaluated for earlier Stylome versions.Stylome 1.2 Large on TOEFL91Nominal calibration FPR targetHuman essaysFalse positivesObserved false-positive rate0.1%9122.20%0.01%9122.20%Evaluated September 16, 2026. The exact two-sided 95% binomial interval for the false-positive rate is 0.267–7.715%, using the independent-essay assumption.","tables":[{"caption":"Stylome 1.2 Large on TOEFL91","rows":[["Nominal calibration FPR target","Human essays","False positives","Observed false-positive rate"],["0.1%","91","2","2.20%"],["0.01%","91","2","2.20%"]]}],"references":[{"title":"the TOEFL91 study","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/"}]}]},{"model":"stylome-1.0","status":"retired","context":["Stylome 1.0 is the previous standard model. These July 2026 results come from 24,505 public-benchmark passages scored with that production model and input contract.","6,068 of 6,068 human passages were cleared across all three EditLens test sections.","5,896 of 5,931 fully generated EditLens passages were detected.","2,605 of 2,606 essays were cleared across the frozen ELLIPSE and PERSUADE evaluations.","Every result below uses the deployed Stylome 1.0 threshold, with no threshold fitting on these benchmarks."],"reports":[{"name":"PAN 2025","report_text":"Stylome 1.0 · PAN at CLEF 2025PAN 2025Human and AI-generated essays, fiction, and news from an independent detection challenge.Human writing cleared99.61%1,272 of 1,277AI-generated detected94.42%2,183 of 2,312Full resultsThe complete labeled validation set, scored at Stylome 1.0’s deployed threshold with no threshold fitting on PAN.PAN 2025 is an academic shared task for distinguishing human writing from generated text. Its validation set spans essays, fiction, and news, with output from 22 model families and variants.Every one of its 3,589 passages is long enough for Stylome’s scanner. None has full-text or 1,000-character-prefix overlap with the final Phase 11 training corpus.1,272 of 1,277 human passages were correctly left unflagged.2,183 of 2,312 AI-generated passages were detected.Stylome 1.0 on PAN 2025 by writing typeWriting typeHuman clearedAI generated detectedEssays99.24% (131/132)98.50% (525/533)Fiction99.57% (924/928)92.41% (840/909)News100% (217/217)94.02% (818/870)Evaluation completed July 2026 on PAN’s labeled validation set, not its hidden official test set. Detection on Llama-3.1-8B-Instruct was 70.25%; the aggregate results above span all 22 model families and variants in the validation set.","tables":[{"caption":"Stylome 1.0 on PAN 2025 by writing type","rows":[["Writing type","Human cleared","AI generated detected"],["Essays","99.24% (131/132)","98.50% (525/533)"],["Fiction","99.57% (924/928)","92.41% (840/909)"],["News","100% (217/217)","94.02% (818/870)"]]}],"references":[{"title":"PAN 2025","url":"https://pan.webis.de/clef25/pan25-web/generated-content-analysis.html"}]},{"name":"ELLIPSE","report_text":"Stylome 1.0 · ELLIPSE CorpusELLIPSEStandardized-test essays written by English learners in grades 8–12.Human essays cleared99.94%1,712 of 1,713Full resultsOnly one false positive across a frozen evaluation of 1,713 learner-English essays.The open-source ELLIPSE corpus contains roughly 6,500 essays written during statewide testing by English learners in grades 8–12.After removing material inherited by an earlier model lineage, we split the remaining 3,425 eligible essays with a fixed random seed: 1,712 for training and 1,713 for this frozen evaluation.1,712 of 1,713 human essays were correctly left unflagged.1 of 1,713 received an incorrect AI-generated result.Stylome 1.0 on the frozen ELLIPSE halfHuman essaysCorrectly clearedFalse positivesClearance rateFalse-positive rate1,7131,712199.94%0.06%Evaluation completed July 2026 at Stylome 1.0’s deployed threshold. The held half has zero full-text or prefix overlap with final model training. This is a Stylome-defined frozen split, not an official detector benchmark split published by the corpus authors.","tables":[{"caption":"Stylome 1.0 on the frozen ELLIPSE half","rows":[["Human essays","Correctly cleared","False positives","Clearance rate","False-positive rate"],["1,713","1,712","1","99.94%","0.06%"]]}],"references":[{"title":"ELLIPSE corpus","url":"https://doi.org/10.1075/ijlcr.22026.cro"}]},{"name":"PERSUADE","report_text":"Stylome 1.0 · PERSUADE 2.0PERSUADEArgumentative essays written by United States students in grades 6–12.Human essays cleared100%893 of 893Full resultsZero false positives across a frozen evaluation of 893 student essays.The open-source PERSUADE 2.0 corpus contains more than 25,000 argumentative essays written by students in grades 6–12 across 15 prompts.After removing material inherited by an earlier model lineage, we split our 1,785-essay eligible sample with a fixed random seed: 892 for training and 893 for this frozen evaluation.893 of 893 human essays were correctly left unflagged.0 of 893 received an incorrect AI-generated result.Stylome 1.0 on the frozen PERSUADE halfHuman essaysCorrectly clearedFalse positivesClearance rateFalse-positive rate95% upper bound8938930100%0%0.34%Evaluation completed July 2026 at Stylome 1.0’s deployed threshold. The held half has zero full-text or prefix overlap with final model training. With no observed errors, 0.34% is the conservative rule-of-three upper bound. This is a Stylome-defined frozen split, not an official detector benchmark split published by the corpus authors.","tables":[{"caption":"Stylome 1.0 on the frozen PERSUADE half","rows":[["Human essays","Correctly cleared","False positives","Clearance rate","False-positive rate","95% upper bound"],["893","893","0","100%","0%","0.34%"]]}],"references":[{"title":"PERSUADE 2.0 corpus","url":"https://doi.org/10.1016/j.asw.2024.100865"}]},{"name":"EditLens","report_text":"Stylome 1.0 · Pangram LabsEditLensHuman-written, AI-edited, and AI-generated text across three test sections.Human writing cleared100%6,068 of 6,068AI-generated detected99.41%5,896 of 5,931Full resultsNo human false positives across 6,068 passages, while Stylome 1.0 detected more than 99% of fully generated text in every test section.EditLens is a dataset from Pangram Labs built to study how much AI has changed a piece of writing. Unlike a simple human-or-AI test, these results use three labels: human written, AI edited, and AI generated.AI-edited writing remains the harder category. Stylome 1.0’s strongest three-class section was the Llama split: 92.5% overall correct, including 84.7% recall on edited text.6,068 of 6,068 human passages were correctly left unflagged.5,896 of 5,931 fully generated passages were detected.3,812 of 6,220 AI-edited passages were detected at the same conservative threshold.Performance at Stylome 1.0’s deployed thresholdTest sectionHuman writing clearedFalse positivesAI edited detectedAI generated detectedStandard test100%0 of 2,03864.11%99.36%Enron email100%0 of 1,99256.37%99.13%Llama100%0 of 2,03864.27%99.71%Research benchmarkThree labels make this a harder testThe live scanner asks whether the evidence is strong enough to flag a passage. EditLens also asks the model to distinguish AI-edited writing from fully AI-generated writing. A random choice among three balanced labels would score about 33%.Three-class performance for Stylome 1.0 on EditLensTest sectionPassagesOverall correctBalanced scoreHuman foundAI edited foundAI generated foundStandard testReviews, news, educational web text, and creative writing6,11589.3%89.1%92.3%77.0%98.7%Enron emailA separate business-email domain6,14788.1%88.2%92.5%76.2%98.2%LlamaAI text produced only with Llama 3.35,95792.5%92.3%92.3%84.7%99.95%Overall correct is the share assigned the right label. Balanced score is macro-F1: it gives each of the three labels equal weight. The final three columns show how often Stylome found each class.The three-way scores complement the deployed-threshold results: together they show Stylome 1.0's binary detection strength and its separate performance on AI-edited writing across each official split.Evaluation completed July 2026 on all three public EditLens test sections. The final training audit found zero full-text and zero 1,000-character-prefix collisions. Official split rows are shown so the evaluation remains comparable and reproducible.","tables":[{"caption":"Performance at Stylome 1.0’s deployed threshold","rows":[["Test section","Human writing cleared","False positives","AI edited detected","AI generated detected"],["Standard test","100%","0 of 2,038","64.11%","99.36%"],["Enron email","100%","0 of 1,992","56.37%","99.13%"],["Llama","100%","0 of 2,038","64.27%","99.71%"]]},{"caption":"Three-class performance for Stylome 1.0 on EditLens","rows":[["Test section","Passages","Overall correct","Balanced score","Human found","AI edited found","AI generated found"],["Standard testReviews, news, educational web text, and creative writing","6,115","89.3%","89.1%","92.3%","77.0%","98.7%"],["Enron emailA separate business-email domain","6,147","88.1%","88.2%","92.5%","76.2%","98.2%"],["LlamaAI text produced only with Llama 3.3","5,957","92.5%","92.3%","92.3%","84.7%","99.95%"]]}],"references":[{"title":"EditLens","url":"https://github.com/pangramlabs/EditLens"}]},{"name":"TOEFL91","report_text":"Stylome 1.0 · Liang et al.TOEFL91Human essays written by people learning English as an additional language.Human essays cleared96.70%88 of 91Full resultsA deliberately difficult fairness check for false positives on human writing by non-native English speakers.TOEFL91 contains 91 human-written TOEFL essays used by Liang and colleagues to study detector bias against non-native English writers.Their 2023 study reported that seven contemporary detectors falsely flagged an average of 61.3% of these essays. Stylome 1.0 cleared 96.70% of the same essays, reducing the false-positive rate to 3.30%.88 of 91 human essays were correctly left unflagged.3 of 91 received an incorrect AI-generated result.Stylome 1.0 on TOEFL91Human essaysCorrectly clearedFalse positivesFalse-positive rate95% interval918833.30%0.69–9.33%Evaluation completed July 2026 at Stylome 1.0’s deployed threshold. The essays have zero full-text or prefix overlap with final model training. This small, single-register sample should not be read as a complete measure of performance for multilingual or non-native English writing.","tables":[{"caption":"Stylome 1.0 on TOEFL91","rows":[["Human essays","Correctly cleared","False positives","False-positive rate","95% interval"],["91","88","3","3.30%","0.69–9.33%"]]}],"references":[{"title":"TOEFL91","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/"}]}]},{"model":"stylome-fast-1.0","status":"retired","context":[],"reports":[{"name":"RAID","report_text":"Stylome 1.0 Fast · RAIDRAIDA large independent test spanning eight kinds of writing, eleven generators, and deliberate attempts to evade detection.Live threshold · human cleared99.90%13,342 of 13,356RAID comparison · assistants detected93.01%Clean assistant output; count not reportedFull resultsTwo operating points answer different questions. Stylome 1.0 Fast’s recorded production threshold shows that model's deployed behavior. RAID’s standard comparison allows a 5% false-positive rate in each domain, as its published evaluation protocol specifies.RAID covers abstracts, books, news, poetry, recipes, Reddit, reviews, and Wikipedia. Stylome scored the complete labeled training release: 467,985 clean passages and all eleven published transformations, or 5,615,820 rows in total.None of the passages had full-text or prefix overlap with Stylome’s final training corpus. The generators date to 2023 and were not represented in Stylome’s training data.99.90% human writing cleared at the live threshold: 13,342 of 13,356 eligible clean passages.74.59% assistant output detected at the same conservative threshold; this rises to 93.01%under RAID’s matched 5%-false-positive comparison.Clean output from 2023 assistant modelsGeneratorLive thresholdRAID comparisonChatGPT · GPT-3.5 Turbo96.48%99.76%Llama 2 70B Chat93.71%99.43%GPT-483.96%98.97%Mistral 7B Instruct74.80%96.35%MPT-30B Chat59.44%85.24%Cohere Chat34.46%76.11%Legacy instruction and base-model completionsRAID also contains older completion systems. These models sample much closer to the human writing they learned from and do not represent the polished assistant products most people use today. They remain in the complete results rather than being folded into the assistant headline.Clean legacy instruction and base-model outputGeneratorLive thresholdRAID comparisonGPT-3 · text-davinci-0025.71%51.69%GPT-2 XL3.28%47.53%Cohere Generate4.62%31.86%Mistral 7B base2.46%33.09%MPT-30B base0.74%25.31%All eleven RAID transformationsEach clean passage was also mechanically altered. These are reported separately because they test deliberate evasion, not ordinary untouched writing.Eligible AI passages detected after transformationTransformationLive thresholdRAID comparisonInserted paragraphs37.60%66.81%Number changes37.15%64.27%Alternative spelling33.28%60.86%Misspellings24.96%53.63%Homoglyphs2.76%53.57%Paraphrasing7.05%52.59%Synonym replacement6.93%52.27%Article deletion15.09%45.28%Alternating case6.21%36.67%Added whitespace3.46%31.25%Zero-width spaces0.003%18.37%Evaluation completed July 2026 on RAID’s labeled training release, not its hidden test set or official leaderboard. RAID’s comparison threshold was selected independently within each domain, following its default protocol. At Fast 1.0's recorded production threshold, clean human clearance was 99.90%; across transformed human passages it ranged from 97.79% to 100%. Passages shorter than Stylome’s 200-character input floor are preserved in the audit but excluded from the product-rate tables above.","tables":[{"caption":"Clean output from 2023 assistant models","rows":[["Generator","Live threshold","RAID comparison"],["ChatGPT · GPT-3.5 Turbo","96.48%","99.76%"],["Llama 2 70B Chat","93.71%","99.43%"],["GPT-4","83.96%","98.97%"],["Mistral 7B Instruct","74.80%","96.35%"],["MPT-30B Chat","59.44%","85.24%"],["Cohere Chat","34.46%","76.11%"]]},{"caption":"Clean legacy instruction and base-model output","rows":[["Generator","Live threshold","RAID comparison"],["GPT-3 · text-davinci-002","5.71%","51.69%"],["GPT-2 XL","3.28%","47.53%"],["Cohere Generate","4.62%","31.86%"],["Mistral 7B base","2.46%","33.09%"],["MPT-30B base","0.74%","25.31%"]]},{"caption":"Eligible AI passages detected after transformation","rows":[["Transformation","Live threshold","RAID comparison"],["Inserted paragraphs","37.60%","66.81%"],["Number changes","37.15%","64.27%"],["Alternative spelling","33.28%","60.86%"],["Misspellings","24.96%","53.63%"],["Homoglyphs","2.76%","53.57%"],["Paraphrasing","7.05%","52.59%"],["Synonym replacement","6.93%","52.27%"],["Article deletion","15.09%","45.28%"],["Alternating case","6.21%","36.67%"],["Added whitespace","3.46%","31.25%"],["Zero-width spaces","0.003%","18.37%"]]}],"references":[{"title":"RAID","url":"https://github.com/liamdugan/raid"}]},{"name":"PAN 2025","report_text":"Stylome 1.0 Fast · PAN at CLEF 2025PAN 2025Human and AI-generated essays, fiction, and news from an independent detection challenge.Human writing cleared99.84%1,275 of 1,277AI-generated detected95.59%2,210 of 2,312Full resultsThe complete labeled validation set, scored at Stylome’s public threshold with no threshold fitting on PAN.PAN 2025 is an academic shared task for distinguishing human writing from generated text. Its validation set spans essays, fiction, and news, with output from 22 model families and variants.Every one of its 3,589 passages is long enough for Stylome’s scanner. None has full-text or prefix overlap with the final model training corpus.1,275 of 1,277 human passages were correctly left unflagged.2,210 of 2,312 AI-generated passages were detected.Stylome 1.0 Fast on PAN 2025 by writing typeWriting typeHuman clearedAI generated detectedEssays100% (132/132)98.87% (527/533)Fiction99.78% (926/928)92.08% (837/909)News100% (217/217)97.24% (846/870)Evaluation completed July 2026 on PAN’s labeled validation set, not its hidden official test set. Detection on Llama-3.1-8B-Instruct was 68.99%; the aggregate results above span all 22 model families and variants in the validation set.","tables":[{"caption":"Stylome 1.0 Fast on PAN 2025 by writing type","rows":[["Writing type","Human cleared","AI generated detected"],["Essays","100% (132/132)","98.87% (527/533)"],["Fiction","99.78% (926/928)","92.08% (837/909)"],["News","100% (217/217)","97.24% (846/870)"]]}],"references":[{"title":"PAN 2025","url":"https://pan.webis.de/clef25/pan25-web/generated-content-analysis.html"}]},{"name":"ELLIPSE","report_text":"Stylome 1.0 Fast · ELLIPSE CorpusELLIPSEStandardized-test essays written by English learners in grades 8–12.Human essays cleared99.94%1,712 of 1,713Full resultsA frozen student-writing check drawn from an established public learner-English corpus.The open-source ELLIPSE corpus contains roughly 6,500 essays written during statewide testing by English learners in grades 8–12.After removing material inherited by an earlier model lineage, we split the remaining 3,425 eligible essays with a fixed random seed: 1,712 for training and 1,713 for this frozen evaluation.1,712 of 1,713 human essays were correctly left unflagged.1 of 1,713 received an incorrect AI-generated result.Stylome 1.0 Fast on the frozen ELLIPSE halfHuman essaysCorrectly clearedFalse positivesClearance rateFalse-positive rate1,7131,712199.94%0.06%Evaluation completed July 2026 at Stylome’s public threshold. The held half has zero full-text or prefix overlap with final model training. This is a Stylome-defined frozen split of ELLIPSE, not an official detector benchmark split published by the corpus authors.","tables":[{"caption":"Stylome 1.0 Fast on the frozen ELLIPSE half","rows":[["Human essays","Correctly cleared","False positives","Clearance rate","False-positive rate"],["1,713","1,712","1","99.94%","0.06%"]]}],"references":[{"title":"ELLIPSE corpus","url":"https://doi.org/10.1075/ijlcr.22026.cro"}]},{"name":"PERSUADE","report_text":"Stylome 1.0 Fast · PERSUADE 2.0PERSUADEArgumentative essays written by United States students in grades 6–12.Human essays cleared100%893 of 893Full resultsA frozen student-writing check drawn from a large public corpus of persuasive essays.The open-source PERSUADE 2.0 corpus contains more than 25,000 argumentative essays written by students in grades 6–12 across 15 prompts.After removing material inherited by an earlier model lineage, we split our 1,785-essay eligible sample with a fixed random seed: 892 for training and 893 for this frozen evaluation.893 of 893 human essays were correctly left unflagged.0 of 893 received an incorrect AI-generated result.Stylome 1.0 Fast on the frozen PERSUADE halfHuman essaysCorrectly clearedFalse positivesClearance rateFalse-positive rate95% upper bound8938930100%0%0.34%Evaluation completed July 2026 at Stylome’s public threshold. The held half has zero full-text or prefix overlap with final model training. With no observed errors, 0.34% is the conservative rule-of-three upper bound. This is a Stylome-defined frozen split, not an official detector benchmark split published by the corpus authors.","tables":[{"caption":"Stylome 1.0 Fast on the frozen PERSUADE half","rows":[["Human essays","Correctly cleared","False positives","Clearance rate","False-positive rate","95% upper bound"],["893","893","0","100%","0%","0.34%"]]}],"references":[{"title":"PERSUADE 2.0 corpus","url":"https://doi.org/10.1016/j.asw.2024.100865"}]},{"name":"EditLens","report_text":"Stylome 1.0 Fast · Pangram LabsEditLensHuman-written, AI-edited, and AI-generated text across three test sections.Human writing cleared99.93%6,064 of 6,068AI-generated detected99.46%5,899 of 5,931Full resultsEvaluated on raw text with Stylome 1.0 Fast. These measurements are not attributed to the newer Stylome 1.0 standard model.EditLens is a dataset from Pangram Labs built to study how much AI has changed a piece of writing. Unlike a simple human-or-AI test, these results use three labels: human written, AI edited, and AI generated.That middle category matters. Lightly or heavily edited human text is a different problem from text written entirely by a model, and the results below show that it is also the harder one.6,064 of 6,068 human passages were correctly left unflagged.5,899 of 5,931 fully generated passages were detected.Performance at Stylome’s public thresholdTest sectionHuman writing clearedFalse positivesAI edited detectedAI generated detectedStandard test99.90%2 of 2,03863.56%99.51%Enron email100%0 of 1,99254.85%99.13%Llama99.90%2 of 2,03860.39%99.71%Research benchmarkThree labels make this a harder testThe live scanner asks whether the evidence is strong enough to flag a passage. EditLens also asks the model to distinguish AI-edited writing from fully AI-generated writing. A random choice among three balanced labels would score about 33%.Three-class performance for Stylome 1.0 Fast on EditLensTest sectionPassagesOverall correctBalanced scoreHuman foundAI edited foundAI generated foundStandard testReviews, news, educational web text, and creative writing6,11588.4%88.1%89.2%76.3%99.6%Enron emailA separate business-email domain6,14789.8%89.8%97.3%75.8%99.2%LlamaAI text produced only with Llama 3.35,95791.3%91.1%89.2%84.3%99.8%Overall correct is the share assigned the right label. Balanced score is macro-F1: it gives each of the three labels equal weight. The final three columns show how often Stylome found each class.The three-way scores complement the public-threshold results: together they show Stylome 1.0 Fast's binary detection strength and its separate performance on AI-edited writing across each official split.Evaluation completed July 2026. Pangram’s dataset release contains 28 training/validation overlaps in Standard test and 27 in Llama (about 0.45% of each); Enron has none. Removing those rows changes reconstructed accuracy by less than 0.04 percentage points. Official-split results are shown here so they remain comparable and reproducible.","tables":[{"caption":"Performance at Stylome’s public threshold","rows":[["Test section","Human writing cleared","False positives","AI edited detected","AI generated detected"],["Standard test","99.90%","2 of 2,038","63.56%","99.51%"],["Enron email","100%","0 of 1,992","54.85%","99.13%"],["Llama","99.90%","2 of 2,038","60.39%","99.71%"]]},{"caption":"Three-class performance for Stylome 1.0 Fast on EditLens","rows":[["Test section","Passages","Overall correct","Balanced score","Human found","AI edited found","AI generated found"],["Standard testReviews, news, educational web text, and creative writing","6,115","88.4%","88.1%","89.2%","76.3%","99.6%"],["Enron emailA separate business-email domain","6,147","89.8%","89.8%","97.3%","75.8%","99.2%"],["LlamaAI text produced only with Llama 3.3","5,957","91.3%","91.1%","89.2%","84.3%","99.8%"]]}],"references":[{"title":"EditLens","url":"https://github.com/pangramlabs/EditLens"}]},{"name":"TOEFL91","report_text":"Stylome 1.0 Fast · Liang et al.TOEFL91Human essays written by people learning English as an additional language.Human essays cleared94.51%86 of 91Full resultsA deliberately difficult fairness check for false positives on human writing by non-native English speakers.TOEFL91 contains 91 human-written TOEFL essays used by Liang and colleagues to study detector bias against non-native English writers.Their 2023 study reported that seven contemporary detectors falsely flagged an average of 61.3% of these essays. Stylome 1.0 Fast cleared 94.51% of the same essays, reducing the false-positive rate to 5.49%.86 of 91 human essays were correctly left unflagged.5 of 91 received an incorrect AI-generated result.Stylome 1.0 Fast on TOEFL91Human essaysCorrectly clearedFalse positivesFalse-positive rate95% interval918655.49%1.81–12.36%Evaluation completed July 2026 at Stylome’s public threshold. The 91 essays have zero full-text or prefix overlap with model training. This small, single-register sample should not be read as a complete measure of performance for multilingual or non-native English writing.","tables":[{"caption":"Stylome 1.0 Fast on TOEFL91","rows":[["Human essays","Correctly cleared","False positives","False-positive rate","95% interval"],["91","86","5","5.49%","1.81–12.36%"]]}],"references":[{"title":"TOEFL91","url":"https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/"}]}]}]}