AI Detectors Are Not Fully Reliable: French Novel Scandal Exposes Detection Errors
Baca dalam 60 detik
- Penulis Kanada-Haiti Thelyson Orelien dituduh memakai AI untuk novel debutnya, memicu perdebatan tentang akurasi alat pendeteksi teks AI.
- Pengujian AFP terhadap kutipan novel menghasilkan beragam kesimpulan, dari 'kemungkinan besar manusia' hingga '100 persen AI', menunjukkan inkonsistensi alat.
- Pakar menyoroti risiko positif palsu dan evolusi AI yang cepat, sementara industri penerbitan dan pendidikan di Indonesia perlu waspada.
Controversy has erupted in the French literary world after Canadian-Haitian author Thelyson Orelien was accused of using artificial intelligence (AI) to write his bestselling debut novel. The accusation surfaced after an X account with few followers claimed that an AI detection program, Pangram, concluded that the story about Haitian migrants heading to Canada was largely generated by text generators such as Claude or ChatGPT. Orelien, 38, strongly denied the claim and promised to prove that his work was purely human-written. "I did not write this novel with my head, but with my gut and my heart," he told AFP.
AFP then tested excerpts from the book through various AI detection tools and obtained contradictory results—ranging from "most likely written by a human" to "100 percent AI-generated." The findings highlight a fundamental weakness in detection technology that is now widely used in the education and publishing sectors. Dozens of similar tools are available online, generally working by comparing the suspected text with large collections of human and AI writing. Although their performance has improved in recent years, a University of Chicago paper in October 2025 said Pangram achieved "near-zero error rates" on medium- to long-form text extracts. However, such accuracy claims are difficult to verify and often overlook the risk of false positives.
Thierry Poibeau, a researcher at the French National Centre for Scientific Research (CNRS) and a specialist in natural language processing, explained that these programs look for certain "linguistic markers." "The classic example is triadic phrases (using three clauses), or em dashes," he said. These patterns were once common in generated text because AI tools were trained on scientific writing or American novels. However, Poibeau also warned of a "moving target problem" because AI programs keep evolving. He added that detection tools can mistake an author's style for an indication of AI. "A tool may pick up something repetitive, but that could actually be the author's distinctive style," he said.
Orelien himself claims his style is rooted in Haitian and Caribbean traditions. To the newspaper Liberation, he stated that the software was "used to the French literary tradition and classical authors." Meanwhile, Benoit Raphael, an entrepreneur and author who specializes in AI, conducted independent research using Pangram to test Orelien's work and the result was also AI-positive. "The fact that the score is around 100 percent AI, or even 90 percent, is a fairly clear sign," he told Nouvel Obs. Although the book received critical praise and sold well in France before the scandal, some doubts had already emerged. Lea Bory on the Torchon podcast may have been the first to publicly say earlier this month that she suspected the novel was written by AI because of the repetitive nature of the sentences and the excessive use of analogies. Fabrice Colin in the newspaper Canard Enchaine also criticized the formulaic sentence structure and said he "searched in vain for a unique voice" in the prose.
The case raises important questions for Indonesia's creative industry. With the growing adoption of generative AI among writers, publishers, and educational institutions, reliance on AI detection tools can create a risk of unfair criminalization. In Indonesia, several education and media platforms have begun using similar software to check the authenticity of works, but there is no standard verification benchmark yet. Pangram CEO Max Spero, in an interview with The Atlantic, stressed that his tool should not be the final determinant, but rather a starting point for deeper investigation. This aligns with experts' calls for AI detection results not to be used as sole evidence, especially in a literary context rich in individual style.
Going forward, the main challenge is to develop more adaptive and transparent detection methods, and to build awareness that human and AI writing cannot always be separated by a clear line. For Indonesia, there needs to be dialogue between AI developers, academics, and creative industry players to formulate fair guidelines. Are we ready to face an era in which the boundary between human and machine work is increasingly blurred, without sacrificing integrity and creativity?
