Study finds nearly two-thirds of AI-generated citations are fabricated or contain errors.
Study reveals AI-generated citations often fabricated or erroneous, posing risks for scientific research.
Why it matters
- AI tools like GPT-4o are increasingly used in research but may produce unreliable citations.
- Accurate citations are crucial for scholarly communication and research integrity.
By the numbers
- Nearly two-thirds of AI-generated citations were either fabricated or contained errors.
- 35 out of 176 citations were entirely fabricated.
- Fabrication rate was 6% for major depressive disorder, 28% for binge eating disorder, and 29% for body dysmorphic disorder.
The big picture
- AI models struggle more with less familiar or specialized topics.
- The reliability of AI-generated content varies significantly based on the topic's prominence in training data.
What they're saying
- Users report similar issues with AI-generated references in their own research.
- Some suggest that while AI-generated citations are often unreliable, AI can still be useful for initial research when paired with online search tools.
Caveats
- The study focused on one AI model (GPT-4o) and may not represent other models.
- Findings are based on a specific set of prompts and topics.
What’s next
- Further research could explore a wider range of topics and AI models to confirm these patterns.
- Academic institutions may need to develop new standards for verifying AI-generated references.