In a recent RAG (Retrieval-Augmented Generation) benchmark analysis, CustomGPT.ai has demonstrated superior performance over OpenAI’s Assistant API V2, underscoring its commitment to high-quality AI solutions. The extensive testing, which included 945 questions across nine diverse datasets, revealed that CustomGPT.ai achieved a 10% lower hallucination rate, a 13% higher accuracy rate, and a 34% faster average response time compared to OpenAI.
The founder and CEO of CustomGPT.ai , Alden Do Rosario, emphasized the importance of reducing AI hallucinations, where AI generates information that is not grounded in reality, which can lead to misinformation, compliance issues, and safety risks. “In today’s AI race, companies must adopt an ‘anti-hallucination first’ focus,” he said. “We founded our company on this premise, and we’re thrilled new research further validates our technology, especially for the 6,000-plus customers we now serve.”
The findings are particularly significant for industries where accuracy is critical, such as legal sectors, finance, healthcare, and education. Do Rosario believes these results will resonate with skeptics in both B2C and B2B sectors who question AI’s reliability and performance.
The research, validated by Tonic.ai, supports the use of RAG to mitigate AI hallucinations and deliver more precise and reliable information. The benchmark was comprehensive, using 945 questions across diverse topics from public health to literature, significantly more than previous studies.
The study introduced an ‘answer consistency binary’ metric, where any deviation from the expected answer was marked as a failure. Do Rosario highlighted the study’s enhanced statistical significance, data diversity, and scoring rigor, stating, “This research significantly ups the ante for statistical significance, data diversity, and scoring rigor.”
Do Rosario concluded by emphasizing the future potential of generative AI. “Gone are the days of organizations needing to settle for chatbots that generate inaccurate responses, especially from short-sighted, underperforming, or overpriced AI vendors. The future is wide open for gen AI to responsibly deliver comprehensive and contextually accurate information in order to truly help organizations advance decision-making capabilities, improve operational efficiency and increase revenues.”
