Poor Document Retrieval Accuracy

In Retrieval-Augmented Generation (RAG) systems, the retrieval phase is critical as it determines which documents or chunks are passed to the language model to ground its responses. When this retrieval step performs poorly, the result is a model that may produce hallucinations, contradictory claims, or entirely irrelevant answers.

Common failure modes include:

- Conflicting Source Documents: When several retrieved passages contain opposing information, the model may merge or alternate between them, causing confusion or factual inaccuracy.
- Poor Update Latency: If the vector database isn't refreshed in a timely manner, recent documents (e.g., updated policies, terms, or prices) may not be included, leading to outdated responses despite recent ground-truth changes.
- Suboptimal Similarity Metrics: Relying on one retrieval strategy (e.g., BM25 or cosine similarity) can underperform in specific query types. Hybrid systems that blend semantic and keyword search tend to perform better across edge cases.
- Incompatible Embedding Spaces: Mixing embeddings generated by different models (e.g., OpenAI vs. Cohere) without alignment can degrade retrieval performance due to distance metric mismatch.

- Incident 639: Customer Overcharged Due to Air Canada Chatbot's False Discount Claims - Air Canada was ordered to pay over $600 in damages for providing inaccurate bereavement discount information via its chatbot, leading to a customer overpaying for flights. The tribunal ruled the airline responsible for the chatbot's misinformation.