Reflect on RAG system quality to identify improvement opportunities. ## Recent Query Samples {{query_samples}} ## Quality Metrics {{quality_metrics}} ## User Feedback {{user_feedback}} Apply structured reflection: **Reflection 1: Retrieval Effectiveness** - Are we retrieving the right documents? - What retrieval failures are we seeing? - How can retrieval be improved? **Reflection 2: Generation Fidelity** - Are generated answers faithful to context? - Where does hallucination occur? - How can we improve grounding? **Reflection 3: User Satisfaction** - Do answers meet user needs? - What patterns in user feedback? - What gaps exist between expectation and delivery? **Reflection 4: System Behavior** - Are there unexpected behaviors? - What edge cases are not handled well? - Where is the system fragile? **Synthesis** Prioritized list of quality improvements with expected impact.
Reflection on RAG Quality
U
@
Apply structured reflection on RAG quality covering retrieval, generation, user satisfaction, and system behavior with improvement synthesis.
94 copies0 forks
Details
Category
AnalysisUse Cases
Quality reflectionRAG improvementSystem evaluation
Works Best With
claude-sonnet-4-20250514gpt-4o
Created Shared