Reflection on RAG Quality

U

@

·

Apply structured reflection on RAG quality covering retrieval, generation, user satisfaction, and system behavior with improvement synthesis.

94 copies0 forks
Reflect on RAG system quality to identify improvement opportunities.

## Recent Query Samples
{{query_samples}}

## Quality Metrics
{{quality_metrics}}

## User Feedback
{{user_feedback}}

Apply structured reflection:

**Reflection 1: Retrieval Effectiveness**
- Are we retrieving the right documents?
- What retrieval failures are we seeing?
- How can retrieval be improved?

**Reflection 2: Generation Fidelity**
- Are generated answers faithful to context?
- Where does hallucination occur?
- How can we improve grounding?

**Reflection 3: User Satisfaction**
- Do answers meet user needs?
- What patterns in user feedback?
- What gaps exist between expectation and delivery?

**Reflection 4: System Behavior**
- Are there unexpected behaviors?
- What edge cases are not handled well?
- Where is the system fragile?

**Synthesis**
Prioritized list of quality improvements with expected impact.

Details

Category

Analysis

Use Cases

Quality reflectionRAG improvementSystem evaluation

Works Best With

claude-sonnet-4-20250514gpt-4o
Created Shared

Create your own prompt vault and start sharing