Semantic Search Quality Monitor

U

@

·

Build a semantic search quality monitor with sampling, relevance evaluation, degradation detection, and comprehensive reporting.

82 copies0 forks
Implement a semantic search quality monitoring system.

## Search System
{{search_system}}

## Quality Baselines
{{quality_baselines}}

## Monitoring Requirements
{{monitoring_requirements}}

Build the monitor:

```python
class SearchQualityMonitor:
    def collect_samples(self, sample_rate: float) -> List[SearchSample]:
        """Sample search queries for evaluation"""
        pass
    
    def evaluate_relevance(self, sample: SearchSample) -> RelevanceScore:
        """
        Evaluation methods:
        - LLM-based relevance judgment
        - Click-through rate proxy
        - Dwell time analysis
        """
        pass
    
    def detect_degradation(self, window: TimeWindow) -> DegradationAlert:
        """Detect quality degradation over time"""
        pass
    
    def generate_report(self, period: str) -> QualityReport:
        """Weekly/monthly quality reports"""
        pass
```

Include:
- Online vs offline evaluation
- Statistical significance testing
- Trend analysis
- Alerting integration

Details

Category

Coding

Use Cases

Quality monitoringSearch evaluationDegradation detection

Works Best With

claude-sonnet-4-20250514gpt-4o
Created Shared

Create your own prompt vault and start sharing