Implement a semantic search quality monitoring system. ## Search System {{search_system}} ## Quality Baselines {{quality_baselines}} ## Monitoring Requirements {{monitoring_requirements}} Build the monitor: ```python class SearchQualityMonitor: def collect_samples(self, sample_rate: float) -> List[SearchSample]: """Sample search queries for evaluation""" pass def evaluate_relevance(self, sample: SearchSample) -> RelevanceScore: """ Evaluation methods: - LLM-based relevance judgment - Click-through rate proxy - Dwell time analysis """ pass def detect_degradation(self, window: TimeWindow) -> DegradationAlert: """Detect quality degradation over time""" pass def generate_report(self, period: str) -> QualityReport: """Weekly/monthly quality reports""" pass ``` Include: - Online vs offline evaluation - Statistical significance testing - Trend analysis - Alerting integration
Semantic Search Quality Monitor
U
@
Build a semantic search quality monitor with sampling, relevance evaluation, degradation detection, and comprehensive reporting.
82 copies0 forks
Details
Category
CodingUse Cases
Quality monitoringSearch evaluationDegradation detection
Works Best With
claude-sonnet-4-20250514gpt-4o
Created Shared