Annotation Guideline Creator

Priya Ramanathan

@priya-ramanathan

·December 31, 2025

Create annotation guidelines for human evaluation.

79 copies0 forks

Share this prompt:

Create annotation guidelines for evaluating {{model}} on {{evaluation_task}}.

Generate guidelines including:
1. Task description and goals
2. Annotation schema with examples
3. Quality criteria definitions
4. Common disagreement scenarios
5. Calibration exercises

Format for {{annotator_expertise}} annotators with {{annotation_tool}} integration notes.

Details

Category

Analysis

Use Cases

Guideline creationAnnotation designEvaluator training

Works Best With

claude-opus-4.5gpt-5.2gemini-2.0-flash

Created December 31, 2025Updated January 2, 2026Shared December 31, 2025

Related Prompts

Annotation Guide Creator

by @dr-sage-mitchell

Develops systematic text annotation practices that support active reading and analysis.

Few-Shot Eval Criteria Generator

by @ethan-park

Learns evaluation patterns from examples to generate consistent, formalized evaluation criteria and frameworks.

Collaborative Annotation Protocol

by @dr-sage-mitchell

Structures shared text annotation that builds collective understanding through collaboration.

Brand Guidelines Document Creation

by @talia-nguyen

Break down brand guidelines document creation into structured sections.

Meta-Prompt Generator for Tasks

by @ethan-park

Creates meta-prompts that generate task-specific prompts with built-in quality checks and self-improvement mechanisms.

Peer Review Protocol Designer

by @dr-sage-mitchell

Develops structured peer feedback systems that promote quality critique and revision skills.

More from @priya-ramanathan

Mitigation Strategy Branching

Instruction Complexity Scoring

Deployment Scenario Analysis

Capability Probe Designer

Create your own prompt vault and start sharing