Tiêu đề Response Evaluation with Trending News [Vendor Guidelines].docx ✅

  Table of Contents 2 Version History 3 How to access the annotation platform 4 Who to Contact 4 Invoicing 4 Project Overview 5 User Interface (UI) 5 Simplified Steps 5 End-to-End Process 6 Step 1: Determine whether the query is answerable. 6 Step 2: Perform an Enhanced Fact Check (EFC) 8 Step 2.1: Identify Claims 9 Step 2.2: Verify Claims 12 Step 3: Determine whether the response is correct. 18 Tips and Notes 19 Step 4: Determine if the response is comprehensive. 32 Step 5: Determine whether the response is natural. 37 Step 6: Determine whether the response contains sensitive information. 40 Best Practices 42 Correctness 42 Edge Cases 44 Sources 45 Sources to use 45 US-Specific Sources 45 UK-Specific Sources 50 Instructions for Reviewers/QAers 53 Reviewer Role and Responsibilities 53 Main Tasks 53 Scoring Workflow 53 When to Grade Below Average and Way Below Average: 54 Feedback to Annotators 54 QA Role and Responsibilities 54 Main Tasks 54 Scoring Guidelines for QA 54 Scoring Workflow 55 Feedback to Reviewer and Annotator 55 Evaluation Criteria for Responses 56 Best Practices 57 Correctness Tolerance Examples 57
  Version History Version Description of changes Date Initials v1 Document creation 09/02/2025 KTH v1 QA team approval 09/02/2025 DSNDK v1 Step 3. Determine whether the response is correct - Added examples to illustrate the differences between the specificity, coverage, and relevance errors and No Response. 9/24/2025 DSNDK v1 Removed Comprehensive and Natural labels Added wikipedia as untrusted source 11/12/2025 DSNDI