Skip to main content

LLM Response Parsing

3 requirements · 16 scenarios

Citation Verification Pipeline

JR-llm-response-parsing-003
The system SHALL verify quoted text against source documents and apply visual indicators based on verification status. Quotes SHALL be verified against primary and secondary sources, with mismatches triggering auto-correction for document sources or preservation for email body sources. Duplicate quotes SHALL be tracked individually.
9 test scenarios
  • Quotes formatted as block or inline JR-llm-response-parsing-003.1
  • Quotes verified against primary source JR-llm-response-parsing-003.2
  • Quotes verified against secondary source JR-llm-response-parsing-003.3
  • Verification status stored as metadata JR-llm-response-parsing-003.4
  • Visual indicators reflect verification status JR-llm-response-parsing-003.5
  • Document mismatches trigger auto-correction JR-llm-response-parsing-003.6
  • Email body mismatches preserved as-is JR-llm-response-parsing-003.7
  • Duplicate quotes tracked individually JR-llm-response-parsing-003.8
  • Corrections propagated through pipeline JR-llm-response-parsing-003.9

Citation Location Auto-Correction

JR-llm-response-parsing-011
When a quoted passage is found in a document but in a different paragraph than cited, the system SHALL auto-correct the citation to reference the actual paragraph. Auto-correction SHALL apply to document and attachment sources only, not email body sources. Correction metadata SHALL be stored for audit and applied during citation rendering.
4 test scenarios
  • Citation corrected for attachment sources JR-llm-response-parsing-011.1
  • Email body citations left uncorrected JR-llm-response-parsing-011.2
  • Correction metadata stored for audit JR-llm-response-parsing-011.3
  • Corrections applied during rendering JR-llm-response-parsing-011.4

Citation Source Eligibility

JR-llm-response-parsing-012
The system SHALL restrict citations to sources that were actually provided to the language model in the current turn. Ineligible sources SHALL be excluded from the prompt context, and any citations referencing ineligible sources SHALL be removed or flagged.
3 test scenarios
  • Ineligible sources excluded from context JR-llm-response-parsing-012.1
  • Ineligible citations removed or flagged JR-llm-response-parsing-012.2
  • Eligibility computed once and reused JR-llm-response-parsing-012.3
All Specs