LLM Pro RSS: give a model a defined evidence set

Prepare a compact evidence bundle

Begin with a clear question, such as which release notes change an installation requirement. Select entries that can answer it, then retain their titles, identifiers, dates, canonical URLs, and relevant passages. Keep the retrieved article text separate from the instructions describing what the model should do.

Give every source a short local label such as S01 or S02. Store the full source details outside the generated prose and ask the output to identify the labels supporting each claim. This makes review easier and helps prevent a correct sentence from being attached to the wrong article.

Ask for an answer you can check

A useful response structure contains the finding, supporting source labels, an explanation of its consequence, and any unresolved detail. Avoid requesting a comprehensive industry conclusion from a handful of feed entries. The scope of the conclusion should remain proportional to the evidence you supplied.

Question: Which updates change installation prerequisites?
Return: finding, source_ids, affected_step, unresolved_detail.
Use only the selected source passages for factual claims.

Include examples of acceptable uncertainty in your evaluation set. An entry that announces an update without describing its requirements should produce a limited answer. It should not inherit details from a similarly titled announcement merely because both records appear in the same batch.

Evaluate the whole workflow

Anthropic's prompt engineering overview emphasizes defining success and testing it. For a feed workflow, score source selection, factual support, omitted qualifications, and usefulness to the intended reader. Check these dimensions independently so a stylistic improvement does not conceal a loss of accuracy.

Retain several difficult examples: overlapping announcements, a correction, an entry with little text, and a long article containing only one relevant paragraph. Repeat the evaluation when changing the prompt, model, or extraction method. Keep the comparison based on the same input snapshots so you can explain what actually changed.

When the selected evidence grows too large, revise retrieval and selection before expanding the request indiscriminately. A smaller set that directly answers the question is easier to inspect and maintain. The token budgeting guide develops that part of the workflow.