Test Cases

Create prompts and assertions for regression testing.

Test Cases define the prompts and expected behavior used to check Sagent. They turn important customer journeys and safety requirements into repeatable release checks.

Use test cases to protect behavior that matters to your business. Update them when you intentionally change Sagent’s configuration or capabilities.

Use Test Cases to create and maintain the prompts and assertions used in regression testing.

1

Create a test case

Open QA → Test Cases. Click New test case.

2

Name the test

Enter a clear name that describes the scenario being tested.

3

Add the prompt

Enter the shopper or adversarial prompt Sagent should answer.

4

Add assertions

Click Add assertion. Select an assertion type, then add a value when that type requires one.

Add more assertions when the response must meet several conditions. Click Save when finished.

Choose an assertion type

Use assertions to define how QA grades Sagent's response.

LLM-graded assertions

• LLM Rubric — grades the response against a custom rubric. Describe expected behaviour in plain English.

• Factuality — compares the response with an expected fact. It checks whether the response agrees, contradicts, or is unrelated.

• Closed QA — checks whether the response correctly answers a closed question using the reference answer.

• Answer Relevance — scores how relevant the response is to the original prompt. No value is required.

String matching

String matching is fast and has no LLM cost.

• Not Contains — checks that the response does not contain an exact, case-sensitive string.

• Not Contains (case-insensitive) — checks that the response does not contain a string, regardless of case.

• Contains — checks that the response contains an exact, case-sensitive string.

• Contains (case-insensitive) — checks that the response contains a string, regardless of case.

Pattern matching

Pattern matching is fast and has no LLM cost.

• Not Regex — checks that the response does not match a regular expression. Use it to block patterns such as prices or URLs.

• Regex — checks that the response matches a regular expression pattern.

Tool usage

Tool-usage assertions are fast and have no LLM cost.

• Used Question Tool — passes when Sagent asks a clarifying question through the question tool. No value is required.

• Used Product Search Tool — passes when Sagent returns product results through the product-search tool. No value is required.

Safety

• Is Refusal — detects whether a response is a refusal. It supports English only and has no LLM cost. Use an LLM Rubric alongside it for multilingual testing.

• Moderation — uses the OpenAI Moderation API to check for harmful, violent, or explicit content. No value is required.

Frequently asked questions

Can a test case have more than one assertion?

Yes. Add an assertion for every condition the response must meet.

When should I use an LLM Rubric?

Use an LLM Rubric when expected behaviour needs plain-language criteria. Use a more specific assertion when you need to check a fact, fixed answer, or exact string.