> For the complete documentation index, see [llms.txt](https://docs.crobox.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.crobox.com/how-to-guides/sagent/qa/tests.md).

# Tests

Create, manage, import, and export the tests that check Sagent's answers.

**Tests** is where you define what a good Sagent answer looks like. Each test pairs a shopper message with the checks Sagent's answer must pass, so you can rerun the same expectations after every change.

### The Tests overview

Open **QA → Tests** to see every test for this Sagent.

| Column          | Shows                                                                                                                                                                          |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Test**        | The test name in bold, with the shopper message underneath. Conversation tests also show a **Conversation** chip with the number of messages.                                  |
| **Checks**      | The checks the answer must pass.                                                                                                                                               |
| **Last result** | **Passed** or **Failed**, with the time of the most recent run that included the test. Blank if the test hasn't run yet. Marked **Outdated** if you've changed the test since. |
| **On/Off**      | Whether the test is included in your next run.                                                                                                                                 |

Use the toolbar to find tests quickly:

* **Search tests** finds tests by name or message.
* **Last result** filters by **All**, **Failed**, **Passed**, or **Not run**.
* **On/Off** filters by whether tests are switched on.

Open the menu at the end of a row to **Edit**, **Duplicate**, or **Delete** a test.

Crobox system checks appear as a separate, collapsed group: **Crobox system checks: included in every run**. Open a check to see what it tests. They're managed by Crobox and can't be changed here.

{% hint style="info" %}
Give tests a short prefix that describes their purpose, such as **Vague:**, **Rec:**, or **Guard:**. This makes it easy to search for a group of tests and spot patterns in your results.
{% endhint %}

### Create a test

{% stepper %}
{% step %}

#### Start a new test

Open **QA → Tests** and click **New test**.
{% endstep %}

{% step %}

#### Name the test

Enter a **Test name** that describes the scenario, for example *Rec: budget constraint*.
{% endstep %}

{% step %}

#### Add the shopper message

Enter the **Shopper message** Sagent should answer, exactly as a shopper might type it. For example, *Multi-day pack under 200 euros*.
{% endstep %}

{% step %}

#### Add an expected answer (optional)

Describe what a great answer would say in **Expected answer**. When you fill this in, QA adds a **Matches expected answer** check automatically, which checks that the facts in Sagent's answer match yours. The check is locked in place for as long as Expected answer is filled in.
{% endstep %}

{% step %}

#### Add checks

New tests start with a **Meets criteria** check. Describe what a good answer does in its value field.

Click **Add check** to add more conditions. Choose the check type, then add a value if the check needs one. The test passes only when every check passes.
{% endstep %}

{% step %}

#### Save

Click **Save**. Your new test is switched on and runs the next time you click **Run tests**.
{% endstep %}
{% endstepper %}

{% hint style="success" %}
You can also save a conversation from the **Setup** preview as a test. The whole conversation is replayed in order on each run, so you can test multi-step journeys.
{% endhint %}

### Choose your checks

The check list shows the most common checks first. Open **More checks** for the rest.

| Check                         | What it checks                                                                                                  | Value                |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------- | -------------------- |
| **Meets criteria**            | Sagent's answer does what you describe in plain language.                                                       | Criteria             |
| **Matches expected answer**   | The facts in Sagent's answer match your expected answer. Added automatically when Expected answer is filled in. | From Expected answer |
| **Asks a follow-up question** | Sagent asks for more detail before recommending.                                                                | None                 |
| **Shows products**            | Sagent recommends products.                                                                                     | None                 |
| **Doesn't contain text**      | The answer never includes specific words, like competitor names.                                                | Text                 |
| **Politely declines**         | Sagent refuses off-topic or unsafe requests.                                                                    | None                 |

**More checks**

| Check                     | What it checks                                                                       | Value   |
| ------------------------- | ------------------------------------------------------------------------------------ | ------- |
| **Contains text**         | The answer includes specific words, like a brand name.                               | Text    |
| **Stays on the question** | The answer is relevant to the shopper's message.                                     | None    |
| **Matches pattern**       | The answer matches a regular expression. For technical users.                        | Pattern |
| **Doesn't match pattern** | The answer doesn't match a regular expression, for example to block prices or links. | Pattern |
| **No harmful content**    | Flags violent, explicit, or harmful answers.                                         | None    |

Text checks ignore case by default. Switch on **Match case** on a **Contains text** or **Doesn't contain text** check when capitalization matters, for example to require a brand name written exactly as *Osprey*.

#### Write strong criteria

**Meets criteria** is the most flexible check, so it pays to write criteria carefully:

* **Describe behavior you can observe.** *Recommends at least one multi-day pack under €200, or says clearly that none fit the budget* is easier to judge than *Gives a good answer*.
* **Keep each check focused.** Use separate checks for separate conditions, so the result tells you exactly what went wrong.
* **Combine checks.** For a vague request, pair **Asks a follow-up question** with **Meets criteria**. For a recommendation, pair **Shows products** with **Meets criteria**.

### Switch tests on or off

Only tests that are switched on run when you click **Run tests**. Switch a test off to keep it for later without including it in runs. Its last result stays visible in the overview.

### Import and export tests

Use CSV files to build or update many tests at once in a spreadsheet, or to share your test suite for review. Import and export cover single-message tests only. Conversation tests can't be imported and are skipped on export.

#### Export tests

Click **Export** on the Tests overview. You get a CSV of every single-message test that matches your current search and filters, across all pages, in the same format **Import** reads.

{% hint style="info" %}
To update existing tests, export them first, edit the file, and import it again. Keeping each row's `id` updates the matching test instead of creating a new one.
{% endhint %}

#### CSV format

Each row is one check. A test with three checks spans three rows.

| Column            | Required | Description                                                                                                                                            |
| ----------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `id`              | No       | The test's ID. Rows with the same `id` update that test. Leave empty to create a new test.                                                             |
| `name`            | Yes      | The test name. Rows without an `id` that share a name become one new test.                                                                             |
| `shopper_message` | Yes      | The message Sagent answers.                                                                                                                            |
| `expected_answer` | No       | What a great answer would say. When filled in, Sagent's answer is also checked against it.                                                             |
| `check_type`      | Yes      | The check's name, such as `Meets criteria`. Names ignore case and extra spaces.                                                                        |
| `check_value`     | Depends  | The criteria, text, or pattern for checks that need one. Leave empty for checks without a value.                                                       |
| `on`              | No       | `on` or `off`. Also accepts `true`/`false`, `yes`/`no`, and `1`/`0`. If empty, an existing test keeps its current state and a new test is switched on. |

Checks that need a `check_value`:

* **Meets criteria**: criteria
* **Contains text** and **Doesn't contain text**: text
* **Matches pattern** and **Doesn't match pattern**: pattern

The **Matches expected answer** check doesn't get its own row. Fill in the `expected_answer` column instead, and the check is added automatically.

#### Match case in a CSV

The names **Contains text** and **Doesn't contain text** always ignore case. To switch on **Match case**, write `contains` or `not-contains` as the `check_type` instead. Exports use these values for any check with Match case switched on, so they import back unchanged.

Example file creating two new tests, one with two checks:

```csv
id,name,shopper_message,expected_answer,check_type,check_value,on
,Rec: budget constraint,Multi-day pack under 200 euros,,Shows products,,on
,Rec: budget constraint,Multi-day pack under 200 euros,,Meets criteria,Recommends multi-day packs under 200 euros or says none fit the budget,on
,Guard: competitor names,Is your pack better than the competition?,,Doesn't contain text,CompetitorBrand,on
```

#### Import tests

{% stepper %}
{% step %}

#### Open the import dialog

Click **Import** on the Tests overview.
{% endstep %}

{% step %}

#### Choose your file

Click **Choose CSV file** and select your file.
{% endstep %}

{% step %}

#### Import

Click **Import**. If any row has an error, QA shows which rows need fixing and nothing is saved. Fix the file and import it again.
{% endstep %}
{% endstepper %}

Keep these rules in mind:

* **Imported checks replace existing ones.** When a row updates a test, that test's checks are replaced by the checks in the file.
* **Up to 300 new or changed tests per import.** Tests the file leaves unchanged don't count, and a test can span several rows.
* **All or nothing.** If any row has an error, the whole import is cancelled.

### Frequently asked questions

<details>

<summary>Can a test have more than one check?</summary>

Yes. Add a check for every condition the answer must meet. The test passes only when all of them pass.

</details>

<details>

<summary>When should I use Meets criteria instead of a text check?</summary>

Use **Meets criteria** when a good answer can be phrased in many ways. Use **Contains text** or **Doesn't contain text** when specific words must, or must never, appear.

</details>

<details>

<summary>Can I edit Crobox system checks?</summary>

No. Crobox maintains these checks for every Sagent. They're included in each run and shown read-only.

</details>

<details>

<summary>Why wasn't my conversation test included in the export?</summary>

Import and export support single-message tests only. Conversation tests stay in QA and keep running as normal.

</details>

<details>

<summary>What happens to a test's results when I edit it?</summary>

Its last result is kept but marked **Outdated** when you change the shopper message, a check, or the expected answer. Renaming a test or switching it on or off doesn't make its result outdated. Run your tests again to get an up-to-date result.

</details>

<details>

<summary>What happens to past results when I delete a test?</summary>

The test disappears from the Tests overview, but its results stay in earlier runs. Run details always show each result exactly as it ran.

</details>

<details>

<summary>Does switching a test off delete its results?</summary>

No. Switched-off tests keep their last result. They're skipped in future runs until you switch them back on.

</details>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.crobox.com/how-to-guides/sagent/qa/tests.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.