Create Experiments in the UI

A guided walkthrough to create experiments in the platform UI using Prompt, LLM generation (chat completion), or Custom tasks.

What is Experiments?

Experiments lets you run repeatable evaluations over a dataset and inspect outputs, evaluator scores, and run status in the UI.

This page focuses on the UI flow. If you prefer code, see Run Experiments via API.

Resources

• Create datasets in the UI
• Run Experiments via API
• Evaluators (UI)

Steps to use

Prompt

1. Step 1: Click New experiment

   Go to Experiments and click New experiment.

   
2. Step 2: Select a dataset

   Choose the dataset you want to run on.

   
3. Step 3: Select task = Prompt

   Pick Prompt as the task type.

   
4. Step 4: Select a prompt

   Choose the prompt you want to test (and the version if applicable).

   
5. Step 5: Select evaluators

   Select one or more evaluators to score outputs.

   
6. Step 6: Create and wait for the run to finish

   Click Create. The run will process in the background. Wait until the status is complete, then inspect outputs and evaluator scores.

   

   

LLM generation (chat completion)

1. Step 1: Click New experiment

   Go to Experiments and click New experiment.

   
2. Step 2: Select a dataset

   Choose the dataset you want to run on.

   
3. Step 3: Select task = LLM generation (chat completion)

   Pick LLM generation (chat completion) as the task type.

   
4. Step 4: Configure the model and parameters

   Choose the model and set parameters like temperature and max tokens.

   
5. Step 5: Select evaluators

   Select one or more evaluators to score outputs.

   
6. Step 6: Create and wait for the run to finish

   Click Create. The run will process in the background. Wait until the status is complete, then inspect outputs and evaluator scores.

   

Custom

1. Step 1: Click New experiment

   Go to Experiments and click New experiment.

   
2. Step 2: Select a dataset

   Choose the dataset you want to run on.

   
3. Step 3: Select task = Custom

   Pick Custom as the task type.

   
4. Step 4: Select evaluators

   Select one or more evaluators to score outputs.

   
5. Step 5: Create and wait for placeholders

   Click Create. The system will create placeholder rows for each dataset entry.

   You can then fill outputs using the API flow (recommended) so evaluators run on your submitted results.

   For custom tasks, the UI is usually used to monitor progress and review results, while outputs are submitted via API.