⋙ List Evals

List evaluations for a project.

msg.payload Properties

after string

Pagination cursor for items after this id.

limit integer

Maximum number of evals to return.

order string

Sort order by creation time. Use asc or desc.

⋙ Create Eval

Create an eval definition with data source config and testing criteria.

msg.payload Properties

name string

The eval name.

data_source_config object

Data source config schema used by eval runs.

testing_criteria array

Grader criteria used to score results.

metadata object

Optional metadata key-value pairs.

⋙ Retrieve Eval

Retrieve an eval by id.

msg.payload Properties

eval_id string

The eval id.

⋙ Modify Eval

Update mutable eval fields.

msg.payload Properties

eval_id string

The eval id.

name string

Updated eval name.

metadata object

Updated metadata.

⋙ Delete Eval

Delete an eval by id.

msg.payload Properties

eval_id string

The eval id.

⋙ List Eval Runs

List runs for an eval.

msg.payload Properties

eval_id string

The eval id.

after string

Pagination cursor for items after this id.

limit integer

Maximum number of runs to return.

order string

Sort order by creation time. Use asc or desc.

⋙ Create Eval Run

Create a run for an eval.

msg.payload Properties

eval_id string

The eval id.

data_source object

Run data source config.

model string

Model used for the run when applicable.

metadata object

Optional run metadata.

name string

Optional run name.

⋙ Retrieve Eval Run

Retrieve a run by id.

msg.payload Properties

eval_id string

The eval id.

run_id string

The run id.

⋙ Cancel Eval Run

Cancel an in-progress eval run.

msg.payload Properties

eval_id string

The eval id.

run_id string

The run id.

⋙ Delete Eval Run

Delete an eval run.

msg.payload Properties

eval_id string

The eval id.

run_id string

The run id.

⋙ List Eval Run Output Items

List output items for a run.

msg.payload Properties

eval_id string

The eval id.

run_id string

The run id.

after string

Pagination cursor for items after this id.

limit integer

Maximum number of items to return.

⋙ Retrieve Eval Run Output Item

Retrieve one run output item by id.

msg.payload Properties

eval_id string

The eval id.

run_id string

The run id.

output_item_id string

The output item id.