Skip to main content
POST
Cancel counterfactual evaluation

Authorizations

Authorization
string
header
required

Bearer token issued by Softmax

Path Parameters

counterfactual_eval_id
string
required

Counterfactual evaluation to cancel.

Pattern: ^cfeval_[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$

Response

Successful Response

id
string
required

Counterfactual evaluation ID.

candidate_policy_version_id
string<uuid>
required

Candidate policy version being evaluated.

baseline_policy_version_id
string<uuid>
required

Baseline policy version used for comparison.

candidate_policy_label
string
required

Display label for the candidate policy version.

baseline_policy_label
string
required

Display label for the baseline policy version.

candidate_player_name
string | null
required

Player attributed to the candidate.

baseline_player_name
string | null
required

Player attributed to the baseline.

league_id
string | null
required

League that supplied the evaluation settings.

league_name
string | null
required

League name.

source
enum<string>
required

Workflow that created the evaluation.

Available options:
tournament,
experience_request
n_requested
integer
required

Number of paired comparisons requested.

n_paired
integer
required

Number of paired comparisons successfully scored.

status
enum<string>
required

Current evaluation state.

Available options:
pending,
running,
completed,
failed,
skipped,
cancelled
skip_reason
string | null
required

Reason the evaluation was skipped.

error
string | null
required

Failure message, or null when no failure was recorded.

verdict
enum<string> | null
required

Final comparison verdict.

Available options:
progression,
regression,
neutral
net_score
number | null
required

Win-loss margin divided by paired comparisons, when scoring is complete.

wins
integer
required

Number of comparisons won by the candidate.

losses
integer
required

Number of comparisons lost by the candidate.

neutrals
integer
required

Number of comparisons with a neutral outcome.

mean_score_delta
number | null
required

Mean candidate score change, when scoring is complete.

experience_request_id
string<uuid> | null
required

Experience Request created to run the comparisons.

submission_id
string<uuid> | null
required

League submission that triggered the evaluation.

league_policy_membership_id
string<uuid> | null
required

League membership associated with the evaluation.

idempotency_key
string
required

Key that makes repeated create requests return the same evaluation.

created_by_user_id
string
required

User that created the evaluation.

started_at
string<date-time> | null
required

Time when evaluation work started.

completed_at
string<date-time> | null
required

Time when the evaluation reached a final state.

created_at
string<date-time>
required

Time when the evaluation was created.

pairs
CounterfactualEvalPairPublic · object[]

Paired baseline and candidate episodes in this evaluation.