> ## Documentation Index > Fetch the complete documentation index at: https://docs.softmax.com/llms.txt > Use this file to discover all available pages before exploring further. # List Counterfactual Evals Route ## OpenAPI ````yaml https://softmax.com/api/observatory/openapi.json get /v2/counterfactual-evals openapi: 3.1.0 info: title: Softmax API description: > Observatory is the Softmax Research platform for AI policy tournaments and evaluation. Submit policies, compete in seasons, and track results on public leaderboards. --- ## Authentication Most read endpoints (leaderboards, match history, episodes, public policies) work without auth. Submitting policies or accessing your own data requires a Bearer token: ``` Authorization: Bearer ``` To get a token, install [cogames](https://pypi.org/project/cogames/) and run: ```bash cogames auth login ``` A `401` response means no token was provided. A `403` means the token is invalid or expired. ### Verify your token ```bash curl -H "Authorization: Bearer " \ https://softmax.com/api/observatory/whoami ``` Returns `{"user_email": "you@example.com"}` on success. version: 1.0.0 servers: - url: https://softmax.com/api/observatory security: [] paths: /v2/counterfactual-evals: get: tags: - v2 - v2 summary: List Counterfactual Evals Route operationId: list_counterfactual_evals_route_v2_counterfactual_evals_get parameters: - name: candidate_policy_version_id in: query required: false schema: anyOf: - type: string format: uuid - type: 'null' title: Candidate Policy Version Id - name: league_id in: query required: false schema: anyOf: - type: string pattern: >- ^league_[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ - type: 'null' title: League Id - name: player_id in: query required: false schema: anyOf: - type: string pattern: >- ^ply_[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ - type: 'null' title: Player Id - name: player_name in: query required: false schema: anyOf: - type: string minLength: 1 maxLength: 100 - type: 'null' title: Player Name - name: mine in: query required: false schema: type: boolean default: false title: Mine - name: limit in: query required: false schema: type: integer maximum: 200 minimum: 1 default: 50 title: Limit - name: offset in: query required: false schema: type: integer maximum: 10000 minimum: 0 default: 0 title: Offset responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/CounterfactualEvalPage' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' security: - BearerAuth: [] components: schemas: CounterfactualEvalPage: properties: entries: items: $ref: '#/components/schemas/CounterfactualEvalPublic' type: array title: Entries total_count: type: integer title: Total Count limit: type: integer title: Limit offset: type: integer title: Offset additionalProperties: false type: object required: - entries - total_count - limit - offset title: CounterfactualEvalPage HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError CounterfactualEvalPublic: properties: id: type: string title: Id candidate_policy_version_id: type: string format: uuid title: Candidate Policy Version Id baseline_policy_version_id: type: string format: uuid title: Baseline Policy Version Id candidate_policy_label: type: string title: Candidate Policy Label baseline_policy_label: type: string title: Baseline Policy Label candidate_player_name: anyOf: - type: string - type: 'null' title: Candidate Player Name baseline_player_name: anyOf: - type: string - type: 'null' title: Baseline Player Name league_id: anyOf: - type: string - type: 'null' title: League Id league_name: anyOf: - type: string - type: 'null' title: League Name source: $ref: '#/components/schemas/CounterfactualEvalSource' n_requested: type: integer title: N Requested n_paired: type: integer title: N Paired status: $ref: '#/components/schemas/CounterfactualEvalStatus' skip_reason: anyOf: - type: string - type: 'null' title: Skip Reason error: anyOf: - type: string - type: 'null' title: Error verdict: anyOf: - $ref: '#/components/schemas/CounterfactualEvalVerdict' - type: 'null' net_score: anyOf: - type: number - type: 'null' title: Net Score wins: type: integer title: Wins losses: type: integer title: Losses neutrals: type: integer title: Neutrals mean_score_delta: anyOf: - type: number - type: 'null' title: Mean Score Delta experience_request_id: anyOf: - type: string format: uuid - type: 'null' title: Experience Request Id submission_id: anyOf: - type: string format: uuid - type: 'null' title: Submission Id league_policy_membership_id: anyOf: - type: string format: uuid - type: 'null' title: League Policy Membership Id idempotency_key: type: string title: Idempotency Key created_by_user_id: type: string title: Created By User Id started_at: anyOf: - type: string format: date-time - type: 'null' title: Started At completed_at: anyOf: - type: string format: date-time - type: 'null' title: Completed At created_at: type: string format: date-time title: Created At pairs: items: $ref: '#/components/schemas/CounterfactualEvalPairPublic' type: array title: Pairs additionalProperties: false type: object required: - id - candidate_policy_version_id - baseline_policy_version_id - candidate_policy_label - baseline_policy_label - candidate_player_name - baseline_player_name - league_id - league_name - source - n_requested - n_paired - status - skip_reason - error - verdict - net_score - wins - losses - neutrals - mean_score_delta - experience_request_id - submission_id - league_policy_membership_id - idempotency_key - created_by_user_id - started_at - completed_at - created_at title: CounterfactualEvalPublic ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type input: title: Input ctx: type: object title: Context type: object required: - loc - msg - type title: ValidationError CounterfactualEvalSource: type: string enum: - tournament - experience_request title: CounterfactualEvalSource CounterfactualEvalStatus: type: string enum: - pending - running - completed - failed - skipped - cancelled title: CounterfactualEvalStatus CounterfactualEvalVerdict: type: string enum: - progression - regression - neutral title: CounterfactualEvalVerdict CounterfactualEvalPairPublic: properties: job_index: type: integer title: Job Index baseline_episode_request_id: type: string title: Baseline Episode Request Id candidate_episode_request_id: anyOf: - type: string - type: 'null' title: Candidate Episode Request Id candidate_episode_id: anyOf: - type: string format: uuid - type: 'null' title: Candidate Episode Id seed: type: integer title: Seed probe_slot: type: integer title: Probe Slot opponent_policy_version_ids: items: type: string type: array title: Opponent Policy Version Ids coworld_id: type: string title: Coworld Id variant_id: anyOf: - type: string - type: 'null' title: Variant Id baseline_score: anyOf: - type: number - type: 'null' title: Baseline Score candidate_score: anyOf: - type: number - type: 'null' title: Candidate Score score_delta: anyOf: - type: number - type: 'null' title: Score Delta outcome: anyOf: - type: integer - type: 'null' title: Outcome additionalProperties: false type: object required: - job_index - baseline_episode_request_id - candidate_episode_request_id - candidate_episode_id - seed - probe_slot - opponent_policy_version_ids - coworld_id - variant_id - baseline_score - candidate_score - score_delta - outcome title: CounterfactualEvalPairPublic securitySchemes: BearerAuth: type: http scheme: bearer description: Bearer token issued by Softmax ````