> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.vapi.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.vapi.ai/_mcp/server.

# Create Eval

POST https://api.vapi.ai/eval
Content-Type: application/json

Creates a reusable eval that defines a mock conversation and checkpoints for evaluating assistant responses and tool calls.

Reference: https://docs.vapi.ai/api-reference/eval/eval-controller-create

## Authentication

- `Authorization` header (bearer token, required) — Authenticate server-side requests with a private Vapi API key. Create or copy a key from the [Vapi Dashboard](https://dashboard.vapi.ai) and send it in the `Authorization` header as `Bearer <token>`. Keep private API keys out of client-side code and public repositories.

## Request

### Body (application/json)

This endpoint expects a CreateEvalDTO.

- `messages` (list of CreateEvalDtoMessagesItems, required) — This is the mock conversation that will be used to evaluate the flow of the conversation. Mock Messages are used to simulate the flow of the conversation Evaluation Messages are used as checkpoints in the flow where the model's response to previous conversation needs to be evaluated to check the content and tool calls
- `type` (enum, required) — This is the type of the eval. Currently it is fixed to `chat.mockConversation`.
  - Allowed values: `chat.mockConversation`
- `name` (string, optional) — This is the name of the eval. It helps identify what the eval is checking for.
- `description` (string, optional) — This is the description of the eval. This helps describe the eval and its purpose in detail. It will not be used to evaluate the flow of the conversation.

## Response

### 201

- `messages` (list of EvalMessagesItems, required) — This is the mock conversation that will be used to evaluate the flow of the conversation. Mock Messages are used to simulate the flow of the conversation Evaluation Messages are used as checkpoints in the flow where the model's response to previous conversation needs to be evaluated to check the content and tool calls
- `id` (string, required) — The unique identifier for the eval.
- `orgId` (string, required) — The unique identifier for the organization that owns the eval.
- `createdAt` (datetime, required) — The ISO 8601 timestamp when the eval was created.
- `updatedAt` (datetime, required) — The ISO 8601 timestamp when the eval was last updated.
- `type` (enum, required) — This is the type of the eval. Currently it is fixed to `chat.mockConversation`.
  - Allowed values: `chat.mockConversation`
- `name` (string, optional) — This is the name of the eval. It helps identify what the eval is checking for.
- `description` (string, optional) — This is the description of the eval. This helps describe the eval and its purpose in detail. It will not be used to evaluate the flow of the conversation.

## Errors

### 403 Forbidden Error

Forbidden

- `any`

## Types

### CreateEvalDtoMessagesItems

### EvalMessagesItems

### ChatEvalAssistantMessageMock

A simulated assistant turn in an evaluation conversation, with optional message content and tool calls.

- `role` (enum, required, default: assistant) — This is the role of the message author. For a mock assistant message, the role is always 'assistant' @default 'assistant'
  - Allowed values: `assistant`
- `content` (string, optional) — This is the content of the assistant message. This is the message that the assistant would have sent.
- `toolCalls` (list of ChatEvalAssistantMessageMockToolCall, optional) — This is the tool calls that will be made by the assistant.

### ChatEvalSystemMessageMock

A simulated system message in an evaluation conversation.

- `role` (enum, required, default: system) — This is the role of the message author. For a mock system message, the role is always 'system' @default 'system'
  - Allowed values: `system`
- `content` (string, required) — This is the content of the system message that would have been added in the middle of the conversation. Do not include the assistant prompt as a part of this message. It will automatically be fetched during runtime.

### ChatEvalToolResponseMessageMock

A simulated tool response in an evaluation conversation.

- `role` (enum, required, default: tool) — This is the role of the message author. For a mock tool response message, the role is always 'tool' @default 'tool'
  - Allowed values: `tool`
- `content` (string, required) — This is the content of the tool response message. JSON Objects should be stringified.

### ChatEvalToolResponseMessageEvaluation

An expected tool-response turn evaluated by a configured LLM judge.

- `role` (enum, required, default: tool) — This is the role of the message author. For a tool response message evaluation, the role is always 'tool' @default 'tool'
  - Allowed values: `tool`
- `judgePlan` (AssistantMessageJudgePlanAI, required) — This is the judge plan that instructs how to evaluate the tool response message. The tool response message can be evaluated with an LLM-as-judge by defining the evaluation criteria in a prompt.

### ChatEvalUserMessageMock

A simulated user message in an evaluation conversation.

- `role` (enum, required, default: user) — This is the role of the message author. For a mock user message, the role is always 'user' @default 'user'
  - Allowed values: `user`
- `content` (string, required) — This is the content of the user message. This is the message that the user would have sent.

### ChatEvalAssistantMessageEvaluation

An expected assistant turn in an evaluation, including the judge plan and how the evaluation should continue afterward.

- `role` (enum, required, default: assistant) — This is the role of the message author. For an assistant message evaluation, the role is always 'assistant' @default 'assistant'
  - Allowed values: `assistant`
- `judgePlan` (ChatEvalAssistantMessageEvaluationJudgePlan, required) — This is the judge plan that instructs how to evaluate the assistant message. The assistant message can be evaluated against fixed content (exact match or RegEx) or with an LLM-as-judge by defining the evaluation criteria in a prompt.
- `continuePlan` (AssistantMessageEvaluationContinuePlan, optional) — This is the plan for how the overall evaluation will proceed after the assistant message is evaluated. This lets you configure whether to stop the evaluation if this message fails, and whether to override any content for future turns

### ChatEvalAssistantMessageMockToolCall

A simulated assistant tool call with the tool name and optional arguments.

- `name` (string, required) — This is the name of the tool that will be called. It should be one of the tools created in the organization.
- `arguments` (ChatEvalAssistantMessageMockToolCallArguments, optional) — This is the arguments that will be passed to the tool call.

### AssistantMessageJudgePlanAI

Evaluates an assistant message with an LLM judge and a configured evaluation model.

- `model` (AssistantMessageJudgePlanAiModel, required) — This is the model to use for the LLM-as-a-judge. If not provided, will default to the assistant's model. The instructions on how to evaluate the model output with this LLM-Judge must be passed as a system message in the messages array of the model. The Mock conversation can be passed to the LLM-Judge to evaluate using the prompt \{\{messages}} and will be evaluated as a LiquidJS Variable. To access and judge only the last message, use \{\{messages\[-1]}} The LLM-Judge must respond with "pass" or "fail" and only those two responses are allowed.
- `type` (enum, required) — This is the type of the judge plan. Use 'ai' to evaluate the assistant message content using LLM-as-a-judge. @default 'ai'
  - Allowed values: `ai`
- `autoIncludeMessageHistory` (boolean, optional) — This is the flag to enable automatically adding the liquid variable \{\{messages}} to the model's messages array This is only applicable if the user has not provided any messages in the model's messages array @default true

### ChatEvalAssistantMessageEvaluationJudgePlan

This is the judge plan that instructs how to evaluate the assistant message. The assistant message can be evaluated against fixed content (exact match or RegEx) or with an LLM-as-judge by defining the evaluation criteria in a prompt.

### AssistantMessageEvaluationContinuePlan

Controls how an evaluation proceeds after judging an assistant message, including failure handling and optional message overrides.

- `exitOnFailureEnabled` (boolean, optional) — This is whether the evaluation should exit if the assistant message evaluates to false. By default, it is false and the evaluation will continue. @default false
- `contentOverride` (string, optional) — This is the content that will be used in the conversation for this assistant turn moving forward if provided. It will override the content received from the model.
- `toolCallsOverride` (list of ChatEvalAssistantMessageMockToolCall, optional) — This is the tool calls that will be used in the conversation for this assistant turn moving forward if provided. It will override the tool calls received from the model.

### ChatEvalAssistantMessageMockToolCallArguments

This is the arguments that will be passed to the tool call.

### AssistantMessageJudgePlanAiModel

This is the model to use for the LLM-as-a-judge. If not provided, will default to the assistant's model. The instructions on how to evaluate the model output with this LLM-Judge must be passed as a system message in the messages array of the model. The Mock conversation can be passed to the LLM-Judge to evaluate using the prompt \{\{messages}} and will be evaluated as a LiquidJS Variable. To access and judge only the last message, use \{\{messages\[-1]}} The LLM-Judge must respond with "pass" or "fail" and only those two responses are allowed.

### AssistantMessageJudgePlanExact

Evaluates an assistant message using case-insensitive exact content matching and expected tool calls.

- `type` (enum, required) — This is the type of the judge plan. Use 'exact' for an exact match on the content and tool calls - without using LLM-as-a-judge. @default 'exact'
  - Allowed values: `exact`
- `content` (string, required) — This is what that will be used to evaluate the model's message content. If you provide a string, the assistant message content will be evaluated against it as an exact match, case-insensitive.
- `toolCalls` (list of ChatEvalAssistantMessageMockToolCall, optional) — This is the tool calls that will be used to evaluate the model's message content. The tool name must be a valid tool that the assistant is allowed to call. For the Query tool, the arguments for the tool call are in the format - \{knowledgeBaseNames: \['kb\_name', 'kb\_name\_2']} For the DTMF tool, the arguments for the tool call are in the format - \{dtmf: "1234\*"} For the Handoff tool, the arguments for the tool call are in the format - \{destination: "assistant\_id"} For the Transfer Call tool, the arguments for the tool call are in the format - \{destination: "phone\_number\_or\_assistant\_id"} For all other tools, they are called without arguments or with user-defined arguments

### AssistantMessageJudgePlanRegex

Evaluates assistant-message content and tool-call arguments using regular-expression patterns.

- `type` (enum, required) — This is the type of the judge plan. Use 'regex' for a regex match on the content and tool calls - without using LLM-as-a-judge. @default 'regex'
  - Allowed values: `regex`
- `content` (string, required) — This is what that will be used to evaluate the model's message content. The content will be evaluated against the regex pattern provided in the Judge Plan content field. Evaluation is considered successful if the regex pattern matches any part of the assistant message content.
- `toolCalls` (list of ChatEvalAssistantMessageMockToolCall, optional) — This is the tool calls that will be used to evaluate the model's message content. The tool name must be a valid tool that the assistant is allowed to call. The values to the arguments for the tool call should be a Regular Expression. Evaluation is considered successful if the regex pattern matches any part of each tool call argument. For the Query tool, the arguments for the tool call are in the format - \{knowledgeBaseNames: \['kb\_name', 'kb\_name\_2']} For the DTMF tool, the arguments for the tool call are in the format - \{dtmf: "1234\*"} For the Handoff tool, the arguments for the tool call are in the format - \{destination: "assistant\_id"} For the Transfer Call tool, the arguments for the tool call are in the format - \{destination: "phone\_number\_or\_assistant\_id"} For all other tools, they are called without arguments or with user-defined arguments

### EvalOpenAIModel

OpenAI model configuration for an LLM judge, including its messages and generation settings.

- `provider` (enum, required) — This is the provider of the model (`openai`).
  - Allowed values: `openai`
- `model` (enum, required) — This is the OpenAI model that will be used. When using Vapi OpenAI or your own Azure Credentials, you have the option to specify the region for the selected model. This shouldn't be specified unless you have a specific reason to do so. Vapi will automatically find the fastest region that make sense. This is helpful when you are required to comply with Data Residency rules. Learn more about Azure regions here https://azure.microsoft.com/en-us/explore/global-infrastructure/data-residency/.
  - Allowed values: `gpt-6-luna`, `gpt-5.6-sol`, `gpt-5.6-terra`, `gpt-5.6-luna`, `gpt-5.5`, `chat-latest`, `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.2`, `gpt-5.2-chat-latest`, `gpt-5.1`, `gpt-5.1-chat-latest`, `gpt-5`, `gpt-5-chat-latest`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1-2025-04-14`, `gpt-4.1-mini-2025-04-14`, `gpt-4.1-nano-2025-04-14`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `chatgpt-4o-latest`, `o3`, `o3-mini`, `o4-mini`, `o1-mini`, `o1-mini-2024-09-12`, `gpt-4o-mini-2024-07-18`, `gpt-4o-mini`, `gpt-4o`, `gpt-4o-2024-05-13`, `gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`, `gpt-4-turbo`, `gpt-4-turbo-2024-04-09`, `gpt-4-turbo-preview`, `gpt-4-0125-preview`, `gpt-4-1106-preview`, `gpt-4`, `gpt-4-0613`, `gpt-3.5-turbo`, `gpt-3.5-turbo-0125`, `gpt-3.5-turbo-1106`, `gpt-3.5-turbo-16k`, `gpt-3.5-turbo-0613`, `gpt-5.6-luna:westus3`, `gpt-5.6-terra:westus3`, `gpt-5.6-sol:westus3`, `gpt-5.4:eastus2`, `gpt-5.4:swedencentral`, `gpt-5.4-mini:eastus2`, `gpt-5.4-mini:swedencentral`, `gpt-5.4-nano:eastus2`, `gpt-5.4-nano:swedencentral`, `gpt-5.2:eastus2`, `gpt-5.2:swedencentral`, `gpt-5.1:eastus2`, `gpt-5.1:swedencentral`, `gpt-5:eastus2`, `gpt-5:swedencentral`, `gpt-5:canadaeast`, `gpt-5:eastus`, `gpt-5:westeurope`, `gpt-5:germanywestcentral`, `gpt-5:polandcentral`, `gpt-5:spaincentral`, `gpt-5-mini:eastus2`, `gpt-5-mini:swedencentral`, `gpt-5-mini:westeurope`, `gpt-5-mini:germanywestcentral`, `gpt-5-mini:polandcentral`, `gpt-5-mini:spaincentral`, `gpt-5-nano:eastus2`, `gpt-5-nano:swedencentral`, `gpt-4.1-2025-04-14:westus`, `gpt-4.1-2025-04-14:eastus2`, `gpt-4.1-2025-04-14:eastus`, `gpt-4.1-2025-04-14:westus3`, `gpt-4.1-2025-04-14:northcentralus`, `gpt-4.1-2025-04-14:southcentralus`, `gpt-4.1-2025-04-14:westeurope`, `gpt-4.1-2025-04-14:germanywestcentral`, `gpt-4.1-2025-04-14:polandcentral`, `gpt-4.1-2025-04-14:spaincentral`, `gpt-4.1-mini-2025-04-14:westus`, `gpt-4.1-mini-2025-04-14:eastus2`, `gpt-4.1-mini-2025-04-14:eastus`, `gpt-4.1-mini-2025-04-14:westus3`, `gpt-4.1-mini-2025-04-14:northcentralus`, `gpt-4.1-mini-2025-04-14:southcentralus`, `gpt-4.1-mini-2025-04-14:westeurope`, `gpt-4.1-mini-2025-04-14:germanywestcentral`, `gpt-4.1-mini-2025-04-14:polandcentral`, `gpt-4.1-mini-2025-04-14:spaincentral`, `gpt-4.1-nano-2025-04-14:westus`, `gpt-4.1-nano-2025-04-14:eastus2`, `gpt-4.1-nano-2025-04-14:westus3`, `gpt-4.1-nano-2025-04-14:northcentralus`, `gpt-4.1-nano-2025-04-14:southcentralus`, `gpt-4o-2024-11-20:swedencentral`, `gpt-4o-2024-11-20:westus`, `gpt-4o-2024-11-20:eastus2`, `gpt-4o-2024-11-20:eastus`, `gpt-4o-2024-11-20:westus3`, `gpt-4o-2024-11-20:southcentralus`, `gpt-4o-2024-11-20:westeurope`, `gpt-4o-2024-11-20:germanywestcentral`, `gpt-4o-2024-11-20:polandcentral`, `gpt-4o-2024-11-20:spaincentral`, `gpt-4o-2024-08-06:westus`, `gpt-4o-2024-08-06:westus3`, `gpt-4o-2024-08-06:eastus`, `gpt-4o-2024-08-06:eastus2`, `gpt-4o-2024-08-06:northcentralus`, `gpt-4o-2024-08-06:southcentralus`, `gpt-4o-mini-2024-07-18:westus`, `gpt-4o-mini-2024-07-18:westus3`, `gpt-4o-mini-2024-07-18:eastus`, `gpt-4o-mini-2024-07-18:eastus2`, `gpt-4o-mini-2024-07-18:northcentralus`, `gpt-4o-mini-2024-07-18:southcentralus`, `gpt-4o-2024-05-13:eastus2`, `gpt-4o-2024-05-13:eastus`, `gpt-4o-2024-05-13:northcentralus`, `gpt-4o-2024-05-13:southcentralus`, `gpt-4o-2024-05-13:westus3`, `gpt-4o-2024-05-13:westus`, `gpt-4-turbo-2024-04-09:eastus2`, `gpt-4-0125-preview:eastus`, `gpt-4-0125-preview:northcentralus`, `gpt-4-0125-preview:southcentralus`, `gpt-4-1106-preview:australiaeast`, `gpt-4-1106-preview:canadaeast`, `gpt-4-1106-preview:france`, `gpt-4-1106-preview:india`, `gpt-4-1106-preview:norway`, `gpt-4-1106-preview:swedencentral`, `gpt-4-1106-preview:uk`, `gpt-4-1106-preview:westus`, `gpt-4-1106-preview:westus3`, `gpt-4-0613:canadaeast`, `gpt-3.5-turbo-0125:canadaeast`, `gpt-3.5-turbo-0125:northcentralus`, `gpt-3.5-turbo-0125:southcentralus`, `gpt-3.5-turbo-1106:canadaeast`, `gpt-3.5-turbo-1106:westus`, `gpt-4.1:australiaeast`, `gpt-4o:australiaeast`, `gpt-5.4-mini:australiaeast`
- `messages` (list of EvalOpenAiModelMessagesItems, required) — These are the messages which will instruct the AI Judge on how to evaluate the assistant message. The LLM-Judge must respond with "pass" or "fail" to indicate if the assistant message passes the eval. To access the messages in the mock conversation, use the LiquidJS variable `{{messages}}`. The assistant message to be evaluated will be passed as the last message in the `messages` array and can be accessed using `{{messages[-1]}}`. It is recommended to use the system message to instruct the LLM how to evaluate the assistant message, and then use the first user message to pass the assistant message to be evaluated.
- `temperature` (double, optional) — This is the temperature of the model. For LLM-as-a-judge, it's recommended to set it between 0 - 0.3 to avoid hallucinations and ensure the model judges the output correctly based on the instructions.
- `maxTokens` (double, optional) — This is the max tokens of the model. If your Judge instructions return `true` or `false` takes only 1 token (as per the OpenAI Tokenizer), and therefore is recommended to set it to a low number to force the model to return a short response.

### EvalAnthropicModel

Anthropic model configuration for an LLM judge, including its messages, generation settings, and optional extended thinking.

- `provider` (enum, required) — This is the provider of the model (`anthropic`).
  - Allowed values: `anthropic`
- `model` (enum, required) — This is the specific model that will be used.
  - Allowed values: `claude-3-opus-20240229`, `claude-3-sonnet-20240229`, `claude-3-haiku-20240307`, `claude-3-5-sonnet-20240620`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`, `claude-3-7-sonnet-20250219`, `claude-opus-4-20250514`, `claude-opus-4-5-20251101`, `claude-opus-4-6`, `claude-sonnet-4-20250514`, `claude-sonnet-4-5-20250929`, `claude-sonnet-4-6`, `claude-sonnet-5`, `claude-haiku-4-5-20251001`, `claude-haiku-5-5`
- `messages` (list of EvalAnthropicModelMessagesItems, required) — These are the messages which will instruct the AI Judge on how to evaluate the assistant message. The LLM-Judge must respond with "pass" or "fail" to indicate if the assistant message passes the eval. To access the messages in the mock conversation, use the LiquidJS variable `{{messages}}`. The assistant message to be evaluated will be passed as the last message in the `messages` array and can be accessed using `{{messages[-1]}}`. It is recommended to use the system message to instruct the LLM how to evaluate the assistant message, and then use the first user message to pass the assistant message to be evaluated.
- `thinking` (AnthropicThinkingConfig, optional) — This is the optional configuration for Anthropic's thinking feature. - If provided, `maxTokens` must be greater than `thinking.budgetTokens`.
- `temperature` (double, optional) — This is the temperature of the model. For LLM-as-a-judge, it's recommended to set it between 0 - 0.3 to avoid hallucinations and ensure the model judges the output correctly based on the instructions.
- `maxTokens` (double, optional) — This is the max tokens of the model. If your Judge instructions return `true` or `false` takes only 1 token (as per the OpenAI Tokenizer), and therefore is recommended to set it to a low number to force the model to return a short response.

### EvalGoogleModel

Google model configuration for an LLM judge, including its messages and generation settings.

- `provider` (enum, required) — This is the provider of the model (`google`).
  - Allowed values: `google`
- `model` (enum, required) — This is the name of the model. Ex. gpt-4o
  - Allowed values: `gemini-3.8-flash`, `gemini-3.5-flash`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite`, `gemini-2.0-flash-thinking-exp`, `gemini-2.0-pro-exp-02-05`, `gemini-2.0-flash`, `gemini-2.0-flash-lite`, `gemini-2.0-flash-exp`, `gemini-2.0-flash-realtime-exp`, `gemini-1.5-flash`, `gemini-1.5-flash-002`, `gemini-1.5-pro`, `gemini-1.5-pro-002`, `gemini-1.0-pro`
- `messages` (list of EvalGoogleModelMessagesItems, required) — These are the messages which will instruct the AI Judge on how to evaluate the assistant message. The LLM-Judge must respond with "pass" or "fail" to indicate if the assistant message passes the eval. To access the messages in the mock conversation, use the LiquidJS variable `{{messages}}`. The assistant message to be evaluated will be passed as the last message in the `messages` array and can be accessed using `{{messages[-1]}}`. It is recommended to use the system message to instruct the LLM how to evaluate the assistant message, and then use the first user message to pass the assistant message to be evaluated.
- `temperature` (double, optional) — This is the temperature of the model. For LLM-as-a-judge, it's recommended to set it between 0 - 0.3 to avoid hallucinations and ensure the model judges the output correctly based on the instructions.
- `maxTokens` (double, optional) — This is the max tokens of the model. If your Judge instructions return `true` or `false` takes only 1 token (as per the OpenAI Tokenizer), and therefore is recommended to set it to a low number to force the model to return a short response.

### EvalCustomModel

OpenAI-compatible custom model configuration for an LLM judge, including its endpoint, headers, messages, and generation settings.

- `provider` (enum, required) — This is the provider of the model (`custom-llm`).
  - Allowed values: `custom-llm`
- `url` (string, required) — These is the URL we'll use for the OpenAI client's `baseURL`. Ex. https://openrouter.ai/api/v1
- `model` (string, required) — This is the name of the model. Ex. gpt-4o
- `messages` (list of EvalCustomModelMessagesItems, required) — These are the messages which will instruct the AI Judge on how to evaluate the assistant message. The LLM-Judge must respond with "pass" or "fail" to indicate if the assistant message passes the eval. To access the messages in the mock conversation, use the LiquidJS variable `{{messages}}`. The assistant message to be evaluated will be passed as the last message in the `messages` array and can be accessed using `{{messages[-1]}}`. It is recommended to use the system message to instruct the LLM how to evaluate the assistant message, and then use the first user message to pass the assistant message to be evaluated.
- `headers` (EvalCustomModelHeaders, optional) — These are the headers we'll use for the OpenAI client's `headers`.
- `timeoutSeconds` (double, optional) — This sets the timeout for the connection to the custom provider without needing to stream any tokens back. Default is 20 seconds.
- `temperature` (double, optional) — This is the temperature of the model. For LLM-as-a-judge, it's recommended to set it between 0 - 0.3 to avoid hallucinations and ensure the model judges the output correctly based on the instructions.
- `maxTokens` (double, optional) — This is the max tokens of the model. If your Judge instructions return `true` or `false` takes only 1 token (as per the OpenAI Tokenizer), and therefore is recommended to set it to a low number to force the model to return a short response.

### EvalOpenAiModelMessagesItems

### EvalAnthropicModelMessagesItems

### AnthropicThinkingConfig

Enables Anthropic extended thinking with a maximum thinking-token budget.

- `type` (enum, required) — Enables Anthropic extended thinking.
  - Allowed values: `enabled`
- `budgetTokens` (double, required) — The maximum number of tokens to allocate for thinking. Must be between 1024 and 100000 tokens.

### EvalGoogleModelMessagesItems

### EvalCustomModelMessagesItems

### EvalCustomModelHeaders

These are the headers we'll use for the OpenAI client's `headers`.

## Examples

**Request**

```json
{
  "messages": [
    {
      "content": "You are a helpful assistant.",
      "role": "system"
    }
  ],
  "type": "chat.mockConversation"
}
```

**Response**

```json
{
  "messages": [
    {
      "content": "You are a helpful assistant.",
      "role": "system"
    }
  ],
  "id": "string",
  "orgId": "string",
  "createdAt": "2024-01-15T09:30:00Z",
  "updatedAt": "2024-01-15T09:30:00Z",
  "type": "chat.mockConversation",
  "name": "Verified User Flow Eval",
  "description": "This eval checks if the user flow is verified."
}
```