POST /chat/completions).
Model
- Key:
model - Type:
string - Requirement:
Required - Description: The ID of the model to use. You can specify a specific model key (e.g.,
"openai/gpt-4o-mini","claude/claude-3-5-sonnet") or use"auto"to let the Semantic Routing Engine dynamically choose the best model.
Messages
- Key:
messages - Type:
array of objects - Requirement:
Required - Description: A list of messages comprising the conversation so far. Each message object contains:
role(string):"system","user", or"assistant".content(string): The text content of the message.
Temperature
- Key:
temperature - Type:
float(0.0 to 2.0) - Default:
1.0 - Description: Influences the creativity and variety of the model’s responses. Lower values (like
0.2) make responses more focused, predictable, and deterministic. Higher values (like1.5) encourage more creative, diverse, and less typical output. At0.0, the model always gives the same response for a given input.
Max Tokens
- Key:
max_tokens - Type:
integer - Default:
500(defaults to 500 at the gateway layer if omitted) - Description: The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model’s context length.
Stream
- Key:
stream - Type:
boolean - Default:
false - Description: If set to
true, token chunks will be sent as data-only server-sent events (SSE) as they become available, instead of waiting for the full response to compile.
Top P
- Key:
top_p - Type:
float(0.0 to 1.0) - Default:
1.0 - Description: An alternative to sampling with temperature, called nucleus sampling. The model considers the results of the tokens with
top_pprobability mass. For example,0.1means only the tokens comprising the top 10% probability mass are considered.
Presence Penalty
- Key:
presence_penalty - Type:
float(-2.0 to 2.0) - Default:
0.0 - Description: Penalizes new tokens based on whether they appear in the text so far. Positive values increase the model’s likelihood to talk about new topics.
Frequency Penalty
- Key:
frequency_penalty - Type:
float(-2.0 to 2.0) - Default:
0.0 - Description: Penalizes new tokens based on their existing frequency in the text so far. Decreases the model’s likelihood to repeat the exact same phrases.