Skip to main content
POST

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
​
model
string
required

ID of the model to use

Example:

"meta-llama/Llama-3.3-70B-Instruct"

​
prompt
required

The prompt to generate completions for

​
best_of
integer<int32> | null
default:1
Example:

1

​
echo
boolean | null
default:false
Example:

false

​
frequency_penalty
number<float> | null

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far

Example:

0

​
logit_bias
object | null

Modify the likelihood of specified tokens appearing in the completion.

Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.

Example:
​
logprobs
integer<int32> | null

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability.

Example:

1

​
max_tokens
integer<int32> | null
default:16

The maximum number of tokens to generate in the chat completion

Example:

4096

​
n
integer<int32> | null

How many chat completion choices to generate for each input message

Example:

1

​
presence_penalty
number<float> | null

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far

Example:

0

​
seed
integer<int64> | null

If specified, our system will make a best effort to sample deterministically

Example:

123

​
stop
string[] | null

Up to 4 sequences where the API will stop generating further tokens

Example:

"json([\"stop\", \"halt\"])"

​
stream
boolean | null

Whether to stream back partial progress

Example:

false

​
stream_options
null | object

Options for streaming response. Only set this when you set stream: true.

​
suffix
string | null

The suffix that comes after a completion of inserted text.

Example:

"json(\"\\n\")"

​
temperature
number<float> | null

What sampling temperature to use, between 0 and 2

Example:

0.7

​
top_p
number<float> | null

An alternative to sampling with temperature

Example:

1

​
user
string | null

A unique identifier representing your end-user

Example:

"user-1234"

Response

Chat completions

​
choices
object[]
required

Array of completion choices response

Example:
​
created
integer<int64>
required

The creation time of the request

Example:

"2021-01-01T00:00:00.000Z"

​
id
string
required

The ID of the request

Example:

"cmpl-1234567890"

​
model
string
required

The model used for the request

Example:

"meta-llama/Llama-3.3-70B-Instruct"

​
object
string
required

The object type

Example:

"text_completion"

​
system_fingerprint
string
required

The system fingerprint

Example:

"system-fingerprint"

​
usage
object
required

The usage information for the request