Cohere: Command R7B (12-2024)

Get your API key

Create a key on the keys page and set it as an environment variable: Keys

export INFAPI_API_KEY=sk-...

Make your first request

Use cohere/command-r7b-12-2024

import requests

response = requests.post(
    "https://infapi.cc/v1/chat/completions",
    headers={
        "Authorization": "Bearer $INFAPI_API_KEY",
        "Content-Type": "application/json",
    },
    json={
        "model": "cohere/command-r7b-12-2024",
        "messages": [
            {
                "role": "user",
                "content": "Introduce yourself in three sentences."
            }
        ],
        "temperature": 0.7,
        "max_tokens": 512
    },
)
response.raise_for_status()
print(response.json()["choices"][0]["message"]["content"])

Using third-party SDKs

Point the OpenAI SDK at https://infapi.cc/v1 with $INFAPI_API_KEY. The HTTP samples above are the request shape this gateway accepts.

from openai import OpenAI

client = OpenAI(
    base_url="https://infapi.cc/v1",
    api_key="$INFAPI_API_KEY",
)

response = client.chat.completions.create(
    model="cohere/command-r7b-12-2024",
    messages=[{"role": "user", "content": "Introduce yourself in three sentences."}],
    temperature=0.7,
    max_tokens=512,
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://infapi.cc/v1",
  apiKey: "$INFAPI_API_KEY",
});

const response = await client.chat.completions.create({
  "model": "cohere/command-r7b-12-2024",
  "messages": [
    {
      "role": "user",
      "content": "Introduce yourself in three sentences."
    }
  ],
  "temperature": 0.7
});

console.log(response.choices[0]?.message?.content);

Supported parameters

• frequency_penalty

  Frequency penalty

  Lowers the chance of a token in proportion to how often it already appeared.

• max_tokens

  Max tokens

  Upper bound on generated tokens. Output stops at the limit.

• presence_penalty

  Presence penalty

  Lowers the chance of any token that has already appeared.

• response_format

  Response format

  For example json_object or json_schema, to constrain the shape.

• seed

  Seed

  A fixed seed helps reproduce sampling. It does not guarantee byte-identical output.

• stop

  Stop

  Stop when these strings are produced. A string or an array of strings.

• structured_outputs

  Structured outputs

  Generate a parseable structure that follows a JSON Schema.

• temperature

  Temperature

  Sampling temperature. Lower is steadier, higher is more varied. Often 0 to 2.

• top_k

  Top-K

  Sample only from the K most likely tokens.

• top_p

  Top-p

  Sample only from tokens whose cumulative probability reaches top_p.