Embeddings

POST /v1/embeddings takes a model and an input (one string, an array of strings, or an array of token arrays), and returns a list of embedding objects with the token usage alongside. It is OpenAI's request and OpenAI's response, so an OpenAI client's embeddings.create reaches it with nothing changed but the base URL and the model slug.

A request

curl https://api.routehook.ai/v1/embeddings \
  -H "Authorization: Bearer $ROUTEHOOK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/text-embedding-3-small",
    "input": ["first document", "second document"]
  }'

Request parameters

The response

{
  "object": "list",
  "model": "openai/text-embedding-3-small",
  "data": [
    { "object": "embedding", "index": 0, "embedding": [0.0021, -0.0138, 0.0074] },
    { "object": "embedding", "index": 1, "embedding": [0.0019, -0.0142, 0.0069] }
  ],
  "usage": { "prompt_tokens": 6, "total_tokens": 6 }
}

Batching

Pass an array and every item is embedded in one call. The results come back in data with an index naming the input each one came from. Rely on index rather than array position if anything in your pipeline reorders. Batching saves round trips and nothing else: the bill is the sum of the tokens either way, so a batch of fifty costs what fifty separate calls cost, minus fifty HTTP handshakes.

Limits worth knowing

Each item is embedded independently, so the model's context length applies to each one and not to the batch. A single document longer than the model's window is a 400, however small the batch. Chunk long documents before you send them. Very large batches are also one all-or-nothing request: if it fails you retry the whole thing, which is a reason to keep batches in the hundreds rather than the tens of thousands.

float or base64

encoding_format: "float" returns each vector as a JSON array of numbers. It is readable and it is large. A 1536-dimension vector is around 20KB of JSON. encoding_format: "base64" returns the same vector as a base64 string of little-endian float32s, roughly a third of the bytes and no decimal parsing on your side. Use base64 for anything at volume; use float when you are reading the response with your eyes.

Decoding base64 vectors

const response = await client.embeddings.create({
  model: "openai/text-embedding-3-small",
  input: ["first document"],
  encoding_format: "base64",
});

const bytes = Buffer.from(response.data[0].embedding, "base64");
const vector = new Float32Array(
  bytes.buffer,
  bytes.byteOffset,
  bytes.length / 4,
);

console.log(vector.length, vector[0]);

dimensions

dimensions truncates the vector, on models that support it. The shorter vector is still usable for similarity, and costs less to store and to search. Not every model offers it. Where one does not, the parameter is forwarded and the upstream decides what to do with it, so check the width you got back before you write it into an index that expects a fixed size.

Cost

Embeddings bill on one axis: the tokens you send. There is no output side, which is why usage carries prompt_tokens and total_tokens and no completion count. The published rate is on the model in GET /v1/models; what a specific call was charged is in GET /v1/generation?id=req_…, keyed on the request id returned in the X-Routehook-Request-Id header.

Failures

Related