Upload API

Use upload sessions to send files directly into the Calypso RAG indexing pipeline with a project API key.

What the Upload API is for

The Upload API turns knowledge ingestion into an automatable workflow. Use it when source material is produced by another system, CI job, crawler, MCP tool, or back-office workflow and should land directly in the Calypso RAG indexing pipeline.

Public uploads use upload sessions only. The client creates a small JSON session, uploads bytes directly to storage, finalizes the session, then polls readiness.

Core flow

Create a single-file or batch upload session with a project API key.
Upload each file directly to the returned upload_url with PUT.
Finalize the single session or batch session with JSON.
Treat the finalize response as durable acceptance, not immediate retrieval readiness.
Poll file, task, or batch status until indexing and bucket sync settle.

Authorization

Use a project API key as a bearer token:

Authorization: Bearer sk_...

To discover valid bucket destinations before upload, call GET /v1/knowledge/buckets with the same project API key. The response is scoped to the key's team and returns bucket ids, slugs, status, counts, and bucket-store readiness.

Single-file upload session

Create a session:

POST /v1/knowledge/files/upload-session
Authorization: Bearer sk_...
Content-Type: application/json
Idempotency-Key: your-stable-upload-key

{
  "filename": "contract.pdf",
  "content_type": "application/pdf",
  "size_bytes": 184233,
  "title": "Customer Contract",
  "tags": ["legal", "customer"],
  "metadata": { "external_id": "crm_123", "source": "local_cli" },
  "bucket": "contracts",
  "create_missing_buckets": true
}

Typical create response:

{
  "session_id": "sess_123",
  "upload_strategy": "gcs_resumable",
  "upload_url": "https://storage.googleapis.com/...",
  "expires_at": "2026-06-08T21:00:00Z",
  "request_id": "req_current"
}

Upload bytes to the returned URL:

curl -X PUT "$UPLOAD_URL" \
  -H "Content-Type: application/pdf" \
  -H "Content-Range: bytes 0-184232/184233" \
  --data-binary @/absolute/path/to/contract.pdf

Finalize:

POST /v1/knowledge/files/upload-session/{session_id}/finalize
Authorization: Bearer sk_...
Content-Type: application/json

{}

Typical finalize response:

{
  "id": "file_123",
  "object": "knowledge_file",
  "status": "queued",
  "title": "Customer Contract",
  "filename": "contract.pdf",
  "content_type": "application/pdf",
  "size_bytes": 184233,
  "sha256": "abc123...",
  "task": {
    "id": "task_123",
    "type": "gemini_file_search_indexing",
    "status": "queued"
  },
  "bucketSyncStatus": "queued",
  "request_id": "req_current"
}

Batch upload session

Create a batch session:

POST /v1/knowledge/files:batch/upload-session
Authorization: Bearer sk_...
Content-Type: application/json

{
  "manifest": {
    "version": 1,
    "batch_idempotency_key": "local-batch-001",
    "bucket": "rag1",
    "create_missing_buckets": true,
    "items": [
      { "client_file_id": "contract_pdf", "filename": "contract.pdf", "title": "Customer Contract" },
      { "client_file_id": "invoice_pdf", "filename": "invoice.pdf", "title": "Customer Invoice" }
    ]
  },
  "files": [
    { "client_file_id": "contract_pdf", "filename": "contract.pdf", "content_type": "application/pdf", "size_bytes": 184233 },
    { "client_file_id": "invoice_pdf", "filename": "invoice.pdf", "content_type": "application/pdf", "size_bytes": 91234 }
  ]
}

Upload every accepted item to its returned upload_url, then finalize:

POST /v1/knowledge/files:batch/upload-session/{batch_id}/finalize
Authorization: Bearer sk_...
Content-Type: application/json

{
  "mode": "finalize_uploaded"
}

Typical batch finalize response:

{
  "batch_id": "batch_123",
  "status": "partial",
  "finalized": [
    { "client_file_id": "contract_pdf", "session_id": "sess_1", "knowledge_id": "file_123", "task_id": "task_123" }
  ],
  "pending": [
    { "client_file_id": "invoice_pdf", "session_id": "sess_2", "reason": "blob_not_uploaded" }
  ],
  "failed": [],
  "replayed": [],
  "request_id": "req_current"
}

Call finalize again after uploading pending items. Already-finalized items are replayed instead of duplicated.

Status endpoints

Poll file status:

GET /v1/knowledge/files/{file_id}

Poll task status:

GET /v1/knowledge/tasks/{task_id}

Poll batch status:

GET /v1/knowledge/batches/{batch_id}?include_items=true

Bucket requirements

Every upload must choose a bucket destination. Use one of:

bucket_ids: stable existing bucket ids.
bucket_slugs: existing bucket slugs.
bucket: shorthand for one bucket slug.
create_missing_buckets: allow slug-based provisioning during session creation.

If no destination is provided, Calypso rejects the session create request with bucket_required.

Readiness

Finalize responses mean the files were stored and indexing was queued. They do not mean the files are immediately retrievable.

For reliable tests, wait until the file or batch item becomes active or indexed, and for bucket-scoped retrieval wait until bucketSyncStatus is active.

Limits and best practices

Keep batch size at or below 100 files.
Respect the 5 create requests per second per team rate limit.
Use stable idempotency keys for retries.
Generate deterministic client_file_id values when retrying the same batch.
Treat upload_url as a short-lived bearer capability and do not log it.