Endpoint: Batch Processing

Submit high-volume, non-time-sensitive workloads at 50% cost savings by leveraging provider-native batch APIs (OpenAI, Anthropic, Gemini). VeriPrompt routes your batch to the optimal provider or splits it across multiple providers for cost optimization.

Base URL: https://<your-domain>/api/v1
Content-Type: application/json or multipart/form-data
Authentication: Session cookie or API key

Core Concepts

Batch Lifecycle

QUEUED -> VALIDATING -> PROCESSING -> COMPLETED
                                   -> FAILED
                                   -> CANCELLED
                                   -> EXPIRED

Status	Description
`QUEUED`	Job created, waiting for background processor to pick it up
`VALIDATING`	Items being validated and formatted for the provider
`PROCESSING`	Submitted to provider, awaiting results
`COMPLETED`	All items processed (some may have individual errors)
`FAILED`	Job-level failure (e.g., no provider found, auth error)
`CANCELLED`	Cancelled by user
`EXPIRED`	Results expired after TTL (default 30 days)

Item Status

Each item within a batch has its own status:

Status	Description
`PENDING`	Awaiting processing
`PROCESSING`	Currently being processed
`SUCCESS`	Completed successfully
`ERROR`	Failed with error
`SKIPPED`	No result received from provider

Split Strategies

When multiple providers are available, batches can be split:

Strategy	Behavior
`single`	All items sent to the best provider (default)
`round-robin`	Items distributed evenly across available providers
`cost-optimal`	More items allocated to cheaper providers

POST /api/v1/batch

Submit a new batch job. Accepts either JSON body with inline items or multipart form data with a JSONL file.

JSON Request Body

typescript

{
  items: Array<{
    id: string;          // Unique ID for this item (used in results)
    prompt: string;      // The prompt text (required)
    systemPrompt?: string;
    variables?: Record<string, string>; // For {{variable}} substitution
  }>;
  config?: {
    name?: string;                    // Display name for the batch
    routingPolicyId?: string;         // Use a specific routing policy
    qualityTier?: "cheap" | "fast" | "quality" | "secure";
    providerHint?: string;            // Prefer this provider (e.g., "openai")
    model?: string;                   // Specific model (e.g., "gpt-4o")
    sanitization?: boolean;           // Enable PII sanitization
    splitStrategy?: "single" | "round-robin" | "cost-optimal";
    callbackUrl?: string;             // Webhook URL for completion notification
    metadata?: Record<string, any>;   // Custom metadata
  };
}

Multipart Form Data

Field	Type	Description
`file`	File	JSONL file (one JSON object per line), up to 100MB
`config`	String	JSON string with configuration options (same as `config` above)

JSONL File Format

Each line must be a valid JSON object with at least id and prompt:

jsonl

{"id": "req-1", "prompt": "Summarize this article in 3 sentences.", "systemPrompt": "You are a helpful assistant."}
{"id": "req-2", "prompt": "Translate to French: Hello, how are you?"}
{"id": "req-3", "prompt": "Extract entities from: {{text}}", "variables": {"text": "Apple was founded by Steve Jobs."}}

Response (201 Created)

json

{
  "batchId": "cm1abc123def456",
  "status": "QUEUED",
  "totalItems": 3,
  "estimatedCostUSD": null,
  "createdAt": "2026-03-05T12:00:00.000Z"
}

Example: cURL (JSON inline)

bash

curl -X POST https://app.veriprompt.tech/api/v1/batch \
  -H "Content-Type: application/json" \
  -H "Cookie: next-auth.session-token=YOUR_SESSION" \
  -d '{
    "items": [
      {"id": "q1", "prompt": "What is 2+2?"},
      {"id": "q2", "prompt": "Capital of France?"},
      {"id": "q3", "prompt": "Explain quantum computing in one sentence."}
    ],
    "config": {
      "name": "eval-run-2026-03",
      "qualityTier": "cheap",
      "splitStrategy": "single"
    }
  }'

Example: cURL (JSONL file upload)

bash

curl -X POST https://app.veriprompt.tech/api/v1/batch \
  -H "Cookie: next-auth.session-token=YOUR_SESSION" \
  -F "file=@batch_requests.jsonl" \
  -F 'config={"name":"file-upload-batch","providerHint":"openai"}'

Example: TypeScript

typescript

const response = await fetch('/api/v1/batch', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    items: [
      { id: 'item-1', prompt: 'Summarize this text: ...' },
      { id: 'item-2', prompt: 'Translate to Spanish: ...' },
    ],
    config: {
      name: 'my-batch-job',
      qualityTier: 'cheap',
      callbackUrl: 'https://myapp.com/webhook/batch-complete',
    },
  }),
});

const { batchId, status, totalItems } = await response.json();
console.log(`Batch ${batchId} created with ${totalItems} items`);

Error Responses

Status	Error	Condition
`400`	Missing items	No `items` array or no `file` in form data
`400`	Validation failed	Items exceed max count (50,000) or file exceeds size limit
`400`	Invalid config	Config fields fail validation
`401`	Unauthorized	No valid session or API key
`429`	CONCURRENT_LIMIT	Too many active batch jobs (default max: 10)

GET /api/v1/batch

List batch jobs for the authenticated user's company.

Query Parameters

Parameter	Type	Default	Description
`status`	string	—	Filter by status (e.g., `COMPLETED`, `PROCESSING`)
`limit`	number	20	Max results (capped at 100)
`offset`	number	0	Pagination offset

Response (200)

json

{
  "jobs": [
    {
      "id": "cm1abc123def456",
      "name": "eval-run-2026-03",
      "status": "COMPLETED",
      "totalItems": 100,
      "completedItems": 98,
      "failedItems": 2,
      "providerName": "openai",
      "estimatedCostUSD": 0.0250,
      "actualCostUSD": 0.0243,
      "createdAt": "2026-03-05T12:00:00.000Z",
      "startedAt": "2026-03-05T12:00:05.000Z",
      "completedAt": "2026-03-05T12:05:30.000Z"
    }
  ],
  "total": 15,
  "limit": 20,
  "offset": 0
}

Example: cURL

bash

curl "https://app.veriprompt.tech/api/v1/batch?status=COMPLETED&limit=10" \
  -H "Cookie: next-auth.session-token=YOUR_SESSION"

GET /api/v1/batch/:batchId

Get detailed information about a specific batch job.

Response (200)

json

{
  "batchId": "cm1abc123def456",
  "name": "eval-run-2026-03",
  "status": "COMPLETED",
  "totalItems": 100,
  "completedItems": 98,
  "failedItems": 2,
  "progress": 1.0,
  "providerName": "openai",
  "providerModel": "gpt-4o-mini",
  "splitStrategy": "single",
  "qualityTier": "cheap",
  "sanitizationEnabled": false,
  "estimatedCostUSD": 0.0250,
  "actualCostUSD": 0.0243,
  "inputTokensTotal": 15000,
  "outputTokensTotal": 12000,
  "createdAt": "2026-03-05T12:00:00.000Z",
  "startedAt": "2026-03-05T12:00:05.000Z",
  "completedAt": "2026-03-05T12:05:30.000Z",
  "expiresAt": "2026-04-04T12:00:05.000Z",
  "errorMessage": null,
  "splits": [
    {
      "id": "split-1",
      "providerName": "openai",
      "providerModel": "gpt-4o-mini",
      "itemCount": 100,
      "status": "COMPLETED",
      "costUSD": 0.0243
    }
  ],
  "metadata": {}
}

Error Responses

Status	Error	Condition
`404`	Not found	Batch ID doesn't exist or belongs to different company
`401`	Unauthorized	No valid session

GET /api/v1/batch/:batchId/results

Download results for a completed batch. Supports JSON and JSONL formats.

Query Parameters

Parameter	Type	Default	Description
`status`	string	—	Filter items by status (`SUCCESS`, `ERROR`, `SKIPPED`)
`limit`	number	—	Limit number of results
`offset`	number	0	Pagination offset

Headers

Header	Value	Effect
`Accept`	`application/json`	Returns JSON array (default)
`Accept`	`application/jsonl`	Returns JSONL file with `Content-Disposition` header

JSON Response (200)

json

{
  "items": [
    {
      "externalId": "q1",
      "sequenceIndex": 0,
      "status": "SUCCESS",
      "response": "2+2 equals 4.",
      "inputTokens": 12,
      "outputTokens": 8,
      "costUSD": 0.0001
    },
    {
      "externalId": "q2",
      "sequenceIndex": 1,
      "status": "ERROR",
      "errorCode": "RATE_LIMIT",
      "errorMessage": "Provider rate limit exceeded"
    }
  ],
  "total": 100,
  "limit": 100,
  "offset": 0
}

Example: Download JSONL

bash

curl "https://app.veriprompt.tech/api/v1/batch/cm1abc123/results" \
  -H "Accept: application/jsonl" \
  -H "Cookie: next-auth.session-token=YOUR_SESSION" \
  -o results.jsonl

POST /api/v1/batch/:batchId/cancel

Cancel an active batch job. Works for jobs in QUEUED, VALIDATING, or PROCESSING status.

Response (200)

json

{
  "batchId": "cm1abc123def456",
  "status": "CANCELLED"
}

Error Responses

Status	Error	Condition
`400`	Cannot cancel	Job is already completed, cancelled, or expired
`404`	Not found	Batch ID doesn't exist

Background Processing

Batch jobs are processed by a background cron job that runs every 60 seconds:

POST /api/internal/batch/process
x-internal-job: <INTERNAL_JOB_TOKEN>

The processor:

Picks up QUEUED jobs (up to 5 at a time)
Resolves the optimal provider from the company's provider group or BYOK credentials
Submits items to the provider's native batch API
Polls PROCESSING jobs for completion
Fetches and stores results on completion
Fires webhook if callbackUrl was configured
Logs usage to billing (UsageLog + CompanyPackage)

Webhook Notification

When a batch completes and callbackUrl was set, VeriPrompt sends:

json

POST <callbackUrl>
Content-Type: application/json

{
  "batchId": "cm1abc123def456",
  "status": "COMPLETED",
  "completedItems": 98,
  "failedItems": 2,
  "actualCostUSD": 0.0243,
  "resultsUrl": "https://app.veriprompt.tech/api/v1/batch/cm1abc123def456/results"
}

Configuration

Environment Variable	Default	Description
`BATCH_MAX_FILE_SIZE_MB`	100	Maximum JSONL upload size in MB
`BATCH_MAX_ITEMS`	50000	Maximum items per batch
`BATCH_RESULTS_TTL_DAYS`	30	Auto-expire results after N days
`BATCH_MAX_CONCURRENT_JOBS`	10	Max active jobs per company
`BATCH_WEBHOOK_TIMEOUT_MS`	10000	Webhook delivery timeout
`BATCH_MAX_RETRY_ATTEMPTS`	2	Max retry attempts for failed items

Provider Support

Provider	Batch API	Discount	Max Items	Notes
OpenAI	Native (`/v1/batches`)	50%	50,000	24h completion window
Anthropic	Native (Message Batches)	50%	10,000	Streaming results
Gemini/Google	Concurrent execution	50%	50,000	20-request concurrency

Gateway Execute — Single prompt execution
Routing Policies — Policy-driven provider selection
Provider Management — Configure providers and API keys
Billing & Usage — Cost tracking and usage logs
PII Sanitization — Data protection for batch items

Endpoint: Batch Processing ​

Core Concepts ​

Batch Lifecycle ​

Item Status ​

Split Strategies ​

POST /api/v1/batch ​

JSON Request Body ​

Multipart Form Data ​

JSONL File Format ​

Response (201 Created) ​

Example: cURL (JSON inline) ​

Example: cURL (JSONL file upload) ​

Example: TypeScript ​

Error Responses ​

GET /api/v1/batch ​

Query Parameters ​

Response (200) ​

Example: cURL ​

GET /api/v1/batch/:batchId ​

Response (200) ​

Error Responses ​

GET /api/v1/batch/:batchId/results ​

Query Parameters ​

Headers ​

JSON Response (200) ​

Example: Download JSONL ​

POST /api/v1/batch/:batchId/cancel ​

Response (200) ​

Error Responses ​

Background Processing ​

Webhook Notification ​

Configuration ​

Provider Support ​

Related Docs ​

Endpoint: Batch Processing

Core Concepts

Batch Lifecycle

Item Status

Split Strategies

POST /api/v1/batch

JSON Request Body

Multipart Form Data

JSONL File Format

Response (201 Created)

Example: cURL (JSON inline)

Example: cURL (JSONL file upload)

Example: TypeScript

Error Responses

GET /api/v1/batch

Query Parameters

Response (200)

Example: cURL

GET /api/v1/batch/:batchId

Response (200)

Error Responses

GET /api/v1/batch/:batchId/results

Query Parameters

Headers

JSON Response (200)

Example: Download JSONL

POST /api/v1/batch/:batchId/cancel

Response (200)

Error Responses

Background Processing

Webhook Notification

Configuration

Provider Support

Related Docs