Guides/Batching videos

Batching videos

Submit many videos against one schema and track them under a single batch ID.

Why batches

There is no separate batch endpoint. videos is a list on every submission, so batching means passing a longer one — nothing about the call changes between one video and two hundred.

It is worth doing. Each video is still extracted on its own, so one video without captions cannot fail the other ninety-nine, but sending them together lets us fetch caption tracks for the whole list in one upstream call. That is where most of the wall-clock time in a large run goes: measured on a 100-video run, batching the fetches took the average from 30–40 seconds a video down to about 15.7.

Submitting

POST /v1
{
"videos": [
"https://youtube.com/watch?v=…",
"https://youtube.com/watch?v=…"
],
"schema": {
"columns": [
{
"name": "city",
"description": "A city the speaker mentions visiting"
}
],
"multiple": true
}
}
201 · created
{
"job_id": "b3f1c0d2-77a4",
"videos": 2
}
NOTE
One id comes back, however many videos went in. That id is what you read, and the only thing you need to keep.

Reading progress

One id means one call to poll, not one per video. counts shows where the job has got to; when queued and processing reach zero the job has settled and results arrives with it.

GET /v1/{job_id}
{
"status": "processing",
"counts": {
"queued": 4,
"processing": 2,
"done": 93,
"failed": 1
},
"results": null
}

The 200 limit

A single call carries at most 200 videos. Over that it is refused outright with batch_too_large and the message 200 video limit per batch exceeded — nothing is queued and nothing is charged, so splitting the list and resubmitting is safe. Split on your side into calls of 200 and run them alongside each other.

Results arrive together

A job hands over its rows once every video in it has settled, not one at a time. That is why polling costs nothing: while anything is still running, results is null and there is nothing to consume by accident.

The consequence for a long run is that the whole list is handed over in one response. If you want rows sooner than that, send smaller calls — fifty videos at a time settles five times sooner than two hundred, at the same price per video.

CAREFUL
The handover happens on one read and one only. Persist in the same code path that reads it.
Was this page useful?Tell us what was missing