Roster Import Queue

Uploading a roster used to mean one long web request: the server created every account before the spinners stopped, and a hiccup halfway through lost the batch. Now the upload becomes a background job: Create saves the rows and returns within a second, and the real work happens afterwards. A spreadsheet that already uses the roster's own columns is kept as it is. One that uses the agency's own names — a creator column, an Instagram handle, a booking email — is lined up with those fields first, so the preview still shows each person with their name, photo, email, and social links. Two tables in the app's Postgres are the source of truth for all of it — roster_import_job (the upload as a whole) and roster_import_item (one row per talent, with its stage, attempts and errors). Nothing important lives only in memory.

Each talent goes live in two steps. First create: the account is created so the talent appears in the rail right away, and the short summary of who they are is written with the lighter model. Then enrich: the worker reads the agency's profile page and files a short profile into the talent's memory in that same read, so the agent knows who it represents. A row with no profile URL skips enrich. A row whose file carries its own ready-made Upload Summary skips the read too: that text is saved as the talent's profile summary, and the profile link is kept on the talent but never opened. That row finishes as done. It does not finish as "Profile page not read". Talent uploads only — a brand spreadsheet ignores the column.

A talent file can also include Upload Summary. Spacing and capitalization in that header do not matter. When that cell has text, the import saves it as the talent's profile summary and does not open the profile link, even if the link cell is filled in. The link stays on the talent. An empty summary, or a file without that column, still reads a filled-in profile link. A brand roster ignores the column. A row that used the uploaded summary finishes as done, with that summary on the talent's profile. It does not finish as "Profile page not read".

A talent file can also include an Upload Summary column. Spacing, punctuation, and capitalization do not matter. When that cell has text, the import saves it as the talent's profile summary and does not open the profile link, even when the link is filled in. The link stays on the talent. An empty summary, or a file without that column, still reads a filled-in profile link. A brand roster ignores the column. A row that used the uploaded summary finishes as done, with that summary in place.

Failures are not fatal. A failed create goes back to pending and is retried up to three times with longer waits between attempts — 30 seconds, then 2 minutes, then 10 minutes — and after the third retry fails it is marked failed for good. The import still finishes; the failed rows are listed in the pill with Retry. A profile page that can't be read never fails the talent: the account already exists, so the row finishes with "Profile page not read".

Crash-safe by construction: before creating anything the worker checks the agency's Talent ID slug. If a previous attempt (or a crash mid-write) already created that person, the existing account is reused, never duplicated — a retry after a crash cannot produce doubles.

The queue

One job runs with K workers in parallel — 5 by default, overridable with the ROSTER_IMPORT_WORKERS env var, capped at 8. Workers claim rows with FOR UPDATE SKIP LOCKED, the Postgres pattern that makes two workers grab different rows even when they claim in the same instant, so two workers can never take the same talent. One agency runs at most 2 imports at a time; a third Create while two are still running gets a 409 instead of a queue nobody watches.

Every worker is its own HTTP request to POST /api/account/managed/roster/worker (Bearer CRON_SECRET), kicked fire-and-forget — never awaited — the same way image-candidates/start kicks its processor. A worker is never a promise left running inside another request: the serverless runtime freezes those the moment the response is sent.

Each worker claims one item at a time, does its create or enrich step, and stops when nothing is claimable or the job was cancelled. When it reaches its 240 s budget (the route allows 300 s) with rows still waiting, it kicks a fresh worker request for the same job and returns, so a large roster never depends on one request.

The tick

Everything is restarted by one alarm clock:

EventBridge (every minute)
  → roster-import tick Lambda (packages/z-agent/srv/infra/roster-import-lambda)
    → POST /api/account/managed/roster/tick   (Authorization: Bearer $CRON_SECRET)

The tick route does three things: it frees claims older than six minutes (a crashed worker's rows go back to pending — a row that has already stopped its worker three times is marked failed instead, so one bad row can't loop forever), it kicks one new worker request for every missing worker of every job that still has claimable rows, and it finishes jobs whose rows are all settled. It is idempotent — running it twice or while another tick is still running changes nothing, because the database decides who claims what.

EventBridge is only the alarm clock; the DB is the truth. Workers started by the Create request do the first work; the tick's job is to restart work that stopped — after a deploy, a crash, or a silent cold start. If a tick fails or a minute is skipped, nothing is lost: the next minute's run finds the same pending rows and continues. A tick that errors returns a 500, the Lambda throws so the run is visible, and the next minute's fire simply retries.

Local testing

Local dev needs none of the AWS machinery — the Create request starts workers itself, and EventBridge only keeps long jobs alive. To trigger a tick by hand (the exact thing the Lambda does every minute):

curl -X POST http://localhost:3000/api/account/managed/roster/tick \
  -H "Authorization: Bearer $CRON_SECRET"

The response reports how many stale claims were freed and which jobs got workers. CRON_SECRET is the app's own env var; never print it.

In the UI you follow the import through the dashboard pill in the header while it runs ("Adding Outshine roster · 42 of 73 talent", then "Reading profiles · 50 of 73"), and the done line when it ends: "73 talent are live." (the number actually created; "brands" for a brand roster). Expanded, it shows added / profiles read / waiting / failed, the failed names with Retry, and Stop (with a confirm) while it runs. It polls every 3 s while an import runs and every 30 s otherwise, and refreshes the roster rail as people are created. Up to two imports show as two pills.

Deploying the alarm clock

The Lambda and its schedule live in packages/z-agent/srv/infra/roster-import-lambda/:

./deploy.sh                  # all three environments
./deploy.sh all --vercel     # also write the Z_AGENT_ROSTER_TICK_* vars to Vercel

It needs one env var pair per environment: Z_AGENT_ROSTER_TICK_SECRET (the app's CRON_SECRET) and Z_AGENT_ROSTER_CALLBACK_URL (the full tick-route URL). See that folder's README for the environment table.

The clock is live in all three environments: one function (zooly-roster-import-tick-dev/-stage/-prod) and one enabled rate(1 minute) schedule per schedule group, all created by the script above (it is idempotent to re-run). Two checks after a deploy:

# the schedule exists and is enabled
aws --profile zooly --region us-east-2 scheduler get-schedule \
  --name zooly-roster-import-tick --group-name zooly-roster-import-prod \
  --query 'State' --output text          # -> ENABLED

# fire it once by hand; {"statusCode":200} means the app answered 2xx
aws --profile zooly --region us-east-2 lambda invoke \
  --function-name zooly-roster-import-tick-prod \
  --payload '{}' --cli-binary-format raw-in-base64-out out.json

A healthy fire logs the app's answer in the function's CloudWatch log: roster tick: {"status":"success","data":{"freedStaleClaims":0,...,"jobs":[]}}. A 405 means the target app predates the tick route — deploy the app before (or right after) wiring its schedule, or every minute's fire fails until it ships. Failed fires are harmless to the queue — the DB is the truth — but they show up as failed invocations in CloudWatch.