perf(webapp): parallelize Phase 2 streaming batch-item ingest (TRI-10273)#3777
Open
matt-aitken wants to merge 2 commits into
Open
Conversation
|
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
💤 Files with no reviewable changes (1)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (3)
📜 Recent review details⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
🧰 Additional context used📓 Path-based instructions (7)**/*.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
{packages/core,apps/webapp}/**/*.{ts,tsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Files:
**/*.ts📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
Files:
apps/webapp/**/*.{ts,tsx}📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Files:
apps/webapp/**/*.server.ts📄 CodeRabbit inference engine (apps/webapp/CLAUDE.md)
Files:
**/*.{js,ts,tsx,jsx,css,json,md}📄 CodeRabbit inference engine (AGENTS.md)
Files:
🧠 Learnings (39)📓 Common learnings📚 Learning: 2026-05-17T08:07:25.757ZApplied to files:
📚 Learning: 2026-03-03T13:07:33.177ZApplied to files:
📚 Learning: 2026-05-17T08:07:36.624ZApplied to files:
📚 Learning: 2026-04-07T14:12:59.018ZApplied to files:
📚 Learning: 2026-04-13T21:44:00.032ZApplied to files:
📚 Learning: 2026-04-23T13:26:31.290ZApplied to files:
📚 Learning: 2026-04-07T14:12:18.946ZApplied to files:
📚 Learning: 2026-04-17T13:20:14.259ZApplied to files:
📚 Learning: 2026-05-05T09:38:02.512ZApplied to files:
📚 Learning: 2026-04-20T15:06:11.054ZApplied to files:
📚 Learning: 2026-05-12T06:43:12.346ZApplied to files:
📚 Learning: 2026-06-01T15:01:35.175ZApplied to files:
📚 Learning: 2026-04-27T16:39:43.098ZApplied to files:
📚 Learning: 2026-04-28T15:57:56.285ZApplied to files:
📚 Learning: 2026-06-06T19:34:44.129ZApplied to files:
📚 Learning: 2026-05-15T08:05:57.683ZApplied to files:
📚 Learning: 2026-05-12T14:13:17.114ZApplied to files:
📚 Learning: 2026-05-04T19:14:44.097ZApplied to files:
📚 Learning: 2026-06-06T19:34:45.521ZApplied to files:
📚 Learning: 2026-04-16T14:21:17.695ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-04-16T14:19:16.330ZApplied to files:
📚 Learning: 2026-06-02T21:20:43.541ZApplied to files:
📚 Learning: 2025-11-27T16:26:37.432ZApplied to files:
📚 Learning: 2025-11-27T16:26:37.432ZApplied to files:
📚 Learning: 2026-03-25T15:29:25.889ZApplied to files:
📚 Learning: 2026-03-22T13:26:12.060ZApplied to files:
📚 Learning: 2026-03-22T19:24:14.403ZApplied to files:
📚 Learning: 2026-05-18T08:21:27.694ZApplied to files:
📚 Learning: 2026-05-18T08:21:27.694ZApplied to files:
📚 Learning: 2026-05-12T21:04:05.815ZApplied to files:
📚 Learning: 2026-05-22T11:50:56.079ZApplied to files:
📚 Learning: 2026-06-04T18:16:35.386ZApplied to files:
🔇 Additional comments (8)
WalkthroughThis pull request refactors the Phase 2 streaming batch ingest endpoint to process NDJSON items with bounded concurrency rather than sequentially. It introduces a configurable Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
…273) Phase 2 of the v3 streaming batch API (POST /api/v3/batches/:batchId/items) processed streamed items strictly sequentially. For batches of many large payloads — each offloaded to object storage inline — this serialized N object-store round-trips inside one request, blowing past Node's default 300s server.requestTimeout. The webapp then returned 408, which the SDK reads as "408 terminated" and retries 5x, turning a slow ingest into a ~26-minute failure. Ingest now runs through p-map over the NDJSON async iterable with bounded concurrency (STREAMING_BATCH_INGEST_CONCURRENCY, default 10). p-map pulls lazily, so at most `concurrency` items are read/in-flight at once — bounding peak memory to roughly concurrency x STREAMING_BATCH_ITEM_MAXIMUM_SIZE while preserving stream backpressure. Set the env to 1 for fully sequential ingestion. Safe by construction: run order derives from each item's index (enqueue timestamp = batch.createdAt + index), and enqueueBatchItem dedups atomically per index — neither depends on processing order. The NDJSON parser now stamps oversized-item markers with their emit position, removing the consumer's sequential lastIndex assumption. The count-check + conditional seal path is unchanged. Tests: bounded-concurrency ingest of a 100-item batch, in-flight cap assertion, concurrent dedup on Phase 2 retry, and emit-position marker indexing. Full existing sealing/idempotency suite still green (42/42). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Enforce positive STREAMING_BATCH_INGEST_CONCURRENCY in the env schema (.int().positive()) — p-map requires concurrency >= 1, so 0/negative would throw at runtime. - Apply the same out-of-range index guard to oversized-item markers as normal items, so an oversized item with index >= runCount returns a 4xx instead of creating a stray pre-failed run. Covered by a new test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
matt-aitken
force-pushed
the
feature/tri-10273-managed-runs-streaming-batchtriggerandwait-fails-with-408
branch
from
4ca9807 to
de3489f
Compare
@trigger.dev/build
trigger.dev
@trigger.dev/core
@trigger.dev/plugins
@trigger.dev/python
@trigger.dev/react-hooks
@trigger.dev/redis-worker
@trigger.dev/rsc
@trigger.dev/schema-to-json
@trigger.dev/sdk
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Phase 2 of the v3 streaming batch API (
POST /api/v3/batches/:batchId/items) processed streamed items strictly sequentially. For a batch of many large payloads — each offloaded to object storage inline — this serialized N object-store round-trips inside a single request, exceeding Node's defaultserver.requestTimeout(300s). The webapp returned408, which the SDK reads as408 terminatedand retries 5×, turning a slow ingest into a ~26-minute failure (BatchTriggerError: Failed to stream items ... 408 terminated).Closes TRI-10273 — https://linear.app/triggerdotdev/issue/TRI-10273
Fix
Ingest now runs through
p-mapover the NDJSON async iterable with bounded concurrency (STREAMING_BATCH_INGEST_CONCURRENCY, default 10):p-mappulls lazily from the stream — at mostconcurrencyitems are read/in-flight at once, so peak memory is bounded to roughlyconcurrency × STREAMING_BATCH_ITEM_MAXIMUM_SIZEand request-body backpressure is preserved.1for fully sequential ingestion (escape hatch).env.server.ts; the service takes a requirednumber.Why this is safe (ordering/idempotency unchanged)
timestamp = batch.createdAt + index), not enqueue order.enqueueBatchItem.lastIndexassumption (the only order-dependent bit).Tests
webapptypecheck clean.Follow-ups (not in this PR)
application/storerefs instead of raw blobs) to remove object-store work from the request hot path and shrink the request body — bigger, protocol-level change.server.requestTimeoutbump as a safety net.🤖 Generated with Claude Code