Skip to main content
Use exports as a complete baseline and observed changes as an invalidation inbox. The change feed is not a lossless database replication stream. In particular, an unchanged inventory check can advance retrieval timestamps without producing an event, and store-directory edits are not covered by product events. For those invalidations, use the separate resource update stream below. Periodically import a new complete snapshot as a reconciliation boundary. Neither stream establishes when DABS updated its source.

Follow resource updates

Use GET /v1/updates/sync when keeping cached product, store, or inventory views up to date. Use /v1/changes/sync for observed value differences and history. Their checkpoints are independent and cannot be interchanged.
Capture the initial checkpoint before importing a baseline. Save only a final page’s completed checkpoint, atomically with handling the invalidations. The same frozen-window, retry/deduplication, cursor-replay and checkpoint-gap rules described below apply. The existing file-based reference consumer follows product changes; it does not automatically process this separate stream. Each record identifies one resource with resource.type, its stable Nektir UUID, and resource.source_key. For inventory, the ID is the product UUID and the key is its six-digit DABS SKU. For product, the key is also its six-digit SKU; for store, it is the four-digit inventory store code. Leading zeros are preserved. An inventory publication produces one invalidation per product, not one per store row. Product and directory publications, including editorial and presence-only updates, invalidate their respective views. A record means re-fetch this view, not that its values necessarily changed. It is not a restock, sale, or stock guarantee. Optionally filter by resource_type=product|inventory|store and resource_id=UUID. An ID requires a type; filters are bound into the checkpoint. limit defaults to 25 and is capped at 100. A product filter does not include inventory records: use the unfiltered stream to maintain all resource types. There is no store-scoped inventory stream; an inventory invalidation identifies the whole product snapshot. Coalesce repeated IDs within a window and re-fetch in bounded batches: products through /v1/products?product_ids=..., inventory through /v1/inventory, and stores through /v1/stores?store_ids=.... Inventory updates can also change store, nearby, family and comparison result lists, including products that were previously absent from them. Revalidate those cached queries or expire them with a bounded TTL; invalidating only rows already displayed can miss a newly matching product. times.retrieved_at_min and times.retrieved_at_max preserve the range of newly published observations. Product times refer to the catalog source; store times to directory retrieval, so an editorial edit can retain older retrieval times. times.published_at is Nektir publication time. times.source_updated_at remains null unless supplied; mixed or partially unknown inventory source times are null. A new invalidation does not prove recent source data or complete store coverage. Re-fetch the resource to inspect its coverage and per-observation timestamps. The lightweight stream retains seven days from publication, independently of longer product history. It starts with publications made after installation, with no historical backfill. Failed refreshes, rolled-back and idempotently repeated publications emit nothing. Retention deletion or a restored/reset stream returns 410 CHECKPOINT_EXPIRED; establish a new baseline. An expired page cursor instead returns 410 CURSOR_EXPIRED; replay from the last completed checkpoint. Responses are no-store, and neither stream triggers DABS collection.

Run the reference consumer

The repository includes api/scripts/examples/consumer-sync.ts, a standalone Node.js TypeScript example with no database dependency. After npm ci, run:
Replace API_HOST with the operator-provided API origin; api.nektir.com currently hosts documentation, not the data service. Use one writer per output directory. The example does not persist credentials or follow redirects to other origins. bootstrap first captures a durable change checkpoint, before requesting the export manifest. It then downloads all three files sequentially into a staging generation. It validates schema version, resource, publication revision, record count, NDJSON framing, and every row’s envelope. Only after all files validate does an atomic rename switch checkpoint.json to the new generation. Interrupted or malformed downloads leave the last complete baseline active. Node fetch decompresses HTTP gzip automatically; the files on disk are plain NDJSON. page commits one change page and its next cursor together. Repeat while the command reports has_more: true; on the next scheduled poll, invoke page again to resume from the last completed publication checkpoint, not the head of retained history. Re-running in a new process resumes the same cursor. Event UUIDs deduplicate retries. The local checkpoint binds the origin, SKU scope, and page size so a different query cannot silently reuse it. The example processes one page per invocation to bound work. It does not contain an infinite retry loop. On a transient 503, retain the checkpoint and retry the same operation after Retry-After with jitter. Persistent export-limit errors need operator attention. An export 410 during import leaves the old generation active; start a new bootstrap. 410 CURSOR_EXPIRED clears only the temporary page cursor and replays from the last completed checkpoint, preserving the baseline and deduplicating any repeated events. 410 CHECKPOINT_EXPIRED imports a new baseline and clears the cursor and inbox. This restores current-state coverage; it does not recover every historical event lost across a retention gap. The command reports rebootstrapped: true and recovery_reason: "checkpoint_expired" so an operator can detect that boundary instead of assuming uninterrupted replay. Cursor-only recovery instead reports restarted_window: true and recovery_reason: "cursor_expired".

Durable checkpoint requests

Start with GET /v1/changes/sync using your intended filters. The response has no events and supplies meta.sync.checkpoint at the current publication. Capture it before downloading your baseline: a publication during import may overlap the baseline and replay, but cannot fall into a gap after it.
The token names above are placeholders; use the opaque strings returned by the API. checkpoint and cursor are mutually exclusive. The service freezes each window’s upper publication and returns events in ascending publication/event order. Newer publications wait for the next poll. A final page, even an empty filtered one, returns the next completed checkpoint. Intermediate pages return a null checkpoint: do not advance your durable position before the window completes. Commit event processing and progress atomically in your storage. Delivery can repeat after an interruption, so deduplicate by event UUID and make downstream side effects idempotent. The API is stateless per consumer; there is no server-side subscription to create or acknowledge. Supported filters are product_id or six-digit sku, store_id, and types. Tokens bind those filters; type ordering and page size do not change server-side scope. Changing filters requires a new initial checkpoint and baseline strategy. Family and retrieval-time filters are not supported by this synchronization route. The ordinary GET /v1/changes remains the newest-first browsing/history endpoint. Completed checkpoints have no short page TTL and survive API process restarts. Their safety depends on retained history and stream identity, not retained product version projections. A deletion watermark conservatively invalidates any checkpoint that might have unread deleted events, even if the deleted event was outside a particular filter. Resetting the event stream also invalidates old checkpoints. All such gaps return 410 CHECKPOINT_EXPIRED, never a silently shortened result. Restoring a database starts a new lineage too: the supported restore command rotates stream identities before completing. Operators using raw database restore tools must reset those identities before serving the restored copy; copying an old stream UUID is not proof of continuous history. Keep the signing secret persistent across service instances. A secret rotation invalidates signed tokens with 400 CHECKPOINT_INVALID; establish a new baseline. Tokens are synchronization positions, not access credentials. Responses use Cache-Control: no-store so an intermediary cannot replay an old synchronization position. None of these operations requests fresh data from DABS.

Apply invalidations in your application

Read the checkpoint’s baseline.directory to locate the three pinned files. Their flat export schema is not identical to the public product/detail DTO schema; keep baseline rows and independently fetched application views separate. For each inbox event, use product.id and the optional store.id to invalidate affected application views. Batch product re-fetches with GET /v1/products?product_ids=... and inventory re-fetches with the bounded matrix endpoint, GET /v1/inventory. Store the revision and observation times returned by each response. Fetching several views sequentially does not produce a single coherent statewide snapshot. Do not replay event payloads as though they contain every field needed to rebuild all product and inventory rows. Do not infer zero inventory from a missing event or a missing row. Preserve null source-update timestamps, missing observations, restricted purchase eligibility, and reported zero quantities as distinct states. Do not set since to the last poll’s wall-clock time. That filter uses retrieval time: an older observation published later could be skipped. Publication-based checkpoints avoid that problem and read only the new event interval. This is still not every-retrieval or directory CDC: use the separate resource update stream for published view invalidations, a transactional inbox for large integrations, and periodically reconcile complete snapshots and current views.

Bounds and durability

This is a small reference consumer, not a production synchronization daemon. The default inbox holds 10,000 distinct events and stops without advancing when full. The library permits up to 50,000, supports page sizes 1–100, and optionally scopes events to a six-digit SKU. Archive/process events transactionally before adapting it for a larger integration; do not drop deduplication IDs and claim exactly-once delivery. Inbox delivery here is replay-safe, not exactly-once business-side-effect execution. For a selected-product invalidation inbox, pass a six-digit SKU as the final CLI argument, using the same scope for bootstrap and subsequent pages. Set NEKTIR_SYNC_PAGE_SIZE if you need smaller pages; keep it unchanged while resuming. SKU scoping limits events only, not the complete baseline export. Each export is limited to two million records, 1 GiB decoded bytes, and 8 MiB per record. Request deadlines are 150 seconds for exports and 15 seconds for change pages. Failed staging directories are removed. Previous successful generations are retained so existing readers are not disrupted; prune inactive generations only after readers release them and budget disk space accordingly. Data files and checkpoint files are synced before publication and checkpoint replacement is atomic on the same filesystem. Restart tests cover process interruption, not filesystem corruption or sudden host power loss. Production consumers should use their own transactional storage and backup policy.

Verification

api/tests/consumer-sync.test.ts covers partial imports, replay deduplication, restart, capacity limits, transient failure, malformed pagination, and expired cursor recovery. api/tests/consumer-sync.postgres.test.ts runs against a compiled API and disposable seeded PostgreSQL database, including a delayed-publication observation and actual cursor-expiry rejection. CI runs that integration test without contacting DABS. api/tests/change-sync.postgres.test.ts also checks exact summary boundaries, rollback/deletion handling, filter binding, frozen publication windows, process restart, independence from version-history TTL, and conservative retention gaps. The local reference checkpoint format is now version 2; if you used version 1, explicitly run bootstrap once rather than reusing its old feed position.