AI Data Engineer

Fuel ETL incident

Incident copilot preloaded with the Fuel ETL duplicate ID failure.

Failed job

Fuel ETL — duplicate transaction IDs

UNIQUE constraint violated on txn_id for 142 rows (card+timestamp collision after vendor file re-send).

Quality issue

qi-fuel-dup

142 duplicates · triaging

Context

Comdata re-send

criticalfact_fuelquarantine

Conversation

AI Data Engineer · Fuel ETL incident context loaded (fail-fuel-dup).

Fuel ETL failed earlier — what's going on with duplicate IDs?

Comdata re-pushed the Jul 22 settlement file at 06:05Z. Their txn_id values were regenerated and collided with the live incremental feed for 142 rows. Pipeline completed the second run with quarantine into `fuel_txn_dupes`; `fact_fuel` uniqueness rule is red (critical).

How bad is the KPI impact if we leave it?

Reported fuel CPM is overstated by ~1.8¢ and OR looks ~0.2–0.3 pts worse until duplicates are removed. Downstream Morning Brief and board pack should cite cleaned fuel until the patch ships.

Propose the fix.

1) Deduplicate on (card_last4, txn_ts, gallons, location_code) keeping the earliest ingest watermark. 2) Assign surrogate `fuel_sk` as PK; treat vendor txn_id as degenerate. 3) Add ingestion guard: reject full-file re-sends without `idempotency_key`. 4) Replay Jul 22–23 partitions and recertify `fact_fuel`. I can draft the dbt test + runbook if you approve.

Approved — draft the runbook and open the quality ticket.

Done. Quality issue `qi-fuel-dup` set to triaging, failed job annotated, and Morning Brief trust badge notes the critical fuel quality exception. ETA to green: ~45 minutes after merge.