Enrichment (Autograb AGVI lookup)
Only one provider has any enrichment logic. enrichData() is the enrichData hook on BaseProviderProcessor's Template Method (see Provider column mapping) — the default implementation is a no-op, and AutograbProvider is the only one of the 12 providers that overrides it.
What triggers it
Runs unconditionally on every Autograb file, for every record. First, it filters to records worth looking up — anything with an agvi value but no redbook_vehicle_key already:
const recordsNeedingLookup = jsonData.filter(record =>
record.agvi && !record.redbook_vehicle_key
);
if (recordsNeedingLookup.length === 0) {
return jsonData; // No lookups needed
}For each of those, it calls Autograb's Tailpipe API:
private async lookupVehicleKey(agvi: string): Promise<string | null> {
const baseUrl = process.env.DRIVE_APIG_BASE_URL || '';
const apiKey = process.env.AUTOGRAB_API_KEY;
const url = `${baseUrl}/ag/tailpipe/lookup?id=${encodeURIComponent(agvi)}`;
if (!apiKey) {
throw new Error('AUTOGRAB_API_KEY environment variable is not configured');
}
const response = await fetch(url, {
headers: { 'x-api-key': apiKey, 'Content-Type': 'application/json' }
});
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const data = await response.json() as TailpipeResponse;
if (data?.success && data?.vehicles?.length > 0) {
return data.vehicles[0].vehicleKey || null;
}
return null;
}The response shape is { success, vehicles: [{ vehicleKey }] } — vehicles[0].vehicleKey becomes the record's redbook_vehicle_key.
Concurrency and failure handling
All lookups for a file are dispatched concurrently, one promise per record, via Promise.allSettled — not Promise.all, deliberately, so one record's failure can't reject the whole batch:
const lookupResults = await Promise.allSettled(lookupPromises);
lookupResults.forEach((result, index) => {
if (result.status === 'fulfilled') {
vehicleKeyMap.set(result.value.agvi, result.value.vehicleKey);
} else {
// rejected — logged, left unresolved, record proceeds with an empty key
const agvi = recordsNeedingLookup[index]?.agvi;
if (agvi) vehicleKeyMap.set(agvi, null);
}
});A rejected promise (missing API key, non-2xx response, network error) is caught individually, logged, and that record's redbook_vehicle_key is left empty — it does not fail the rest of the file.
There's no retry, no feature flag, and no fallback provider. A failed lookup is a silent per-record skip, not a hard error.
No caching
Every record missing a key hits the live Autograb API, every file, every time — there's no persistent cache of previously-resolved AGVI-to-key lookups.
This is a genuine difference from how the marts-side colour lookup works: the exterior-colour standardisation model (stock_colours_ref.sql) checks a cache table and has a skip_colour_api dbt var to bypass the external call entirely. No equivalent cache or bypass flag exists for the AGVI lookup — it's a live call by design, not an oversight to fix here, just a fact worth knowing before assuming both lookups behave the same way.
The separate, non-API fallback: NVIC mapping
stock_nvic_ref (src/stock/dbt/models/intermediate/stock/stock_nvic_ref.sql) is a dbt model, not part of ingestion — it runs later, downstream in the stock pipeline. For records where redbook_vehicle_key is still null, empty, or one of a few known-bad sentinel values, but an nvic_code is present, it joins against a static seed table (src/redbook/dbt/seeds/) to resolve a key. No external call, no caching concern — it's a plain table join against a fixed reference file.
A third fallback exists on top of AGVI and NVIC — resolving a key from the dealer-supplied VIN directly. See VIN-based fallback (VIN lookup) for how it's triggered and what it does.