Enrichment (Autograb AGVI lookup)
Only one provider has any enrichment logic. enrichData() is the enrichData hook on BaseProviderProcessor's Template Method (see Provider column mapping) — the default implementation is a no-op, and AutograbProvider is the only provider that overrides it.
What triggers it
Runs unconditionally on every Autograb file, for every record. First, it filters to records worth looking up — anything with an agvi value but no redbook_vehicle_key already:
const recordsNeedingLookup = jsonData.filter(record =>
record.agvi && !record.redbook_vehicle_key
);
if (recordsNeedingLookup.length === 0) {
return jsonData; // No lookups needed
}For each of those, it calls Autograb's Tailpipe API:
private async lookupVehicleKey(agvi: string): Promise<string | null> {
const baseUrl = process.env.DRIVE_APIG_BASE_URL || '';
const apiKey = process.env.AUTOGRAB_API_KEY;
const url = `${baseUrl}/ag/tailpipe/lookup?id=${encodeURIComponent(agvi)}`;
if (!apiKey) {
throw new Error('AUTOGRAB_API_KEY environment variable is not configured');
}
const response = await fetch(url, {
headers: { 'x-api-key': apiKey, 'Content-Type': 'application/json' }
});
if (!response.ok) {
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
}
const data = await response.json() as TailpipeResponse;
if (data?.success && data?.vehicles?.length > 0) {
return data.vehicles[0].vehicleKey || null;
}
return null;
}The response shape is { success, vehicles: [{ vehicleKey }] } — vehicles[0].vehicleKey becomes the record's redbook_vehicle_key.
Concurrency and failure handling
All lookups for a file are dispatched concurrently, one promise per record, via Promise.allSettled — not Promise.all, deliberately, so one record's failure can't reject the whole batch:
const lookupResults = await Promise.allSettled(lookupPromises);
lookupResults.forEach((result, index) => {
if (result.status === 'fulfilled') {
vehicleKeyMap.set(result.value.agvi, result.value.vehicleKey);
} else {
// rejected — logged, left unresolved, record proceeds with an empty key
const agvi = recordsNeedingLookup[index]?.agvi;
if (agvi) vehicleKeyMap.set(agvi, null);
}
});A rejected promise (missing API key, non-2xx response, network error) is caught individually, logged, and that record's redbook_vehicle_key is left empty — it does not fail the rest of the file.
There's no retry, no feature flag, and no fallback provider. A failed lookup is a silent per-record skip, not a hard error.
No caching
Every record missing a key hits the live Autograb API, every file, every time — there's no persistent cache of previously-resolved AGVI-to-key lookups.
This is a genuine difference from how the marts-side colour lookup works: the exterior-colour standardisation model (stock_colours_ref.sql) checks a cache table and has a skip_colour_api dbt var to bypass the external call entirely. No equivalent cache or bypass flag exists for the AGVI lookup — it's a live call by design, not an oversight to fix here, just a fact worth knowing before assuming both lookups behave the same way.
The separate, non-API fallback: NVIC mapping
stock_nvic_ref (src/stock/dbt/models/intermediate/stock/stock_nvic_ref.sql) is a dbt model, not part of ingestion — it runs later, downstream in the stock pipeline. For records where redbook_vehicle_key is still null, empty, or one of a few known-bad sentinel values, but an nvic_code is present, it joins against a static seed table (src/redbook/dbt/seeds/) to resolve a key. No external call, no caching concern — it's a plain table join against a fixed reference file.
A third fallback exists on top of AGVI and NVIC — resolving a key from the dealer-supplied VIN directly. See VIN-based fallback (VIN lookup) for how it's triggered and what it does.