Made/MADE-ID
Internal
Stage ② · Lift & ER map

How Reporting data gets into Transfer — and becomes one MADE-ID

Where it sits: the second stage. Stage ① proved everything lands in Reporting; this is the next hop — the lift (Reporting → Transfer), the ER map that resolves each row to a canonical made_id, how it’s meant to stay in sync, and the human-in-the-loop match & merge on /made-id-review — ending at what the custodian agent must own (page 3).

8
sync_*_from_reporting lift functions
er_identity_map
the canonical crosswalk (Transfer)
never built
ids.external_ids account writer (PROJ-057 Ph3)
full re-pull
lift has no change-delta today

1 · The lift

Reporting → Transfer

The “lift” is a family of pg_cron SQL functions that run on the Transfer DB. Each calls Reporting’s PostgREST via the Postgres http extension (auth = vault secret reporting_anon_key), pages the source table (limit 1000 offset…), and upserts into a rpt_*_stage staging table on Transfer. Critically: they only stage — none of them writes er_identity_map, contact_made_id, campaign_dim, or mints/merges/links. And they re-pull the whole table every run (no changed-since cursor) — the change-sync is a full refresh, not a delta.

Lift function (runs on Transfer)Reads (Reporting)Writes (Transfer staging)Then applied by
sync_persons_from_reportingpersons (contacts+leads+HS)rpt_persons_stageapply_sfdc_contact_identity → contact_made_id
sync_external_ids_from_reportingexternal_ids crosswalkrpt_external_ids_stagerefresh_crm_identity_bridge
sync_crm_campaign_members_from_reportingcampaign members (SFDC+HS)rpt_campaign_members_stagecampaign_membership build
sync_lifecycle_history_from_reportinglifecycle_stage_historyrpt_lifecycle_stage_history_stagelifecycle timeline
sync_accounts_from_reportingaccounts (unified)rpt_accounts_stageaccount identity
sync_sfdc_from_reportingsfdc_* (accounts/opps/…)Transfer sfdc mirror / companies joinaccount matching ladder
sync_hubspot_from_reporting · sync_hubspot_companies_from_reportinghubspot_contacts / hubspot_companiesTransfer HS mirror / companies joinaccount + contact matching

Gap — the lift is a full re-pull, not a change-feed

Every run re-pages the entire source table into staging. It works at today’s volumes, but there is no “what changed since last run” — so the custodian can’t (yet) react to just new/changed CRM rows. The incremental change-sync is a thing to build, not a thing that exists.

2 · The ER map

resolving a row to a canonical made_id

Identity is built into a small set of Transfer tables. The anchor is companies (canonical, one made_id each, with a merged_into_company_id tombstone for merges). er_identity_map is the crosswalk that ties an external record to that anchor; contact_made_id keys people; campaign_dim/campaign_membership key campaigns.

Table (Transfer)What identity it holdsKey columnsState
companiesCanonical company anchor — one row per real firmid, made_id, made_company_id, normalized_name, hubspot_id, merged_into_company_id (tombstone), field_provenanceexists · 4,719
er_identity_mapCrosswalk: (system, external_id) → company_id → made_idsystem, external_id, external_name, company_id, made_id, match_method, confidence, status, confirmed_at/by, last_verified_atexists · 9,336
contact_made_id (+ er_contact_identity_map)Person identity (SFDC contacts/leads + HS contacts)contact_made_id, person keys, match_method, status112,617 mapped
campaign_dim / campaign_membershipCanonical campaign + who was in itcampaign_dim.id, member→(contact_made_id, company made_id), has_respondedcrosswalk cosmetic

The real bottleneck — the account crosswalk was never wired (not merely “stalled”)

From the sync registry (verbatim, 2026-08-10/13): ids.external_ids has no writer — “it is not a broken sync, the sync was never built.” It was populated once on 2026-06-11 (7,783 SFDC / 21,019 HS) as PROJ-057 “MADE ID Unifier” Phase 3, and never got a maintaining job. It splits into a person half (fresh — apply_sfdc_contact_identity runs every 6h) and a dormant account half: external_ids_account (7,030 rows, ~2,396 accounts with both a SFDC + HS id) whose internal_id resolves to nothing, and whose consumer propose_hubspot_account_identity is not scheduled.

Scope correction — 2026-09-09

Earlier versions of this page implied this dormant crosswalk meant account identity as a whole had stopped. It does not. er_identity_proposals is still being written daily (most recently 2026-09-09 08:30) and er_identity_map holds 9,336 live links. Two further points matter: ids.external_ids_account is registered as a monitor key pointing at a Reporting view, not a table in an ids schema — so nothing should be built named after it; and because its readers are unknown, the right first move is to trace active consumers, then decide maintain / bypass-behind-a-view / retire. “Repairing” a key whose consumers are unidentified risks breaking another project’s feed.

3 · Keeping it in sync

new/changed CRM row → proposal

Today — what runs

every 6h + nightly
  • Lift re-pulls Reporting → rpt_*_stage (full)
  • refresh_crm_identity_bridge applies the contact identity
  • made_id_* engine scores + proposes matches
  • Auto-merge only at ≥0.999; else a review row

Today — what’s broken/missing

the account leg
  • Account crosswalk unwired (writer never built; consumer unscheduled)
  • No change-delta — full re-pull, can’t react to just-changed rows
  • field_provenance has 0 salesforce / 0 hubspot stamps

Target — the change-sync

to build
  • Watermark on SystemModstamp/hs_lastmodifieddate
  • Changed row → custodian → propose match/mint/merge
  • Stamp salesforce/hubspot provenance on write
  • Enable the account identity (PROJ-057 Ph3)

4 · Match & merge — the human in the loop

/made-id-review

When identity isn’t certain, the custodian writes a proposal and a human decides on /made-id-review. Auto-confirm is reserved for the near-certain; everything else is banded for review; merges re-point onto a senior MADE-ID and land losers in a census (never a DELETE).

Review surface (tab)What the human resolvesServed by
Coverage / CensusUnmatched CRM rows awaiting a decisionmade-id-census-review EF
DuplicatesTwo canonical companies that are one firm → merge onto senior made_idmade_id_dedup_scan · preview_company_merge
CRM MatchingAccount proposals (SFDC/HS → company)made-id-review EF · er_identity_proposals
ContactsContact→identity + contact→account rollup (D3b bands, bulk-accept)made-id-contact-review EF · made_id_contact_* RPCs
New AccountsMint a net-new company (human-approved) → MADE-ID + enrichmentmade-id-new-accounts EF · orphan-seeder
CampaignsSFDC↔HubSpot campaign crosswalkmade-id-campaign-review EF · campaign_dim

The rule that never moves

Propose-only. Auto-merge only at ≥0.999. Never made_id written directly, never a silent DELETE — sub-floor candidates become census rows so the next run knows they were already examined. This is what the custodian agent enforces (page 3).

5 · What exists vs what’s missing

PieceStatusNote
Lift Reporting→Transfer (contacts/persons/campaigns/lifecycle)existsfull re-pull into rpt_*_stage; person identity applied every 6h
er_identity_map + companies anchor + merge tombstoneexists9,336 links · 4,719 companies · merged_into_company_id ready
Contact identity + rollup (D3b)partial112,617 mapped; ~31% contact→account gap; two proposal tables to consolidate
Legacy account crosswalk (external_ids_account)dormanta monitor key over a dormant Reporting subset, stalled since 2026-06-11; consumer unscheduled. Not proof account matching is broken — proposals still write daily. Trace consumers before repair/retire
Abandoned match queuesretirematch_queue 3,731 pending (last enqueued 2026-05-15) · contact_match_queue 2,963 pending (last 2026-07-29) — drains still run every 10 min over stale rows
Canonical rich person record (public.contacts)0 rowsperson identity is in made_id_contacts (109,324); the rich table was never filled and consumers routed around it
Change-delta sync (react to just-changed rows)missinglift is a full re-pull; no watermark
salesforce / hubspot provenance stampingmissingfield_provenance has 0 of each
Campaign crosswalk (SFDC↔HubSpot merged)cosmeticnever truly merged; attribution 19% / 6%
Method: lift-function bodies read from supabase/migrations/20260729b_phase1_reporting_identity_sync_functions.sql; ER-map + crosswalk truth verbatim from the sync_registry seed (20260813_sync_registry_seed.sql) + Transfer OpenAPI RPC inventory; counts read live from Transfer/Reporting 2026-09-09. Companion pages: upstream_truth_map.html (page 1), 03_custodian_agent.html (page 3). Internal — not for external share. No system changed.