A self-contained containered agent (pydantic-ai + FastAPI) with no access to the brain or anything else — so it must hold everything it needs to know, operate a small set of skills on that knowledge, watch and manage its own runs, and keep a memory that learns over time. It consumes pages ① and ②, and it owns the MADE-ID system: watching the inputs, verifying the ER mapping, and matching/minting/merging every account, contact, opportunity and campaign to one canonical made_id — propose-only, a human always in the loop for anything uncertain.
One agent, one job: make Salesforce, HubSpot and the Hub agree on one company / one contact / one MADE-ID, and keep them agreeing as data flows in. It reads the inputs (①) and the ER map (②); it writes proposals; a human clicks in /made-id-review. It never writes a committed identity link directly.
SFDC + HubSpot land in Reporting (feeds + mechanisms + freshness)
Reporting → Transfer; resolve to a canonical made_id
watch · verify · match · mint · merge · learn
/made-id-review — a person confirms the uncertain
one companies.made_id / contact_made_id
The work is overwhelmingly deterministic SQL — the one LLM seam is an adjudicator that returns a signal, never a verdict. The agent proposes; a human commits. The only fully-automatic write is the ≥0.999 auto-merge primitive it calls. Nothing is ever silently deleted — sub-floor candidates land in a census so the next run knows they were already examined.
| Harness | pydantic-ai + FastAPI container (made-id-custodian) on bc-made — self-contained, no external/brain access |
| Archetype | propose-only resolver; deterministic-SQL-first (historically 26 of 9,261 decisions used a model) |
| Trigger | cron + POST /run; default mode dry_run. 69 runs to date, most recent 2026-09-09 22:00 UTC — and an apply run did occur on 2026-08-26, so “it has never written” is no longer true. Goal: an incremental trigger on changed rows |
| Model | claude-haiku via LiteLLM at one seam (the ambiguity adjudicator) — one of 15 signals, ceiling 0.85, cannot auto-confirm |
| Databases | Transfer (canonical, writes proposals) · Reporting (read-only by construction) |
| Guardrails | PROTECTED_TABLES raises on any committed-identity write · no unfiltered UPDATE · never a silent DELETE (census) |
| Cost | $0.00 all-time — deterministic-first is the cost control |
| State | goal-level one skill runs today (contact→account); the rest are designed here and being built |
The agent can’t phone the brain. Every skill runs on knowledge baked into the container: what the inputs are, how the ER map + schema work, and the matching methodology (its own tab). Below is the schema and system knowledge; the matching model is in Methodology.
Salesforce (the sync-sfdc Edge Function, describe-generated) + HubSpot (contacts EF + campaigns/touchpoints EFs; companies via legacy n8n) all land in Reporting xyopyttkhoxvnyeyijzb. It must know each feed’s mechanism + cadence to tell when one didn’t fire. Full map →
That figure comes from a code comment, not an executed reconciliation, and earlier versions of this page presented it as measured. Selected API fields, unique destination columns, accessible fields and populated fields are four different inventories. A real catalog needs a describe run under a genuine integration identity — which is blocker 1. Until then, treat coverage as inferred.
| Table (Transfer) | What it holds | Key columns the agent must know |
|---|---|---|
| companies | Canonical company anchor — one MADE-ID each | id, made_id, normalized_name, company_domain, aliases, name_embedding (1024-d), relationship_type, is_icp_cgt |
| er_identity_map | Cross-system crosswalk | system, external_id, external_name, company_id, made_id, match_method, confidence, status, confirmed_by/at |
| er_account_census | Durable “what lives where + resolution state” — nothing dropped | system, external_id, external_domain, company_id, resolution_state, match_method, triage_class, examined_by, evidence |
| er_identity_proposals | Proposed links awaiting a human | action (assign_external_id / create_canonical / merge_candidate), primary/secondary_company_id, evidence, status |
| er_orphan_candidates | Net-new companies awaiting mint | crm_name/domain, triage, bc_hub_match, clay_status, proposed_primary_category, created_company_id/made_id |
| campaign_dim / campaign_membership | Canonical campaign + person-in-campaign | campaign_dim.id, member → (contact_made_id, company made_id), has_responded |
CRM row with no canonical match → er_orphan_candidates
bc_hub match check; classify primary_category
Clay firmographics ($0 if Hub-matched)
New Accounts tab → create_company_from_orphan
new companies row + made_id
A confidence score is not a similarity number — it is a log-odds fusion of 15 independent signals, each with a weight, an evidence tier, and a ceiling. Git is authoring truth; er_scorer_config is the runtime mirror. Weights are uncalibrated priors until the agent has enough of its own decisions to fit against.
| Signal | Weight | Tier | Ceiling | Discriminating |
|---|
fused = base(−4.0) + Σ(weight × strength × damping). Damping is 1.0 for a discriminating signal, 0.35 for a corroborating one. raw = sigmoid(fused); score = min(raw, ceiling) where the ceiling is the highest among contributing signals.
0.49: corroboration alone can never clear the propose floor. An ambiguity penalty 3.0 × (1−margin) is subtracted when a runner-up also clears the floor.Per candidate, given score · tier · rank · cardinality:
exact domain → exact normalized name → trigram + mandatory LLM/human adjudication. A trigram hit can never carry a match by itself. For contacts, two independent deterministic routes run — the CRM ID chain and the email domain — and a route conflict goes to a human, never a proposal. Role addresses (info@, sales@, 30+ localparts) are suppressed as identity evidence.
The target: auto-confirm on any single deterministic signal (exact external-id / email / domain / learned alias / unique normalized name) — except when that domain/email is shared by >1 company (CDMOs, incubators, universities, shared inboxes), which routes to review. The shared-domain guard is the safety valve.
This is a target, not current behavior. It stays shadow-only until it has per-rule guard coverage and precision evidence from a labeled holdout, plus an explicit policy approval — see the plan (M-Q1, M-Q7). A score is not a calibrated probability until something has calibrated it.
made_id_match_by_embedding and the pgvector neighbour search exist, and companies.name_embedding is on the schema — but there is no embedding API key on the box, so the semantic route does not run. Matching is therefore deterministic + trigram only today.
We are treating this as a declared degraded capability, not a blocker: exact external IDs plus normalized name, domain and email cover the large majority of a ~31k-record universe. It raises recall on hard cases. Blocker 5.
On a confirmed merge: pick the senior MADE-ID (earliest assigned). The loser inherits the survivor’s made_id; every er_account_census and er_identity_map row is re-pointed loser→survivor; the loser’s name is learned as an alias; er_decisions is audited. Identity-only today (both physical rows remain — no tombstone/FK re-point yet; that’s a known gap).
Not “lanes” — skills. Each is a thing the agent can do, defined by the knowledge it uses. Together they cover the whole custodial job: keep the data flowing, keep it mapped, and keep every entity resolved to one MADE-ID.
Make sure every SFDC/HubSpot feed fires on its cadence. When one doesn’t, detect it (freshness ledger), investigate the cause, and do the simple fix — or raise it with the exact feed + last-good time.
sync_freshness_state / sync_logAs data syncs, confirm each value maps correctly through the ER system to its canonical home — and flag values that land nowhere. Guards against the “lift ran but the account crosswalk didn’t” failure (page ②).
Per object (account · contact · opportunity · campaign): score confidence, auto-match the near-certain (single deterministic + shared-domain guard), and route to the MADE-ID human-in-the-loop when unsure — making sure the right evidence is captured for the reviewer.
Mint canonical new contacts/accounts (orphan → enrich → human-approve → MADE-ID); merge duplicates onto the senior MADE-ID and re-point everything to it. Holds the whole database schema so it never orphans a reference.
Improve from its own runs over time: every confirmed match becomes a learned alias/rule so the next run doesn’t re-pay; every rejection is remembered and never re-proposed; the census means nothing is examined twice. Feeds calibration of the scorer priors.
~80 deterministic Postgres functions on Transfer do the actual work; the agent orchestrates them and adds the one LLM seam. It writes only proposal + census rows; five Edge Functions serve them to six review tabs where a human clicks.
| Family | Functions (grouped) |
|---|---|
| Normalize | made_id_norm_company/name/email/phone/title/linkedin · made_id_email_domain · made_id_is_role_email · made_id_is_junk_name |
| Match | made_id_company_matches · made_id_match_by_embedding · made_id_resolve_contact · made_id_resolve_by_linkedin · made_ralph_pgvector_neighbors · made_ralph_trigram_matches |
| Score / route | made_id_band · made_id_set_fused_scores · made_id_sufficiency_gate_reason · made_id_name_stats · made_id_refresh_scorer_stats |
| Propose / apply | made_id_generate_contact_proposals · made_id_contact_review_queue · made_id_bulk_accept_deterministic · made_id_apply_review · made_id_mint_contact_identity |
| Merge / dedup | made_id_auto_merge (Joe’s ≥0.999 primitive) · made_id_dedup_scan · made_id_detect_drift · preview_company_merge · made_id_backup_object/restore_object |
| Surface (HITL) | EFs → 6 tabs: made-id-review · -census-review · -contact-review · -new-accounts · -campaign-review |
build the candidate set; skip negative memory, junk, and anything already in the census (idempotent)
15 signals → log-odds fusion → ceiling + ambiguity penalty (deterministic, no I/O)
auto-confirm (gated) / propose / review / new_canonical / hold-to-census
er_custodian_runs ledger + batch_tag (the rollback handle) + heartbeat
BLITZ — a one-time, max-effort pass that clears the entire finite CRM backlog (every SFDC/HubSpot account, contact, opportunity, campaign) against the canonical set: match or mint. The review queue drains via band-scoped bulk-accept + mint approvals. A project with an end.
INCREMENTAL — after the blitz, react only to new/changed CRM rows, re-propose, and detect drift. The perpetual full-rescan drains retire. This is the long-term custodial job.
er_custodian_runs records examined / proposed / routed / held / cost / scorer version / host / mode / error per run. batch_tag = lane slug + UTC + random suffix — everything a batch wrote can be found and reversed by that one string.
With no brain to lean on, the Custodian’s only way to improve is to learn from its own runs. Memory is four durable stores it reads before it spends effort, and writes after every human decision. Two of them are the single highest-leverage fix in the whole system — they’re empty today.
| Memory store | What it remembers | State |
|---|---|---|
| er_custodian_runs | Every run it has ever done — examined/proposed/routed/held/cost. It watches its own history for drift + throughput. | exists |
| er_decisions | Every human decision (confirm/reject/merge) — the audit trail + the training set for calibrating the scorer priors. | 1,919 |
| er_negative_decisions | Pairs a human rejected — never re-proposed. Checked first, before any scoring cost. | 9 |
| er_account_census | “Already examined, and when” — so a re-run costs nothing on settled rows; nothing is examined twice. | 22,400 |
| er_learned_rules + name_aliases | Learn-on-confirm: every confirmed match should write an alias/rule so the same reference resolves free next time. | 0 — the loop to build |
After 14k+ merges, er_learned_rules and name_aliases are still 0 — every match re-pays full price and none of the human effort compounds. Wiring learn-on-confirm (each confirmed decision → an alias/rule) is what turns the Custodian from a re-scanner into an agent that gets cheaper and smarter every run. It also builds the labelled set that finally calibrates the 15 scorer weights (priors → fitted).
Before a run: read negative memory + census + learned aliases → skip settled work. During: score with the current (learning) weights. After a human clicks: write the decision, learn the alias, and (over time) re-fit the priors. That’s the whole self-improvement cycle — no external brain required.
wiki/systems/made-id-spine/madeid-custodian-deep-dive.md + skills/identity/* + the live container; made_id_* inventory from the Transfer PostgREST OpenAPI; counts live-measured 2026-09-09 22:00 UTC. Companion pages: Inputs (①), Lift & ER map (②), Plan → production (◇). Goal-level design — being built. Internal, not for external share.