AI-assisted match and merge

Layer AI on top of MDG matching

MDG already has the duplicate check in central governance (typically HANA-based fuzzy search) and consolidation (standardize, match, best record, activate). AI adds three narrow things: recognizing differently written names, explaining pairs so stewards decide fast, and tuning thresholds from decisions. Replacing MDG matching with an LLM is a red flag.

Embeddings and blended scores

An embedding turns text into a vector; similar meanings land close together (cosine similarity near 1). 'Larsen & Toubro Ltd' and 'L&T Limited' score high. Embeddings handle abbreviations, legal forms and transliterations but are bad at numbers ('Plant 12' vs 'Plant 21'), so tax numbers, IBANs and postcodes stay deterministic. Blocking first limits comparisons to plausible pairs.

import numpy as np

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

def score_pair(rec_a, rec_b, vec_a, vec_b):
    # vec_a / vec_b: name embeddings from your chosen model (e.g. via the hub)
    name_sim = cosine(vec_a, vec_b)
    same_tax = bool(rec_a["tax_no"]) and rec_a["tax_no"] == rec_b["tax_no"]
    same_post = rec_a["postcode"] == rec_b["postcode"]
    # Simple, explainable blend. Tune weights on steward decisions.
    score = 0.5 * name_sim + 0.35 * same_tax + 0.15 * same_post
    return round(score, 3)

Try it: be the steward

The LLM explains, it does not decide

For middle-band pairs, ask for a fixed JSON shape: agreeing and conflicting fields, patterns found, and what to check before merging. Mask contact persons, emails and bank details first. The result is shown as a note on the steward's work item and logged.

{
  "pair_id": "SUP-004211|SUP-118907",
  "verdict": "likely_duplicate",
  "confidence": 0.86,
  "agreeing_fields": ["postcode", "street", "city", "tax_no"],
  "conflicting_fields": ["name1"],
  "patterns": ["abbreviation (L&T = Larsen & Toubro)", "legal form (Ltd = Limited)"],
  "check_before_merge": ["open items in both company codes",
                         "different bank accounts: confirm with AP"],
  "explanation": "Same tax number and address; names differ by a known abbreviation."
}

Getting results back into MDG

Three routes: consolidation for cleanups and new sources (AI standardizes names before the load and adds explanations), the central governance duplicate check for ongoing creation (validate enhancement options for the release), and mass processing to block confirmed losers as governed CRs.

Why stewards stay in the loop

A wrong merge moves open items, POs and payment history, replicates through DRF, and can send payments to the wrong legal entity. Explanations make the control affordable: 3,000 pairs at four minutes each is 200 hours; at one minute, 50. Watch precision per band, minutes per decision and reversals within 90 days.

Try it: the auto-merge request

Survivorship: AI helps, rules decide

Best-record survivorship must be explainable and repeatable, so it stays in configured rules. AI standardizes inputs before consolidation and flags conflicts rules cannot resolve, such as two active bank accounts.

Try it: who handles which field?

Check yourself