01
Multi-Source Data Warehouse
Many messy sources, one player record you can trust.
The problem
Every source has its own IDs, spellings and seasons, and they break quietly. One open dataset once wiped the market value of 10,021 players while keeping the same row count.
How it works
- Each import passes five checks before it goes live: the shape of the data, no more than 15% of rows lost, minimum counts, filled columns, and dates that never go backwards. If a check fails, the live data stays as it was.
- Players are matched in three passes with a confidence score: exact ID (1.0), name and birth date (0.95), a near name with one clear candidate (0.8). Manual links are never overwritten.
- Clubs are matched by vote: at least three shared players and a clear margin. Agreement is 99.7%.
- 21 reference transfers must match within 1% before anything loads. Sources are reconciled every week, and disagreements become tickets.
Status
- Live: running in production529,137 transfer records from 1996 to 2026, 72,828 players, 15,961 clubs.
Also fits
Retail catalogues from many suppliers, property data, medical registries, KYC, logistics.