Hello All,
We have SSIS Packages that move data from Staging Tables to Destination Tables.
The current design pattern generates a Unique Identifier for every Business Key in the Staging table, and this Business Key is used by the destination table as its Primary identifier (to track inserts/updates/deletes). The ETL as it takes the record from the Staging checks for the business rules and flags them as appropriate (Foreign Key Check, Required attribute checks, etc.)
A change in the User system, forces the data model to have unique Entity names (The Business Natural key and the Identifiers are not exposed and cannot be used by the end system)? Can you suggest a pattern that helps me check if the names are unique in a Initial Load Scenario (where the destination tables are empty) and also in a incremental load scenario? The ultimate goal would be to flag them in the Staging Database as Potential Duplicate records?
Regards, Dinesh