All,
I have a flat file that loads data in to a staging table, and then my staging table loads data in to a main table. But the problem is that data might already exist in my main table. So, for example, my main table before getting loaded looks like this:
Name Sps Va John 4 10 John 6 12 John 8 10 Tim 4 12 And the file that I run today loads my staging table, so data in staging table looks like this: Name Sps Va John 4 10 John 6 12 John 8 10 John 9 15 -- New Tim 4 12 Tim 5 10 -- New I just want to insert the new records that are coming in today. So, when this staging table loads to my main table. i want to see just the distinct data and no duplicates: Name Sps Va John 4 10 John 6 12 John 8 10 John 9 15 Tim 4 12 Tim 5 10
There's no KEY structure in the table. Files that I get daily will have around 2000 records. And the records in today's file compared to yesterday's file might be same as well. Or, today's file might have just 5 records extra compared to yesterday's file. How to get through this?