We have a Lookup to match incoming records from temp table and filter out duplications before entering the destination table. The temp and destination tables have exactly the same definition. The matching compares 3 columns with type of text, numeric, and datetime.
It works correctly under full cache mode, but will run out of memory when data grows to realistically massive size. The Lookup will load the entire destination table into memory and triggers too much swapping.
For performance tried to change it to partial cache and no cache, and the changes breaks correctness because the equal values on datetime column fails to match and repeats are escaping the filter into destination data table.
The other columns are in text and numeric types, and they seem to be OK. Just for test, if exclude the datetime column the Lookup matches as expected with the rest columns.
I suspect there might be something wrong in collation or other stuff related, I am not sure how to confirm and do not have a solution.
The environment is SQL Server 2016, Visual Studio 2015, and Windows Server 2016. And the matching failure happens in Visual Studio debugging. More information available if needed, and thanks a lot in advance.