Previously the staging data was getting lookup with full cache to find the Match and No Match for Insert and Update Operation.
Now the Lookup data holds around 1 Million records and Data Flow Task failed to execute .
Since with Full cache is using Memory Operation it gives the error.
Decided to load the data in Cache File and use in Lookup to do minimum changes but that too fails
The buffer Manager failed a memory allocation call for 1638000 bytes , but was unable to swap any buffer to remove memory pressure . 2290 buffer were considered and 2 were locked
tried to change the Setting for the data Flow task DefaultmaxBufferRow -500 and Defaultbuffersize value is as it is but it failed
Decide to divide the data into series and tried to load in cache file but cache file can be used once . please suggest