School of hard-knocks... wanted to share some pain to help others avoid it.
1) 64 Bit 2 processor dev box running windows 2003 server with 2gb
2) reasonably large data-set, package with cached lookups etc (normal sort of stuff)
3) Execution on the 2gb machine in debug had issues with memory allocation. (memory pressure and allocation failures, then swapping to temp drive, and ran out of space there).... everything worked as it should and failed for obvious reasons, due to resource limitations.
1) 64 bit 8 processor server running windows 2003 server with 64 gb
2) Same reasonably large data-set, package with cached lookups etc (normal sort of stuff)
3) Executed in debug without issues.
...... this is where the pain starts.....
Wanted to see it run (not in debug), but wanted to get progress messages etc without cranking up the logging.... so ran the package by selecting "run package" from management studio.
Would run for a little.... then fail with a message about not being able to allocate the next 100mb buffer, and reported 61gb free memory...
Did a bit of digging about, and found that dtexecui.exe (which is what executes the package from management studio) is ONLY available as a 32bit app, and does NOT get access to anything above 2gb.
Built a database job that calls the package.... (which runs it with dtexec.exe) runs like a champ, I see it allocate 6.5gb - 7gb of memory, no issues.
Things to know..... BIDS (visual studio) is 64 bit on a 64 bit machine
SSMS (management studio) is 32 bit on a 64 bit machine (32 bit will not access above the 2gb memory limits)
DTEXECUI.exe (called from management studio) is 32 bit on a 64 bit machine
DTEXEC.exe is 64 bit on a 64 bit machine
If your going to access large quantities of buffers, caches etc.... if your getting memory allocation problems.... be SURE your not accessing the package from a 32-bit app.
1) 64 Bit 2 processor dev box running windows 2003 server with 2gb
2) reasonably large data-set, package with cached lookups etc (normal sort of stuff)
3) Execution on the 2gb machine in debug had issues with memory allocation. (memory pressure and allocation failures, then swapping to temp drive, and ran out of space there).... everything worked as it should and failed for obvious reasons, due to resource limitations.
1) 64 bit 8 processor server running windows 2003 server with 64 gb
2) Same reasonably large data-set, package with cached lookups etc (normal sort of stuff)
3) Executed in debug without issues.
...... this is where the pain starts.....
Wanted to see it run (not in debug), but wanted to get progress messages etc without cranking up the logging.... so ran the package by selecting "run package" from management studio.
Would run for a little.... then fail with a message about not being able to allocate the next 100mb buffer, and reported 61gb free memory...
Did a bit of digging about, and found that dtexecui.exe (which is what executes the package from management studio) is ONLY available as a 32bit app, and does NOT get access to anything above 2gb.
Built a database job that calls the package.... (which runs it with dtexec.exe) runs like a champ, I see it allocate 6.5gb - 7gb of memory, no issues.
Things to know..... BIDS (visual studio) is 64 bit on a 64 bit machine
SSMS (management studio) is 32 bit on a 64 bit machine (32 bit will not access above the 2gb memory limits)
DTEXECUI.exe (called from management studio) is 32 bit on a 64 bit machine
DTEXEC.exe is 64 bit on a 64 bit machine
If your going to access large quantities of buffers, caches etc.... if your getting memory allocation problems.... be SURE your not accessing the package from a 32-bit app.