Hi,
We have an SSIS job that runs well in our datacenter servers but is failing on strong Amazon AWS windows servers (hi1.4xlarge, which use SSD storage).
We have tried everything - we had Amazon AWS support on this case for several weeks and they can't see evidence that the problem is with their infrastructure. They have produced trace files of performance monitor but I can't attach to this post. What we see is that SSIS job which is using "SQL Destination" runs fine for a few good minutes and then it hangs. We see a wait PREEMPTIVE_SHAREDMEM_GETDATA which keeps accumulating. It never hangs at the same place.
When it hangs, sometimes there is enough free memory (like 8GB) so it doesn't seem like a lack of memory problem.
We tried everything - increased the amount of virtual memory to the recommended amount of 93GB; Reduced the batch commit size; Installed the most recent service packs and rollup updates (SP2 + rollup update 4), etc. Nothing helped.
We are using SQL Server enterprise edition and the destination table is partitioned and page compressed. As I said earlier, it works well on our datacenter and need to understand why it doesn't work and how to work around the problem to get this data to load.
I did see a few other MSDN support groups mentioning this wait type but the responses on these cases are not leading to a solution.
AWS Support team has just asked me to get Microsoft support involved since they can't seem to find the problem. I have "perf monitor" trace of many performance indicators which I can send (I wish I could attach this to the ticket). We have an MSDN license and I believe that we qualify for a few support cases per year. We have not used Microsoft support during last year. Please help.