SSIS Tries So Hard To Make You Hate It

George

Posted on: 12/23/2008 5:34 PM

I've got a solid 3 years with SSIS having written at least a couple hundred packages and have been using SSIS 2008 for the last 6 months. I can develop packages very quick, some in just a few hours. It has some real performance advantages over using plain old SQL for CERTAIN kinds of jobs (usually where many tables are joined) but I have to admit that the posts on this thread are spot on.

The nuances never stop! It seems like no matter how many times I do something similar, I always experience some weird behavior or bug that makes me have to work around it. For example, recently I was implementing a large dataflow with an OLEDB source using parameter substition (you know where you use the "?" question mark). The OLEDB source would take an hour to start producing data for the rest of the dataflow after I clicked go. It's a simple query with a ? for a single condition in the where clause. To troubleshoot, I copy paste the same query to a query window (replacing the "?" with a value) and I get the whole dataset in less than 5 seconds. I hard coded the value in the OLEDB source, removing the "?", and guess what, I get query results in less than 5 seconds. I discovered that if I use an SQL expression, substituting the variable into the expression, I get results in seconds. For some reason, for this particular package, it does not like parameter substition. I have several other pacakges where I didn't need to do this. The question is WHY? Like everyone else I have a job to get done and unfortunately can't spend lots of time debugging a product my client bought (that is supposed to have been tested).

I just discovered that placing a UNION ALL after a LOOKUP in the dataflow creates a permanent "package contains duplicate column...blah blah blah." I had an ETL that could match on anyone of multiple keys so I was trying to lookup one at a time, if it didn't match, it would go to next match criteria. I was hoping to UNION ALL the matches but guess what, you can't.

By far the worst is the merge join. Connect and dataflow with 100's of columns to a merge join. Whenever you need to modify the merge join, it will take several minutes and all your CPU (and there are 4 CPUs on the dev box I use!!!) to close the merge join after you click ok. I have a package with many merge joins. So guess what. when I have to add a column to the dataflow, it becomes a several hour chore of opening the next merge join, clicking on the column to pass it through, and then clicking to close the merge join. It's horrible for productivity. What's worse is this problem existed in 2005 and they did not fix it for 2008.

Maybe it will be fixed in SQL 2011 or whatever one is next. But many times I have contemplated going back to plain old stored procs and SQL code. At least I know that works 100% of the time and I won't waste a day trying to work around some obscure bug.