Quantcast
Channel: SQL Server Integration Services forum
Viewing all articles
Browse latest Browse all 24688

A try to speed up a package. Will it work out?

$
0
0
Hello guys,

I have to denormalize a flat file.
Each row of the file contains information about a customer:
 - Customer's Name;
 - Customer's Address;
 - Customer's Phone;
 - etc...
My DataFlow consists of an OLE DB Source and a long chain of OLE DB commands.
Each of the OLE DB Commands executes a Stored Procedure.
The logic of the package is the next:
1) Insert the customer           (Customer table is involved);
2) Insert the customer's address (Address table is involved);
3) Insert the customer's phone   (Phone table is involved);
4) ...
Each stored procedure has its input parameters, e.g:
 - inserting a customer accepts customer's FirstName, LastName, etc;
 - inserting an address accepts HomeAddress, WorkAddress, etc...;
The original input row is quite "heavy" - it has a lot of columns and moreover, most of the columns are of NVARCHAR(between 100 and 300) type.

I am NOT SATISFIED with the package speed and I want to help it a little.
I noticed that many "heavy" columns are used once only, and the subsequent procedures (OLE DB Commands) do not need them at all.
I decided to delete them from the pipe in order to have more memory avaliable to the package - I use a UNION ALL task to accomplish that.
So, the process looks like this:
1) OLE DB Source;
2) OLE DB Command (Insert Customer), wich takes all the columns avaliable from the source;
3) UNION ALL task (named UA1) eliminating the columns (FirstName, LastName, BirthDate, etc);
   The UNION ALL task has only one input = OLE DB Command (Insert Customer), its purpose is just to drop unnecessary columns;
4) OLE DB Command (Insert Address), wich takes all the columns avaliable from the source except for columns having been dropped in task UA1;
5) UNION ALL task (named UA2) eliminating the columns (HomeAddress, WorkAddress, City, Country, etc);
6) ...


So, my questions are:
 - Do you think I will gain anything from that modification?
 - Am I on the right way when using UNION ALL to discard unnecessary columns?
I really interested in your authoritative opinion!

Thanks a lot,
BorkaS

Viewing all articles
Browse latest Browse all 24688

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>