Quantcast
Channel: SQL Server Integration Services forum
Viewing all articles
Browse latest Browse all 24688

Paramterized bulk copy - Disabling indexes for performance causes timeout exception

$
0
0

My requirement is to sling a rowset from one place in SQL server into a table in another place in the most performant way. I want this to be parameterizable -  I want to provide just a connection string and some SQL for the source and a connection string and a table name for the destination.  The package should do the rest. 

The solution I chose was an 2014 SSIS package with source and destination as ADO.NET connections configured from project variables.  The package has a script task to bulk copy the data.  For performance I disable the non-clustered indexes first.  But this preformance precaution causes the bulk copy to timeout after delivering the correct rowcount to the destination table.  Does anyone know what I can do to avoid this error?

Here's my script code:

                //get hold of the source and a data reader from it
                SqlConnection sqlconnSource = new SqlConnection();
                sqlconnSource = (SqlConnection)(Dts.Connections["source"].AcquireConnection(Dts.Transaction) as SqlConnection);
                SqlCommand sourcesqlCommand = new SqlCommand(SourceSQL, sqlconnSource);
                sourcesqlCommand.CommandTimeout = 1500;
                if (sqlconnSource.State==ConnectionState.Closed ) sqlconnSource.Open();
                SqlDataReader reader = sourcesqlCommand.ExecuteReader();



                //get hold of the destination and put the data in it
                SqlConnection sqlconnDestination = new SqlConnection();
                sqlconnDestination = (SqlConnection)(Dts.Connections["destination"].AcquireConnection(Dts.Transaction) as SqlConnection);
                SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlconnDestination);
                bulkCopy.DestinationTableName = DestinationTable;
                bulkCopy.BulkCopyTimeout = 1500;
                bulkCopy.WriteToServer(reader);
                reader.Close();

This takes 128 seconds to put 13 million thin rows into my empty destination table and then throws an exception with this message:

 Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding. 

Any help gratefully received.


Viewing all articles
Browse latest Browse all 24688

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>