My requirement is to sling a rowset from one place in SQL server into a table in another place in the most performant way. I want this to be parameterizable - I want to provide just a connection string and some SQL for the source and a connection string
and a table name for the destination. The package should do the rest.
The solution I chose was an 2014 SSIS package with source and destination as ADO.NET connections configured from project variables. The package has a script task to bulk copy the data. For performance I disable the non-clustered indexes first. But this preformance precaution causes the bulk copy to timeout after delivering the correct rowcount to the destination table. Does anyone know what I can do to avoid this error?
Here's my script code:
//get hold of the source and a data reader from it SqlConnection sqlconnSource = new SqlConnection(); sqlconnSource = (SqlConnection)(Dts.Connections["source"].AcquireConnection(Dts.Transaction) as SqlConnection); SqlCommand sourcesqlCommand = new SqlCommand(SourceSQL, sqlconnSource); sourcesqlCommand.CommandTimeout = 1500; if (sqlconnSource.State==ConnectionState.Closed ) sqlconnSource.Open(); SqlDataReader reader = sourcesqlCommand.ExecuteReader(); //get hold of the destination and put the data in it SqlConnection sqlconnDestination = new SqlConnection(); sqlconnDestination = (SqlConnection)(Dts.Connections["destination"].AcquireConnection(Dts.Transaction) as SqlConnection); SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlconnDestination); bulkCopy.DestinationTableName = DestinationTable; bulkCopy.BulkCopyTimeout = 1500; bulkCopy.WriteToServer(reader); reader.Close();
This takes 128 seconds to put 13 million thin rows into my empty destination table and then throws an exception with this message:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Any help gratefully received.