Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

SSIS 2008 Package hangs on OLE Data Source - [SSIS.Pipeline] Information: Execute phase is beginning.

$
0
0

Hello All,

My SSIS package hangs on an OLE Data Source with this message ' [SSIS.Pipeline] Information: Execute phase is beginning.

The package hangs for hours and actually doesn't finish. The behavior has begun in the last to weeks.  Prior to that it ran as scheduled weekly.

OLE Data Source  (Production Server X64)

  •  IsSorted=True
  • DefaultBufferMaxRows=10000000
  • DefaultBufferSize=80971520
  • Access Mode - SQL Command
  • Run Time - Never finishes
  • Database located on the SSIS server
  • Observer the SQL commands via Profiler

OLE Data Source (UAT Server X64)

  • Default values - Everything is default
  • Access Mode - Table - View
  • Run Time - 4 minutes
  • Database located on SSIS server

 The View returns results in 10 secs on SSMS and returns, 185K rows (100 chars wide). 

Coping the database and package to a smaller server and running the identical package (Default Settings) it completes in 4 minutes like I would expect. 

Any ideas to improve the performance would be very helpful and much appreciated

Thanks - Covi


Mark Covian


Export Fixed Width File

$
0
0

Hello I am exporting a fixed width text file and it works fine until the end users try importing it into there system. I have the format set to Ragged Right so it separates each record to its own row. When i try importing the file into an external system it would throw an error then i messed around with the file and removed one space at the end. That seemed to do the trick. Is there something that i am configuring incorrectly or need to do in SSIS to fix this? I am using SSIS 2012.

Do you think the using Ragged Right format is adding another column?


Thanks !!

Package Variable Values in For Each Contaner

$
0
0
I have a For Each File Container that loops through Excel files, executing a child package for each file found. I am using the RowCount transform to log the number of rows at various points in a DataFlow. When one file is processed the numbers are correct but when multiple files (e.g. 5) are processed some counts are zero when they shouldn't be. I have tried a 1 second sleep in the loop but that didn't fix the problem.

R Campbell

Dts.Runtime LoadPackage exception

$
0
0

Hi all

I am working with SQL Server 2012 and Visual Studio 2013.
I have installed Integration Services Project Templates in Visual Studio.
Then I have created a new, very simple, Package in Visual Studio.
When I run this package in Visual Studio, it works fine.

When I then use "Microsoft.SQLServer.ManagedDTS.dll" (Version 11.0.3000.0)
to execute the package in a new WinForms Project, I get the following exception (at LoadPackage):
"The package failed to load due to error 0xC0011008 "Error loading from XML. No further detailed error information can be specified for this problem because no Events object was passed where detailed error information can be stored.". This occurs when CPackage::LoadFromXML fails."

What am I doing wrong ?

Thanks for any feedback :-)
Best regards
Frank

P.S. Here is my code:

Microsoft.SqlServer.Dts.Runtime.Package _Package;
Microsoft.SqlServer.Dts.Runtime.Application _Application;
Microsoft.SqlServer.Dts.Runtime.DTSExecResult _DTSExecResult;

_Application = new Microsoft.SqlServer.Dts.Runtime.Application();
_Package = _Application.LoadPackage(@"D:\Package.dtsx", null);
_DTSExecResult = _Package.Execute();

SSIS DTEXEC Memory Issue

$
0
0

I have 4 SSIS packages running on my Windows 2012/ SQL Server 2012 machine to load data from other LAN SQL Servers to local SQL Server database.

Recently I noticed receiving an odd error from data flow task. The error message is just "Unspecified Error" . Investigating the cause of the error, I noticed that DTEXECs constantly eats memory and as you can see below, has high RAM usage (each DTEXEC around 4GB and going up).

Is this common for SSIS? How can I prevent the increase in memory usage and ultimately the error?

Get Variable (Excel Destination Path String) From Script Task to Use For Destination Path In Excel Destination Plus Object

$
0
0

I am dynamically crating excel reports and file names for data input for SSIS package. The following is my VB code for Script Task to copy and rename Excel template:

PublicSub Main()

   
Dim sourcePathAsString="\\server\Dir1\Dir2\Dir3\FileName_TEMPLATE.xlsx"

   
Dim destPathAsString="\\server\Dir1\Dir2\Dir3\FileName"+CDate(Date.Today.Date).ToString("yyyyMMdd")+".xlsx"


   
If File.Exists(destPath)=TrueThen

        File
.Delete(destPath)      'delete existing file'
        File
.Copy(sourcePath, destPath)        'copy template file and rename'

   
EndIf

    Dts
.TaskResult= ScriptResults.Success

EndSub
EndClass

After the Script Task is completed the SSIS process moves into the Data Flow Task which is starts with an Ole DB Source that executes a  stored procedure to select the data I need to import into the new Excel File that the Script Task created.  The Ole DB Source is connected to the Excel Destination Plus (CozyRoc Tool) to import the data into excel.

How do I pass the

destPath

variable that holds the destination path for the new Excel file from the Script Task to the Excel Destination Plus for the data to be imported?

Thanks.



Not able to Access Shared Folder from SQL Server using SSIS

$
0
0

I have created a OLE DB Connection and Created a Flat File Connection pointing to a Shared Folder Location.
When i execute in BIDS, its working fine in placing the file, But when i deploy the code in SSISDB, its errors out, "Cannot Open Datafile", Access Denied

We are using SSIS 2012 and executing the package through stored procedure.

Any Idea!!


SSIS Foreach Loop Container not processing all files

$
0
0

Morning

Currently there are 3 csv file in the directory. The for-each loop container only process the first file. Everything was working fine then all of a sudden only the first file get's processed. 

Ty

W


DTS_E_INDUCEDTRANSFORMFAILUREONERROR

$
0
0

Hi,

I am facing following error. 

[ABC ORACLE insert [2]]  Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available.  Source: "OraOLEDB"  Hresult: 0x80004005  Description: "ORA-03113:
 end-of-file on communication channel
Process ID: 0
Session ID: 1177 Serial number: 348".

[ORACLE insert [2]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "ABC ORACLE insert.Inputs[OLE DB Destination Input]"
 failed because error code 0xC020907B occurred, and the error row disposition on "JFSL ORACLE insert.Inputs[OLE DB Destination Input]" 
specifies failure on error. An error occurred on the specified object of the specified component.  
There may be error messages posted before this with more information about the failure.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "ABC ORACLE insert" (2)
 failed with error code 0xC0209029 while processing input "OLE DB Destination Input" (15).
 The identified component returned an error from the ProcessInput method. 
The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. 
There may be error messages posted before this with more information about the failure.

Kindly give your input.

Thanks,
Vivek Singh

Unable to upgrade SSIS packages to SQL Server 2014

$
0
0

Hi there,

I'm trying to upgrade our SSIS packages from SQL Server 2012 to 2014 (in Visual Studio 2013).  The wizard pops up the minute the project loads and I leave everything to default.

It errors immediately though on almost every package with the following error:

Error 0xc001000e: MyPackage: The connection "{5F69F91E-8CD7-49A7-A76A-815244411037}" is not found. This error is thrown by Connections collection when the specific connection element is not found.

That connection is a project connection manager and most definitely exists.  In fact, I can open the individual package and it upgrades it for me and the package is completely fine.  It seems to me that the upgrade wizard has a bug whereby it doesn't look for project connection managers.

Any help would be greatly appreciated, otherwise I might just post it to connect as a bug.

How to pass values to SQL server Agent Job which internally calls SSIS

$
0
0

Hi,

I have a requirement where i need to call remotely SSIS package from ASPX. After following MSDN form, I have creaed one SQL server agent job and calling that job from ASPX. I am ablt call SQL server agent job from ASPX. Only issue i am facing is: I need to pass 2 variables from ASPX to SQL Agent job so that from Job i can map the same to SSIS Package.

I am using system SP (sp_start_job) for calling SQL server agent job as mentioned in MSDN forum.

below is the code , which i am using to call SQL serve agent job.

Code:

SqlConnection

jobConnection;

SqlCommand jobCommand;

               

SqlParameter jobReturnValue;

               

SqlParameter jobParameter;

               

int jobResult;

                jobConnection =

newSqlConnection("Server=ServerName;Initial Catalog=msdb;Integrated Security=SSPI;user id=username;Password=password");

               

                jobCommand =

newSqlCommand("sp_start_job", jobConnection);

                jobCommand.CommandType =

CommandType.StoredProcedure;

                jobReturnValue =

newSqlParameter("@RETURN_VALUE",SqlDbType.Int);

                jobReturnValue.Direction =

ParameterDirection.ReturnValue;

                jobCommand.Parameters.Add(jobReturnValue);

                jobParameter =

newSqlParameter("@job_name",SqlDbType.VarChar);

                jobParameter.Direction =

ParameterDirection.Input;

                jobCommand.Parameters.Add(jobParameter);

                jobParameter.Value =

"ImportCMTData";

                jobConnection.Open();

                jobCommand.ExecuteNonQuery();

                jobResult = (

Int32)jobCommand.Parameters["@RETURN_VALUE"].Value;

                jobConnection.Close();

               

switch (jobResult)

                {

                   

case 0:

                       

Console.WriteLine("SQL Server Agent job, RunSISSPackage, started successfully.");

                       

break;

                   

default:

                       

Console.WriteLine("SQL Server Agent job, RunSISSPackage, failed to start.");

                       

break;

                }

               

Console.Read();

Thanks

Pankaj

SSIS Merge Join Error

$
0
0

Hi all,

I have a simple data flow.  I'm trying to join two sets of data from two different databases.  The TXT database is collation SQL_Latin1_General_CP1_CI_AS. The other dataset (Cost) is coming from Dynamics AX which is Latin1_General_CI_AS.

AX columns are Unicode.  So I have put data conversion on TXT columns to Unicode

However no matter what I do I cannot get rid of the STUPID MERGE JOIN ERROR and it's driving me insane. 

TITLE: Microsoft Visual Studio
------------------------------

Both inputs of the transformation must contain at least one sorted column, and those columns must have matching metadata.

Both datasets are set to sort by ITEMID (and I have set sort keys in both).  Any ideas???

How to Run SSIS Package through SQL AGENT Job

$
0
0

I am good with SSIS, I created a package which will load the excel files to SQL SERVER  tables. To Implement this I wrote the C# code to extract the first sheet name only, and then package will load the data present the first sheet only. The package working fine when i run this through BIDS. But the same package throwing error at C# code step when i am running this package through SQL AGENT JOB. When i searched for this, i have seen many posts saying that, this is happening becoz of using Microsoft.interop.excel references inside the c# code. Please help me with this. Is there any alternate way in c# to extract the first sheet name(not based on ascending order) of an excel file with out using interop. Or

How to configure SQL AGENT job to run suceessfully when using interop inside the code.

My UAT server is of 64 bit. And i have to run this in the same server.

I tried creating  "Desktop" folder in "C:\Windows\System32\config\systemprofile\"

Error:Description: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object.     at ST_ecfa668f250a45e18c95639c9ffd64d4.csproj.ScriptMain.Main()    --- End of inner exception stack trace ---     at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)     at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)     at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)     at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, CultureInfo culture)     at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()  End Error  Error: 2015-02-27 11:24:00.23     Code: 0x00000001    Source: User Mail      Description: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)     at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)     at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)    at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, SqlConnection owningObject)    at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject)     at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart)     at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)     at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)     at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)    at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)     at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owning... The package execution fa...  The step failed

using System;
using System.Data;
using System.Diagnostics;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Data.OleDb;
using Microsoft.Office.Interop;
using System.Runtime.InteropServices;

namespace ST_ecfa668f250a45e18c95639c9ffd64d4.csproj
{
    [System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        #region VSTA generated code
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion


        public void Main()
        {   /*Passing the file path via User::File_Name Variable*/
            string FileName = Dts.Variables["User::File_Name"].Value.ToString();
            Microsoft.Office.Interop.Excel.Application xlApp = null;
            Microsoft.Office.Interop.Excel.Workbook excelBook = null;
            try
            {
                xlApp = new Microsoft.Office.Interop.Excel.Application();
                excelBook = xlApp.Workbooks.Open(FileName, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing);
                string[] excelSheets = new string[excelBook.Worksheets.Count];
                int i = 0;
                foreach (Microsoft.Office.Interop.Excel.Worksheet wSheet in excelBook.Worksheets)
                {
                    excelSheets[i] = wSheet.Name;
                    i++;
                }
                Dts.Variables["User::WorkSheetName"].Value = excelSheets[0] + "$";
            }
            catch (Exception ex)
            {
                excelBook.Close(false, FileName, null);
                Marshal.ReleaseComObject(excelBook);
                string error = ex.Message;
            }
            finally
            {
                excelBook.Close(false, FileName, null);
                Marshal.ReleaseComObject(excelBook);
            }

            Dts.TaskResult = (int)ScriptResults.Success;
        }
    }
}



Sql Server agent Jobs running

$
0
0

Hi All,

We have a scenario where we have to run 30 pkgs at same second. Each pkg is designed to fetch 60 lakh per day records and load it into Oracle/OLEDB destination. For which we have created 30 SQL agent jobs for each pkg running every 5 mins 24/7. We want to test running with option(1) these 30 jobs on single sql server instance and option(2) 10 jobs each on 3 sql server instances. Can someone please help me out to understand which would give me the better results.

a) The main advantage of multiple sql server instance scenario when compared with single sql server instance?
b) How much does the hardware configurations impact on the performance?

Thanks in Advance....

Try to run SSIS pack as an SQL Job

$
0
0

I have created an SSIS pack using SQL Server Business Intelligence Development studio for SQL2008.

When I run the package in VS, it runs perfect without errors. I make sure that the users and paswoord on the datasources is filled in, set the protection level to ServerStorage, and then I save a copy tot the MSDB in the SQL server.

I have 1 solution file with two packs, that I save separtly.

When I then logon to the Integration Services, I can run the package by right click and then run package.

But when i try to run the package as a scheduled job by the SQL server agent, the the package starts and then it says :

execute job XXX, but the status comes to "in progress" and never changes. If I stop the job and go to the history it just tells nothinh, no errors, just nothing.

I have been using these things before and never had an issue, but now I am getting totally puzzeld.

What am I doing wrong ?


is there a how-to on ssis web services with salesforce.

$
0
0

I would like to know more step by step examples on using WebServices with SSIS.

I've gotten the WSDL but, doesn't help me getting data and don't know what else to look.


-- IBM has jobs for USA employee to work oversea (India) and get their wages.

Client framework within SSIS to Access OData Remote Host

$
0
0

I need some advice on which direction to go regarding the best way to import data from an OData remote host to an mssql2012 instance. On average the data that I would need to import consists of 15 data objects, 8 of which require about 14,000 rows averaging about 1,200 bytes per row. The other remaining objects average about 1/3 of that. This will execute about 4 times per day.

I have created an SSIS controller package using C# scripting with the System.Net.Http classes as a POC, which manages the thresholds and parallelism of accessing and writing the data. This was developed on the Visual Studio version 12 IDE, some of which I made visible here: (Stackoverflow Code Review ) . The code works, however, there are timeouts that manifest as thread drops/task cancellations, etc. during peak operational hours.

It is time for me to refactor this package in such a way, that it can be used as an OData pattern we can use in our shop going forward.  I am uncertain how to address this challenge. There are conflicting references online--some approaches lean toward WCF, others point out newer asp.net OData assemblies, etc. I am unclear on Microsoft's direction regarding OData consumption of remote data. There is more published clarity on providing OData services as a provider for clients to consume.

The more ideas, thoughts, etc. that I hear from this community the better! :)




DTS packages

$
0
0
I have migrated the DTS packages (which was running in SQL 2000 Server) to SSIS Packages (2008) (through
Migrate 2000 DTS Package option in Visual Studio).
The below is error I am getting when we execute the SSIS package in SQL Server 2008 R2 Server.
Environment : SQL Server
2008 R2
Version : 64 bit
Version
Date  2/18/2015 5:10:15 PM
Log  Job History (Oracle to Access - PROD)
Step ID  1
Server  CSVP000A1037\WWSHRD02
Job Name  <TestJob>
Step Name  Test Oracle to Access
Duration  00:00:01
Sql Severity  0
Sql Message ID  0
Operator Emailed 
Operator Net sent 
Operator Paged 
Retries Attempted  0
Message
Microsoft (R) SQL Server Execute Package Utility
Version 10.50.6000.34 for 64-bit
Copyright (C) Microsoft Corporation 2010. All rights reserved.
Started:  5:10:15 PM
Error: 2015-02-18 17:10:16.42
   Code: 0x00000000
   Source: TEST_TABLE
   Description: This task does not support native Win64 environment. Please run the package in 32-bit WOW environment instead.
End Error
Error: 2015-02-18 17:10:16.42
   Code: 0xC0024107
   Source: TEST_TABLE
   Description: There were errors during task validation.
End Error
DTExec: The package execution returned DTSER_FAILURE (1).
Started:  5:10:15 PM
Finished: 5:10:16 PM
Elapsed:  0.546 seconds
 
The below one is the SQL script command which we have used in 2008 Server we are trying to execute through SQL Job scripts.
 
@command = N'dtexec /ser servername\instancename /sq Oracle2Access_PROD', @database_name = N'', @server = N'', @database_user_name = N'', @subsystem = N'CmdExec', @cmdexec_success_code = 0, @flags = 0, @retry_attempts = 0, @retry_interval = 0, @output_file_name = N'', @on_success_step_id = 0, @on_success_action = 3, @on_fail_step_id = 0, @on_fail_action = 2

 

I have set this package property Run64BitRunTime = False

 Please help me how to resolve the issue.

Thanks,

Vijay

Script Task to apply styling to Excel Sheet

$
0
0

Hi Everyone,

I have a "Execute SQL Task" writing full result set to a variable. I have to export this data to Excel sheet(destination) and the excel sheet needs to have styling (Medium 2). 

So far I was able to exported data to a template (Using OLE DB source and Excel Destination) but I am not getting the designated results.

Please Help.

Running SSIS from dtexec

$
0
0
I have a project that extracts data from a sql 2008 r2 database and writes it to an excel (2007) file.  When I run it in debug everything works fine but when I run it from a command line I get lookup errors (32 bit) or odbc errors (64 bit).  Any ideas on how to configure or run the package?
Viewing all 24688 articles
Browse latest View live


Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>