Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

ssis hell issue

$
0
0

Good day:

SSIS is probably one of the most frustrating ETL tool I've used and the documentation really is bad. I've configured a environmental variable to point to a debug.dtsConfig file whenever running the package through BDIS. However, I'm trying to run it through command line using dtexec with a different dtsConfig file however, it doesn't seem to run to my target database and for some reason I think it's still using my environmental variable configured to debug.dtsConfig instead of:

dtexec /f Package.dtsx /ConfigFile dev.dtsConfig

the target database isn't being populated with data. This is extremely frustrating!!



Ldap data load failed after loading 1000 records

$
0
0

Hi All,

I have defined an LDAP connection in SSIS and used it in my package. It is getting failed after loading 1000 records. Is there any limit for the LDAP Connection manager? If yes, is there a way to load all the records? There are records more than 1000 but not sure why this is failing. Any help regarding this issue is appreciated.

Not able to Access Shared Folder from SQL Server using SSIS

$
0
0

I have created a OLE DB Connection and Created a Flat File Connection pointing to a Shared Folder Location.
When i execute in BIDS, its working fine in placing the file, But when i deploy the code in SSISDB, its errors out, "Cannot Open Datafile", Access Denied

We are using SSIS 2012 and executing the package through stored procedure.

Any Idea!!


SSIS Package and CheckPoints when Moving Large DataSets

$
0
0

We are using SSIS Package (SQL Server 2012) to move data from Source Oracle DB to a Destination Oracle DB using OleDBSource and OleDBDestination Components. We are planning to set the batch size at 10,000 per batch.

The expected volume is about 500,000 (max). In this scenario when SSIS is moving the data in batches of 10,000 - if we enable checkpoints - would SSIS resume from the batch that failed and not have to start over from scratch? Please clarify?  

FileDialog not working in SSIS - Script

$
0
0

Hi,

I seek to work out in SSIS with multiple files most of the same updating on daily basis. However, I can't find the solution to be working for opening the File Dialog to select the files to be considered in this regard.

Please see if any of you can simply provide a code for Script Task so as to Open the File Dialog and thereafter reflecting in the msgbox the fullpath & name of the file in VB.Net.

What I have come upon yet is simply empty msgbox.

Any sample code would be appreciated.

SSIS - File System Task - Move file using variables

$
0
0

Hello,

My configuration contains all information i.e. Folder + Files to load into table and then move these files into archive folder.

So my Foreach loop for files works perfect and now once i am done with import, i need to move these files into archive folder

I have file path and archive folder path.

While using File System task to move, i am getting below error at compile time in BIDS

Error at Move file to Archive folder [File System Task]: "SourcePath" is not valid on operation type "MoveFile".
Error at Move file to Archive folder [File System Task]: "DestinationPath" is not valid on operation type "MoveFile".

I have set Source and destination using Expression with file path and archive path and i don't see anything like "SourcePath" OR "DestinationPath" in expression

I am not sure, what i am missing here.

Can anybody help me to debug the same?


Technical Project Manager cum Architect


SSIS - Multiple Oracle statements

$
0
0

I have created a SSIS package which connects to an Oracle database. My code runs multiple SQL statements, how do I seperate the statements so that it run in 1 Execute SQL task (the ones I have tried do not work)?

Any help much appreciated.

Chris

SSIS BULK INSERT non-standard column delimiter

$
0
0

I just can't seem to figure out how to get SSIS to specify a non-standard column delimiter.

The following BULK INSERT query processes my data file without issue:

BULK INSERT   NZ_POC.[dbo].[v_STG_DCLK_DSM_PROPOSAL_LINE_ITEM]
      FROM 'W:\NZ_POC\ETL_Testing\WORK\dsmapi_ProposalLineItem_20161011_56476.txt'
     WITH   (   CODEPAGE = 'RAW' -- { 'ACP' | 'OEM' | 'RAW' | 'code_page' } ]   
                , DATAFILETYPE = 'char' --   { 'char' | 'native'| 'widechar' | 'widenative' } ]   
                , FIELDTERMINATOR = '0xFE' -- Latin THORN
                , FIRSTROW = 2        -- Skip the header row
                , TABLOCK  
        --, ERRORFILE = 'W:\NZ_POC\etl_error.txt'
    );

After working that out I assumed it would be easy to set the column delimiter / field terminator in SSIS as an expression. 

Does anyone know how to set a high value ASCII character (in this case, the decimal ascii value is 254 = 0xFE)?

I've tried casting the value using DT_STR with several different code pages but they usually result in a truncation error.

Just about every example I've seen online uses either comma or pipe delimiters.

Once I have this nugget figured out I should be all set.

Thanks,

PS: I know I could implement this in SSIS as a SQL Task but it would have to use dyanamic sql, since the package handles multiple input data files and target tables.  It just seems much cleaner to use the built in BULK INSERT task. 

Here's a working prototype of the TSQL dynamically called code:

DECLARE    @INPUT_FILE        NVARCHAR(4000)
        , @FQ_TABLENAME    NVARCHAR(4000)
        , @TSQL            NVARCHAR(4000);

SELECT @INPUT_FILE = QUOTENAME (N'W:\NZ_POC\ETL_Testing\WORK\dsmapi_ProposalLineItem_20161011_56476.txt', '''')
        , @FQ_TABLENAME = N'NZ_POC.[dbo].[v_STG_DCLK_DSM_PROPOSAL_LINE_ITEM]';

SELECT @TSQL = N'    BULK INSERT   ' + @FQ_TABLENAME + N'
      FROM  ' +  @INPUT_FILE + N'
     WITH   (   CODEPAGE = ''RAW''
                , DATAFILETYPE = ''char''    
                , FIELDTERMINATOR = ''0xFE''
                , FIRSTROW = 2        
                , TABLOCK  
    );'

EXECUTE sp_executesql @TSQL;



In SSIS,Script task how to read values from object and writes into DataTable

$
0
0

Hi,

I have some data which I get by running Execute Sql Task ,and I populate this in a  object  called "ResultTest".So my main aim is to check that if there something in Object then Mail it else leave it.

Below is the steps which I performed but no luck in getting proper output.

1.  I have a Object called "ResultSet where I store the value"

2.  I use Script task where I use  this object ,and try to  read the object and pass it into various variable(no.of columns) of type object .

3.Suppose I have data in object like this

Now I want to Read the data and pass the values each column into variable

For Example I have created TimeDifferenceTotal Object so I want to add in values 314,313 and similarly other columns also.   

 public void Main()
        {
            DataTable dtTotal = new DataTable();
            OleDbDataAdapter adapter = new OleDbDataAdapter();
            DataTable dt = new DataTable();

            adapter.Fill(dt, Dts.Variables["ResultSet"].Value);
            // In the first Run dtTotal is created
            if (Dts.Variables["ApplicationNameFinal"].Equals(""))
            {
                foreach (DataColumn dc in dt.Columns)
                {
                    dtTotal.Columns.Add(dc.ColumnName, dc.DataType);
                }
            }

            else // In the next runs dtTotal is retrieved from variable
            {
                dtTotal = (DataTable)Dts.Variables["TimeDifferenecTotal"].Value;
                dtTotal = (DataTable)Dts.Variables["UpdateTime"].Value;
                dtTotal = (DataTable)Dts.Variables["ApplicationNameFinal"].Value;
                dtTotal = (DataTable)Dts.Variables["ServernameFinal"].Value;
                dtTotal = (DataTable)Dts.Variables["DatabaseNameFinal"].Value;
            }

            foreach (DataRow dr in dt.Rows)
            {
                DataRow newDR = dtTotal.NewRow();


                foreach (DataColumn dc in dt.Columns)
                {
                    newDR[dc.ColumnName] = dr[dc.ColumnName];
                }

                dtTotal.Rows.Add(newDR);
            }

            Dts.Variables["TimeDifferenecTotal"].Value = dtTotal.Columns[0];
            Dts.Variables["UpdateTime"].Value = dtTotal.Columns[1];
            Dts.Variables["ApplicationNameFinal"].Value = dtTotal.Columns[2];
            Dts.Variables["ServernameFinal"].Value = dtTotal.Columns[3];
            Dts.Variables["DatabaseNameFinal"].Value = dtTotal.Columns[4];

}






SSIS: 2 server data transfer using dynamic sql

$
0
0

We have two (unlinked) servers: Src and Dst and want to run this statement in an SSIS package

INSERT INTO  <Dst>.<DB2>.<dbo>.[Table2]

           SELECT * FROM <Src>.<DB1>.<dbo>.]Table1]

where  DB1, DB2, Table1, Table2 come from Expressions (we need to run this statement in a loop). Src and Dst are fixed. 

We can't run this in an Execute SQL task as it allows only a single server\DB connection and our statement involves 2 servers and DBs.  Does SSIS offer anything that can handle this type of query?

TIA,

edm2




bulk insert: ssis 2008 checks client for file location, runs on server

$
0
0
Hi,   I was just playing with ssis 2008 bulk insert.   It was very insistent that the input file for bulk insert exist on my laptop.  SSIS wouldn't accept an input file name that did not exist on the laptop.   I was watching the execution of the package in a dos window and it became obvious to me that ssis was dumping the bulk insert code off to the server to run, therefore the input file had to exist on the server to run.   I found a common directory between my server and my laptop and with renames and deletes proved to myself that ssis was checking for the file on the client and actually using the input file that was on the server.   I can't believe that microsoft ships the product with this as the default.  Does my organization have some default install for the ssis client wrong to have this work so poorly?

"Do nothing" Dummy task

$
0
0
Hello

I want to make a first step preexecution event handle.
But later perhaps the first task will not to be the first task.
So I thought I do a "Do nothing" Dummy task and link to this the event handler.
But how can I do this "Do nothing" step? I didn't found like this task.

thnx
shade3

How to expect error msg when SSIS fails

$
0
0

I m new to SSIS 2012. I m using below method in SQL agent job step to run an SSIS pkg, but the same do not return error/failure when SSIS pkg errors and stops.  Basically, I want job step to fail and throw proper error msg of SSIS error.

Declare @execution_id bigint
EXEC [SSISDB].[catalog].[create_execution] @package_name=N'EmailPaystoreETL.dtsx', @execution_id=@execution_id OUTPUT, @folder_name=N'PayrollReportETL', @project_name=N'PayrollETL', @use32bitruntime=False, @reference_id=Null
Select @execution_id
EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,  @object_type=50, @parameter_name=N'SYNCHRONIZED',  @parameter_value=1; -- true
EXEC [SSISDB].[catalog].[start_execution] @execution_id
GO

 Thanks


Mahesh


[Execute SQL Task] Error: Executing the query failed with error: "The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION."

$
0
0

I have an package with source , data flow and destination to execute on begin and commit.

in the below i have just used for questioning purpose with diagram removing my Data Flow task explaining my problem

source

on execution, i get error as

[Execute SQL Task] Error: Executing the query "COMMIT TRAN DimCompanyCode" failed with the following error: "The COMMIT TRANSACTION request has no corresponding BEGIN TRANSACTION.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

as mentioned , i have checked the ResultSet, parameters , connection establish.. all are fine.

execution

Kindly advice on what has to be done.


ShanmugaRaj


SSIS 2014: not able to find 'SSIS Data Flow Items' reg,.

$
0
0

Hi,

I was about to use the SharePoint List adapter. But i couldn't find a way to include it in my solution SSIS 2014 package. I searched many threads but didn't get a proper solution. 

If anyone had got a solution, please let me know. - Thank you!


--------------------------- Radhai Krish | Golden Age is no more far | --------------------------


Web Service Task in SSIS causing Issues when using https URL

$
0
0

I am getting the below error when I use the Web Service Task in SSIS.

Can some one help me on this?

FYI... This error is coming when I am using https:\\.... only but when the Web Service is changed to http:\\... there is no issue.

Error Message:

SSIS package "C:\SSIS Projects\MSSQL\EDI\BoldVan\EDI_BoldVan\EDI_BoldVan\EDI_810.dtsx" starting.

Error: 0xC002F304 at Web Service Task, Web Service Task: An error occurred with the following error message: "Microsoft.SqlServer.Dts.Tasks.WebServiceTask.WebserviceTaskException: The Web Service threw an error during method execution. The error is: Error on verifying message against security policy Error code:1000.

   at Microsoft.SqlServer.Dts.Tasks.WebServiceTask.WebMethodInvokerProxy.InvokeMethod(DTSWebMethodInfo methodInfo, String serviceName, Object connection)

   at Microsoft.SqlServer.Dts.Tasks.WebServiceTask.WebServiceTaskUtil.Invoke(DTSWebMethodInfo methodInfo, String serviceName, Object connection, VariableDispenser taskVariableDispenser)

   at Microsoft.SqlServer.Dts.Tasks.WebServiceTask.WebServiceTask.executeThread()".

please  ask me for more info and I will be happy to provide you.

Thanks

using xml source

$
0
0

Hi 

i have a sp that spits out an execution plan as an xml file, for readability i want to load into the table. When i used xml source and pointed to the xml file, there is something called xsd , can some one please tell me where to look for this xsd file?

Thanks

SSIS - run container based on today's dates.

$
0
0

Hello there - hoping if someone can help.

Im building an SSIS 2014 package that has two sets of tasks in two different containers. I am trying to set it up so that one container will run it's tasks if todays date equals the 15ht (something like DATEPART( "dd" , getdate())=15), and if today is not the 15th (basiclaly every other day of the month) run the other container. These two containers are the only thing inside the package.

Is there a simple way of this?

Handle optional field in flat file loading

$
0
0

Hi All,

I have a requirement to load a flat file (csv), where a particular field is optional (I may /may not get the field in source file)

Scenario1 :

Col1,Col2,Col3,Col4

1,2,3,4

1,2,5,6

Scenario2 :

Col1,Col2,Col4

1,2,4

1,2,6

From the above example, col3 is optional. I need to dynamically map according to the source file (based on the presence of Col3)

Please let me know your suggestions to handle this in SSIS without Metadata Approach

Thanks,


Regards, Dineshkumar,
Please Mark as Answer if my post answers your question andVote as Helpful if it helps you

Dineshkumar's BI Blog


In SSIS,How to read the data from Object and mail in the form of Table

$
0
0

HI,

My main objective is:

1. To read the object which I get through Execite Sql task

2. Then  Using Script task I want to mail it ,but I need to format the values in object in form of table as the body of mail.

3. And IF Time TimeDifference Column had value =<100 Then Row should be Green Else Red.


So I have an Object called "ResultSet",I pass to Script Task, I convert it into c# table structure and place it in a variable called "ApplicationTotal". Below is the code for that which works fine.

public void Main()
{
            DataTable dtTotal = new DataTable();
            OleDbDataAdapter adapter = new OleDbDataAdapter();
            DataTable dt = new DataTable();

            adapter.Fill(dt, Dts.Variables["InactiveSet"].Value);

            // In the first Run dtTotal is created
            if (Convert.ToInt32(Dts.Variables["InsertedRowCountTotal"].Value) == 0)
            {
                foreach (DataColumn dc in dt.Columns)
                {
                   dtTotal.Columns.Add(dc.ColumnName,dc.DataType);
                }
            }
            else // In the next runs dtTotal is retrieved from variable
            {
                dtTotal = (DataTable)Dts.Variables["InactiveSetTotal"].Value;
            }

            foreach (DataRow dr in dt.Rows)
            {
                DataRow newDR = dtTotal.NewRow();
                
                foreach( DataColumn dc in dt.Columns)
                {
                    newDR[dc.ColumnName] = dr[dc.ColumnName];
                }

                dtTotal.Rows.Add(newDR);
            }

            Dts.Variables["InactiveSetTotal"].Value = dtTotal;
            Dts.Variables["InsertedRowCountTotal"].Value = Convert.ToInt32(Dts.Variables["InsertedRowCountTotal"].Value)  + Convert.ToInt32(Dts.Variables["InsertedRowCount"].Value);

Dts.TaskResult = (int)ScriptResults.Success;
}

Then I pass ApplicationTotal object to script task where i have to read the object and mail the details in form of table.I am successful in sending mail but I am not able to format the data in table and change the colour

I get the output in mail as 

enter image description here

 if (Convert.ToInt32(Dts.Variables["InsertedRowCount"].Value) == 0)
        {
            return;
        }


        #region BuildingEmailBody

        StringBuilder sb = new StringBuilder();
        sb.AppendLine(string.Format("Monitor Application Report"));
        sb.AppendLine();
        sb.AppendLine();
        //sb.AppendLine(string.Format("Following are the Details:\n\n<TABLE><TR>\n<TH>{0}</TH><TH>{1}</TH><TH>{2}</TH> <TH>{3}</TH> <TH>{4}</TH>\n</TR>\n", "TimeDifferences(Minutes) ", "UpdateTime", "ApplicationName", "ServerName", "DatabaseName"));

        OleDbDataAdapter adapter = new OleDbDataAdapter();

        if (Convert.ToInt32(Dts.Variables["InsertedRowCount"].Value) > 0)
        {
            sb.AppendLine();
            DataTable ApplicationTotal = new DataTable();
            ApplicationTotal = (DataTable)Dts.Variables["ApplicationTotal"].Value;

            foreach (DataRow dr in ApplicationTotal.Rows)
            {
                sb.AppendLine(string.Format("{0}     {1}      {2}      {3}        {4}  ", dr[0], dr[1], dr[2], dr[3], dr[4]));
            }




            sb.AppendLine();
        }


        # endregion


Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>