Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Data extraction from text file

$
0
0

Hi Experts,

I need to regularly load data from a text files saved on a given FTP location. The text file contains numerous records of data.

4 records each together make a single row of data.So for example excluding the headers starting the second record in the text file

record position starting at 2 till 5 

2-5 = 4 records forming a single record

6-9 = 4 records forming another set of single record.

So please assist in accomplishing the below given requirements.

1. Form single record from a set of 4 records.

2.Perform condition based extraction of data from the single row of data formed from the set of 4 records.

3.The records meeting the required conditions need to be loaded into a folder.

4.The records failing to do from the text file need to be moved together in a separate folder.

5. The read source text file after finishing all the above conditions need to be moved to a different location.

Please advice.

Thanks

Priya


OntaskFailed event handler issue in SSIS

$
0
0

Hello

I am trying to send a message with only the information of the tasks that fail in a sequence container. I have two packages: Parent-Child. In the first one the task Query_2 and child package are going to fail. The child package only have one single sql task that provokes its fail.

In the event handler I have created an ontaskfailed event for the sequence container. The idea is to receive a message for the query_2 and the child package steps. But I'm receiving a message for the child query as well (query_4_child). And this is want I want to avoid.

I have proved setting the value of the propagation variable to false. But this did not work. I also have tried to implement an Onerror event but without any succeed.

What can I do in order to reach this?


Script or tool to identify DW tables used in a SSIS Project

$
0
0

Hi,

im a SSIS newbie and i was asked to try to find a tool or script that identifies the DW tables used in a SSIS Project.

I don't know if there is any (free) toll that does some report like this, or a script?...

Thanks and regards.

Updating Oracle destination using staging table statement error

$
0
0

Hi,

I'm trying to add only new rows and update rows that have changes instead of truncating data each time I load the data.

I managed to perform the task from Oracle to SQL using Lookup transformation but an error occurred from SQL to Oracle updating. Upon further research, it seems that Oracle have trouble performing row by row update using OLE DB Command. Thus, I followed a guide (http://www.techbrothersit.com/2014/04/ssis-how-to-create-use-temp-table-in.html) and stored the changed rows in a ##Temp OLE DB Destination table(SQL) and execute an SQL task after my data flow task to update it into Oracle. However, it seems that Oracle do not support JOINS like SQL.

I tried it with this statement from(http://dba.stackexchange.com/questions/3033/how-to-update-a-table-from-a-another-table) but there's still error in it. Please help me thanks :(

EMPSQL is my Oracle table and ##Temp is the temporary SQL staging table. Data that needs to be updated are EmployeeName, EmployeeAge, EmployeeEmail and EmployeeAddress.

MERGE INTO EMPSQL
USING 
(
SELECT 
EmployeeID,
EmployeeName,
EmployeeAge,
EmployeeEmail,
EmployeeAddress
FROM ##Temp
)SRC ON (SRC.EmployeeID = EMPSQL.EmployeeID)
WHEN MATCHED THEN UPDATE 
SET 
EMPSQL.EmployeeName = SRC.EmployeeName,
EMPSQL.EmployeeAge = SRC.EmployeeAge,
EMPSQL.EmployeeEmail = SRC.EmployeeEmail,
EMPSQL.EmployeeAddress = SRC.EmployeeAddress

Send Mail component hangs in SSIS

$
0
0

I have a very simple SSIS package that moves data from a source table to a target table.  I have created an event handler for OnError that sends the Error Description via "send Mail" component for notification.  I have purposely created a failure to try to insert a null value into a not nullable field which should have triggered the OnError.  As it moves down the control flow the send mail just hangs with no messages or errors.  This basically causes my workstation to lock up that I need to hard boot each time.  I have 3 other packages with the exact OnError code within the same solution which is working fine except for this package. I have tried to recreate the package manually but seem to have the same issues.  Not sure what is causing the the send mail component to hang but would be great if someone could help me understand how I could trouble shoot this issue or identify why my workstation is hanging when trying to send an email.

Using #tempTables in SSIS

$
0
0

First I am very new to SSIS as I learn more about its capabilities I choose to use it more often. I am creating a SSIS package to load records to a Database. I load records to a temporary table and use that table to build the parent table records. I have a table called tenp_Updates which is a real table I use for this. It is cleared before each run. My table and log sit out there until the next run taking up 20gb of space. I was wondering if there is a way to create these as #temp_tables so at the end of the control flow task they just disappear. Below is my control flow. Data is loaded in the Import Raw Data step and then extracted to create the jobs table and of course fill the history table. I would love to not have to keep the 20gb of data as I don't care about it after it is loaded to the main tables. How I would create the table, would I create or use an existing connection? If I create it in one task can I use it in another? 

How to Edit Data Flow Task Programmatically using C#

$
0
0

How to Edit Data Flow Task Programmatically using C#

hi i have SSIS Package & I wan to add oledb source in data flow task which is already present in SSIS Package How to do that? any body please help.

Thanks in Advance

Script Task not picking the correct values

$
0
0

Hi All,

I have an Excel source which is loading data to an OLE DB destination. Before loading data, there is a script task which finds out the first column and the first row within the excel sheet. The logic is that wherever the word 'Table' is present in the file, it should be the first column.

For some reason, the script task is not picking the correct start and end columns. For now, I have hard coded the values but this needs to be fixed asap. 

Below is my code :

/* Microsoft SQL Server Integration Services Script Component

*  Write scripts using Microsoft Visual C# 2008.

*  ScriptMain is the entry point class of the script.*/

using System;

using System.Data;

using System.Windows.Forms;

using Microsoft.SqlServer.Dts.Pipeline.Wrapper;

using Microsoft.SqlServer.Dts.Runtime.Wrapper;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]

publicclassScriptMain :UserComponent

{

    //class variables declaration

    int intRow;

    int intColumn;

    int intStartRow;

    int intEndRow;

    int intStartColumn;

    bool bolCheck;

    publicoverridevoid PreExecute()

    {

       base.PreExecute();

       //initializing the variables

        intRow = 0;

        intColumn = 0;

        bolCheck =false;

    }

    int Asc(char chrReturnMyAscii)

    {

       //return the character's ASCII value of the given character

       return (int)System.Convert.ToChar(chrReturnMyAscii);

    }

    Char Chr(int intReturnMyChar)

    {

       //return the character of the given character's ASCII value

       returnConvert.ToChar(intReturnMyChar);

    }

    publicoverridevoid Input0_ProcessInputRow(Input0Buffer Row)

    {

       //incrementing the row counter each time a row is processed.

        intRow = intRow + 1;

       if (intRow > 12) { MessageBox.Show(intRow.ToString()); }

       //checking if the "Table" word has already been found.

       if (bolCheck == false)

        {

           //checking if the end of rowset has been reached or not.

           if (Row.EndOfRowset() == false)

            {

               //resetting column counter to 0 each time a row is processed.

                intColumn = 0;

               //loop to check the columns for the current row to find the "Table" word.

               foreach (System.Reflection.PropertyInfo piColumnin Row.GetType().GetProperties())

                {

                   //incrementing the column counter each time a column is processed.

                    intColumn = intColumn + 1;

                   if (intRow > 12)

                    {MessageBox.Show(intColumn.ToString());

                   MessageBox.Show((Convert.ToString(piColumn.GetValue(Row,null))));

                    }

                   MessageBox.Show((Convert.ToString(piColumn.GetValue(Row,null))));

                    //checking if the column's value is "Table"

                   if (Convert.ToString(piColumn.GetValue(Row,null)) == "Table")

                    {

                       MessageBox.Show((Convert.ToString(piColumn.GetValue(Row,null))));

                       //setting the variables when the "Table" word is found in the source file.

                        bolCheck =true;

                        intStartRow = intRow + 1;

                        intStartColumn = intColumn;

                        MessageBox.Show(intStartColumn.ToString());

                       MessageBox.Show(intStartRow.ToString());

                       MessageBox.Show(bolCheck.ToString());

                       break;

                    }

                }

            }

        }

       //storing the last row number into the variable.

        intEndRow = intRow + 1;

       //MessageBox.Show(intEndRow.ToString());

    

    }

    publicoverridevoid PostExecute()

    {

       int stepno = 0;

       try

        {

           base.PostExecute();

           //declaration of the variables used in the PostExecute method.

           int intStartColumnIndex;

           string strStartColumnDesc;

           int intEndColumnIndex;

           string strEndColumnDesc;

           int intStartColumnNumber;

           int intTotalColumn;

           // Lock variable for read

           VariableDispenser variableDispenser = (VariableDispenser)this.VariableDispenser;

            variableDispenser.LockForRead("User::vintTotalColumns");

           IDTSVariables100 vars;

            variableDispenser.GetVariables(out vars);

           // Fill the internal variable with the value of the SSIS variable

            intTotalColumn = (int)vars["User::vintTotalColumns"].Value;

            stepno = 0;

           //calculating and storing the actual column number where the word "Total" is found.

            intStartColumnNumber =Convert.ToInt16(Math.Round(Convert.ToDecimal(intStartColumn + 1) / 2));

            intStartColumnIndex = intStartColumnNumber % 26;

           MessageBox.Show(intStartColumnNumber.ToString());

           if (intStartColumnIndex == 0)

            {

                intStartColumnIndex = 26;

            }

            stepno = 1;

           //logic to get the column name of the columns.

           if (Convert.ToInt16(Convert.ToInt16(intStartColumnNumber - intStartColumnIndex) / 26) == 0)

            {

                stepno = 2;

                strStartColumnDesc =Convert.ToString(Chr(Asc('A') + (intStartColumn - 1)));

                   //Convert.ToString(Chr(Asc('A') + (intStartColumn - 1) / 2));

               MessageBox.Show(strStartColumnDesc.ToString());

            }

           else

            {

                stepno = 3;

                strStartColumnDesc =Convert.ToString(Chr(Asc('A') +Convert.ToInt16((intStartColumnNumber - intStartColumnIndex) / 26) - 1) + Chr(Asc('A') + intStartColumnIndex - 1));

            }

            intEndColumnIndex = (intStartColumnNumber +Convert.ToInt16(intTotalColumn) - 1) % 26;

            stepno = 4;

           if (intEndColumnIndex == 0)

            {

                intEndColumnIndex = 26;

            }

           if (Convert.ToInt16(((intStartColumnNumber +Convert.ToInt16(intTotalColumn)) - intEndColumnIndex) / 26) == 0)

            {

                stepno = 5;

                strEndColumnDesc =Convert.ToString(Chr(Asc('A') + intStartColumnNumber +Convert.ToInt16(intTotalColumn) - 1));

               MessageBox.Show(strEndColumnDesc);

            }

           else

            {

                stepno = 6;

                strEndColumnDesc =Convert.ToString(Chr(Asc('A') +Convert.ToInt16((intStartColumnNumber + Convert.ToInt16(intTotalColumn) - intEndColumnIndex) / 26) - 1)) + Chr(Asc('A') + intEndColumnIndex - 1);

               MessageBox.Show(strEndColumnDesc);

            }

            stepno = 7;

           //storing the values into the packages variables.

           this.ReadWriteVariables["vstrStartColumnTOT"].Value ="G"; // strStartColumnDesc;

           this.ReadWriteVariables["vstrEndColumnTOT"].Value ="AW"; // strEndColumnDesc;

           this.ReadWriteVariables["vintStartRowTOT"].Value = intStartRow;

           this.ReadWriteVariables["vintEndRowTOT"].Value = intEndRow;

           // Unlock variable

            vars.Unlock();

        }

       catch (Exception ex)

        {//Dts.Events.FireError(0, "Script Task Example", ex.Message ,0);

            bool pbCancel =false;

            this.ComponentMetaData.FireError(0,"", "An error occurred: at"+ stepno.ToString() + ex.Message.ToString(),"", 0, out pbCancel);

        }

    }

}

Please help!

Thanks,

A


Regarding Excel Destination, duplicate worksheet created

$
0
0

Hi, I have to output the results to an excel destination. I read that unlike flat file, excel connection does not offer the function of clear and load. Hence i have created 2 Execute SQL Task to perform DROP and CREATE the table before each data flow task.

Drop query: 

DROP TABLE `Report`

Create table query: 

CREATE TABLE `Report`(
    `Brand` LongText,
    `Location` LongText,
    `Cost` Decimal(19,2)
)

In the excel destination table, i selected "New" and paste in the same code as above, for creation of Report table. 

Not sure what is the root cause, but Report1 always gets created and data written there instead of Report. 

Anybody know where my mistake could lie in? Thank you.

Use SP or SQL Query

$
0
0

I've written a SQL query to generate .txt file using SSIS pkg.  One of the column value in select query needs to be examine and generate email if certain conditions are not met and send that entire rec in the email.

I need suggestions on below questions

1. Shall I use SP or direct SQL query in the pkg, If I use SP how can I check the value of the column to generate email

2. Do I've to use script task for generating the .txt file and placing the SQL select result init. Is it possible to create a .txt file with out using script task.  

 

Thanks!

 



Neil

How to use "%" in value, for a @FileName variable with chaning name on the file. (SSIS)

$
0
0

Hi,

I have Filenames with names ABC1501, ABC1502, ABC1503... The last 4 numbers are for years and month and will change every month.

I want to set the value for a variable to "ABC%" but it doesn't work. 

Is it any other symbols that works in SSIS?

I'm using the variable in a script task.

Best regards

Baden 

ways of creating SSIS package for below query

$
0
0

Hi - what are different ways to create an SSIS package for this ?

and what is the easiest way to create an SSIS package from below query ? I was suggested using a pwershell but any other approach ideas will be appreciated ?

Server:                                 xyz-a-SQL-03
Database:                            msR
Stored Procedure:           sprTblGetUserAccessabc
Query:
select        UserName             =      ums.UserName,
                     FName                =      usr.FName,
                     LName                =      usr.LName,
                     GrpName                    =      grp.GrpName,
                     ModuleName           =      grp.ModuleName,
                     RoleName             =      grp.RoleName,
                     Email                =      ums.Email,
                     Status               =      case
                                                       when ums.StatusId = 1 then 'Enabled'
                                                       else 'Disabled'
                                                       end,
                     LastLogDate          =      convert(varchar(10),ums.LastLogDate , 111) + ' ' + convert(varchar, ums.LastLogDate , 114),
                     RegCode                    =      ums.GrpCode,
                     RegName                    =      ums.GrpName,
                     CpnyCode             =      ums.CpnyCode,
                     CpnyName             =      ums.CpnyName

from          mdm.TblUserMemberShip             as     ums           with (nolock)
                     inner join
                     rpt.LstADUsers                           as     usr           with (nolock)        on     usr.UserId = ums.UserName
                     inner join
                     rpt.LstADGroups                          as     grp           with (nolock)        on     grp.AppCode = 'MDM'
                     inner join
                     rpt.LstADGrpsUsrs                 as     gxu           with (nolock)        on        gxu.UserId = usr.UserId and
                                                                                                                                  gxu.GrpName = grp.GrpName
where         ums.BatNo            =      (select max(BatNo) from mdm.TblBatches with (nolock))
                     and
                     ums.CpnyCode  =      case
                                                       when @CpnyCode is null then ums.CpnyCode
                                                       else @CpnyCode
                                                       end
order by      ums.Name, grp.GrpName, ums.CpnyCode



MBH

Partitioning - rebuild indexes question

$
0
0

Hi all,

Another question that is not really clear to me yet.

I recently introduced a partition strategy. What I wonder now, is that when we rebuild our indexes every load, I assume that the benefit of partitioning is that we can now rebuild our index on the latest partition only.

Is this a good idea? Why yes/no?

How do I tell SQL server to only rebuild the latest partition? By default when looking at the rebuild script he takes all partitions.

USE [LNL_STG]
GO
ALTER INDEX [NCI_X_DT_EXTRACT] ON [itvm].[TL_ITVM_SALES] REBUILD PARTITION = ALL WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO

Thanks!

How To Convince Mgt to Enable MSDTC

$
0
0

Good morning, oh esteemed SSIS Superiors!

In another thread, I detailed a need to use TransactionOption to make use of Sequence Containers.

I learned that, in order to use that setting, I must have MSDTC enabled on my local box and the server.

If this service is not running, then an error, "Failed to acquire connection 'Connection'. Connection may not be configured correctly or you may not have the right permissions on this connection," occurs when I haveTransactionOption set to "Required."

I was easily able to set MSDTC on my local box, but management has decided NOT to enable MSDTC on the server, blocking my use of Sequence Containers.

These seem to be useful tools, but I don't have the experience with the package to be able to advocate enabling MSDTC sufficiently. Seems that the ability to roll back events in a failed sequence wasn't enough - though, it seems to be a pretty darn good argument to this novice.

So, my question for the group is, what other arguments might I use to gently nudge management into turning on MSDTC without their being aware of being nudged?

How to achieve successful data loading using package, where lengthy data field of table1 inserts into table2 having small length field

$
0
0

Hi,

I have a scenario in which I need to successfully load the 250 char field in Table1 inserts into Table2 of corresponding field length of 50 char. Can anyone help how can achieve this package loading without any error using SSIS ?

Thanks,

Srinivas


SSIS connector to non azure cloud sql database

$
0
0

Hi,

Can I use SSIS to connect to non azure cloud like verizon cloud(sql server Database)  and do all the general ETL  accomplished in SSIS with the Non azure based sql DB too?

Thanks!

Samir

Cannot see Integration Services Catalogs node in SQL Server Management Studio (SSMS)

$
0
0

I wanted to create a SSIS catalog and the way to create a catalog is to right click on "Integration Services Catalogs" node and select "Create Catalog" option.

However, I am not able to see the "Integration Services Catalogs" node in SQL Server Management Studio (SSMS) as shown in the following image.

What is missing so that I can see this node in SSMS? Please help.

FYI, I have SQL Server 2014 (Developer edition) and Integration Services 12.0 installed on my machine.

ProductVersion = 12.0.2269.0 (SELECT SERVERPROPERTY('productversion') AS ProductVersion)


References:

Same question on Stackoverflow


SSIS data conversion Numeric[DT_NUMERIC]

$
0
0

ETL from SQL to Excel 

I used the Data conversion tools to transfer the data from SQL to Excel

the issue is in the numeric field, when transfer to the excel, the columns  show up as text field

in the data conversion the output columns datatype some are Numeric[DT_NUMERIC],four-byte signed integer [DT_I4] or double-precision float [DT_R8] .. 

but those field in EXCEL_DESTINATION  under External Columns Datatype come as Unicode string [DT_WSTR].. the issue here is when i change this data type to Numeric[DT_NUMERIC] or any . Don't accept it or don't save it.. when I close and open again the data type still show [DT_WSTR]


Is SSIS the "right" way to export to a EDI X12 flat-file?

$
0
0

Hi

Most of the documentation I've found deals with parsing an EDI file.   I need to *write* to an EDI file, not read it.

The flat-file end result should look like this (note the 3 headers and 3 footers ):

ISA^00^          ^00^          ^ZZ^SUPPLIER       ^ZZ^ABCCorpT    ^151125^1002^U^00602^000000002^0^P^>~
GS^PT^SUPPLIER^ABCCorpT^20151125^1002^2^X^006020~
ST^867^0002~
BPT^00^20151125100241^20151125^SS~
CUR^SE^CAD~
...
...
...
SE^18908^0002~
GE^1^2~
IEA^1^000000002~


The data is coming from SQL Server.  The columns are all different for each row, and there are loops (eg. N1, N2, N4 fields for address may all get repeated if shipping and address are different)

If SSIS is capable of producing this, can anyone refer me to a walkthrough or tutorial that shows how to serialize data into this format?

Sidenote:  I understand that it's possible in Biztalk, we 

Extract (Parse) the data from SQL Server as Xml  
Map (xslt transformation) to the EDI Schema
Serialize XML into EDI

Is Biztalk overkill, and/or is SSIS underkill?

Thanks for any guidance..

Cheers!



Thanks! Dan

Developers keep changing data source for my SSIS package

$
0
0

I am just getting into building a data warehouse for a BI project. We also have acquired a new company so the homegrown MRP/ERP system databases (AS400) are getting changed to account for the integration.

 Every week or so my SSIS package fails becasue of the adding of fields in the table I am retrieving. Is there any way within SSIS (Visual Studio 2013, SSIS 2014 I think) that could help me account for these changes? Once my job fails I look in the SQL job log, find the package / step that failed ( I have created a separate package and step in the SQL agent job for each table I want).

I open SSIS and delete the SSIS package innards and the destination table on my MS SQL data warehouse server (with MS SQL  management studio) and recreate the innards and the destination table with SSIS, redeploy SSIS project to the SQL server and all is well.

I am not a DBA or a programmer so am looking for a way within SSIS if possible but willing to learn if I need to script something.

Please do not offer change management or those types of recommendations becasue you would be preaching to the choir:)

Any help is appreciated.

Jeff

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>