Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

To bulid SSIS package taking more time after providing the value through expression to arguments property in execute process task

0
0

Hi ,

I am using Execute process task in my SSIS package, by using this task I am providing the password to the destination Excel file. In execute process task we have property Arguments, while providing the value to  argument through manually it is executing with in 2mins while providing the value to argument through expression it is taking more time(12 to 15 minutes).

For better understanding please see the bellow image.


SSIS text file import skips some rows

0
0
Hi Guys,

I have some flat files (.txt file)which are loaded into the database on a daily basis. Each file contains about 500 rows  ,The interesting thing is that the SSIS doesn't import the correct rows for some flat files, but for others it can import the rows correctly. For example, when i import a 348 rows file into my DB, the job works well with no error, but the info of  package execution progress tells that "The total number of data rows processed for the file is 285" .waht's wrong? I am so frustrated with this problem, Any tips are appreciated~

XML to database, very difficult requirement - Please help!

0
0
Hi,

I have 10k xml files in a folder. 

The sampe file may be somehting like this

<policy>
  <SubPolicy>
     <Customer1>
          <id>1000</id>
          <name>Peter</name>
          <term>12months</term>
          <vehicletype>car</vehicletype>
          <city>Charlotte</city>
          <state>NC</state>
      </customer1>
     <Customer2>
          <id>2000</id>
          <name>John</name>
          <term>12months</term>
          <vehicletype>Bike</vehicletype>
          <city>Charlotte</city>
          <state>NC</state>
      </customer2>
  </SubPOlicy>
  </POlicy>

I want to store the values in the xml files in to differnet tables in a database. Can you please help me on this..

Example, in the customer_maintable, I manually  insert the records like this 

create table dbo.customer_Maintable
(ID int,
[Type] varchar(20)
)

insert into dbo.customer_Maintable values ('1','name')
insert into dbo.customer_Maintable values ('2','Term')
insert into dbo.customer_Maintable values ('3','VehicleType')
insert into dbo.customer_Maintable values ('4','City')
insert into dbo.customer_Maintable values ('5','State')


then, in the dbo.customer_Transactiontable, the corresponding values should populate like the below output.

create table dbo.customer_Transactiontable
(
customer_TransactionID int identity (1,1),
CustomerID int,
CustomerMainTable_ID int,
Value varchar(20)
)

insert into  dbo.customer_Transactiontable values (1000,1,'Peter')
insert into  dbo.customer_Transactiontable values (1000,2,'12months')
insert into  dbo.customer_Transactiontable values (1000,3,'car')
insert into  dbo.customer_Transactiontable values (1000,4,'Charlotte')
insert into  dbo.customer_Transactiontable values (1000,5,'NC')

insert into  dbo.customer_Transactiontable values (2000,1,'John')
insert into  dbo.customer_Transactiontable values (2000,2,'12months')
insert into  dbo.customer_Transactiontable values (3000,3,'Bike')
insert into  dbo.customer_Transactiontable values (4000,4,'Charlotte')
insert into  dbo.customer_Transactiontable values (5000,5,'NC')


Can you please help me on this?  In the future, if another tag/xpath comes, example <salary>5000</salary>, then , I should 
do a insert an entry in the main table, like the below 
insert into dbo.customer_Maintable values ('6','Salary') , then the salary should be correctly loaded in dbo.customer_Transactiontable table, using a 
SSIS package or from a stored procedure.

Can yo please help me on this?

SSIS Running a Parent package in SQL Agent will not run the child package

0
0

Hello

I have two SSIS packages, a Parent package that just runs a SQL script to a recordset and then runs a second package and passes a location to the child package.

When I run in visual studio it runs fine, but when I run in SQL Agent I get the following error.

Message
Executed as user: TELFORD\Sql. ...0.5058.0 for 32-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  16:40:18  Error: 2016-05-20 16:40:20.11     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.13     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.15     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.15     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specific connection element is not found.  End Error  Error: 2016-05-20 16:40:20.15     Code: 0xC001000E     Source: Child_Package      Description: The connection "{97198076-5026-4B8C-A6FB-30FD129E93D1}" is not found. This error is thrown by Connections collection when the specif...  The package execution fa...  The step failed.

The child package location is;

\\ttt-vsp02\SSIS\Trading Report\Trading Report\Child_Package.dtsx

I think it is complaining about the connection to the child package, but I don't know how to resolve. Any suggestions?

Regards

SQLSearcher99

sqwl server query question

0
0

I have the below code in my table which was working before.

The tables Program and Program Description did not have datetime columns before and I have a datetime columns now. But I am not fetching the datetime columns, but it keeps throwing the error of 

Msg 241, Level 16, State 1, Line 1
"Conversion failed when converting date and/or time from character string."

Why is it throwing the error, what do I need to modify in the query so that I can get it working

Below is the query

SELECT DISTINCT
rtrim(prg.TMSId) TMSId,
description = LTRIM(ISNULL(STUFF((SELECT ' ' + convert(varchar(301),dsc2.Description)
FROM ACQ_TMS.ProgramDescription dsc2 WITH (NOLOCK)
WHERE dsc2.ProgramTMSId = dsc.ProgramTMSId
and (dsc2.Type = 'plot'
and dsc2.[Size] = dsc.[Size]
                  or dsc2.Type in ('source', 'sell'))
ORDER BY DSC2.TYPE
FOR XML PATH('')),1,1,''), ''))
FROM ACQ_TMS.Program prg WITH (NOLOCK)
LEFT OUTER JOIN ACQ_TMS.ProgramDescription dsc WITH (NOLOCK)
ON (dsc.ProgramTMSId = prg.TMSId
and dsc.[Size] = (CASE
WHEN EXISTS (SELECT 1 FROM ACQ_TMS.ProgramDescription dsc2 WITH (NOLOCK) where [size] = 255
       and dsc2.ProgramTMSId = dsc.ProgramTMSId
       and dsc.type = 'plot') then 255                   
WHEN EXISTS (SELECT 1 FROM ACQ_TMS.ProgramDescription dsc2 WITH (NOLOCK) where [size] = 250
and dsc2.ProgramTMSId = dsc.ProgramTMSId
and dsc.type = 'plot') then 250
WHEN EXISTS (SELECT 1 FROM ACQ_TMS.ProgramDescription dsc2 WITH (NOLOCK) where [size] = 100
and dsc2.ProgramTMSId = dsc.ProgramTMSId
and dsc.type = 'plot') then 100
else 60
END))

Any help would be appreciated. Thanks

SSIS execution window

0
0

Hello,

I have a SSIS package which runs from 6am to 10am everyday, how can I make sure that no other process runs on the server during this time other than the SSIS package.

Thank You 


sree

Converting value to datetime in SSIS

0
0

Hi 

I have a date field value as 201605100001(unicode DT_WSTR as data type) and i want to move this value to 
a field whose data type is datetime(2016-05-10 00:00:01). I read this can be done by splitting the records. what is the best way to achieve this in SSIS?

Thanks

kicking off a sql agent job from ssis

0
0
Hi. I'm running a 2008 r2 std ssis pkg.  From this pkg, I want to kick off a sql agent job that sits on a 2005 sql box.   So far I'm having difficulty finding how to do this even in the same version of sql with off the shelf ssis but am aware of sp_start_job, osql and dmo (which aren't so bad).  Is there an off the shelf ssis component that starts sql agent jobs? 

Fill in Column Data using Script Transform

0
0

All,

I'm trying to figure out a way to dynamically populate a column as it passes through a data flow task.  My data source is Excel 2007 and the first row of data has data in the say, column B, and then rows 2-50 let say contains blanks for this column. I want to grab this date on the first row and assign it as my column value for all subsequent rows.

I've tried doing this in a script transform and assigning the value to a variable but it seems like it can only write to the variable after all rows have passed through it.  This is where it get's a little more complicated.  Let's say on row 51 there is a new value in column B.  Now I'd like to use this new data as the column B value for rows 52-100.  If the value couldn't possible change, I'd be good with setting this variable at the end of the script transform, but alas no.  

So how can I dynamically set this value in column B as the transform is rolling through the data?

Example:

Original Data

ColumnAColumnB

15/1/2016

2

3

4

5

65/2/2016

7

8

9

10

Final Transform

ColumnAColumnB

15/1/2016

25/1/2016

35/1/2016

45/1/2016

55/1/2016

65/2/2016

75/2/2016

85/2/2016

95/2/2016

105/2/2016

Thanks for any guidance anyone can provide.  

MB

Database Table column to XML destination - Please help

0
0
Hi,

I have a source table - the DDL is as below

CREATE TABLE [dbo].[PolicyDetail]
(
[PolicyDetailId] [bigint] IDENTITY(1,1) NOT NULL,
[PolicyNumber] [bigint] NULL,
[OriginalXml] xml,
[CreatedBy] [nvarchar](50) NOT NULL,
[CreatedDate] [datetime] NOT NULL,
)

This table has 10k records. In the '[OriginalXml]' column, actual xml file is stored as it is.
So, that means, 10k xml files are stored in the table. Now, I want to pull the originalxml  column and create 10k individual xml files in a folder.

Can you please help me for this?

In the SSIS package, I have selected, Data Flow task. In the data flow task, for source, I have selected the table.
What should I select for Destination?

My target is to get the 10k individual files from the originalXML column from the table. Please help!

Thanks in advance for your help!

c sharp console application code to ssis script task

0
0

Hello,

I am having below console appliaction which i put it into ssis script task but its gving me below error. 

I added dll from  the link ( from the folder net-20)

http://www.icsharpcode.net/OpenSource/SharpZipLib/Download.aspx

using System;
using System.Text;
using System.Collections;
using System.IO;

using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;


class Program
{
    public static void Main()
    {
        // Perform simple parameter checking.
        //if (args.Length < 1)
        //{
        //    Console.WriteLine("Usage UnzipFile NameOfFile");
        //    return;
        //}

        //if (!File.Exists(args[0]))
        //{
        //    Console.WriteLine("Cannot find file '{0}'", args[0]);
        //    return;
        //}

        using (ZipInputStream s = new ZipInputStream(File.OpenRead(@"C:\ZipFiles\xxx_2013_04_25.zip")))
        {

            ZipEntry theEntry;
            while ((theEntry = s.GetNextEntry()) != null)
            {

                Console.WriteLine(theEntry.Name);

                string directoryName = @"C:\ZipFiles\processed\xxx_2013_04_25\" + Path.GetDirectoryName(theEntry.Name);
                string fileName = Path.GetFileName(theEntry.Name);

                // create directory
                if (directoryName.Length > 0)
                {
                    Directory.CreateDirectory(directoryName);
                }

                if (fileName != String.Empty)
                {
                    using (FileStream streamWriter = File.Create(directoryName+theEntry.Name))
                    {

                        int size = 2048;
                        byte[] data = new byte[2048];
                        while (true)
                        {
                            size = s.Read(data, 0, data.Length);
                            if (size > 0)
                            {
                                streamWriter.Write(data, 0, size);
                            }
                            else
                            {
                                break;
                            }
                        }
                    }
                }
            }
        }
    }
}


In the script task :

             

#region Namespaces
using System;
using System.Text;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Collections;
using System.IO;
using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Zip.Compression.Streams;
#endregion

namespace ST_a5f0461bdf9440b491ea28167c4cb782
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to 
         * either the list contained in the ReadOnlyVariables property or the list contained in 
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and 
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         * 
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         * 
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         * 
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         *  
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         * 
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         * 
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         * 
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         * 
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic 
         * "Working with Connection Managers Programatically" for details.
         * 
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion


/// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
public  void Main()
        {
        // Perform simple parameter checking.
        //if (args.Length < 1)
        //{
        //    Console.WriteLine("Usage UnzipFile NameOfFile");
        //    return;
        //}

        //if (!File.Exists(args[0]))
        //{
        //    Console.WriteLine("Cannot find file '{0}'", args[0]);
        //    return;
        //}

        using (ZipInputStream s = new ZipInputStream(File.OpenRead(@"C:\ZipFiles\xxxxx_2013_04_25.zip")))
        {

            ZipEntry theEntry;
            while ((theEntry = s.GetNextEntry()) != null)
            {

                Console.WriteLine(theEntry.Name);

                string directoryName = @"C:\ZipFiles\processed\xxxxxx_2013_04_25\" + Path.GetDirectoryName(theEntry.Name);
                string fileName = Path.GetFileName(theEntry.Name);

                // create directory
                if (directoryName.Length > 0)
                {
                    Directory.CreateDirectory(directoryName);
                }

                if (fileName != String.Empty)
                {
                    using (FileStream streamWriter = File.Create(directoryName+theEntry.Name))
                    {

                        int size = 2048;
                        byte[] data = new byte[2048];
                        while (true)
                        {
                            size = s.Read(data, 0, data.Length);
                            if (size > 0)
                            {
                                streamWriter.Write(data, 0, size);
                            }
                            else
                            {
                                break;
                            }
                        }
                    }
                }
            }
        }

Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        /// 
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

}
}

I am getting the below error.

Please help me.

Thanks,

Anilkumar


compatibility of excel source

0
0

Hi All,

I have developed the ssis package in VS 2008. source is excel, loading to SQL table. It works fine in Development environment.

Now can i run the package in - SQL SERVER Deployment utility 2014? Do I need to do any code changes or upgrade package to 2010? Please advise.

I am trying to run the package from Deplyment utility 2014- having OS - WINDOWS SERVER 2012 R2 

Getting below errors. Please advise.

The requested OLEDB Provider SQLNCLI10.1 is not registered. If 32 bit driver is not installed
0x80040154 : class not registered.

acquire connection manager failed with error code 0xC0209302.

Destination - error code 0XC020801C

CAN WE RUN THE PACKAGE WHERE MS OFFICE IS NOT INSTALLED.(BECAUSE SOURCE IS EXCEL)??


Thanks,

Raghavendra 

Multiple CSV Upload

0
0

We have multiple CSV files each containing different data and mapping to different tables. These tables also have various relationships which need to be included in the ETL process. 

I am thinking how best to go about this as I do not want a separate SSIS package for each.

My thoughts are that we need the following logic:

1. For each CSV file in the file system

2. Get the matching entity name (table)

3. For each row in the current file

4. Retrieve FK IDs using lookups 

5. Use the CSV header names to map to the table column names

6. Insert each row

7. Move to next CSV file

Looking for expert advice on how best to tackle this using SSIS? i.e. components to use etc. Does this type of requirement require a script component?

how to do data cleansing ssis2008

0
0
how to do data cleansing ssis2008 ?

SSIS Package Deployment Parent - Child packages

0
0

Hello

I have a SSIS project that has a Parent package and a Child package, the parent package runs a SQL script that gets 5 locations and it passes the location to the child package and the child package run several queries based on the location.

When I run in Visual Studio the parent package passes the parameter to the child package.

However when I change the settings of the "Execute Package Task" from Project Reference to External Reference, I can no longer pass the parameter to the child package.

I need to be able to schedule the package in SQL Agent, how do I do this and pass the parameter to the child package? 

Regards

SQLSearcher99


Connecting Oracle using SSIS

0
0

We are using SSIS in SQL Server 2014, trying to conenct to Oracle. We are not able to do it. I see Attunity connectors can help it but it says it works with Enterprise & Developer Edition only.

Can any one help me with Step by step how to connect as I a new to SSIS and MS BI

Regards

Surya

Visual Studio 2015 - SSIS Development - Backward Compatibility - Can we develop 2008, 2012, 2014 packages?

0
0

Hello and good day to all

I'm trying to determine if Visual Studio 2015 (VS2015) Ultimate or any other version VS2015 will allow for SSIS packages for 2008, 2012, 2014 environments.

Any direction is appreciated

Thank You Forum


Mark Covian

Data Load of a HUGE National Database File and any hints...pointers

0
0

We have tasked to load the Centers for Medicare & Medicaid Services(CMS) NPI file...or otherwise know as the "National Plan and Provider Enumeration System(NPPES)" file. The file is created and posted as a monthly full file cut and then weekly incrementals. We are attempting to load the full monthly file which is 5.7GB. Soooo leads me to my questions...plural...

  1. How much space do I need to allocate to my SQL Server Database and Table? 5.7GB or is there overhead?
  2. For efficiency purposes, what do I need to do to my SSIS Package that will run this efficiently? And the most timely
  3. Should I load this with SSIS or some other means because of the huge volume?
  4. Do I need to explicitly define my data fields on my Flat File Connection Manager? I think I saw somewhere that you should explicitly define your data columns and lengths to avoid any overhead

If there are any additional suggestions, please let me know.

Thanks for your review and am hopeful for a reply.

SSIS 2008 script tasks won't compile after SQL 2014 install

0
0
I have SQL 2008 installed on my local machine, and have quite a few SSIS packages that I have created.  I recently installed SQL 2014 Management Studio.  Since then, my script tasks no longer compile (in BIDS 2008).  After doing some research and testing, I *think* that the issue is that BIDS 2008 no longer recognizes the Microsoft.SqlServer.manageddts dll.  I'm not certain that this is the case, though.  I have tried several iterations of repair/uninstall/reinstall of both SQL versions, and nothing seems to work.  Has anyone else encountered, and more importantly FIXED, this issue?  Thanks in advance!

Cannot see ODBC oracle connection in SQL bids 2012

0
0

Hello all,

i have WIN 7 64 bit machine, SQL server BIDS 2012 Version.

i installed Oracle 11.2.0.4 both 64bit and 32 Bit . and then i tried to setup DSN in ODBC admin.. i was able to see Oracle 64..so i setup dsn for oracle 64 bit, but i am not able to see odbc 32 bit oracle driver and cannot setp dsn

Now.. in my bids.. i am not able to see any see any Oracle DSN/odbc , so i wen into bids sol properties-> degub mode and set "64bitrunmode" to false.. i still cannot see any thing.. help plsss




Viewing all 24688 articles
Browse latest View live




Latest Images