Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

After installing SSMS 2014 I can no longer add a source or destination in SSIS VS2010

$
0
0

I really don't know what is happening but I have been using VS2010 and SSMS 2014 for a couple of months now with no issue. Yesterday when I opened an SSIS package that I had created in VS2010 I was no longer able to add a source or destination to any of the data flows. I can copy an existing one and change the properties but when I try to drag it from the tool bow I get the following error -

TITLE: Microsoft Visual Studio
------------------------------

The component could not be added to the Data Flow task.
Could not initialize the component. There is a potential problem in the ProvideComponentProperties method.

------------------------------
ADDITIONAL INFORMATION:

Error at "WORKFLOW NAME" [SSIS.Pipeline]: Component ", clsid {GUID}" could not be created and returned error code 0x80070005 "Access is denied.". Make sure that the component is registered correctly.

Error at "WORKFLOW NAME" [ [128]]: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".

------------------------------
Exception from HRESULT: 0xC0048021 (Microsoft.SqlServer.DTSPipelineWrap)
------------------------------
Program Location:

   at Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass.ProvideComponentProperties()
   at Microsoft.DataTransformationServices.Design.PipelineTaskDesigner.AddNewComponent(String clsid, Boolean throwOnError, Boolean select)

NOTE:I replaced the workflow name and the GUID because I don't think it is important for the post.

When I search for the GUID in regedit it leads me to an entry that has the folder DTSInfo and the value of

Microsoft.DataTransformationServices.DataFlowUI.OleDbSourceUI, Microsoft.DataTransformationServices.DataFlowUI, Version=12.0.0.0, Culture=neutral, PublicKeyToken=********** 'This was not the original value, just hiding it for the post.

for UITypeName

No one here could figure out what the exact issue and how to fix it so my pc was rebuilt. Everything was working just fine until I installed SSMS 2014. Now I am right back where I started.

I really need to know what is going on because I wasted an entire day on trying to figure out this issue and I have already tried reinstalling/repairing VS2010. Removing SSMS 2014. Registering OleDbSrc.dll but get an error when trying to do that.


use variable in child package which is set in parent package

$
0
0

Hi
using ssis 2012

1- In parent package, there is a variable called vFileNameFullPath
2- This variable is assigned using variable1 and variable2
3- Parent package calls child package
4- How can child package use the variable vFileNameFullPath ?

Thank you

How to undo SSIS.ReplacementTask?

$
0
0

It seems that in the 2016 SSIS tooling, if you open a package and the task cannot be found, the package XML is changed and any invalid creation name references are replaced by "SSIS.ReplacementTask". So if I then fix the issue, such as installing the missing task, I'm still stuck with a broken package, because nothing changes "SSIS.ReplacementTask" back to "NowInstalledTask" or whatever it was before.

Have I misunderstood this, or was this a by design that the designer will corrupt your package, and assume you have your own backup to revert to?

Thanks


http://www.sqlis.com | http://www.konesans.com

Emergency.......Loding data into DB2 from SQL Server 2005 through SSIS data flow task????????

$
0
0

Greetings....

When I'm trying to load the data from SQL Server into DB2 database. It's trowing an below error. Can anyone help me what would be the possible resaon and workaround for this. ANy help would be much much appreciated.

Error: [OLE DB Destination [16]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available.  Source: "IBM OLE DB Provider for DB2"  Hresult: 0x8000FFFF  Description:" CLI0125E  Function sequence error. SQLSTATE=HY010"

What exactly I'm doing is ......Loading data into

Loading the data from sql server into the view on DB2

Why to use SSIS , when same task can be in many other ways in a Microsoft/Non microsoft shop?

$
0
0

Data moving can be done in many different ways.

Before/After ETL came in ; it was a windows service,phython/php/shell scripting,BCP,Linked server.
There is always an argument with why to use SSIS .SSIS has its own flaws like no brainer work it does or meta data &mapping issues or non dynamical mapping limitation as per source file/system.
Ex: https://ayende.com/blog/2637/ssis-15-faults .
http://www.jamesserra.com/archive/2011/08/when-to-use-t-sql-or-ssis-for-etl/

On Pro side; one single integrated control process which orchestrates the functionality,logging and alerting/emailing built in along with bulk data moving with out much load on OLTP server or transaction log ,supposing the shop has its own ETL server to alleviate the load on resources.

For data warehousing efforts, I know why SSIS.But for a single table/file load .Why SSIS is needed or better ?


Linked servers, it is obvious and we know why.But you can elaborate if needed.
http://www.sqlservercentral.com/Forums/Topic769610-148-1.aspx
https://www.toadworld.com/platforms/sql-server/b/weblog/archive/2013/05/14/top-3-performance-killers-for-linked-server-queries


BCP same thing , not that easy to write or change.Extra steps for alerting or logging.
http://geekswithblogs.net/LifeLongTechie/archive/2014/09/23/to-bcp-or-not-to-bcp.aspx
http://sqlserverplanet.com/data-warehouse/quick-table-transfers-imports-using-ssis-bulk-insert-or-bcp


Not being a non database service developer, so you know it better on c# or other scripting way to do it.
One of the .Net developer challenged me on why should a database developer be given this task  or how can he do better than a non DB way of implementing same requirement.
I have few answers for myself as mentioned above, but didnt feel enough confident to go back and explain him why.

I am sorry if this topic is already covered somewhere.You can redirect me there.
I couldnot find one stop point for what I am looking for.




Load Multiple files with different structure in different tables

$
0
0

Hi,

I have been searched in the net but i didn't found any result that resolve my problem .

I have a folder that contains some journly data the name of files are like this:

productBought_28_03_2014.LED.txt ,productBought_28_03_2014.LEDPRO.txt ,productBought_28_03_2014.LEDPROBN.txt

productBought_29_03_2014.LED.txt ,productBought_29_03_2014.LEDPRO.txt ,productBought_29_03_2014.LEDPROBN.txt 

Every day i have the same file with different day .

What i want to do is doing a foreach loop container and loop the files in different table each the file have different structure 

I have been tried some solution but didn't work  please How can i do it ?

Thanks for Help.

How can I pass a ConnectionString as a variable

$
0
0

I have an SSIS Package that goes out to the web to access a file. The file name will change on a monthly basis and I can derive this and place the values into SSIS Variables.

I found this web site to assist me in creating the pieces to go out to the web and access the file.

http://microsoft-ssis.blogspot.com/2011/05/download-source-file-from-website-with.html

And I have created my C# Edit Script accordingly...

#region Help:  Introduction to the script task
/* The Script Task allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services control flow.
 *
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script task. */
#endregion


#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
#endregion

namespace ST_df61caf5e38f4ac98f0fc667443e62b4
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
	[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]

	public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
	{
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to
         * either the list contained in the ReadOnlyVariables property or the list contained in
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         *
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         *
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         *
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         *
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         *
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         *
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         *
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         *
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic
         * "Working with Connection Managers Programatically" for details.
         *
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion

 		/// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
		public void Main()
		{
			// TODO: Add your code here

            String CMSNPIFullyQualifiedFileName = Dts.Variables["User::CMSNPIFileFullyQualifiedName"].Value.ToString();
            //String CMSNPIURLConnectionString = Dts.Variables["User::CMSNPIFileConnectionString"].Value.ToString();
           //String CMSNPIURLConnectionString =

            try
            {
                // Logging start of download
                bool fireAgain = true;
                Dts.Events.FireInformation(0, "Download File", "Start downloading " + Dts.Connections["HTTP Connection Manager"].ConnectionString, string.Empty, 0, ref fireAgain);

                // Get your newly added HTTP Connection Manager
                Object mySSISConnection = Dts.Connections["HTTP Connection Manager"].AcquireConnection(null);

                // Create a new connection
                HttpClientConnection myConnection = new HttpClientConnection(mySSISConnection);

                // Download file and use the Flat File Connectionstring (D:\SourceFiles\Products.csv)
                // to save the file (and replace the existing file)
                myConnection.DownloadFile(Dts.Connections["CMSNPIFile"].ConnectionString, true);

                // Logging end of download
                Dts.Events.FireInformation(0, "Download File", "Finished downloading " + Dts.Connections["CMSNPIFile"].ConnectionString, string.Empty, 0, ref fireAgain);

                // Quit Script Task succesful
                Dts.TaskResult = (int)ScriptResults.Success;
            }

            catch (Exception ex)
            {
                // Logging why download failed
                Dts.Events.FireError(0, "Download File", "Download failed: " + ex.Message, string.Empty, 0);

                // Quit Script Task unsuccesful
                Dts.TaskResult = (int)ScriptResults.Failure;
            }


		}

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        ///
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

	}
}

Because my file name is going to change on a monthly basis, like "FileName_June_2016", I can derive this as a SSIS Variable but then how can I pass this as a variable within my C# Edit Script as a ConnectionString variable?

               // Logging start of download
                bool fireAgain = true;
                Dts.Events.FireInformation(0, "Download File", "Start downloading " + Dts.Connections["HTTP Connection Manager"].ConnectionString, string.Empty, 0, ref fireAgain);

                // Get your newly added HTTP Connection Manager
                Object mySSISConnection = Dts.Connections["HTTP Connection Manager"].AcquireConnection(null);

                // Create a new connection
                HttpClientConnection myConnection = new HttpClientConnection(mySSISConnection);

Thanks for your review and am hopeful for a reply.

csv file with 'old' and 'new' file name

$
0
0

Not sure how to write this in SSIS

I have a csv file, with 2 columns.  Column 1 is "old" file name, Column 2 is "new" file name.  The csv file looks like this
Photo12630.JPG,0734917.JPG
Photo12631.JPG,0279630.JPG
Photo12632.JPG,0743277.JPG

I have a folder with individual image files like this
Photo12630.JPG
Photo12631.JPG
Photo12632.JPG

I need to read each line of the csv file, and copy (or rename) the image file to the new file name
Photo12630.JPG  -->  0734917.JPG
Photo12631.JPG  -->  0279630.JPG
Photo12632.JPG  -->  0743277.JPG



SSIS 2012, reading (small) nvarchar(max), sluggish performance

$
0
0

I'm reading some data from a SQL Server 2012. The source table has some 600 million rows, and two of the headache columns are nvarchar(max). Max datalength is 250 bytes and avg datalength is 50 bytes. I have another table where the average size is beyond the value of nvarchar(2000) or varchar(4000), so I need to get this sorted out.

If I read the data without the two columns in question, the throughput is adequate (ca 100K per sec). The destination is simply a RowCount.

If I include these two blob columns, the performance drops to something like, 200 per second (!). What I see is that SSIS works on the data in chunks. While SSIS is working, the source server is idle: No CPU, IO or Pagelookups. I don't know what SSIS is doing internally. I have 40 GB of RAM on the server, SSD disks, should be enough resources. I've tried to set the Blob and Buffer Storage Path to a explicit folder, but I can't see anything going on there. The working set of the SSIS package fine, nothing going on there. It consumes only some 300-400 MB, that's it. No paging going on. I'm using the SQLNCLI11.1 provider, 32K packet size. Networking is fine, I've ruled out that part. The waitstats on the source server says vaguly it's waiting for network, but that's simple because the client app is not consuming anything. Again, if I leave out the blob column the network delivers 400 mbpbs constantly (1 gb network ).

It seems to me as SSIS simply doesn't handle blobs very well. It's very very conservative (because it cannot know how much space the data will occupy from the buffer). From the doc I can read the SSIS handles blobs differently, that they may be stored on disk, read back to memory again and then sent down the pipeline. Whatever happens, the performance is simply a disaster.

I've played around with the buffer size and maxbufferrows, it doesn't really make that much difference. The average performance is crap whatever I choose to do.

I've googled "everything", this is my last resort. What have I missed so far?

Error while parsing deployment manifest: The element DTSDeploymentManifest has invalid child element ConfigurationFile.

$
0
0

Hi

I am am running VS2012 on a 2012 windows server

I have designed a package in VS2012 and it uses a config file to store password of oracle schema to which one of the connection managers refers to. I build the package, and when I run the deployment manifest file (so I can access the package from sql server and run it with an sql agent job) I get error:

Error while parsing deployment manifest: The element DTSDeploymentManifest has invalid child element ConfigurationFile.

I have not found this exact error on a google search, and to my knowledge I have not done anything different with this deployment........Any ideas?

ezAPI: how to add add package to project with a project oledb connection

$
0
0

I'm using ezAPI to build a lot SSIS packages. However,   I have a hard time understanding how to properly create a project package db connection.

            EzProject pro = new EzProject();          
            ConnectionManager cm = new EzSqlOleDbCM(pro, "SSIS_Audit.conmgr");
            cm.Name = "SSIS_AuditParent";
            cm.ConnectionString = "Data Source=localhost;Initial Catalog=SSIS_Audit;Provider=SQLNCLI11.1;OLE DB Services=-2;Integrated Security=sspi;Connect Timeout=300;Auto Translate=False;";
            pro.AddConnectionManager(cm);

...but how do I make a package with a reference to the connection of the project (and not ref to a package connection)?

Thanks.

SSIS: Change Code page 1252 ANSI to UTF-8 65001

$
0
0

Hi All;

I have created SSIS Package to import data from OLE DB to a flat file. The code page in the flat file connection manager was set to default1252 ANSI - Latin I. Now I have been asked to output the flat file inUTF-8 format. I have changed the code page to 65001 UTF-8 but the package will not run. I got the following message:

the code page on input column COLUMN_NAME (184)is 1252 and is required
to be 65001

I have searched around and i have read that i need to change the column data type fromVARCHAR to NVARCHAR!! Is this the solution?

 

Is there is any other option? Thanks.


Connecting to Exchange Server

$
0
0

Anyone had any luck connecting to an Exchange Server via SSIS using either the CDATA component (http://cdn.cdata.com/help/CEA/rssis/pg_connectionr.htm) or COZYROC component (http://www.cozyroc.com/ssis/exchange-connection)?

I know next to nothing about Exchange and have been provided with a CAS server as my Exchange Server along with user/password. I've tried both the CAS address and the OWA access with the above components: CDATA gives a "301: Moved Permanently" while COZYROC gives an immediate "440: Login Timeout".

Since there are limited options and all I'm trying to do for now is get the connection (no need for proxy or specific impersonation), I've only entered server, user, pass and specifically for CDATA, a Auth Scheme and Platform (for which I've tried every combination without luck)

Has anyone found any gotchas that need to be set to work around this?

Thanks

PGP/SSIS Agent

$
0
0

Greetings!

I am using GPG command line tool with SSIS to encrypt a file and post on FTP site.The issue i am facing is that package runs fine on the dev env but its not running on the Server Agent.I am aware from the doing several web searches that its a security issue but i just can't figure out where.

The server agent has a proxy account and it's used to run a couple of other jobs that run fine under that account.How do I check if my package just runs a cmd.exe that opens up the window that i can see then run a .bat file for a second step.

Thanks in advance....

Conditional Concatenation in Derived Column

$
0
0

Hi,

for example i have 4 columns a, b, c and d

if a is null or blank than i want to get this value b-c-d

if b is null or blank than i want to get this value a-c-d

if c is null or blank than i want to get this value a-b-d

if d is null or blank than i want ot get this value a-b-c

if all columns have values it will be a-b-c-d.

(DT_STR,100,1252)((ISNULL(a) || LEN(a) == 0 ? "" : a) + (DT_STR,50,1252)(ISNULL(b) || LEN(b) == 0 ? "" : "-" + b) + (DT_STR,10,1252)(ISNULL(c) || LEN(c) == 0 ? "" : "-" + c) + (DT_STR,10,1252)(ISNULL(d) || LEN(d) == 0 ? "" : "-" + d))

in this expression i get some time -b-c-d if a is null i do not want - in the begining or if d is null it comes a-b-c- similarly i do not want - in the end

the values should be  b-c-d and a-b-c

Thanks

MH


MH


Script Task Corrupt error

$
0
0

Hi,

  Just installed Visual Studio 2015. I was getting error with Script Task.

   So created a New project with just script task (no code ) in try 2 added message box . I was able to run it in development studio in both cases.

   Tried to execute the package with task scheduler or at command prompt I get Script task is corrupt error.

Thanks,

AJ

Why to use SSIS , when same task can be done in many other ways in a Microsoft/Non microsoft shop?

$
0
0

Data moving can be done in many different ways.

Before/After ETL came in ; it was a windows service,phython/php/shell scripting,BCP,Linked server.
There is always an argument with why to use SSIS .SSIS has its own flaws like no brainer work it does or meta data &mapping issues or non dynamical mapping limitation as per source file/system.
Ex: https://ayende.com/blog/2637/ssis-15-faults .
http://www.jamesserra.com/archive/2011/08/when-to-use-t-sql-or-ssis-for-etl/

On Pro side; one single integrated control process which orchestrates the functionality,logging and alerting/emailing built in along with bulk data moving with out much load on OLTP server or transaction log ,supposing the shop has its own ETL server to alleviate the load on resources.

For data warehousing efforts, I know why SSIS.But for a single table/file load .Why SSIS is needed or better ?


Linked servers, it is obvious and we know why.But you can elaborate if needed.
http://www.sqlservercentral.com/Forums/Topic769610-148-1.aspx
https://www.toadworld.com/platforms/sql-server/b/weblog/archive/2013/05/14/top-3-performance-killers-for-linked-server-queries


BCP same thing , not that easy to write or change.Extra steps for alerting or logging.
http://geekswithblogs.net/LifeLongTechie/archive/2014/09/23/to-bcp-or-not-to-bcp.aspx
http://sqlserverplanet.com/data-warehouse/quick-table-transfers-imports-using-ssis-bulk-insert-or-bcp


Not being a non database service developer, so you know it better on c# or other scripting way to do it.
One of the .Net developer challenged me on why should a database developer be given this task  or how can he do better than a non DB way of implementing same requirement.
I have few answers for myself as mentioned above, but didnt feel enough confident to go back and explain him why.

I am sorry if this topic is already covered somewhere.You can redirect me there.
I couldnot find one stop point for what I am looking for.





How to get sequence container name inside a script task using C# code

$
0
0

Hi,

I have to get the list of sequence container names in a SSIS Package pro-grammatically using C# in a script task. Please help me.

Thanks in advance.

Not able to set the paramater of Stored prodedure in SSIS execute sql task connected too Sybase database destination

$
0
0

Exception from HRESULT: 0xC020204A
Error at Data Flow Task [OLE DB Destination [87]]: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E10.

Error at Data Flow Task [OLE DB Destination [87]]: Unable to retrieve column information from the data source. Make sure your target table in the database is available.

Please help to set the values of the parameters of the stored procedure when connected to Sybase

SSIS package - Error: 0x1 at Script Task: Cannot load script for execution.

$
0
0

I have a Script Task in my SSIS project that is failing with the following error - 0x1 at Script Task: Cannot load script for execution.

When I originally added the task and wrote the code it had the following message - Target Of Invocation on Scrip Task. I have read few forums and followed suggested fixes:

- Replaced Microsoft.SQLServer.ManagedDTS.dll from version 11 to version 12

- Replaced Microsoft.SQLServer.ScriptTask.dll from version 11 to version 12

- changed .NET Framework from 4 to 4.5

After all those changes I do have now cannot load script for execution.

The code inside the scrip task:

	public void Main()
		{
            try
            {
                String RemoteFileName = "/folderx/export_new.csv";

                SessionOptions sessionOptions = new SessionOptions
                {
                    Protocol = Protocol.Sftp,
                    HostName = "xyz.example.testing.com",
                    UserName = "username1",
                    Password = "password1",
                    PortNumber = 22,
                    SshHostKeyFingerprint = "ssh-rsa 4096 xx:xx:xx:xx:xx:xx..."
                };

                using (Session session = new Session())
                {
                    session.Open(sessionOptions); //Attempts to connect to your sFtp site

                    if (session.FileExists(RemoteFileName) == true)
                    {
                        Dts.Variables["FileExists"].Value = true;
                    }
                    else
                    {
                        Dts.Variables["FileExists"].Value = false;
                    }

                    session.Close();
                }

                Dts.TaskResult = (int)ScriptResults.Success;

            }
            catch
            {
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
		}

Any ideas?

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>