# SSIS package for Microsoft SQL Server 2014 (SP2) (KB3171021) - 12.0.5000.0 (X64)
# PSVersion 5.0.10586.117      32 bit

$ErrorActionPreference="Stop"
Set-PSDebug -Strict

Add-Type -AssemblyName "Microsoft.SQLServer.ManagedDTS, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91"
Add-Type -AssemblyName "Microsoft.SqlServer.DTSPipelineWrap, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91"

[string]$srcConnStr = "Provider=SQLNCLI11;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=Test;Data Source=XYZ;"

[Microsoft.SqlServer.Dts.Runtime.Package]$Package = New-Object Microsoft.SqlServer.Dts.Runtime.Package
[Microsoft.SqlServer.Dts.Runtime.Application]$App = New-Object Microsoft.SqlServer.Dts.Runtime.Application
[Microsoft.SQLServer.Dts.Runtime.Executable]$exec = $Package.Executables.Add("STOCK:PipelineTask")
[Microsoft.SqlServer.Dts.Runtime.TaskHost]$th = [Microsoft.SqlServer.Dts.Runtime.TaskHost]$exec
$th.Name = "Data Flow Task"
[Microsoft.SqlServer.Dts.Pipeline.Wrapper.MainPipe]$DataPipe = [System.Runtime.InteropServices.Marshal]::CreateWrapperOfType($th.InnerObject, [Microsoft.SQLServer.DTS.pipeline.Wrapper.MainPipeClass])
[Microsoft.SqlServer.Dts.Runtime.ConnectionManager]$SourceConn = $package.Connections.Add("OLEDB") -As [Microsoft.SqlServer.Dts.Runtime.ConnectionManager]
$SourceConn.Name = "OLEDBConnectionSource"
$SourceConn.set_ConnectionString($srcConnStr)

$oledbSource = $DataPipe.ComponentMetaDataCollection.New()
$oledbSource.Name = "OLE DB Source"
$oledbSource.ComponentClassID = "DTSAdapter.OLEDBSource.1";

[Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapper]$InstanceSource = [System.Runtime.InteropServices.Marshal]::CreateWrapperOfType($oledbSource.Instantiate(),[Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass])
$instanceSource.ProvideComponentProperties()

If I change DTSAdapter.OLEDBSource.1 -> DTSAdapter.OLEDBSource.4

Thank you

Hi. We run 2012 enterprise. I just finished a poc that showed if I insert my finder keys to db2 thus allowing my joins on db2 to render 10's of thousands of lookup records instead of 10's of millions, that my job can run in about half the time.  The run time reduced from 21 minutes to 10.  The db2 datasets are used in merge joins for lookup purposes.  SQL Server datasets serve as the left side of most of these merge joins.  In one case a sql server dataset is inner joined with the db2 data.

I noticed that the finder inserts themselves were taking a substantial amount of that 10 minutes.  Anywhere from 3-6 minutes.  And the source sql server query was only 8 seconds of that 3-6 minutes in the insert DF.  There are only 46k records that need to be inserted into the db2 finder table.

My db2 connection says my provider is Native OLE DB\IBM OLE DB Provider for DB2 - DB2COPY1.  My db2 table name is from a variable because for some reason the dropdown of db2 table names didn't seem to go far enough into the alphabet for me to pick my table.

I poked around and concluded that while fastload rowset and fastload rowset from variable could be chosen in the properties of the target oledb db2 component, choosing them over "rowset from var" wasn't legal and in fact turned the destination component red. 

Does the community have any recommendations for making these inserts perform faster?

Hi,

I am wondering, if I can search any of SSIS package from my SQL Server Data Tools.

(using SQL Server 2012 data tool/VS 2010 shell)

Scenario:

I have around more than 1000 SSIS package which is consists of Source ETL, Stage ETL, Mart ETL, Cube process ETL in same solutions.

When I want to change/update/modify something in any of the ETL package, I always need to scroll down the scroll bar to go to the Package and sometime due to similar names, it takes even 5 Minutes to find my actual SSIS package.

However, I looked into Visual Studio 2012 (.NET) where you can actually, search any object with in a search bar as well as in side the code. this is really acool feature.

1. Just imagine a feature, if I have 1000's of package, I only type a word "Customer" in Solution Explorer, it will search only packages which contains "Customer" is a part of package name.

2. Just imagine a feature, If I have 1000's of package, I make a global search in XML, to search for table "Customer", it brings up all the package which is having "Customer" table used. that would be so more cool na.

A developer no need to be worried, if any changes in any of the package, he not only sure about impact analysis, he can more idea how overall data flow works.

This is suggestion to MS if not exists, If exists, then Help :)

Thanks,

SDM

I have two sequence containers “A” and “B” in package “Z”. They run from a preceding expression task which sets a variable value. Depending on the value set in the expression task:

1.       Only “A” sequence container can run.
2.       Both “A” and “B” sequence containers can run in parallel.

 Next, there is an Execute SQL task “C” that follows immediately afterwards. It is the “constrained executable” which follows “A” and “B”. If both “A” and “B” are determined to run, they must both finish (“success”) before “C” runs.  In the constraint editor for precedence constraints for the constrained executable “C” we use a logical “AND” to ensure that is what occurs.  How do we get “C” to execute if only “A” is executed.   The logical “AND” keeps “C” from executing if “B” doesn’t execute. Which is a condition  we may optionally want.  Can we dynamically alter the constraints “LogicalAnd” propert y, depending on the variable value?    Am I missing something? Am I going down a fruitless rabbit hole for trying to alter the constraints at run time?  thanks Michael

Hi,

I have a dataflow and in my source component i have a query like as bellow :

select * from mytable where cat in (?)

the ? is my parameter that i have the value in variable.The variable is a string and i have a value 'computer','Laptop' when i execute the project ,the project executed successfully but i didn't have a result i have nothing what i want is doing the query as bellow with SSIS using parameters it's possible ?

select * from mytable where cat in ('computer','Laptop')

ss

Hi,

I have a different text files that need to import into different tables. For example C:/Folder/Hello.txt should be imported to tblHello and C:/Folder/Hi.txt should be imported to tblHi. I am new to SSIS i dont know how to achieve this.

Can you please guide me accomplish this tast, it would be nice if anyone provide me a step by step method?

Thanks,

Karthik

I'm using SSIS 2014 and want to make use of the package/project validation features as part of our DEV testing procedures.  We have environment variables set up to handle all connection strings, and we also have several project parameters that are set at run-time.  When I right click on a deployed package/project and select "validate", the parameters that are mapped to environment variables are populated as expected.  However, the "run-time" project parameters are blank, and I can't figure out how to populate them with values for the purposes of validation.  Here is a screenshot:

I'd expect to see the "..." button that would allow me to enter a value, as it does when I execute the package instead of validate.

I'm trying to create a view from the SSISDB. I'm querying the standard Catalog.* views. So I'm displaying the pertinent run info for our team:

• Project.Name
• Operations.Operation_Id Max Id associated to respective Project.Name
• Operations.StartTime 
• Operatons.EndTime
• Operations.Status
• Operation_Messages.Message_Type
• Operation_Messages.Message_Source_Type

My issue is that i need to capture the previous operations.operation_id so that I can display the previous Operations.StartTime.

i.e run today at 2:00 pm. Previous Run today 1:00 pm (or whenever the last time it ran.

I realize this is more of a T-SQL type question but I thought that anyone who might have had this request in the past might be able to pass along a snippet. Cheers Michael  

Hello,

I'm quite new to using SSIS and I was hoping someone would guide me on the best practice in handling my issue.

I have a process where I need to break out records to a more detailed level than they currently exist by joining them against several low level sources, but each record only needs to be joined once and then spit out to a final table while the remaining records that don't exist in the low-level table continue on to the next join...

I'm doing this in SSIS by using left joins in my Merge Join transform and then using Conditional Splits to single out the records with the NULL value in the field I joined against. These records continue onto the next step while the records that matched move to the output table.

This all works fine, however, after several of these and a lot of necessary manipulation in between, my data flow is looking very unwieldy.

What I would like to do is to break the sequence up into multiple data flows passing the data between them. I attempted to do this with a raw file but ran into issues in the second dataflow because it said the file it was trying to access was in use (by itself, apparently). I used delay validation and everything and still the same problem... I just can't get the Destination in the first Data Flow to let go of the file before the second data flow runs.

Ideally, I'd like to do this in place by simply removing the records that matched in each join from the staging table and leaving the rest in there for the next Data Flow to access without fear of duplicating records. I just can't see a way to do that... I was hoping there was an obvious way to simply truncate the staging table within the dataflow just prior to reloading the records coming out of my Conditional Split, but I don't see how... if someone could guide me or give me a better, more typical solution, I'd greatly appreciate it.

Thanks!

-- Bill

EDIT: Also, I'd prefer solutions, if possible, that don't require the following because they've already been considered:

1. Using another staging table to hold remaining records in between each data flow task. We try to avoid creating and removing temporary tables from our SQL Server. I'm absolutely OK with staging the records somewhere as I attempted with the RAW file, I just have to know how to get around the access issues.

2. Deleting from the staging table against records NOT IN the final table between each data flow task.

When I press the edit script button it opens VS and I can see the script components code for a brief moment but it closes it self almost immediatly. It only does that when I add an input column when I don't specify an input column, it opens as expected and I can edit the script.

Anybody an idea what might cause this ?

Thanks in advance

Rbie

Hi All,

I have created a ssis package which I have deployed in SSIS catalog.

I have created a environmental variable to configure the connection string and pass the value to package.

Same code working on Dev environment very well , but when I deployed in QA it is giving me error like below

Thankx & regards, Vipin jha MCP

Hi ,

We are migrating SSIS packages from SQL Server 2014 to SQL Server 2016.  I'm using SSISUpgrade.Exe to upgrade my packages.

I have two questions :

1) I see provider name as  'Provider=SQLNCLI11.1;Integrated Security=SSPI' which is same as for SQL Server 2014 . What is correct name for SQL Server 2016 provider name for SSIS and should this be updated in all packages in SSIS.

2) When I open a solution after upgrading the packages in folder using Visual studio 2015 , I see that project opens with name as it is still in 2014 version ( Ex ProjectA(SQL Server 2014) ) . how can I upgrade packages in such a way that when I open a solution file , I should be able to open and see all packages when project in 2016 version .

any help is appreciated.

Hi,

I am trying to open a text file in script task and reading line by line in a loop and update each like and then write to new file. I came across an exception in loop. It exits fine but, running wheel keeps running on script task and wouldn't go into failure precedence and start another task upon failure. This is how I am doing. I tried Try,catch block with setting results to failure etc. Nothing is working.

 string line = "";
 int rowCount = 1;
 string sequenceNumber = "";StreamReader reader = null;
 StreamWriter writer = null;

 reader = new System.IO.StreamReader(sourceFile);
 writer = new StreamWriter(targetFle);

 while ((line = reader.ReadLine()) != null)
            {
                sequenceNumber = CreateSequenceNumber(rowCount, 5);
                // get total shares
               totalQty += int.Parse(line.Substring(26, 7));
                //replace spaces with sequence number
                line = line.Remove(0, 5).Insert(0, sequenceNumber);
                writer.Write(line + "\r\n");
                rowCount++;
            }

The exception is happening on bolded line because the data has dash. I tried

try

{

    totalQty += int.Parse(line.Substring(26, 7));

}

catch (Exception ex)

{

    Dts.TaskResult = (int)ScriptResults.failure;

    break;

}

Please suggest how to break that running wheel and go to failure task.

Thank You

Hi all,

I have an SQL script that selects images taken by specific users. I want to create a package that will extract all these images and place them in appropriate folders, based on the user names.

For example, my data is as follows:

I want the package to create folders based on the user names (the user name is part of the image name). So , in this example, the folders created would beuser1 (3 images), user4  (2 images), user5  (1 image), user6  (4 images).

Please assist

Regards

Good day

I've tried to run my package through the command line using `dtexec /f Package.dtsx /ConfigFile debug.dtsConfig /Decrypt ######`.

When I set the security mode, I generate my dtsConfig file that results in:

<?xml version="1.0" encoding="UTF-8"?><DTSConfiguration><DTSConfigurationHeading><DTSConfigurationFileInfo GeneratedBy="INFOTECH-CARIB\dclarke" GeneratedFromPackageName="Package" GeneratedFromPackageID="{1F85FCBF-3B69-4958-8D3A-CABC59BAC841}" GeneratedDate="10/14/2016 7:10:59 PM" /></DTSConfigurationHeading><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[ConnectionString]" ValueType="String"><ConfiguredValue>Data Source=tcp:sssssssssss.database.windows.net,1433;User ID=blah@ssssssssssssssssss;Initial Catalog=blah_test;Provider=SQLNCLI11.1;Persist Security Info=True;database=blah_test;encrypt=yes;trustservercertificate=no;connection timeout=30;</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[Description]" ValueType="String"><ConfiguredValue /></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[InitialCatalog]" ValueType="String"><ConfiguredValue>blah_test</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[Name]" ValueType="String"><ConfiguredValue>idb</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[Password]" ValueType="String"><ConfiguredValue /></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[ProtectionLevel]" ValueType="Int32"><ConfiguredValue>1</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[RetainSameConnection]" ValueType="Boolean"><ConfiguredValue>0</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[Scope]" ValueType="Int32"><ConfiguredValue>0</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[ServerName]" ValueType="String"><ConfiguredValue>tcp:ssssssssssssssss.database.windows.net,1433</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[idb].Properties[UserName]" ValueType="String"><ConfiguredValue>ssssssssss@ssssssssssss</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[ConnectionString]" ValueType="String"><ConfiguredValue>Data Source=ICWKS-022\SQLEXPRESS;User ID=;Initial Catalog=test;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[Description]" ValueType="String"><ConfiguredValue /></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[InitialCatalog]" ValueType="String"><ConfiguredValue>test</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[Name]" ValueType="String"><ConfiguredValue>test</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[Password]" ValueType="String"><ConfiguredValue /></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[ProtectionLevel]" ValueType="Int32"><ConfiguredValue>1</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[RetainSameConnection]" ValueType="Boolean"><ConfiguredValue>0</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[Scope]" ValueType="Int32"><ConfiguredValue>0</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[ServerName]" ValueType="String"><ConfiguredValue>ICWKS-022\SQLEXPRESS</ConfiguredValue></Configuration><Configuration ConfiguredType="Property" Path="\Package.Connections[portaldb].Properties[UserName]" ValueType="String"><ConfiguredValue /></Configuration></DTSConfiguration>

However, when I try to run it using the command line option mentioned above, I get: