Script Component task for my goal?

September 23, 2016, 6:10 am

≫ Next: text file + add header, footer and changes to data rows

≪ Previous: How to connect to Teradata from SSIS

Hi all of you,

I need to create a large bunch of plain files from different SQL tables (approx 60 tables in the first drop). Every SQL table has the same number of columns and names. And some tables are in different servers and databases.

So far so good. Problem is how can I achieve this by using SSIS? I mean,I don’t want to create dozens of data flows by using the usual couple of "ADO .Net Source" and "Flat File Destination" tasks.. I would like to automate everything from the scratch using parameters.

I checked (let me know if I am wrong) that SQL Command Property on ADO .Net Source / OLE Db Source does not allow me parameterize the name of the tables.

So I’ve thought store all the info in a SQL table:

Server
Database
Staging Table
Destination

And then retrieve all the data by using .NET.

Does anyone has some sample by using Script Component + VSTA?

Idea here is just use an unique Data flow per environment (development->UAT->and finally reuse the same package when go live.

Thanks for your thoughts an ideas!!

↧

text file + add header, footer and changes to data rows

September 23, 2016, 8:06 am

≫ Next: accomplishing two joins in one merge join

≪ Previous: Script Component task for my goal?

Hi,

We get a txt file (fixed positions) from one vendor. We need to add header, footer and update each data row with ID and send this updated txt file to another vendor.

Hearder - is dynamic. Has certain pattern with static data and dynamic data. Dynamic data is I have to include total number of data records that file has.

Data rows - Beginning of each row, I need to add ID (1, 2, 3 ) etc padded with zeros. The position to add ID is from 1-5th position. so if ID is 1 then it should be 00001. If ID is 10, then ID should be 00010. Rest of the data in data row is same.

Footer: Data rows have quantity as part of data. I need to keep track of it, add it and create a footer with total quantity.

How can I do it in SSIS package.

Thank You

↧

accomplishing two joins in one merge join

September 23, 2016, 8:13 am

≫ Next: db2 query in lookup seems to be complaining about "with ur" phrase

≪ Previous: text file + add header, footer and changes to data rows

Hi we run 2012 enterprise.

My merge join from sql server (left, 50,000 records) to db2 (20 million records) joins on 3 columns.

A second merge join to assign defaults (should 1st lookup fail on any given row in first merge join) joins on 2 of those 3 columns and could theoretically use the same queries sorted the same way if 1) a merge join can somehow do such a thing, 2) I could somehow avoid duplicates because the 2 columns can repeat.

I like the performance of the merge join. But shaving some run time would be desirable. We don't allow linked servers and openrowset. I'm already talking to my db2 dbas about a poc that would allow insertion of a temporary 50,000 row finder table to db2 so key to db2 attribute joins can be done on db2 before I ask ssis to do anything with the db2 data.

can the merge join do double duty as I described?

↧

db2 query in lookup seems to be complaining about "with ur" phrase

September 22, 2016, 2:49 pm

≫ Next: SSIS :How to get OUTPUT $action, Inserted.*,Deleted.* records in In SSIS

≪ Previous: accomplishing two joins in one merge join

Hi we run 2012 enterprise. I am using a var in a lookup to generate a resultset from db2. I'm getting the error u see below...

error code 0x80004005...ibm ole db provider for db2...db2 sql0199n the use of the reserved word "with" following ''' is not valid. Expected tokens may include : "and or having group intersect...sqlstate 42601

the evaluated query runs on my db2 client. It also runs as an oledb source in a DF when I preview. The latter uses the very same var.

For now I'm not going to post the whole query, just the last few lines modified for privacy reasons. I'm hoping that's enough for someone to recognize what seems to be totally different behavior between a lookup being sourced (via the df properties) by a var vs an oledb source in a df being sourced by the same var. I should mention that I have mixed lower and upper case letters in the query and am not sure if that is the problem on the lookup side.

order

by a.CPU_STMP_DT_YR ,

a.ACCT_NBR ,

b.LINE_NBR

fetch

first 1000000 rows only

WITHUR;

↧

SSIS :How to get OUTPUT $action, Inserted.,Deleted. records in In SSIS

September 23, 2016, 11:15 am

≫ Next: How do I configure the CozyRoc Parallel Task?

≪ Previous: db2 query in lookup seems to be complaining about "with ur" phrase

hi all

i am inserting record into table i need value of identity column

using Sql we can do using OUTPUT clause but i want to do in SSIS how i can do that?

↧

How do I configure the CozyRoc Parallel Task?

September 23, 2016, 11:10 am

≫ Next: SSIS - Create ID Field when two fields are identically

≪ Previous: SSIS :How to get OUTPUT $action, Inserted.*,Deleted.* records in In SSIS

I'm hoping somebody has experience with this on SQL Server 2012, as it seems really handy.

I followed the instructions on https://www.linkedin.com/pulse/ssis-parallel-using-cozyrocs-loop-task-eric-whitley

However when I plugged in the parallel task, the results I got were not right.

I have a loop that has a startdate and enddate, and for each date combination it runs three similar queries to populate a temp table.

Previous the temp table was truncated on each loop, so this time I created a new table for each iteration. It also had an inner loop to run the same query three times with different params, however I was getting odd errors, and just changed it to three separate dataflows.

The parallel task completes however the results are not correct.

Each temp table has the exact same data, and instead of seeing data from the above three queries, only the first query appears to have run.

Do I need to do something with the expressions on the parallel task to somehow link the package variable with the loop variable?

Wish there was more documentation on this.

Cheers

Jon

↧

SSIS - Create ID Field when two fields are identically

September 22, 2016, 4:57 am

≫ Next: dtexec command line 2015 connection guid not found

≪ Previous: How do I configure the CozyRoc Parallel Task?

Hi friends,

i have a flat list with many values. For an import in another system i need to create a id value, when two values match a rule. For example

Name, Streeet

Name1, Street1
Name 1, Street1
Name 1, Street1

All theese values should get a new column with an id. ID1 or so. Becuase its the same data.

For the next values i need to repeat theese step, so that i have in the end a list.
The matching rule is fix in this case.

Is there any way to do this with SSIS?

Thanks for your help.

Regards,
Sebastian

↧

dtexec command line 2015 connection guid not found

September 23, 2016, 3:49 pm

≫ Next: SSIS timeout issues

≪ Previous: SSIS - Create ID Field when two fields are identically

Hi there,

I am using Visual Studio 2015 integration services project creating packages.

The server is on another machine and is SQL Server 2014.

I am trying to use dtexec.exe command line.

I am using the format:

"C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\dtexec.exe" /f "D:\SSIS Project\Convert FDDE.dtsx"

I get this error:

Started: 3:41:33 PM
Error: 2016-09-23 15:41:33.70
   Code: 0xC001000E
   Source: Convert FDDE
   Description: The connection "{3D5D8946-652E-4AE3-94BD-F6987B235113}" is not found. This error is thrown by Connections collection when the specific connection element is not found.
End Error

How can I make this work?

Thanks,

Ric

Ric Miller

↧

SSIS timeout issues

September 23, 2016, 1:06 pm

≫ Next: SSISDB log file is increasing very fast. how we can reduce the size of log file.

≪ Previous: dtexec command line 2015 connection guid not found

Hi Experts,

Need small help on this SSIS issue. Check if there is any workaround or better way to alleviate the issues.
We have SSISDB Catalog configuration in place and we see lot of timeout issues.
When I dig deeper we came to know we are hitting the above bug.
Initially, we thought the queries are getting timed out due to performance issues for heavy blocking and so we have changed the remote query timeout.

But in reality we are seeing below errors.
package_name event_name      message_source_name               message
Master.dtsx       OnError                Create Package Execution           Create Package Execution:Error: Executing the query "EXECUTE [SSISDB].[catalog].[create_execution]     ..." failed with the following error: "The operation failed because the execution timed out.". Possible failure reasons: Problems with the query,"ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Master.dtsx       OnError                Create Package Execution           Create Package Execution:Error: Executing the query "EXECUTE [SSISDB].[catalog].[create_execution]     ..." failed with the following error: "The operation failed because the execution timed out.". Possible failure reasons: Problems with the query,"ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Master.dtsx       OnError                Create Package Execution           Create Package Execution:Error: Executing the query "EXECUTE [SSISDB].[catalog].[create_execution]     ..." failed with the following error: "The operation failed because the execution timed out.". Possible failure reasons: Problems with the query,"ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

Basically, we are hitting this BUG in our environment.
https://connect.microsoft.com/SQLServer/feedback/details/783291/ssis-package-fails-to-start-application-lock-timeout-in-ssisdb-catalog-create-execution

Environment details
=======================

SQL Server details
===================
Microsoft SQL Server 2014 (SP1-CU5) (KB3130926) - 12.0.4439.1 (X64)
                Feb 15 2016 12:12:43
                Copyright (c) Microsoft Corporation
                Enterprise Edition: Core-based Licensing (64-bit) on Windows NT 6.3 <X64> (Build 9600: ) (Hypervisor)

OS information
=====================
OS Name                                             Microsoft Windows Server 2012 R2 Datacenter
Version                                6.3.9600 Build 9600
Other OS Description     Not Available
OS Manufacturer                             Microsoft Corporation
System Name                   xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
System Manufacturer    Microsoft Corporation
System Model                   Virtual Machine
System Type                      x64-based PC
System SKU                        Unsupported
Processor                            Intel(R) Xeon(R) CPU E5-4650L 0 @ 2.60GHz, 2600 Mhz, 8 Core(s), 8 Logical Processor(s)
Installed Physical Memory (RAM)             56.0 GB
Total Physical Memory                  56.0 GB
Available Physical Memory          21.1 GB
Total Virtual Memory                     75.9 GB
Available Virtual Memory             38.2 GB
Page File Space                                 19.9 GB

Thanks,

Sam

↧

SSISDB log file is increasing very fast. how we can reduce the size of log file.

September 22, 2016, 11:56 am

≫ Next: Running dtexec on Azure Jenkins Server command line.

≪ Previous: SSIS timeout issues

Hi,

I am using SQL Server 2014 and deployed SSIS packages in SSISDB(SSIS Catalog database)

SSISDB log file(.ldf) is increasing very fast so disk space is being reduced.

how we can reduce the size of log file. Can you advice please ?

Thanks

Bijay Maurya

Tech Mahindra

Regards, Bijay Maurya

↧

Running dtexec on Azure Jenkins Server command line.

September 23, 2016, 3:03 pm

≫ Next: Different Versions of SSIS

≪ Previous: SSISDB log file is increasing very fast. how we can reduce the size of log file.

We're having some technical issue running dtexec on Azure Jenkins server. When we try to run the command, we're getting this:

C:\Program Files (x86)\Jenkins\workspace\BLAH API Deploy - Stage\BLAHBLAH.IntegrationServices>dtexec /f Package.dtsx /ConfigFile stabilization.dtsConfig
Microsoft (R) SQL Server Execute Package Utility
Version 12.0.4100.1 for 32-bit
Copyright (C) Microsoft Corporation. All rights reserved.

Started: 5:35:53 PM
Error: 2016-09-23 17:35:53.52
Code: 0xC0016016
Source: Package
Description: Failed to decrypt protected XML node "DTS:Password" with error 0
x8009000B "Key not valid for use in specified state.". You may not be authorized
to access this information. This error occurs when there is a cryptographic err
or. Verify that the correct key is available.
End Error
Error: 2016-09-23 17:35:53.52
Code: 0xC0016016
Source: Package
Description: Failed to decrypt protected XML node "DTS:Password" with error 0
x8009000B "Key not valid for use in specified state.". You may not be authorized
to access this information. This error occurs when there is a cryptographic err
or. Verify that the correct key is available.
End Error
Warning: 2016-09-23 17:35:53.88
Code: 0x80012011
Source: Package
Description: Cannot load the XML configuration file. The XML configuration fi
le may be malformed or not valid.
End Warning
Warning: 2016-09-23 17:35:53.89
Code: 0x80012011
Source: Package
Description: Cannot load the XML configuration file. The XML configuration fi
le may be malformed or not valid.
End Warning
Warning: 2016-09-23 17:35:53.89
Code: 0x80012011
Source: Package
Description: Cannot load the XML configuration file. The XML configuration fi
le may be malformed or not valid.
End Warning
Progress: 2016-09-23 17:35:53.89
Source: consume claim customer response
Validating: 0% complete
End Progress
Error: 2016-09-23 17:35:54.02
Code: 0xC0202009
Source: Package Connection manager "idb"
Description: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred.
Error code: 0x80040E4D.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0
" Hresult: 0x80040E4D Description: "Login failed for user 'ansadbadmin'.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0
" Hresult: 0x80040E4D Description: "Invalid connection string attribute".
End Error
Error: 2016-09-23 17:35:54.02
Code: 0xC020801C
Source: consume claim customer response Lookup [2]
Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAG
ER. The AcquireConnection method call to the connection manager "idb" failed wi
th error code 0xC0202009. There may be error messages posted before this with m
ore information on why the AcquireConnection method call failed.
End Error
Error: 2016-09-23 17:35:54.02
Code: 0xC0047017
Source: consume claim customer response SSIS.Pipeline
Description: Lookup failed validation and returned error code 0xC020801C.
End Error
Progress: 2016-09-23 17:35:54.02
Source: consume claim customer response
Validating: 25% complete
End Progress
Error: 2016-09-23 17:35:54.03
Code: 0xC004700C
Source: consume claim customer response SSIS.Pipeline
Description: One or more component failed validation.
End Error
Error: 2016-09-23 17:35:54.03
Code: 0xC0024107
Source: consume claim customer response
Description: There were errors during task validation.
End Error
DTExec: The package execution returned DTSER_FAILURE (1).
Started: 5:35:53 PM
Finished: 5:35:54 PM
Elapsed: 0.813 seconds

C:\Program Files (x86)\Jenkins\workspace\BLAH API Deploy - Stage\BLAHBLAH.IntegrationServices>

↧

Different Versions of SSIS

September 23, 2016, 12:38 am

≫ Next: Record count value in SSIS generated output file is not getting updated in few systems.

≪ Previous: Running dtexec on Azure Jenkins Server command line.

Hello Experts,

I have confusions regarding SSIS different versions and which is capable of what.
What is the difference between Shell Isolated and Shell Integrated.

In our office we have Microsoft Visual Studio 2010 Professional but with that I can not open SSIS Packages.

But we have Microsoft Visual Studio 2012 Shell (Integrated) with which we can open the SSIS Packages.

So can anyone please explain the differences and when to install what and why Professional edition can not open SSIS Package is SHELL (Integrated or Isolated) are not installed.

Thanks and Regards,
Pratik

- Pratik
Please use Marked as Answer if my post solved your problem and useVote As Helpful if it was...

↧

Record count value in SSIS generated output file is not getting updated in few systems.

September 20, 2016, 4:13 am

≫ Next: Split text into 4 characters each

≪ Previous: Different Versions of SSIS

SSIS package has been built to generate data from SQL DB to a flat file. After generating the file an audit information should get printed at the end of the file stating record count of the rows loaded.I am using a Script task to print the record count value at the end of the file.
Now the record count value is not getting printed when I execute the package through Integration Service Catalogs in few servers while it is working fine in few others.
For getting the record count I have used a variable in my package which is getting called in the script task to print the output.Also there is no error message in the output file, just that record count variable is giving the output as 0.Any suggestion or workaround on this is very much appreciable as this is required by our clients urgently.

↧

Split text into 4 characters each

September 20, 2016, 1:03 am

≫ Next: SSIS Use Lookup transformation to flag a variable to either flow through update statement or insert new records.

≪ Previous: Record count value in SSIS generated output file is not getting updated in few systems.

Hi, I have a requirement that requires a long string to be split into 4 characters each. Any idea how i can achieve this?

Basically the high level description of the problem is..

Source File:

S/NoValue

1112233445566

212345678901234567890

311123223344456780001112

I would want to split the value column by 4 characters:

S/NoValue1 Value2Value3 Value4Value5 Value6

1112233445566

212345678 901234567890

3111222333444567800011112

Problem now also that the longest i have is about 10,000 text letters and i will need to split them as well. There is not separator within the value column. Therefore, the number of "Value" columns can shoot up depending on the length of the text.

Anybody has any idea please?

Thank you.

↧

SSIS Use Lookup transformation to flag a variable to either flow through update statement or insert new records.

September 22, 2016, 1:27 pm

≫ Next: how to redirect user variable to the error table after derived column

≪ Previous: Split text into 4 characters each

I'm seeking a design and set variable assistance in my SSIS package.
I currently have a merge join and I pass data through a lookup and I need to set a variable @updateflag = 1 if there is a match or set @updateflag = 0 if there is no match(new records). I will then need to have this data flow by a conditional split join to Update a table on set of fields if the flag is set to 1 or insert new records if there are new records. Please let me know if you need additional information.

Please see design of my package in the link.

https://1drv.ms/i/s!AsAeRbH-asQpdw8lVk7MG1kCWS4

Paul Hinostroza

↧

how to redirect user variable to the error table after derived column

September 15, 2016, 4:35 pm

≫ Next: Retrieving Column wise result set from Row wise result set in sql....

≪ Previous: SSIS Use Lookup transformation to flag a variable to either flow through update statement or insert new records.

hi folks:

I have a simple package that loops flatfile data for each year and dump into a table. I've created a user variable @user:: CounterLoop to control the loop. My goal is to redirect errors whenever an error occurs in the derived column .

However, when I double click "OLE DB Destination 4 " and go to the Mappings part, I am unable to find this user variable in the available Input Columns.

My question is how to direct this user variable to the error output for trouble shooting purpose?

Thanks

--Currently using Reporting Service 2000; Visual Studio .NET 2003; Visual Source Safe SSIS 2008 SSAS 2008, SVN --

↧

Retrieving Column wise result set from Row wise result set in sql....

September 26, 2016, 4:46 am

≫ Next: Processing commands contain DDL that is not valid

≪ Previous: how to redirect user variable to the error table after derived column

I had a table which consists of questions and answers as below.....

Question_fk Answers

1 Yes

1 NO

1 N/A

2 Yes

2 No

I want the result set to be.....

Question_fk Answer1 Answer1 Answer3

1 Yes No N/A

2 Yes No N/A

↧

Processing commands contain DDL that is not valid

September 26, 2016, 2:08 am

≫ Next: SSIS Package Config File Localtion

≪ Previous: Retrieving Column wise result set from Row wise result set in sql....

Hi guys,

I tried to process a cube using processing command (Variable) in ssis but I got the following error:

Processing commands contain DDL that is not valid.

Kindly assist.

Regards

↧

SSIS Package Config File Localtion

September 26, 2016, 2:06 am

≫ Next: oledb Source not returning data in ssis

≪ Previous: Processing commands contain DDL that is not valid

Hi,

I have a number of SSIS packages that use a configuration file for database connections.

I have updated the file to point to a new database. However, the package continues to try to connect to the old database.

SSIS is running on SQL Server 2008R2. The packages are called from a different server.

The package config file I updated is on the SSIS server @ C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Packages\

Is this the correct location?

Thanks!

↧

oledb Source not returning data in ssis

September 26, 2016, 12:30 am

≫ Next: Verificar arquivos (Local/FTP)

≪ Previous: SSIS Package Config File Localtion

I have worked on an SSIS solution wherein I migrated the package from SQL Server 2008 R2 to SSDT 2014. After migration, I faced the below issue:

We have 4 OLeDB sources in the Dataflow task. Though all of them get executed without any error, two of them return 0 rows. Upon further inspection of the underlying query at the corresponding OLeDB source, I found that it returns data when I execute it on SQL query window. The frustrating part is that I don't face this issue every time. 2 out of 10 times, this happens.

↧

Latest Images