Building a data flow task, within a foreach loop for dynamic table name, but ole db source not allowing variable

December 19, 2012, 11:53 am

≫ Next: Secure Package deployment. Protecting IP

≪ Previous: Deploying standard changes to SSIS packages

In my control flow, I set up a variable for the table name, enumerated by SMO, following the instructions from the link here:
http://www.bidn.com/blogs/mikedavis/ssis/156/using-a-for-each-loop-on-tables-ssis

Now, I put a data flow task inside the foreach. I selected the OLE DB connection manger for my database, set the Data access mode to "Table name or view name variable", and selected my variable name from the drop down. So far so good. When I click on OK, it gives me an error 0x80040E37, basically saying it can't open the rowset for "my variable", Check that the object exists in the database.

So, I assume I won't be able to do this "that' easily, and I will need to build a "SQL command from variable" or some such thing. Any advice on how to build this Source editor to dynamically name my columns from the variable?

Thanks in advance!

mpleaf

↧

Secure Package deployment. Protecting IP

April 1, 2014, 12:33 pm

≫ Next: Spaces in UNC Path when used in a File System Task

≪ Previous: Building a data flow task, within a foreach loop for dynamic table name, but ole db source not allowing variable

Is there a secure way to deploy an SSIS package to a client site so that the client / client admins can not open or modify the package. It's a very complex package that contains what we consider our companies intellectual property and we do not want the clients to have access.

↧

Spaces in UNC Path when used in a File System Task

March 28, 2014, 7:15 am

≫ Next: Execute SQL Task with Parameter DB2 to SQL Server

≪ Previous: Secure Package deployment. Protecting IP

Hello,

I have a simple SSIS package which moves a template Excel file from one folder to another folder:

Both folders are referenced in a UNC. The natural UNC path is: \\A\B\C D\F E\filename.xlsx

When storing the path in a variable, the expression is: "\\\\A\\B\\C D\\F E\\filename.xlsx"

The SSIS package runs under my account and gives the following error: (could not find a part of the path)

I read in other threads that a lot of times, it's related to access permissions but I verified that it's not. I actually created a file with no spaces under the following parent folder:\\A\B\filename.xlsx and the package ran fine.

How do we handle ignoring UNC path spaces in SSIS? I tried enclosing the whole UNC path expression in double quotes using the escape sequence: ""\\\\\A\\B\\C D\\F E\\filename.xlsx"\" to produce "\\\\A\\B\\C D\\F E\\filename.xlsx" but it doesn't work.

Thank you

↧

Execute SQL Task with Parameter DB2 to SQL Server

April 1, 2014, 1:31 pm

≫ Next: Example of Fact table dataloading

≪ Previous: Spaces in UNC Path when used in a File System Task

I am pulling data from DB2 to SQL Server.

1st Execute SQL task runs the following against DB2:

SELECT TIMESTAMP(CHAR(CURRENT_DATE - (DAY(CURRENT_DATE)-1) DAYS - 1 MONTH)) as START_MONTH
            FROM SYSIBM.SYSDUMMY1;

I'm storing it as a Result Set.
Next I have a Data Flow Task. This pulls data from DB2 where the date matches the parameter.

FROM SCHEMA.TABLE t
WHERE DATE_TIME_COLUMN= ?

This works fine. Guessing, because the parameter source is DB2 and the Data Flow source is DB2.

The problem is, I want to remove existing data for the same date in the SQL table. IE, if I'm pulling March 2014 data from DB2, I want to be sure there is no March 2014 data in the SQL table. The main use is for re-runs.

So, I added another Execute SQL task after the first one that assigns the variable, and before the Data Flow Task. This runs the following statement against SQL Server:

DELETE FROM 
database.dbo.table
WHERE DATE_TIME_FIELD >= ?

The package fails at the new Execute SQL Task with the following error message:

Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "DELETE FROM

SBO.dbo.tblHE_MSP_FEE_BUCKETS

WHERE T..." failed with the following error: "Parameter name is unrecognized.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

Task failed: Execute SQL Task

SSIS package "Load_MSP_Fee_Buckets_SQL.dtsx" finished: Success.

The program '[14240] Load_MSP_Fee_Buckets_SQL.dtsx: DTS' has exited with code 0 (0x0).

I am assuming this is something to do with the Parameter source being DB2, and trying to use against SQL?
Any suggestions on how to make this work??

Thanks,

-Al H

↧

Example of Fact table dataloading

April 1, 2014, 1:31 pm

≫ Next: SSIS Issue loading the data from SQL table into .csv file - Comma is within the data

≪ Previous: Execute SQL Task with Parameter DB2 to SQL Server

Hi Experts

I'm looking to get an example of data loading in fact table that will help me to understand fact table concept . Really appreciate if someone provide my mentioned stuff. Thanks

Regards

Muz

↧

SSIS Issue loading the data from SQL table into .csv file - Comma is within the data

April 1, 2014, 8:39 am

≫ Next: Flat file connection: The file name "\server\share\path\file.txt" specified in the connection was not valid

≪ Previous: Example of Fact table dataloading

Hi,

My source is a SQL table and i'm fetching some data in which there are few fields which has comma within the data. I'm trying to load this data into csv file (flat file destination).

While loading the data, say for example, i have a field with data "Boston,MA" the csv file outputs it as:

"Boston in one field

and MA" in another field.

My field is not fixed length.

I don't want any special character to be replaced for comma and i need the data as it is in from the SQL table. I tried enclosing the column values within double quotes but even that didn't help out.

Is there any way of handling this issue in SSIS ? Is this a known issue that SSIS can't handle while loading into CSV ?Please help me understand this load process

↧

Flat file connection: The file name "\server\share\path\file.txt" specified in the connection was not valid

March 27, 2014, 8:01 pm

≫ Next: SSIS 2012 Project Deployment Wizard "Out of memory" Exception

≪ Previous: SSIS Issue loading the data from SQL table into .csv file - Comma is within the data

I'm trying to execute a SSIS package via SQL agent with a flat file source - however it fails with Code: 0xC001401E The file name "\server\share\path\file.txt" specified in the connection was not valid.

It appears that the problem is with the rights of the user that's running the package (it's a proxy account). If I use a higher-privelege account (domain admin) to run the package it completes successfully. But this is not a long-term solution, and I can't see a reason why the user doesn't have rights to the file. The effective permissions of the file and parent folder both give the user full control. The user has full control over the share as well. The user can access the file (copy, etc) outside the SSIS package.

Running the package manually via DTExec gives me the same error - I've tried 32 and 64bit versions with the same result. But running as a domain admin works correctly every time.

I feel like I've been beating my head against a brick wall on this one... Is there some sort of magic permissions, file or otherwise, that are required to use a flat file target in an SSIS package?

↧

SSIS 2012 Project Deployment Wizard "Out of memory" Exception

April 1, 2014, 4:37 pm

≫ Next: Data Conversion issue

≪ Previous: Flat file connection: The file name "\server\share\path\file.txt" specified in the connection was not valid

Hello,

I encounter "Out of Memory" exception when trying to deploy SSIS Projects to SSISDB catalog using the Deployment wizard invoked through SSMS. A screenshot is attached. The error occurs at "Changing protection level" step, and when I click the "Failed" status, a pop up message shows the out of memory exception. This happens when large packages (combined dtsx file size is over 50MB) are in the project, although the *.ispac file is under 3MB.

I notice that I can successfully deploy the same project (*.ispac file) using the 64-bit deployment wizard which I have to invoke manually (C:\Program Files\Microsoft SQL Server\110\DTS\Binn\ISDeploymentWizard.exe). I tested this with increasing dtsx file sizes and when the combined dtsx file size reaches around 500MB, 64-bit Deployment Wizard also fails.

My desktop is Windows 7 64-bit and has 8GB RAM.

I would like to hear from others who have encountered the same error and whether they have found workarounds.

Thanks

↧

Data Conversion issue

April 1, 2014, 4:37 pm

≫ Next: Stuck

≪ Previous: SSIS 2012 Project Deployment Wizard "Out of memory" Exception

Environment: SQL Server 2008 R2
Problem: an error message occur, when I tried to convert test_dt data type using data conversion transformation. I named it test_dt_ database timestamp [DT_DBTIMESTAMP].

1. [Data Conversion [385]] Error: Data conversion failed while converting column "test_dt" (495) to column "test_dt_" (524). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
2. [Data Conversion [385]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "output column "test_dt_" (524)" failed because error code 0xC020907F occurred, and the error row disposition on "output column "test_dt_" (524)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.

3. [SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Data Conversion" (385) failed with error code 0xC0209029 while processing input "Data Conversion Input" (386). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.

I also checked the data type in column destination which they match.

and I don't see an issue. How I could overcome this puzzle

↧

Stuck

April 1, 2014, 5:04 pm

≫ Next: Help me to solve error

≪ Previous: Data Conversion issue

I have a situation where I want to load the Excel file dynamically, and the excel file have different columns or even worksheet name. Any idea how I could approach this? I believe there's no way to modify the meta data (specifically the mapping) in the data flow.

↧

Help me to solve error

April 1, 2014, 5:07 pm

≫ Next: SSIS Upsert Primary Key Violation error

≪ Previous: Stuck

Hi Experts

I'm facing the below attached error. Please help me out. Thanks in advance

Refards

Muz

↧

SSIS Upsert Primary Key Violation error

April 2, 2014, 8:04 am

≫ Next: Unable to understand datekey concept.

≪ Previous: Help me to solve error

Hello,

I created an SSIS package with look up transform. My source is OLE DB Source and destination is OLEDB destination with Lookup no match output and OLEDB command with Lookup match output.

I am inserting data in destination database if records donot exist or running a stored proc in OLEDB command if the records already exist.

Still I am receiving a

Violation of PRIMARY KEY constraint 'PK_'. Cannot insert duplicate key in object 'dbo.tablename'. The duplicate key value is (734eb1e6-9987-41b6-a143-0039726df29d)."

Can some one tell me where I am doing it wrong. Shd I change the lookup transform to a different transform?

Experts I Need your inputs.

Thanks a ton

↧

Unable to understand datekey concept.

April 2, 2014, 10:05 am

≫ Next: What is mean by Ghoust queries in SSIS oledb Source

≪ Previous: SSIS Upsert Primary Key Violation error

Hi Experts

I'm unable to understand the concept of date key while loading data in fact table. I'm reading a book and as per book I created an OLE DB Source and write the following query:

Query

SELECT QuotaDate, SalesQuota, NationalIDNumber as EmployeeNationalIDAlternateKey FROM Sales.SalesPersonQuotaHistoryINNER JOIN HumanResources.EmployeeON SalesPersonQuotaHistory.BusinessEntityID = Employee.BusinessEntityID

I have got the employee key (Surrogate key) from Dim Employee by using the below query:

Query

SELECT EmployeeKey, EmployeeNationalIDAlternateKey FROM DimEmployeeWHERE EndDate IS NULL

Now as per book, I have to get Date key and in order to get it the following steps mentioned :

Steps to get Date Key

Step 1- For the DateKey, a Lookup is not needed because the DateKey is a “smart key,” meaning the key is an integer value based on the date itself in YYYYMMDD format.Therefore, you will use a Derived column to calculate the Date Key for the fact table. Add a Derived Column Transformation to the Data Flow and connect the blue data path output of the Employee Lookup to the Derived Column Transformation. When prompted, choose the Lookup Match Output from the Lookup Transformation. Name the Derived Column Date Keys.

Step 2- Double-click the Derived Column Transformation and add the following three new Derived Column columns and their associated expressions, as shown below:
DateKey: YEAR([QuotaDate]) *10000 + MONTH([QuotaDate]) *100 +DAY([QuotaDate])
CalendarYear: (DT_I2) YEAR([QuotaDate])
CalendarQuarter: (DT_UI1) DATEPART(“q”,[QuotaDate])

I'm too confused what is the logic behind these 2 steps and I really appreciate if someone explain this to me. Thanks

Regards

Muz

↧

What is mean by Ghoust queries in SSIS oledb Source

April 2, 2014, 10:18 am

≫ Next: Execute SQL Task with Parameter - from DB2 to SQL Server

≪ Previous: Unable to understand datekey concept.

Dear All,

What is mean by Ghoust query, how can you write a ghoust query in OledbSource.

Can any one help on this. How can I use this one,

Regards,

Praveen

Regards, Praveen

↧

Execute SQL Task with Parameter - from DB2 to SQL Server

April 1, 2014, 1:31 pm

≫ Next: Is it possible to see the count of files downloaded from FTP site using FTP Task?

≪ Previous: What is mean by Ghoust queries in SSIS oledb Source

I am pulling data from DB2 to SQL Server.

1st Execute SQL task runs the following against DB2:

SELECT TIMESTAMP(CHAR(CURRENT_DATE - (DAY(CURRENT_DATE)-1) DAYS - 1 MONTH)) as START_MONTH
            FROM SYSIBM.SYSDUMMY1;

I'm storing it as a Result Set.
Next I have a Data Flow Task. This pulls data from DB2 where the date matches the parameter.

FROM SCHEMA.TABLE t
WHERE DATE_TIME_COLUMN= ?

This works fine. Guessing, because the parameter source is DB2 and the Data Flow source is DB2.

So, I added another Execute SQL task after the first one that assigns the variable, and before the Data Flow Task. This runs the following statement against SQL Server:

DELETE FROM 
database.dbo.table
WHERE DATE_TIME_FIELD >= ?

The package fails at the new Execute SQL Task with the following error message:

Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "DELETE FROM

SBO.dbo.tblHE_MSP_FEE_BUCKETS

Task failed: Execute SQL Task

SSIS package "Load_MSP_Fee_Buckets_SQL.dtsx" finished: Success.

The program '[14240] Load_MSP_Fee_Buckets_SQL.dtsx: DTS' has exited with code 0 (0x0).

I am assuming this is something to do with the Parameter source being DB2, and trying to use against SQL?
Any suggestions on how to make this work??

Thanks,

-Al H

↧

Is it possible to see the count of files downloaded from FTP site using FTP Task?

March 28, 2014, 2:20 pm

≫ Next: SSIS package Upgrade from 2005 to 2012

≪ Previous: Execute SQL Task with Parameter - from DB2 to SQL Server

Hi All,

Question: How to confirm multiple files were downloaded during a single FTP Task execution with neither a flat file connection manager nor the names of the files themselves?

Background: Every day, our vendor generates a new data file for us to download. Due to the file name standard that includes create date and time ("ftp_file_2014-03-27-10-59-59.csv"), we have no idea what the file name will be unless we go looking for it manually. I've remedied this problem in the FTP Task Editor by using an Expression Variable for the remote path that builds a file mask (for example, "ftp_file_2014-03-27-*.csv"). This works fine for downloading the right file at the right time.

The download package doesn't always get called daily, so I put the FTP task in a loop that loops through days from the last successful import to today. All's still fine.

Unfortunately, the vendor can't be trusted to provide us only one file per day. This is possible because the create time also appears in the file name, ensuring uniqueness in all file names. The FTP Task will download all the files that match the file mask, and if that is more than one file, I need to hit the kill switch on this package: delete the 2+ files downloaded [but not the previous days' files], log the error, and don't download any more files for later dates.

Thanks,
Eric B.

↧

SSIS package Upgrade from 2005 to 2012

April 2, 2014, 9:26 am

≫ Next: Why is SSIS Conditional Split including unwanted data regardless of condition?

≪ Previous: Is it possible to see the count of files downloaded from FTP site using FTP Task?

Hi,

We have a Solution file with 8 projects for SSIS packages. All of them are in VSS. They all have been created in 2005. Now we installed VS 2012 and SSDT on our local machines. I got latest of solution and projects from VSS on new machine. I tried to open solution file which took me SSIS package Upgrade wizard. I followed the steps and in the last step of conversion, it displayed that it had errors. The errors are:

Error 0xc001f429: Package Upgrade: The file, "C:\SSIS\compay\hello.dtsx", could not be opened for writing. The file could be read-only, or you do not have the correct permissions.

and for some packages, it is like this:

Messages
* Information 0x40019316: : The provider name for the connection manager "OLEDBConnection" has been changed from "SQLNCLI.1" to "SQLNCLI10".

* Information 0x40019316: : The provider name for the connection manager "OLEDBConnection" has been changed from "SQLNCLI10" to "SQLNCLI11".

* Information 0x40016019: : The package format was migrated from version 2 to version 6. It must be saved to retain migration changes.

* Warning: PackageName: Found SQL Server Integration Services 2005 Script Task "ScriptTask_49664f73e0f440bbb0e7d44f01036bff" that requires migration!

* Information 0x4001601a:PackageName: The Script Task "ScriptTask_49664f73e0f440bbb0e7d44f01036bff" has been migrated. The package must be saved to retain migration changes.

* Error 0xc001f429: Package Upgrade: The file, "C:\SSIS\company\Hi.dtsx", could not be opened for writing. The file could be read-only, or you do not have the correct permissions.

But solution and all projects and packages show that they are checked out. I checked properties also. Read only checkbox is not checked.

Please suggest what to do. I appreciate your response.

Thanks,

↧

Why is SSIS Conditional Split including unwanted data regardless of condition?

April 2, 2014, 4:03 am

≫ Next: ssis 2010 visual studio

≪ Previous: SSIS package Upgrade from 2005 to 2012

I have the following condition set in SSIS Conditional Split to send all invalid address to the Output NameInvalidOutput.

However, I keep getting some addresses with NullAddress1 column or with length less than 5. I am not sure why.

ISNULL(LEN(TRIM(Address1)) > 5) || 
ISNULL(LEN(TRIM(ZipCode)) == 5) || 
ISNULL(LEN(TRIM(City)) >= 5) || 
ISNULL(LEN(TRIM(Province)))

Is there also an alternative way to check for empty string?

Thanks!

↧

ssis 2010 visual studio

April 2, 2014, 11:57 am

≫ Next: How to insert another VB section (i.e. AcqureConnections(Object)) into existing VB code of script component

≪ Previous: Why is SSIS Conditional Split including unwanted data regardless of condition?

Right now I want to work with an SSIS package that was written in 2010. On my workstation, I have bids installed for visual studio 2008. I also have visual studio 2012 on my workstation to obtain C# programming code. I am also told that I should be able to look at an SSIS package written in 2010 while working with visual studio 2012.

Thus I have the following questions:
1. What do I need to obtain from the Visual studio 2012 install so that I can look at the ssis 2010 package? Where would I obtain this portion of the install package that I am missing?
2. If I want to obtain the visual studio 2010 environment to work with SSIS 2010 package, where would I obtain this donwload from the MSDN site?

↧

How to insert another VB section (i.e. AcqureConnections(Object)) into existing VB code of script component

April 2, 2014, 10:37 am

≫ Next: SSIS creates empty flat fie, how to avoid that?

≪ Previous: ssis 2010 visual studio

Hello,

I am working with VB within a Script Component of my SSIS Package.I am
modeling my code after another working package. I have several sections of code (i.e.
Declarations, PreExecute, PostExecute, etc.). I need to insert a new section in
between these existing sections and I don't understand how to do that. The new
section will be AcquireConnections(Object)
once I figure out how to add it.

Thanks!

DaveDVF

↧