Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Foreach loop that loops over all files in a folder does not consider first file found

0
0

Hello,

I have a foreach loop in SSIS (as part of Visual Studio 10 and with SQL Server 2012).

It loops over all files in a folder (ForEach File Enumerator).

I set the folder: OK
Traverse Sub folders: OK
I have set up a Variable in Variable Mappings called FileName so it shows User::FileName: OK
Index of variable is 0: OK

It almost works ok. Except that the first file it finds is never considered.

I set a breakpoint then and the first file it finds, is shown in black in the watch variable window. All subsequent files found are shown in red font. When I change the names of the files so another file is the first file found then it skips the other one which now is the first file.

What is going on here? Why does SSIS skip the first file it finds in a foreach file loop?

Any suggestions highly appreciated.
Thank you

Andi


Andreas


Error: 0xC020902A at Data Flow Task, XML Source [24515]: The "component "XML Source" (24515)" failed because truncation occurred, and the truncation row disposition on "output column "MsgLev1" (26196)" specifies failure on truncation. A truncation error o

0
0

When I was Importing data from XML to SqlServer using SSIS , I am getting this error. The import is working if i use small file and not working if I use large XMl file. Can any one of you guys help me out with the issue?

 

Error: 0xC020902A at Data Flow Task, XML Source [24515]: The "component "XML Source" (24515)" failed because truncation occurred, and the truncation row disposition on "output column "MsgLev1" (26196)" specifies failure on truncation. A truncation error occurred on the specified object of the specified component.
Error: 0xC02092AF at Data Flow Task, XML Source [24515]: The component "XML Source" (24515) was unable to process the XML data. Pipeline component has returned HRESULT error code 0xC020902A from a method call.
Error: 0xC0047038 at Data Flow Task: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on component "XML Source" (24515) returned error code 0xC02092AF.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "SourceThread0" has exited with error code 0xC0047038.  There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047039 at Data Flow Task: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047039 at Data Flow Task: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread1" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047039 at Data Flow Task: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread3" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047039 at Data Flow Task: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread4" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047039 at Data Flow Task: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread2" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread0" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread1" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread2" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread3" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047021 at Data Flow Task: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread4" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.

Create table and import Flat file data based on flat file

0
0

Hi ,

I have a requirement where i have import flat files to SQL Server 2012 DB Table dynamically. I have already used Script task to read the first row of the flat file and generated the create table script. I was able to execute the create table from the Execute SQL task using variable. But i'm not able to map the flat file source columns to the OLE-DB destination (recently generated table). Column names have special charaters. Please suggest me to overcome this issue.



Thanks, Thiya


Can someone PLEASE help me with an import of a .xlsx spreadhseet to SQL Server utilizing SSIS?

0
0

Ok...here's what I've tried...and I had this working at one time and just don't remember how I got it to work. I have a .xlsx spreadsheet that I need to pump into a SQL Server Table with some UpSert logic and some data transformations. My problem is I can't even get the sourcing to work to start this.

  1. I created an Excel Connection Manager and an Excel Source
  2. I then tried an OLE Db Source and defined a Microsoft Office Access Database Engine OLE DB Provider...found this web site...http://hrvoje.piasevoli.com/2010/09/01/importing-data-from-64-bit-excel-in-ssis/

When I try previewing the data I get this error in both cases. Can anyone PLEASE help me out here? I am at my wits end.

Thanks for your review and am hopeful for a reply.

Changing SQL Server Table names and related related references in SSIS Packages

0
0

Hi All,

I am changing some table names in my SQL Database.  We also have a lot of packages and they depend on these tables. Few questions .

What is the impact of changing the table names on the packages? Is there any way to find where the table names have been used ? ie , in the properties, expressions or SQL queries etc on the package? 

Thanks

KangrinPoqe

Multiple runs of same stored procedure

0
0

I am an SSIS newbie so please excuse how little I know.

We have a manual job that I would like to automate in SSIS.

We run the same stored procedure six times with different parameters. Each run creates a csv file that we load into excel and then mail the attachments to the user. Can SSIS run the SP, and in a send mail task, send the six outputs?

Data Conversion Issue

0
0

I've inherited some SSIS code, and not having some trouble modifying a data conversion task.

Here are the original conversion statements. These columns are actually in MySQL - which I have never worked with in the past. Currently, they're being converted to what I thought were strings, but when trying to join the data columns to other data columns in a Lookup task, I get an error saying they're BLOB's.

REPLACE(LEFT((DT_WSTR,50)processdate_date,19),"T"," ")

REPLACE(LEFT((DT_WSTR,50)receiveddate,19),"T"," ")

I need to actually convert these into dates - with no time. I tried replacing DT_WSTR with DT_DATE, but that didn't work.

If someone could provide some insight into what I may be missing, that would be great!

Thanks!!


A. M. Robinson

ssis --how to handle the situation of update and insert

0
0

hi fiends:

       I want to have a advice on a problem.  now ,I have a flat file :file1  and a sqlserver database table : t_1,

they both record the vendor of the company,and they column number and column name is the same. and the id column is the primary key. I want to do a ssis project , send the record of file1 to the table t_1,if the t_1 have the same ID record with file1,then update the record . and if the record is new ,insert it into the t_1. what componet of dataflow should I use. can you give me advice?

thanks a lot!

ming


Pass Variable to ADO.Net Data Source For Use in Query

0
0

I have a situation where I need to take data from a SQL Server query and pass it to an ADO.Net data source, which uses a query to return records from a MySQL database.

I need to retrieve the maximum date from a table in SQL Server, then filter the MySQL table on that date.

I currently have a variable created in SQL Server called @MaxProcessedDate which is a datetime. I also have a variable called MaxProcessedQuery, which is a string, and looks like this:

"SELECT MAX(processedDate) FROM FactActivity"

My ADO.Net source query is this:

SELECT id,
       consumerid,
       keyword,
       receiveddate,
       processeddate,
       createdappid,
       field,
       unit,
       value,
       insertdate,
       enabled,
       persistid,
       lastupdated
FROM  mnemonic

I need to pass the following to the source query:

SELECT id,
       consumerid,
       keyword,
       receiveddate,
       processeddate,
       createdappid,
       field,
       unit,
       value,
       insertdate,
       enabled,
       persistid,
       lastupdated
FROM  mnemonic

WHERE processeddate > @MaxProcessedDate

How does one accomplish this with the ADO.Net data source. I've been looking everywhere but can't seem to find anything. Any help would be appreciated!!


A. M. Robinson

dead locks caused by truncate

0
0

I have one table one row with one column, this table will always have one row only(date). I am getting this table populated from csv file using SSIS. I have a truncate in place before loading this table using execute sql task (having just truncate on this table) then followed by data flow task to load this table from csv.

However occasionally this is causing deadlock, and getting hanged forever. I have few questions

1. Replacing this with delete then commit(to avoid trans log) will be a good option?

2. I see we can have no lock option with select statement, do we need to add(or can we add) no locks for the insert and delete also?

I read online that truncate can cause the lock on not only table but entire schema is it because of this.

Thanks!


Neil

Multiple HTML files

0
0

Hi,

I have a scenario where in a table , I have clientID, filename and the html_text.

Using these two, I need to create multiple html files for a given client ( the filenames will be from the filename column).

These output html files needs to be created in a directory (given by the business user)

We want to give the link to this report/utility in the business user's sharepoint .

Can you please help in strategizing the approach ?

P.S: I have access to develop this using either SSRS or SSIS.

Thanks,

Anu

[Solved] Foreach loop that loops over all files in a folder does not consider first file found

0
0

Hello,

I have a foreach loop in SSIS (as part of Visual Studio 10 and with SQL Server 2012).

It loops over all files in a folder (ForEach File Enumerator).

I set the folder: OK
Traverse Sub folders: OK
I have set up a Variable in Variable Mappings called FileName so it shows User::FileName: OK
Index of variable is 0: OK

It almost works ok. Except that the first file it finds is never considered.

I set a breakpoint then and the first file it finds, is shown in black in the watch variable window. All subsequent files found are shown in red font. When I change the names of the files so another file is the first file found then it skips the other one which now is the first file.

What is going on here? Why does SSIS skip the first file it finds in a foreach file loop?

Any suggestions highly appreciated.
Thank you

Andi


Andreas


SQL Jobs are running long time

0
0

Hi Team,

I have one job which has 5 steps and earlier 4th and 5th steps were taking 5 minutes time to complete and now 4ht and 5th steps are taking 10 minuts of time. there is a increase of 5 minuts and delaying for our reports.

Both these steps are executing SSIS packages

Please help me to understanad what things should i consider to complete these steps in less time

I have a scenario like this please give me any suggestion hw to i approach?

0
0

I am having ErrorCode ,ErrorMessage,ETLCreatedBySourceID columns in my table..

At the time of new  data inserting the system will generate sourceId  will generate automatically and also 
ErrorCode column data type is bit, so based on the Errorcode I have to generate ErrorMessage..
I didn't get how to do this..
So can u please help me..
My output like attached document...

please give me a solution for this

Pls suggest for this...

ErrorErrorMessage
1Fact not mapping to following dimensions: Circle
1Fact not mapping to following dimensions: Channel
1Fact not mapping to following dimensions: Handset
1Fact not mapping to following dimensions: Channel
1Fact not mapping to following dimensions: Circle
1Fact not mapping to following dimensions: Channel
1Fact not mapping to following dimensions: Handset
1Fact not mapping to following dimensions: Circle
1Fact not mapping to following dimensions: Handset

Thanks!



Unable to rename file for archive files using filesystemtask

0
0

Hi,

I have to move and rename files using file system task. I can move files but unable to rename it with timestamp.

My variables are:  

user::filepath   ----> its having total file path of the .csv file

user::movefiles  ----> d:\dir\dir1\archive\data\

for this path files are moving.

for adding time stamp : user::movefiles---->properties--->eveluate exp:true

EXp : @[User::FileName] +"_" +(DT_WSTR,4)YEAR(GETDATE())+(DT_WSTR,2)MONTH(GETDATE())+(DT_WSTR,2)DAY(GETDATE()) + "_" + REPLACE(SUBSTRING((DT_WSTR,30)GETDATE(),11,9),":","") + ".csv"

I am getting error as could not find the part of the path.

please help on it...

Thanks

AVS


Data Import using SSIS 2008 and Excel Source.

0
0

While Importint data from Excel xlsx/xls , if some column has invalid values
say the column has "TRUE/FALSE" , but some how (manual entry) it is changed to "abc" , if the column has many values as Date but some how user by mistake make it as "abc"
then the while reading from Excel through ssis script Task the column turns out to be blank, as while the data type cannot be able to validate and report data error issue to end user.

 

                if (ext.ToUpper() == ".XLS")
                {

                    sourceConnString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + excelFile + ";Extended Properties=\"Excel 8.0;HDR=Yes;IMEX=1\"";
                }
                else if (ext.ToUpper() == ".XLSX")
                {
                    sourceConnString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + excelFile + ";Extended Properties=\"Excel 12.0;HDR=Yes;IMEX=1\"";
                }

Have used IMEX=1 to treat all column as string , but still it is not working. So if alll column becomes string it will accept all values
irrespective of TRUE/FALSE or abc , but IMEX=1 doesnot work out.

 

                    excelSheetName = dt.Rows[0]["TABLE_NAME"].ToString();

                sourceDA = new OleDbDataAdapter("SELECT * FROM [" + excelSheetName + "]", new OleDbConnection(sourceConnString));
                sourceDS = new DataSet();
                sourceDA.Fill(sourceDS);

The Excel file has been read using Script Task in SSIS 2010

When we view the data set "sourceDS" it turn blank so , no data is available while the user has provided "abc"

please suggest how we can retain all the values and validate and produce the relevant data error to the user

How to migrate DTS package to SSIS 2012? I was thrown with an error when destination server address has given in Package migration wizard

0
0

Hi respected developers,

After much R & D also, I could not find the right solution.Can anyone please help me in how to migrate DTS packages to SSIS 2012 packages . When I am trying migration using Package Migration Wizard, it throws an error (under Choose destination location window) as failed to connect. I have given the correct server name & credentials.I also checked that package name does not contain any spaces in between.Please help me.


sammy

Package Validation Error "The operation could not be completed. (Microsoft.DataTransformationServices.VSIntegration)"

0
0

Help

I am trying to execute a package within Visual Studio 2008 Business Intelligence Developer.

However there are no errors, warnings etc. just a dialog that pops up with this message:

Package Validation Error

Additional Information:

> The operation could not be completed. (Microsoft.DataTransformationServices.VsIntegration)

How can I troubleshoot this if there are no errors displayed?


I.W Coetzer

loop through folders name via sourcepath

0
0

Hi,

How to Loop through folders and get each file inside  a particular folder.

I have a sourcePath : D:/Myfolder

FolderName : App1,App1

FileName : file.csv,file1.csv    inside each folder


How can i get all folder names from sourcepath from folder need to get .csv files

Thanks

AVS

SSIS/Oracle occasionally truncating the last character on one row, one column

0
0

This problem is driving me up a wall. It will typically just be one record out of many (1000's ++), and seems like it happens on one day out of 10. If i rerun the extract all records come through fine. For the longest time I thought it was some mystery process running on the source database that was messing with the data while I was trying to extract it. Now it's happening on a new table... this is going to be a big problem for me very soon.

It's not consistent in any way that I can see other than it seems to be the same column.  However that could be just because of the data mart logic, other columns could miss a character on just one row and no one would notice.  In this case it causes a duplicate in a key.

Does anyone have any leads for me to follow?  Oracle OLEDB to MS-SQL OLEDB.  Have now seen the issue in SSIS 2008R2 & SSIS 2012.

-Ken

Viewing all 24688 articles
Browse latest View live




Latest Images