SSIS Update Column on Database from the current recordset in memory

April 8, 2019, 11:47 pm

≫ Next: Execute dynamically references SSIS packages

≪ Previous: duplicate error message SSIS package

Hello All

I have a package that runs after a previous insert made to the database. After that execute another dataflow task and do another select and combine different things to update a specific column in the same table. Once i get the result i want, how can i update the same records on the fisical table with the values i have on the record that is in memory.

thanks in advance!

↧

Execute dynamically references SSIS packages

April 9, 2019, 2:28 am

≫ Next: Got "System.Runtime.InteropServices.COMException (0x80040427): Execution was canceled by user." when execute a SSIS 2008 package

≪ Previous: SSIS Update Column on Database from the current recordset in memory

Hello,

I have a master package with execute package task. I should execute dynamically SSIS package stored in same project.

but I had a error in execute SSIS task (packge is not specified).

I use project reference in reference type.

↧

Got "System.Runtime.InteropServices.COMException (0x80040427): Execution was canceled by user." when execute a SSIS 2008 package

April 9, 2019, 2:58 am

≫ Next: Native Client 11.0 error when importing.

≪ Previous: Execute dynamically references SSIS packages

I have a SSIS package which was migrated from DTS SQL2000 to SSIS 2008.

The package designed to perform following jobs:

Do the data cleansing by truncate the tables in SQL Server
Transform data from Oracle Database into SQL Server tables

The package is working fine until the server (Windows Server 2008 R2) which install SQL Server 2008 enabled TLS1.2 and disabled TLS 1.0 & TLS 1.1

Error "System.Runtime.InteropServices.COMException (0x80040427): Execution was canceled by user." will be thrown when executing task 2 (Oracle to SQL server transformation). Data cleansing part was done successfully.

Provider for Oracle used = OraOLEDB

Provider for SQL Server = MSOLEDBSQL

Any help will be very much appreciated.

↧

Native Client 11.0 error when importing.

April 4, 2019, 10:56 am

≫ Next: .csv load, target table has more rows than source

≪ Previous: Got "System.Runtime.InteropServices.COMException (0x80040427): Execution was canceled by user." when execute a SSIS 2008 package

I am getting the below message when importing data into a database on a remote server:

This just started happening after the installation of Windows 365. Any ideas?

client unable to establish connection
TCP Provider: An existing connection was forcibly closed by the remote host. (Microsoft SQL Server Native Client 11.0)

------------------------------
Program Location:

   at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnectionString constr, OleDbConnection connection)
   at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)
   at System.Data.ProviderBase.DbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionInternal.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
   at System.Data.OleDb.OleDbConnection.Open()
   at Microsoft.SqlServer.Dts.DtsWizard.DTSWizard.GetOpenedConnection(WizardInputs wizardInputs, String connEntryName)
   at Microsoft.SqlServer.Dts.DtsWizard.Step2.OnLeavePage(LeavePageEventArgs e)

↧

.csv load, target table has more rows than source

April 9, 2019, 5:17 am

≫ Next: Calling oracle statements from Script Task

≪ Previous: Native Client 11.0 error when importing.

this is weird, I have loaded a series of .csv files into one sql server table, each file belongs to one day of the month

so when i check the counts for one of the days, the target table has more rows than the source csv file

1. how to find out why target table has more rows

2. in csv, everything is text, right now my table has even date columns as char, can i have date columns in my target table

when loading from csv

↧

Calling oracle statements from Script Task

April 9, 2019, 7:07 am

≫ Next: Populate and Import From Webform

≪ Previous: .csv load, target table has more rows than source

I am executing oracle statements from Script task in my SSIS 2008 package

OleDbConnection oraConnection = (OleDbConnection)(Dts.Connections["ABCConnection"].AcquireConnection(Dts.Transaction)asOleDbConnection);

oraConnection =

newOleDbConnection(oraConnection.ConnectionString);

oraConnection.Open();

DataSet ds =newDataSet();

OleDbDataAdapter adapter =newOleDbDataAdapter("select count(*) from emp", oraConnection);

adapter.Fill(ds);

Int32 dataCount =Convert.ToInt32(ds.Tables[0].Rows[0][0]);

oraConnection.Close();

My code is returing 0 recrods. But I have data in that table.

Can you please help me on this.

Thanks.

↧

Populate and Import From Webform

April 9, 2019, 7:32 am

≫ Next: DTLoggedExec Util

≪ Previous: Calling oracle statements from Script Task

Good morning, all -

I am working on an SSIS tool to import a csv file from a web source. The particular data to be imported is created by a web form to select the particular fields, the date range and the source database.

Unlike other site I import data from, this one does not retain the field selection from the prior day's run. So, I have to go through the whole list of checkboxes, checking the ones I need individually. This can become tedious and opens up a potential error, in case a check box is mistakenly checked or a needed one is missed.

Is there any way to have SSIS programmatically fill out the web form and then import the csv?

Thanx in advance for any assistance!

↧

DTLoggedExec Util

December 30, 2015, 5:30 pm

≫ Next: Employee Dimension Truncated everyday in Datawarehouse

≪ Previous: Populate and Import From Webform

I am working on a new logging framework and trying to move our current packages and processes away from hardcoded configs and connection strings. I've upgraded DTLoggedExec utility to work with MSSQL 2012 and 2014 and would like to build a lot of my framework around it.

I like its external process; lightweight; and allows changes outside of the package.

The drawback I have with it is it right now provides 2 log's (Console or CSV). I'd prefer to log in to the MSSQL database directly so that for a long running process; I can leverage SQL/SSRS reports from a dashboard perspective. I need to wait for the process to finish before I load in the files to the database currently.

Before I go down this road; has anyone else run in to this situation or written their own customer LogProvider? Any input would be appreciated!

↧

Employee Dimension Truncated everyday in Datawarehouse

April 9, 2019, 2:19 pm

≫ Next: How to check SSIS failed jobs error messages

≪ Previous: DTLoggedExec Util

I am developing a new data warehouse and my source tables for the employee dimension gets truncated every day and reloaded with all history and updates,deletes and new inserts.

The columns which tracks these changes are effective date & effective sequence.We also have a audit table which helps us determine which records are updated,inserted and deleted every day by comparing table from today & previous day.

My question is to how can I do a incremental load on the table in my staging layer so the surrogate key which is a identity columns remains same.If I do a truncate on my final dimension then I get new surrogate key each time I truncate and hence it mess up my fact table.

.I found a way to keep track of the identity column DBCC CHECKIDENT ('.dimEmployee' , RESEED, 1) .Does this approach have any loop holes or scenarios which it does not captures.I will also truncate and reload the fact table which will use the employee key.

↧

How to check SSIS failed jobs error messages

April 9, 2019, 7:36 pm

≫ Next: how to compare columns structure before processing files in ssis

≪ Previous: Employee Dimension Truncated everyday in Datawarehouse

Hi Msdn, I am unable to find exact ssis error message either from job history or from SSIS report. Could you help me with this.

↧

how to compare columns structure before processing files in ssis

April 9, 2019, 10:04 pm

≫ Next: What should be recovery model for SSIS DB and what data does SSISDB stores ?

≪ Previous: How to check SSIS failed jobs error messages

Hi I have one doubt in ssis

I want load every day 3 different metadata files into 3 different table using ssis.
day by day in the source path files name are coming different

for sample files.
1st day files are a.txt and b.txt and c.txt
2nd day files are xy.txt and yz.txt and za.txt
3rd day files are comes some other different name.
above 3 days are sample filesnames.file names not come exactely it will come different names.
here inside files should be diffrent columns for 3 different files.
so here I need to find columns structure comparision(source to target) before processing the source files using ssis.

sourcepath : c:\test\
source path have files 1)a.txt
2)b.txt
3)c.txt
a.txt file structure : id |name
b.txt file structure : id | name | deptno
c.txt file structure : id |name | Locaiton|sal|deptno
Target have 3 tables : 1)emp and structure : id|name
2)emp_dept and structure : id|name |deptno
3)emp_sal and structure : id|name |location|sal|deptno

some times file names are comes different name, that time before processing we need compare source to target columns structure
then process it.
here I follow the steps like below using ssis

in controal flow: drag and drop 1 dft and go the dataflow here I used 3 differet flatfile source and create 3 diferent flat file connections
2nd way I took 3 dft and each dataflow task I create one flat file soruce connection and configure to destionaton tables.

then conffigure to 1flat file source to emp tabe and 2flatfilesource connection to emp_dept and 3rd flatfile soure connection to emp_sal

after that used file system task for move processed files into archive folder .

its working file when i run the first day

if i run same package next day package will failed due files names are comes different and flatfile connection configruation file names are not exist in the sourcepath.

day by day file names are comes different ,
can you please tell me how to compare columns structure before loading to the table and how will I decide which source file which table need to load
please tell me how to achive this task in ssis .

↧

What should be recovery model for SSIS DB and what data does SSISDB stores ?

April 10, 2019, 1:28 am

≫ Next: While populating computed field data from list in sharepoint into table through SSIS package, getting error

≪ Previous: how to compare columns structure before processing files in ssis

Please provide answers to the following questions :

1. What should recovery model configuration of SSISDB

2. What data does SSIDB stores ?

3. Is it required to take back up SSISDB ? If yes what should be the method of storing the back up

↧

While populating computed field data from list in sharepoint into table through SSIS package, getting error

March 27, 2019, 1:48 am

≫ Next: SSIS Data Flow Task hanging

≪ Previous: What should be recovery model for SSIS DB and what data does SSISDB stores ?

While populating computed field data from list in sharepoint into table through SSIS package, getting below error

cannot access a disposed object.
Object name:'System.Net.HttpWebResponse'.

Please let me know how to resolve this issue.

Thanks in Advance.

↧

SSIS Data Flow Task hanging

April 4, 2019, 5:27 am

≫ Next: How to test for SUB (ctrl Z)

≪ Previous: While populating computed field data from list in sharepoint into table through SSIS package, getting error

I am busy running a package that brings in data from a number of mariaDBs. I built the package with SSIS and it has been running fine for a few months. Last week Friday the package started hanging on one of the Data flows. It normally hangs on one of the data flows (the first) but occasionally hangs on another. It doesn't always hang on the same table in each flow and if you keep trying to run it, it eventually works. To test the issue I picked one table that it hangs on often and created a package to load that table. Sometimes it hangs and sometimes it works. When it hangs it doesn't always hang at the same point. The data in the table doesn't appear to be an issue but there may be something that I am missing.

There is no error message in the logging (See below). The output from the debugging in visual studio states: "SSIS package [path] finished: Canceled". Does anyone have any ideas as to how I can fix this?

Tried:

Change transactions from required to supported

Change validate external metadata to false

Running in visual studio and SSMS

Separating the load of the table into different tasks

Increasing the size of the columns that the data is inserted into

Setting delay validation to false

last lines in the logging:

Execute phase is beginning.

PrimeOutput will be called on a component. : 2 : ODBC Source : ODBC Source

Data flow engine will call a component method. : 2 : ODBC Source : PrimeOutput

Rows were provided to a data flow component as input. : : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] :