Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Subroutine Call In SSIS

April 4, 2019, 4:24 am
$
0
0

Hi Experts,

Is it possible in SSIS, Subroutine call? going into details

I am creating a package in which I want if data like Account number of the bank is present it will verify(like Account number can't be a string or account number should be 16 digits). 

This can be done using conditional split but as per client requirement, he wants something like a subroutine call. Account number will be verified if it matches then it will go to the databases. if not matches job will stop(or just write file)

Thanks in Advance

Best Regards,

Ravi

↧
Search

SSIS Data Flow Task hanging

April 4, 2019, 5:27 am
$
0
0

I am busy running a package that brings in data from a number of mariaDBs. I built the package with SSIS and it has been running fine for a few months. Last week Friday the package started hanging on one of the Data flows. It normally hangs on one of the data flows (the first) but occasionally hangs on another. It doesn't always hang on the same table in each flow and if you keep trying to run it, it eventually works. To test the issue I picked one table that it hangs on often and created a package to load that table. Sometimes it hangs and sometimes it works. When it hangs it doesn't always hang at the same point. The data in the table doesn't appear to be an issue but there may be something that I am missing. 

There is no error message in the logging (See below). The output from the debugging in visual studio states: "SSIS package [path] finished: Canceled". Does anyone have any ideas as to how I can fix this?

Tried:

Change transactions from required to supported

Change validate external metadata to false

Running in visual studio and SSMS

Separating the load of the table into different tasks

Increasing the size of the columns that the data is inserted into

last lines in the logging:

Execute phase is beginning.  
PrimeOutput will be called on a component. : 2 : ODBC Source : ODBC Source
Data flow engine will call a component method. : 2 : ODBC Source : PrimeOutput
Rows were provided to a data flow component as input. :  : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] : 
Data flow engine will call a component method. : 87 : OLE DB Destination : ProcessInput
Data flow engine has finished a call to a component method. : 87 : OLE DB Destination : ProcessInput : 131988512852893389 : 131988512853483038
Rows were provided to a data flow component as input. :  : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] : 
Data flow engine will call a component method. : 87 : OLE DB Destination : ProcessInput
Data flow engine has finished a call to a component method. : 87 : OLE DB Destination : ProcessInput : 131988512855951629 : 131988512856551276
Rows were provided to a data flow component as input. :  : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] : 
Data flow engine will call a component method. : 87 : OLE DB Destination : ProcessInput
Data flow engine has finished a call to a component method. : 87 : OLE DB Destination : ProcessInput : 131988512861458464 : 131988512862327957
Rows were provided to a data flow component as input. :  : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] : 
Data flow engine will call a component method. : 87 : OLE DB Destination : ProcessInput
Data flow engine has finished a call to a component method. : 87 : OLE DB Destination : ProcessInput : 131988512867584956 : 131988512869313954
Rows were provided to a data flow component as input. :  : 148 : ODBC Source Output : 87 : OLE DB Destination : 100 : OLE DB Destination Input : 2691 : OLE DB Destination : Paths[ODBC Source.ODBC Source Output] : OLE DB Destination.Inputs[OLE DB Destination Input] : 
Data flow engine will call a component method. : 87 : OLE DB Destination : ProcessInput
Data flow engine has finished a call to a component method. : 87 : OLE DB Destination : ProcessInput : 131988512872702027 : 131988512873901325

↧
↧

Column cannot convert between unicode and non-unicode string data types

January 31, 2014, 12:47 pm
$
0
0

I am converting SSIS jobs from SQL Server 2005 running on a Windows 2003 server to 2008R2 running on a Windows 2008 server.  I have a dataflow with an OLE DB Source which is selecting from an Oracle view.  This of course worked fine in 2005.   This OLE DB Source will not even read the data from Oracle without the error "Column "UWI" cannot convert between unicode and non-unicode.  The select is:

SELECT SOME_VIEW.UWI AS UWI,

             CAST(SOME_VIEW.OIL_NET AS NUMERIC(9,8)) AS OIL_NET

FROM SOME_SCHEMA.SOME_VIEW

WHERE OIL_NET IS NOT NULL AND UWI IS NOT NULL

ORDER BY UWI

When I do "Show Advanced Editor" on this component, in the Input and Output Properties, I show the OLE DB External Column as DT_STR length 40 for the UWI column and for the Output Columns I see the UWI as the same DT_STR.

How can I get past this?  I have tried doing a cast...cast(SOME_VIEW.UWI AS VARCHAR(40)) AS UWI and this gives the same error.  The column in Oracle is a varchar2(40).

Any help is greatly appreciated.  Thanks.







↧

FEATURE REQUEST: Enable execution of a subset of a control flow

April 4, 2019, 6:07 am
$
0
0

It would be very useful to be able to debug a control flow by executing a certain portion of its tasks by:

1. Multi selecting them first

2. Defining a starting point (run until the end)

↧

FEATURE REQUEST: Enable multiple selection/change of precedence constraints

April 4, 2019, 6:13 am
$
0
0

Sometimes I find myself in need to multi select many precedence constraints in order to:

1. Delete and recreate them in a different direction.

2. Change all their constraint options.

Please add a "select all precedence constraints" option and a shared dialog window to change all of their options in one action.

↧
↧

Issue using Merge Join transformation after an aggregate transformation - SSIS 2017

March 29, 2019, 8:58 am
$
0
0

Hi,

I've implemented a SSIS 2017 package having a merge transformation and an aggregate transformation.

In the aggregate transformation I've specified a group by operation for a DT_WSTR code and a count distinct for the same code.

In the merge join transformation as join Keys I've specified the code from the aggregate transformation, on the right side, and the equivalent code on the left side: for a specified value for the matching code no values are returned from the right side (aggregate transformation). I've queried the corresponding source table and I can see that the two codes match rightly.

How can I solve this issue? Can I use an aggregate transformation before a merge transformation? Is there a bug? Thanks

Obviously, I've ordered the two sources for the merge join transformation.

↧

Error - cannot convert between unicode and non-unicode string data types

April 4, 2019, 9:57 am
$
0
0


The packages were working fine till we upgraded to SQL server 2016 from SQL Server 2012.

Using the Oracle OleDB driver 12.10

Now I am getting this error - "cannot convert between unicode and non-unicode string data types"

The packages did not change. So there has to be some setting in the driver/server that needs to be changed, I think.

Can someone help.

↧

Native Client 11.0 error when importing.

April 4, 2019, 10:56 am
$
0
0

I am getting the below message when importing data into a database on a remote server:

This just started happening after the installation of Windows 365.  Any ideas?

client unable to establish connection
TCP Provider: An existing connection was forcibly closed by the remote host. (Microsoft SQL Server Native Client 11.0)

------------------------------
Program Location:

   at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnectionString constr, OleDbConnection connection)
   at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)
   at System.Data.ProviderBase.DbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionInternal.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
   at System.Data.OleDb.OleDbConnection.Open()
   at Microsoft.SqlServer.Dts.DtsWizard.DTSWizard.GetOpenedConnection(WizardInputs wizardInputs, String connEntryName)
   at Microsoft.SqlServer.Dts.DtsWizard.Step2.OnLeavePage(LeavePageEventArgs e)

↧
Search

With the advent of Spark and Microsoft adopting it in the latest version of SQL server is SSIS now outdated technology?

April 4, 2019, 11:59 am
$
0
0

With the advent of Spark and Microsoft adopting it in the latest version of SQL server is SSIS now outdated technology?

Can anyone confirm if this is true? I would like to know what is the future of SSIS . As per Microsoft there will be some ETL use cases but in general there will be a shift to new way of ingesting data with high data transmission speed.

I see in the cloud offering both Databricks( cloud spark) and Datafactory ( cloud ssis)

Also will you do ETL with SPARK or SSIS or any other popular ETL tool?




↧
↧

how to connect multiple XSD files to one XML Source file in SSIS

July 30, 2010, 9:27 am
$
0
0

Hi all,

I am new to deal with XML files.  My problem is, I have a XML file as a source. and my client provided the XSD files. but the problem is, he provived 3 xsd files for one xml file. Each xsd is inter related. the 1st xsd is connected to 2 and 3 and 2nd is connected to 3.

But in SSIS in the xml source i can pass only one xsd file for my source. Can anyone please give me some suggestions to procced.

Thank you.

↧

Loading from table to History and email notify

April 4, 2019, 12:36 pm
$
0
0

Hi

I get source files and load in to a table on a daily basis. the data on this table would have 5 different fields with 2 fields as date columns . 

The unique column would be date1 and Col2. When ever i see the last business day of month i need to grab those records and populate in to history table. 

There can be updates on this source table and need to maintain history of data under historical table and send out email to customers the old and new value.

i can use slowly changing dimension and store the historical attribute with start and end_dt where end_dt would be updated when there is a change in value. 

Question, How can i grab the old and new value and send email notifications

Thanks

raj

↧

SSIS: When set the task to ignore errors, don't log the errors

March 27, 2019, 9:06 am
$
0
0

Hello there. Ignoring any discussion of 'coding it right in the first place' (I have my reasons)….I have an SSIS package that loops through a task and sometimes it fails and other times it doesn't.

I've got it functionally working, i.e. errors are ignored, parents + package succeed (as designed) but I can't seem to supress the error logs.

Is there a way to stop these logs?

Thanks.

↧

Sales Order and Detail Issue

April 1, 2019, 9:09 am
$
0
0

Greeting 

I'm working with the AdventureWorks DataBase I do have 3 Tables 

Table1  : OrderSales (OrderSaleslID "PK" , OrderDate , CustomerID )

Table 2 : OrderSaleReason (OrderSaleslID "PK" , SalesReason "PK")

Table 3 : OrderSalesDetail  (OrderSalesDetail ID "PK" , OrderSaleslID "PK" , ProductID , QTE ) 

My fact table "Sales" is based on the OrderSalesDetail   

FactSales (OrderSalesDetail ID "PK" , OrderSaleslID  , ProductID , QTE Order , OrderDate , CustomerID)

My Question is there a possibility to integrate the SalesReason in my fact table without creating another table or having a duplicate in my fact table 
PS: 1 sale order can have many sales Reason 

and thanks!  

↧
↧

Version incompatibility for script task called from a Dataflow

August 28, 2017, 7:14 pm
$
0
0

I have a SSIS 2016 package I can run fine from Visual Studio 2015 but when I run from the catalogue or a step in a SQL Server Agent job it fails.

There are a number of error message the first of which is "The version of ..... is not compatible with this version of the DataFlow. [[The version or pipeline version or both for the specified component is higher than the current version. This package was probably created on a new version of DTS or the component than is installed on the current PC.]]

The next error says "The component metadata for .... could not be upgraded to the newer version of the component. The PerformUpgrade method failed."

Firstly, what exactly is causing the version problem - the dataflow, the component or some other software on the SSIS server? I'm not entirely sure from these error messages.

Secondly, could this be an issue around the .NET version on my Visual Studio development machine and the server running the package? It is 4.6.2 on the SSIS server (which is Windows Server 2012 R2) and 4.6.1 on my development machine (also 2012 R2)? To me that doesn't seem likely given the small difference in versions.

What other software should I be looking at around this?

Would be wonderful if somebody has the answer to this one as it's driving me nuts. :-)

Nick Ryan MIS Programmer Analyst, ANZ Bank

↧

SQL 2016 Tabular Model processing fails when running from Integration Services package

July 5, 2016, 12:21 am
$
0
0

When I run a SSIS package to do a full processing of a SQL 2016 Tabular Model deployed on SQL 2016 with compatability set to 1200, it fails and gives me the following error:

"[Analysis Services Execute DDL Task] Error: This command cannot be executed on database 'TabularDatabaseName' because it has been defined with StorageEngineUsed set to TabularMetadata. For databases in this mode, you must use Tabular APIs to administer the database."

How do I change the StorageEngineUsed to the Tabular API's so that the I can process it correctly?

When manually processing this model within SSAS, it works 100% correctly.

↧
Search

SSiS package to set users in their correct timezone

April 1, 2019, 12:53 pm
$
0
0
I would like to setup a scheduled SSIS Package to scan users and to correct their timezone. I have a table that contains user name and timezone. My question is this possible and where do I check if there timezone is incorrect, and if it is possible to change to correct timezone.
↧

SSIS package Just fails...not compatible with this version of the DataFlow?

January 9, 2013, 9:39 am
$
0
0

Executed as user: x ...0.00 for 64-bit  Copyright (C) Microsoft Corp 1984-2005. All rights reserved.    Started:  11:27:08 AM  Error: 2013-01-09 11:27:09.74     Code: 0xC0048020     Source: import file import file (DTS.Pipeline)     Description: The version of component "Flat File Source" (1) is not compatible with this version of the DataFlow.  End Error  Error: 2013-01-09 11:27:09.75     Code: 0xC0048020     Source: import file DTS.Pipeline     Description: The version of component "Flat File Source" (1) is not compatible with this version of the DataFlow.  End Error  Error: 2013-01-09 11:27:09.75     Code: 0xC004706C     Source: import file DTS.Pipeline     Description: Component "component "Data Conversion" (773)" could not be created and returned error code 0x80070005. Make sure that the component is registered correctly.  End Error  Error: 2013-01-09 11:27:09.75     Code: 0xC0048021     Source: import file Flat File Source [1]...  The package execution fa...  The step failed.

I am unsure what i am running into here.... Any ideas?

↧
↧

Replace the special character in a flat file using SSIS

April 5, 2019, 7:12 am
$
0
0

Hello,

I have a csv which has a column called "Value" and in it there are a combination of strings & special characters.

Eg:

Value

abc

xyz

"

pqr

'

Now, I want to replace the " & ' with nothing in the output file.I used the Derived column "Replace" function as 

Replace(Value," ' ","") & Replace(Value," \" ","") this doesn't seem to work

Please suggest some solution

↧

SSIS job/package fails with "Unexpected Termination" error

April 5, 2019, 7:27 am
$
0
0

Hello,

We have an SSIS package that used to run on a SQL 2008 server.  We upgraded it to run on a SQL 2016 server.  The package runs several stored procedures and writes the results to different tabs on an Excel spreadsheet.  The package runs fine locally but when we run either the package or the job on the 2016 server, it fails with an Unexpected Termination error.  It starts writing to the Excel file and stops at random points.  It doesn't appear to be after a certain number of records or a certain amount of time.

↧

SSISDB - Rebuild Indexes?

April 5, 2019, 8:14 am
$
0
0

Fellow SQLers,

I am on sql 2016. Is it best practice to rebuild the indexes on ssisdb?

MG

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>