I am trying to copy records from a table (table A) from one system (SQL Server) to a table (table B) in another system (Oracle).  Table B does not allow duplicates, has 20 fields, with two indices (one with 8 fields in the index and the other with 9 fields in the index; 8 of the fields in both indices are the same). One of the fields is named SequenceNumber which is included in both indices.

For Table A I use a query that uses the RowNumber function to increment the SequenceNumber field if there are duplicates.  For Table B I have a query that provides only the fields in the smaller index (8 fields) which I group together and use the MAX function on the SequenceNumber field so I will only have the records which have the greatest SequenceNumber for that index.  Both tables are sorted by the index fields.

In my SSIS package

1. I first create a temp table which matches the Oracle table using an Execute SQL task. 
2. Then I have a Data Flow task where i use a Merge Join transform to left outer join Table A with Table B based on the smaller index (8 field).  Then using a Conditional Split transform I separate New Records from Existing Records.  I am checking whether SequenceNumber is null from Table B to tell which type of record it is (New or Existing).  The New Records I connect to a Merge transform but the Existing Records I connect to a Derived Column transform to increment the SequenceNumber field depending on which SequenceNumber is greater from Table A or Table B.  If Table B SequenceNumber is greater than Table A SequenceNumber I add them together and make that the new SequenceNumber.  If not I add 1 to Table A SequenceNumber.  Then I connect it to the Merge transform.  This is inserted into the temp table.
3. Then I connect this Data Flow task to another Data Flow Task where I copy the temp table records into Table B.
4. I use another Data Flow task to update a Posted field in Table A to mark the record as posted.
5. Finally I use an Execute SQL task to drop the temp table.

I am finding it is not increment the SequenceNumber so that it does not allow the records to be inserted.

Where am I going wrong in my logic?

Thanks for any help that can be provided.

Fred

Hi,

File System Task - it is copied one excel file from one folder to another folder.

both source and destination are same server but different folder.

got the error - Access path "Destination file path" is denied.

Regards,

Abdul Khadir

Hi ya, I'm using my own Windows user. Such user belong to Admin group. What's more, my Integration Services is started using this same user.

What's happening here? 

Any advice or help would be very appreciated,

Version:

Microsoft SQL Server Management Studio11.0.3153.0

Enric

I have a SSIS package with a script task, I get the following error when i try to run it in my local system. It works fine for my collegues as well as in production. However, I am not able to run it locally, to test. I keep a debug point in the main method, but it is never reached, I get the error before it goes to main method.

I am using VS 2010, .Net framework 4.5.

The script task does compile. I get the following messages SSIS package "..\Test.dtsx" starting. Error: 0x1 at Test: Exception has been thrown by the target of an invocation. Task failed: Test SSIS package "..\Test.dtsx" finished: Success. The program '[2552] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).

Hi,

i have loaded the csv file to table using ssis, it is working file but record is mismatch.

source data :

col1 col2 col3 col4

aa, bb, this is sample,data for the details, this is 

column delimiter - comma

col3 - record contains comma. so col3 record after the comma is added with the as col4

i need the col3 record with comma.

so what is the column delimiter will use ?

Regards,

abdul Khadir

Hi friends,

I have a small requirement here.

we have 10 child packages and one parent package. Calling the child packages from parent.

irrespective of that child package fails of success , parent package should pick the next package..

EX.. Parent package calling the 5th child package, due to some error it got failed. Now parent package shouldn't fail it will skip the 5th child package and  call the next 6th child package ...

if the child package success parent should have to call the next one..

if the child package fails parent package shouldn't fail and it calls the next child package..

How we can implement in this in ssis..

Hello,

Can anybody please let me know which is the best option to implement incremental loading (Type 2), History should be kept.

I know there are many approaches

• SCD transformation, its not good for performance
• MERGE JOIN and Lookup and updaing using execute SQL Task
• Using CCD

I am bit confused which is the best way to implement. Your suggestions are appreciated and it would be great if you have any link which explains the same.

Thanks!

Shamsuddeen

Hi All,

Can anyone please help out with the following requirement?

There is a XML file data which I need to pull it to database. Please see the attached XML file.

Question: When I am trying to pull the root level data (i.e. CreationDateTime and FileNumber) from XML source in the dataflow, the respective tag is not seen in the input. Can anyone help me how can I get those values in my input to pull the data to Database?

Thanks & Regards,
Sri

Hi All,

I have a SSIS 2012 package deployed to the Integration Services Catalog. I believe the package involves a double-hop authentication.  If I right click the package in the Integration Services Catalog (ISC) through SSMS on the server and select execute it fails with the 'Access Denied' error - Can't open the file in the shared location. However, when I try to call the package using a SQL Server agent job, it works perfectly fine.  

I understand that the Agent uses the service account while executing the package and when I manually execute the package through SSMS it uses my credentials. However, my credentials has full access to both the database server and the shared locations. In fact, if I execute the package within SQL Server Data tools using my credentials, the package runs fine without any issues.

Package Descriptions: The package simply copies a text file from one location to another.  I also tried creating another package in which data is extracted from one of the SQL server tables and is written to a text file and I encounter the same error.

My double hop scenario: (the one which FAILS)

Shared location in Server A

Catalog database is hosted on Server B

RDP to Server A, connect to SQL Server B using SSMS and execute the package from the Integration services catalog. – Fails with ‘Access Denied error’

Another scenario: (the one which SUCCEEDS)

Shared location in Server A

Catalog database is hosted on Server A

RDP to Server A, connect to SQL Server A using SSMS and execute the package from the Integration services catalog. – Bingo! Works fine.

Steps taken: 

* Enabled Kerberos authentication. Followed article: http://www.sqlscientist.com/2014/01/setup-kerberos-authentication-for-sql.html . However, it has no bearing as the package call through the SQL agent job works perfectly fine even without enabling Kerberos.

* The shared location has full control to everyone.

Any thoughts or insights would be really helpful? I have read articles, which suggests that SQL Server Integration Services does not support the delegation of credentials, sometimes referred to as a double hop. However, I fail to understand how it succeeds when I try to call the package through SQL server agent?

It makes me think if I should simply execute the package by doing an RDP to the server which also hosts the catalog database for manual execution of the packages.

Thanks for your time.

Hi guys,

I was trying to load data from a source database that has over 500Million records into a target database using the a Script Task and SqlBulkCopy.

I used the following code;

On running the SSIS package, it executes successfully but no record was populate into the target db from the source db.

Please can someone help me to check with could be the cause of it. 

me

I am trying to perform an Inner Join in a lookup task in SSIS. This is the schema of the original query I need to execute:

UPDATE SQLSERVER
SET
SQLSERVER.COLUMN1 = ORACLETABLE_2.VAL
FROM
SQLSERVER INNER JOIN (ORACLETABLE_1 INNER JOIN ORACLETABLE_2 ON
ORACLETABLE_1.ID = ORACLETABLE_2.ID) ON
SQLSERVER.COLUMN3 = ORACLETABLE_1.COLUMN3
WHERE
(((SQLSERVER.COLUMN1)='A' Or (SQLSERVER.COLUMN1) Is Null) AND
((SQLSERVER.COLUMN2)='R') AND
((ORACLETABLE_2.CAR)=34) AND
((ORACLETABLE_2.VAL) Is Not Null));

As you can see I have 3 different sources, one in a sql server and the other 2 in the same oracle database.

This is the work flow that I have created for this purpose:

OLE DB SOURCER -> DATA CONVERSION -> LOOKUP -> OLE DB COMMAND

For the lookup table I've got this query, for the part of Oracle:

WITH FS AS
(
SELECT ORACLETABLE_2.VAl
FROM ORACLETABLE_1 INNER JOIN ORACLETABLE_2 ON
ORACLETABLE_1.ID = ORACLETABLE_2.ID
WHERE
((ORACLETABLE_2.CAR)=34) AND
((ORACLETABLE_2.VAL) Is Not Null
)
SELECT *
FROM FS;

But when I try to prove this query I've got this error message:

So, at this point I just can ask if someone knows a possible solution for this. I have decided to use a CTE because in different forums I have seen this as a possible solution, but I am not so sure if I am using it in the right way, this is the first time that I use a Lookup task. If there is a BETTER way to create this query I will appreciate any advice or if you need more information about the workflow please ask.

- I have executed the CTE using Oracle SQL developer without any problem.

- If I choose the "Use a table of view" option on the connection section of the lookup task I can see without any problem a single table. If I choose the "Use result of an SQL query" opticon and I work with only one table I do not have any problem with the preview option, but when I try to create the query using "Build query" I've got this error message: "ORA-01005: null password given; logon denied"

- I am using MVS 2010

Hi Gurus, 

I have created a package and the flow is like below. 

1. Copy the file in folder flatfiles, Rename and move it archive folder

2. Create a file in DIFFERENT directory

3. FTP the file from DIFFERENT directory to flatfiles directory. 



When running the step 3 individually, the package is running with no issues. But, when running all the three steps together my package fails at step3 saying: "The process cannot access the file because it is being used by another process. "

Seems like to me that one of my task or operation is holding on that file. I'm not sure what it is. Can you please help me troubleshoot this issue. 

Thanks

Hi,

I have a problem with a SSIS package. I am using SQL Server 2008 R2. 

• I want to join two different OLE DB sources
• The columns that I want to join have the same datatype and same length 
• I tried to sort the sources in two ways. Via ORDER BY in the source query setting isSorted in the advanced properties and the other way via with the Sort Transformation (which is no option at the end, because it is definitly to slow) in the SSIS Package 

Depending on which way I sort the data, I get some NULL values in my result table. But I have checked the original data, and there are no NULL values...

So I have no idea about

1. why I get NULL values in general with the Merge Join transform

2. why it depends on the sort which values are NULL...

When I do the same JOIN as TSQL command, there are no NULL values. 

I hope any of you can give me some new idea about that :)

Thanks in advance

I've got eight 2005 SSIS packages that were accessing SQL Server 2018 for bulk data and a SQL Server 2000 for some data also.

Both connection strings use the Provider=SQLNCLI.1.  These packages all are working daily without trouble even though the DB03 box had been upgraded to SQL2008 but I never changed that Provider to the SQLNCLI10. 

SQL Server 2000 to SQL Server 2012 and I've converted the VS2005 projects to VS2008.  It converted the SQL Server 2008 connection string provider to SQLNCLI10 but since our SQL Server 2000 box is still in place it doesn't convert that Provider.  I thought well I'll just manually change the Provider to SQLNCLI10, recompile and then we'd have to DTSX to quickly swap on the new SQL Server 2012 box.  But on recompile now it changes it back to the SQLNCLI.1 Provider.

Question:  Must I wait til the Server is actually upgraded to be able to change and recompile the DTSX with the new Provider?  How is this usually handled gracefully?

TIA

Harry

I have created a XML configuration file which has Initial catalog and server name in it selected under connection manager, when I try to change the server name it doesn't reflect on my package. Am I doing it right? 

Can someone help me how to change the server name through configuration files ?

Thanks,

sree

Hi, 

• VS2008 
• SQL 2008 R2 

Bit of a newbie with SSIS and script tasks.  I have a simple package that defines two package variables. 

I'm trying to read these in the Main() method of the C# script task. 

However this gives the following error. 

System.Runtime.InteropServices.COMException (0xC0010009): The element cannot be found in a collection.

This would imply that the variable does not exist in the variables collection, however it does.  I've tried both Package:: and User::, same result. 

Googling this seems to show that square brackets are correct for C#. 

Any ideas? 

Hi all,

I'm getting the above error when trying to execute a custom lookup.  What I'm trying to do is lookup the minimum promotional price between date ranges.  The SQL executes fine in SSMS

SELECT  refTable.PRICEFROM(SELECT    MIN(PRICE)AS PRICE ,
                    NAME ,
                    ITEMRELATION ,
                    FROMDATE ,
                    TODATEFROM[dbo].[AllCustPrices]GROUPBY  NAME ,
                    ITEMRELATION ,
                    FROMDATE ,
                    TODATE)[refTable]WHERE[refTable].[NAME]=?AND[refTable].[ITEMRELATION]=?AND?BETWEEN[refTable].[FROMDATE]AND[refTable].[TODATE]

but errors in SSIS?

A third-party vendor is sending us a pipe-delimited flat file with about  500 fields. The largest field is 250 bytes. For a given row the first byte of data starts in column 1 and the last one, possibly near column 21,000. Several of the fields could be empty -- but sometimes not. I need to import this data into a "staging" data (sql 2008) so we can extract the data of value to us. Given the sql server max data row size of 8060 I need a schema design recommendation for the staging table. (A table where every column was varchar(max) seems absurd but I don't have any clear idea how large a row in the data file can be. My concern is that it may well exceed 8060)

TIA,

edm2

Hi,

I have a xmla file which I'm trying to put it in a variable. While evaluating the expression I'm getting the error as 

 The expression might contain an invalid token, an incomplete token, or an invalid element. It might not be well-formed, or might be missing part of a required element such as a parenthesis.

Here is my xmla expression 

"Select '<Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine"><ErrorConfiguration xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200" xmlns:ddl300="http://schemas.microsoft.com/analysisservices/2011/engine/300" xmlns:ddl300_300="http://schemas.microsoft.com/analysisservices/2011/engine/300/300" xmlns:ddl400="http://schemas.microsoft.com/analysisservices/2012/engine/400" xmlns:ddl400_400="http://schemas.microsoft.com/analysisservices/2012/engine/400/400"><KeyNotFound>IgnoreError</KeyNotFound></ErrorConfiguration><Parallel><Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200" xmlns:ddl300="http://schemas.microsoft.com/analysisservices/2011/engine/300" xmlns:ddl300_300="http://schemas.microsoft.com/analysisservices/2011/engine/300/300" xmlns:ddl400="http://schemas.microsoft.com/analysisservices/2012/engine/400" xmlns:ddl400_400="http://schemas.microsoft.com/analysisservices/2012/engine/400/400"><Object><DatabaseID>IIS_Version2</DatabaseID><DimensionID>Application</DimensionID></Object><Type>ProcessAdd</Type><DataSource xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200" xmlns:ddl300="http://schemas.microsoft.com/analysisservices/2011/engine/300" xmlns:ddl300_300="http://schemas.microsoft.com/analysisservices/2011/engine/300/300" xmlns:ddl400="http://schemas.microsoft.com/analysisservices/2012/engine/400" xmlns:ddl400_400="http://schemas.microsoft.com/analysisservices/2012/engine/400/400" xmlns:dwd="http://schemas.microsoft.com/DataWarehouse/Designer/1.0" xsi:type="RelationalDataSource" dwd:design-time-name="1a3cb292-9bce-4c59-a182-177d6b3506ff" xmlns="http://schemas.microsoft.com/analysisservices/2003/engine"><ID>IISDW</ID><Name>IISDW</Name><ConnectionString>Provider=SQLNCLI11.1;Data Source=CO1MSFTSQLHKT02;Integrated Security=SSPI;Initial Catalog=IISDW</ConnectionString><!--<ImpersonationInfo><ImpersonationMode>ImpersonateAccount</ImpersonationMode><Account>phx\v-chveng</Account><ImpersonationInfoSecurity>passwordremoved</ImpersonationInfoSecurity></ImpersonationInfo><Timeout>PT0S</Timeout>--></DataSource><DataSourceView xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ddl2="http://schemas.microsoft.com/analysisservices/2003/engine/2" xmlns:ddl2_2="http://schemas.microsoft.com/analysisservices/2003/engine/2/2" xmlns:ddl100_100="http://schemas.microsoft.com/analysisservices/2008/engine/100/100" xmlns:ddl200="http://schemas.microsoft.com/analysisservices/2010/engine/200" xmlns:ddl200_200="http://schemas.microsoft.com/analysisservices/2010/engine/200/200" xmlns:ddl300="http://schemas.microsoft.com/analysisservices/2011/engine/300" xmlns:ddl300_300="http://schemas.microsoft.com/analysisservices/2011/engine/300/300" xmlns:ddl400="http://schemas.microsoft.com/analysisservices/2012/engine/400" xmlns:ddl400_400="http://schemas.microsoft.com/analysisservices/2012/engine/400/400" xmlns:dwd="http://schemas.microsoft.com/DataWarehouse/Designer/1.0" dwd:design-time-name="b0b61205-c64d-4e34-afae-6d4d48b93fb3" xmlns="http://schemas.microsoft.com/analysisservices/2003/engine"><ID>IISDW</ID><Name>IISDW</Name><DataSourceID>IISDW</DataSourceID><Schema><xs:schema id="IISDW" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:msprop="urn:schemas-microsoft-com:xml-msprop"><xs:element name="IISDW" msdata:IsDataSet="true" msdata:UseCurrentLocale="true" msprop:design-time-name="a6676059-1f9d-4cf2-8dcf-40b8f31e3332"><xs:complexType><xs:choice minOccurs="0" maxOccurs="unbounded"><xs:element name="Application" msprop:design-time-name="6022ab52-0da7-49db-a3d3-b38704d0559b" msprop:IsLogical="True" msprop:FriendlyName="Application" msprop:DbTableName="Application" msprop:TableType="View" msprop:Description="" msprop:QueryDefinition="SELECT        Application, DATEADD([hour], DATEDIFF([hour], 0, [Timestamp]), 0) AS [Timestamp], ServerName, CAST(ServerName AS char(3)) AS DataCenter, CS_URI_Stem, CAST(HashBytes("'MD5'",Application + ServerName + CS_URI_Stem + CAST(DATEADD([hour], DATEDIFF([hour], 0, [Timestamp]), 0) AS varchar(24))) AS uniqueidentifier) AS Server_URI_Identity&#xD;&#xA;FROM            IIS_6 AS a LEFT OUTER JOIN&#xD;&#xA;Dimension_Pointer AS b ON a.Application = b.Application&#xD;&#xA;WHERE        (b.ProcessedFlag = 0) AND (a.Application = "+(DT_WSTR, 100) @[User::Application]+")&#xD;&#xA;GROUP BY Application, DATEADD([hour], DATEDIFF([hour], 0, [Timestamp]), 0), ServerName, CS_URI_Stem" msprop:QueryBuilder="SpecificQueryBuilder"><xs:complexType><xs:sequence><xs:element name="Application" msprop:design-time-name="8b794def-ecf7-48c5-8f41-8829e00cf235" msprop:DbColumnName="Application" msprop:FriendlyName="Application" minOccurs="0"><xs:simpleType><xs:restriction base="xs:string"><xs:maxLength value="255" /></xs:restriction></xs:simpleType></xs:element><xs:element name="Timestamp" msdata:ReadOnly="true" msprop:design-time-name="00274b1e-7459-4197-8a22-f2f5de0c4af1" msprop:DbColumnName="Timestamp" msprop:FriendlyName="Timestamp" type="xs:dateTime" /><xs:element name="ServerName" msprop:design-time-name="793900a3-304a-45a4-8f3f-87352b3fb70f" msprop:DbColumnName="ServerName" msprop:FriendlyName="ServerName" minOccurs="0"><xs:simpleType><xs:restriction base="xs:string"><xs:maxLength value="255" /></xs:restriction></xs:simpleType></xs:element><xs:element name="DataCenter" msdata:ReadOnly="true" msprop:design-time-name="6cfa5bef-833b-4ac1-bfde-1b7a89516140" msprop:DbColumnName="DataCenter" msprop:FriendlyName="DataCenter" minOccurs="0"><xs:simpleType><xs:restriction base="xs:string"><xs:maxLength value="3" /></xs:restriction></xs:simpleType></xs:element><xs:element name="CS_URI_Stem" msprop:design-time-name="4cee9248-c545-4ff1-8887-c553e2932403" msprop:DbColumnName="CS_URI_Stem" msprop:FriendlyName="CS_URI_Stem" minOccurs="0"><xs:simpleType><xs:restriction base="xs:string"><xs:maxLength value="4000" /></xs:restriction></xs:simpleType></xs:element><xs:element name="Server_URI_Identity" msdata:DataType="System.Guid, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" msdata:ReadOnly="true" msprop:design-time-name="6666d716-ac3c-4a5a-a331-9398a6bbd8cc" msprop:DbColumnName="Server_URI_Identity" msprop:FriendlyName="Server_URI_Identity" type="xs:string" /></xs:sequence></xs:complexType></xs:element></xs:choice></xs:complexType><xs:unique name="Constraint1" msprop:IsLogical="True" msdata:PrimaryKey="true"><xs:selector xpath=".//Application" /><xs:field xpath="Server_URI_Identity" /><xs:field xpath="Timestamp" /></xs:unique></xs:element></xs:schema><IISDW xmlns="" /></Schema></DataSourceView><WriteBackTableCreation>UseExisting</WriteBackTableCreation></Process></Parallel></Batch>' as ProcesAdd