Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

ssis and postgres

December 6, 2012, 10:19 am
$
0
0

Hi.  We see older posts like the one at http://social.msdn.microsoft.com/Forums/en-US/adodotnetdataproviders/thread/c5561541-2193-440c-8d4f-1a731e0a4798 and we know one moderator would like us to join a different community (with 55 followers) that has a related post but we're looking for more current, simpler and less time consuming ways of getting the latest thinking on ssis and postgres.

We still run std 2008.   We use sql server for most of our data stores.  We use Postgres 9.2 for one situation.  We're hoping to find a somewhat seamless Postgres connector (for input and output, maybe a plug in or new tool box item) for use in ssis.   We'd prefer that it be a good performer, unlike the one we tried out in a competitor's java based etl tool.   Actually we're not sure if the etl tool or the connector (or both) was to blame but we know the etl oem is Talend.   Not sure if they also provided the connector.

We'd also prefer to check it out on more than 100 rows before buying it.  We did experiment with one connector mentioned in the link above but they want $300 before we know if it satisfies our needs.   We want to see how available connectors perform loading 300 million rows before laying out a dime.

We'd prefer staying away from customizing our own connector.   Does anyone have any info that can be posted in this forum?  

↧
Search

The option to Set Breakpoints does not show up when I right-click any step

December 6, 2012, 9:48 am
$
0
0

Hi All,

I am working with a SSIS package and trying to set breakpoints to view the contents of package scoped variables. I am running SQL Server 2008 R2 on Windows Server 2008 R2 and developing in Visual Studio 2008.

When I right-click on an Execute SQL Task (or ANY step in the package) I see no menu pick to "Edit Breakpoints" or "Set Breakpoint". Can someone tell me why that option is missing?

TIA

↧
↧

How can I see what an SSIS package is doing at run time

December 5, 2012, 11:13 am
$
0
0

Hi all,

I have an SSIS package that connects to an SFTP site copies many XML files, then parses each file into SQL 2008 tables.  I have it scheduled thru SQL Agent, sometimes the job takes 5 minutes to complete and other times it can take over 2 hours.  the size of the files are within a 2MB of each other.

Is there a way to see what the SSIS package is doing during runtime?  I started a SQL trace, but there was nothing going on, I saw the start of the package and the end but nothing in between.  I added the SQL Logging within the SSIS package to log each process to event log, but again, there is only the start of the job and the end of the job. 

TIA,

Joe

↧

FTP Password Issue

December 6, 2012, 9:18 am
$
0
0

Hello

I am using FTP task in my SSIS Package. And every time, I make changes in this package. I have to enter password for FTP connection each time. In case, if i forget to put password after my change, FTP task fails in this package because it will not able to connect to FTP site.

Is something wrong with my package or am i missing something?

Please help

Thanks

↧

Importing mutiple query tables data into single sheet CSV file

November 26, 2012, 9:50 am
$
0
0

Hi ,

I have a situation  where we have  to move data from multiple sources data into one CSV file like

for eg:- first table data ( select * from table1 ) oledb source goes to csv destination file1

            second table data ( select * from table2 ) oledb source goes to csv destination file1

            third table data ( select * from table3 ) oledb source goes to csv destination file1

            Fourth table data ( select * from table4 ) oledb source goes to csv destination file1

            Fifth table data ( select * from table5 ) oledb source goes to csv destination file1

 and for each tab in the CSV file  i need name of the file name with date.

can anyone got in this situation. can you please guide me how to start?

thanks,

aravind-

↧
↧

SSIS package connect to a SharePoint secure link (https)

December 6, 2012, 11:41 am
$
0
0

Hi,  I need to access an excel file on a secured SharePoint library via SSIS package.  I'd like to either read it directly via Excel datasource or copy it to local drive via File System Task.  Either solution works for regular SharePoint site.  However, when it comes to a secured link (https://), it won't work anymore.  Can anybody give me some advice on how to deal with https?  URL:https://xxx.xx.com/sites/mysite/test.xls. the UNC path I used:\\xxx.xx.com\sites\mysite\test.xls. 

Thanks in advance.

↧

CASE statement in SSIS

December 6, 2012, 1:28 pm
$
0
0

Hi Friends

I have my data in flat file, Now I'm loading it into table.

Now I have a column in my flat file which has int values. If the value is '1' then I want it to put it in destination table as '1' other wise I want to put it as NULL.

How to do this. 

Royal Thomas

↧

SSIS - Sybase Destination - FastLoad

May 23, 2011, 2:56 am
$
0
0

I am trying to migrate data from Sql Server to Sybase, using SSIS (Sybase ASE OLEDB Provider).

When I set AccessMode to "OpenRowset Using FastLoad", get eh following  error:

********************

[OLE DB Destination [16]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E21.

An OLE DB record is available.  Source: "ASE OLE DB Provider"  Hresult: 0x80040E21  Description: "Errors occurred".

 

[OLE DB Destination [16]] Error: Failed to open a fastload rowset for "dbo.blahblah". Check that the object exists in the database.

********************
Any ideas? Does anyone know a way of fast loading data into Sybase from SQL Server?

↧
Search

Performance Issue while Inserting data in to DB2 from SQL Server using SSIS package

November 29, 2012, 12:03 pm
$
0
0

Hi,

I am hoping someone will have a solution for my problem. 

I am having a performance issue while inserting data from SQL Server to DB2 using SSIS package. To give an idea a table with only 3 columns with 50K rows is taking around 50+minutes to insert the data in DB2 Table. 

Here are the Details of my SSIS package/Environment:
SQL Server 2005 SP4 (Standard Edition) developed the package using BIDS on a machine with Standard Edition. This package will run on production server with Enterprise edition installed.

I am using IBM OLEDB Provider for Db2 to connect to DB2 database(IBMDADB2.Db2Copy is the provider name in the connection string).

For the Data flow destination. I am using OLEDB Destination task. 

We are researching Network traffic issue as I am typing this. Since our SQL Server is in one city and our DB2 server is in another city. But I highly doubt if thats the problem because I am not having this kind of performance issue when we get data from DB2 server and insert into SQL Server. Everything works fine in that case even though the servers are in 2 different locations.

After reading multiple posts some one had suggested that MS OLEDB Db2 provider is faster than IBM OLEDB Db2 provider. So I downloaded the Microsoft OLEDB Db2 provider version 3.0 from the below link.
http://blogs.msdn.com/b/sjackson/archive/2010/06/11/microsoft-ole-db-provider-for-db2-v3-0-is-available.aspx

But I had issue with the installation. Apparently its only supported for Developer and Enterprise edition sql server. My machine is standard edition. Its not practical to directly install this on our staging or production server which has Enterprise edition sql server installed. Although I was able to successfully install it on a machine which only has SQL Server 2005 Express. (Funny its available for free version of SQL Server but not the Standard).


Has any one come across similar issue and have found a solution. any other ideas on improving the performance?

Thanks
Ashwini

↧
↧

Difference between SSIS 2005 and SSIS 2008

May 9, 2010, 7:00 pm
$
0
0

Hi All,

Can I get lists of all major difference between ssis 2005 and ssis 2008.

Thanks in advance

↧

Failing the package

December 6, 2012, 2:18 pm
$
0
0

How can I fail the task and inturn the package(I am hoping the package will fail anyway if this task fails as this is the first task and all the remaining tasks are connected to this task by precedence constraint )

I have a ID field and status field.

I want to see if the max(id ) in the table has batch status value of '2'. If not I want to fail the package.

↧

Same Package - Different Machines - Incompatible !!

December 6, 2012, 10:11 pm
$
0
0

Dear All,

I have a package given to us by a third party as part of a software suite.  Works great in our production environment.

However, I want to run it in our dev environment, test it and see if we can improve performance somehow.  Anyway, I copy the package (.dtsx file) to our dev machine and it won't run!!  (I'm running the package via the command line utliity)

I get a whole heap of errors such as

   Description: The component is missing, not registered, not upgradeable, or missing required interfaces. The contact information for this component is "".
   Description: The component metadata for "component "TableDifference" (58)" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.
   Description: The managed pipeline component "SqlBi.SSIS.Components.TableDifference, TableDifference, Version=10.0.0.0, Culture=neutral, PublicKeyToken=729551dd7e47caaf" could not be loaded.  The exception was: Could not load file or assembly 'TableDifference, Version=10.0.0.0, Culture=neutral, PublicKeyToken=729551dd7e47caaf' or one of its dependencies. The system cannot find the file specified..


When I try to open the .dtsx file in VS 2008, i get a dialog box that says 'There were errors while the package was being loaded.  The package might be corrupted.  See the error list for details.'

There are 84 errors all up.  Most of them with something like '....component XXXcould not be upgraded to the newer version of the component.  The PerformUpgrade method failed.'

As far as I can see, both production machines have the same software  ..

Windows Server 2008 R2 Enterprise SP1 (64 Bit)

Microsoft SQL Server 2008 (SP2) - 10.0.4330.0 (X64)   Mar  1 2012 18:55:29   Copyright (c) 1988-2008 Microsoft Corporation  Developer Edition (64-bit) on Windows NT 6.1 <X64> (Build 7601: Service Pack 1)

Visual Studio 2008 Version 9.0.30729.1.SP  (.Net Framework 3.5 SP1)

I can open the file in VS 2008 on the production machine, no problem. 

Can anybody please explain why this is happening and what I can do to resolve the issue.

Many Many thanks in advance.

Basil


↧

Moving a file to a sharepoint folder using SSIS

December 4, 2012, 12:21 pm
$
0
0

I have been searching the forums trying to find a way of moving a file to a specific folder in Sharepoint.

I worked out that using the file system task you can plug in the UNC address of the folder as a destination.

Its not working - I was getting an 'access denied' error message - I'm guessing the sharepoint folder requires some sort of access?  How can I add this (I don't control the sharepoint folder).

[File System Task] Error: An error occurred with the following error message: "Access to the path is denied.".

I also read that you could use a web service so I tried the wsdl command and got this:


D:\Program Files\Microsoft Visual Studio 10.0\VC>wsdl http://spserveraddress/_vti_bin
/Copy.asmx
Microsoft (R) Web Services Description Language Utility
[Microsoft (R) .NET Framework, Version 4.0.30319.1]
Copyright (C) Microsoft Corporation. All rights reserved.
Error: There was an error processing 'http://spserveraddress/_vti_bin/Copy.asmx'.
  - There was an error downloading 'http://spserveraddress/_vti_bin/Copy.asmx'.
  - The request failed with HTTP status 401: Unauthorized.

If you would like more help, please type "wsdl /?".

So I'm at a bit of a loss here.  Any help greatly appreciated.


↧
↧

Memory Buffer Problems SSIS 2008 64bit

April 26, 2011, 4:58 pm
$
0
0

Hello -

First - this is configuration

SQL Server 2008 10.0.4000
Integration Services 10.0.4000
Windows Server 2008 64bit

HP DL 380
59 GB RAM
12 Cores
Lots of disk

So, I'm getting memory errors of all things during one particular Aggregation function.

The row length is 12 bytes.  (3 columns, 4 bytes each)

There are 200,000,000 rows,

Here is the error:

Error:  A buffer failed while allocating 196608 bytes
Error:  The system reports 17% memory load.  There are 60118847488 bytes of physical memory with 49395331072 bytes free.   There are 2147352576 bytes of virtual memory with 207876096 bytes free.  The paging file has 146016305152 bytes with 135324930048 bytes free

Error: The attempt to add a row to the data flow task failed with error code 0x8007000E

Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on "OLE DEB Source" (1) returned error code 0xC02020C4.  The component returned a failure code when the pipeline engine called the PrimeOutput().  The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be more errors messages posted before this with more information about the failure.

 

Just before these errors there is a long sequence of these:

Information: The buffer manager failed a memory allocation call for 65520 bytes, but was unable to swap out any buffers to relieve memory pressure.  71090 Buffers were considered and 71090 were locked. Either not enough memory is available to the pipeline because not enough installed, other processes were using it, or too many buffers are locked.

 

So, I think it's safe to say that there IS enough memory installed.  So this leaves "other processes were using it" or "too many buffers are locked".  Either way, I am at a loss on how to trouble shoot this.


Any help is greatly appreciated.

Joe

 

 

↧

Fuzzy Grouping Transformation - is it possible to specify the source/target data sets being compared.

December 3, 2012, 1:13 pm
$
0
0
I'm working on a project for a client that uses the Fuzzy Grouping Transformation component to identify potential duplicate records in the system. It appears that you can define a query to use as the source of records look through... but as far as I can tell, the FGT uses that query to then look through the rest of the database and I need to restrict that to look at just a subset of the data in the database... sometimes that subset will be the whole database, other times it won't be (more often than not it'll be a true subset) ... Does anyone have any experience using the FGT? Is there a way to limit what the logic looks at? or do I need to look at everything first, then remove from the results data that isn't in the subset?
↧
Search

Name 'dts' is not declared

December 4, 2012, 12:06 pm
$
0
0

I've inherited a SSIS 2005 task that I've moved into SSIS 2008. In the task besides the ScriptMain.vb there is a Reference.vb. The reference.vb was modified to the following: 

Me.Url = dts.Variables("ListUrl").Value.ToString() 'Global.LotNumberExtract.My.MySettings.Default.LotNumberExtract_wsLotNumbers_Lists

It is part of the following:

 Public Sub New()
            MyBase.New()
            Me.Url = dts.Variables("ListUrl").Value.ToString() 'Global.LotNumberExtract.My.MySettings.Default.LotNumberExtract_wsLotNumbers_Lists
            If (Me.IsLocalFileSystemWebService(Me.Url) = True) Then
                Me.UseDefaultCredentials = True
                Me.useDefaultCredentialsSetExplicitly = False
            Else
                Me.useDefaultCredentialsSetExplicitly = True
            End If
        End Sub

Since the DTS object is not a static global object like in SSIS 2005, it is throwing the "Name 'dts' is not declared" error.

How can I correct this so it works?

↧

Passing Filename Arguments in Execute Task Process

November 30, 2012, 12:52 am
$
0
0

Hi All,

Having an issue trying to pass a filename as an argument to a batch file from a Execute Task Process.  The filename passes fine if the filename contains no spaces but as soon as a space is invloved it truncates the filename.

EG:

Filename1.txt would pass fine and be echoed out to a test file by my simple batch below:

echo %1 > file.txt

However, if my filename is like "Filename1 with spaces.txt" only Filename1 is getting output.

Any ideas how to cater for the spaces?

Many Thanks,

Slash.

↧
↧

[SharePoint List Source [1]] Error: System.ServiceModel.ProtocolException:

December 7, 2012, 3:34 am
$
0
0

I am using SSIS sharepoint list adapter to get the data. I am getting the below error...Please help me out...

[SharePoint List Source [1]] Error: System.ServiceModel.ProtocolException: The content type text/html; charset=utf-8 of the response message does not match the content type of the binding (text/xml; charset=utf-8). If using a custom encoder, be sure that the IsContentTypeSupported method is implemented properly. The first 1024 bytes of the response were: '<HTML dir="ltr">
<HEAD><meta name="GENERATOR" content="Microsoft SharePoint" /><meta name="progid" content="SharePoint.WebPartPage.Document" /><meta HTTP-EQUIV="Content-Type" content="text/html; charset=utf-8" /><meta HTTP-EQUIV="Expires" content="0" /><meta name="ROBOTS" content="NOHTMLINDEX" /><title>
 
 Error

</title><link rel="stylesheet" type="text/css" href="/_layouts/1033/styles/core.css?rev=5msmprmeONfN6lJ3wtbAlA%3D%3D"/>
<script type="text/javascript" language="javascript" src="/_layouts/1033/init.js?rev=SKi7C%2FTrsh1U%2FCnIwkB9Ag%3D%3D"></script>
<script type="text/javascript" language="javascript" src="/_layouts/1033/core.js?rev=mHKsOQ0iU3Q5jdm9OZNDdg%3D%3D"></script>

↧

How to deal with comma values in CSV file into one column while exporting to Sqlserver table using ssis?

December 6, 2012, 9:28 pm
$
0
0

Hi All,

This is very urgent requirement.

We have CSV file which contains the data as LIC Finance Limited,Inc.

Here we facing the issue like Inc is moving to next column and that column value goes to next column and so on to all the other columns.

So,Can any one guide me how to fix this issue ASAP.

Earlier response is appreciable.

Regards,

Sudha

sudha

↧

how to used dynamic excel file in SSIS 2005

December 7, 2012, 2:25 am
$
0
0

Hello everyone,

I have excel file whose column will change everytime.i want to copy the data of excel file into SQL table.

i am not getting how to create package for this scenario.

If i am using sql source component then its giving me error metatdata its bcoz everytime excel file colums are changing.

so for this i thought using script component. as i am new to script component i m not getting how to do it.

i am asking this question from last three days but not getting any  proper solution.

Is this possible using BIDS 2005.

kindly suggest or provide link or code in vb.net

↧
Viewing all 24688 articles
Browse latest View live
Search