Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

SSIS: Dynamic destination table creation with dynamic input columns

March 11, 2015, 9:56 am
$
0
0

Hi,

I have several CSV files that need to be added to an SQL database, each having its own separate table with the same name as the CSV file. The files do not all have the same columns.

To summarize, what I need to do is:

  1. Get the file name (This gives the table name for my DDL script)
  2. Get column information from file (The Flat File Connection Manager gives both i.e. the column names as well as suggested types for each of the columns. I want to use these in my dynamically generated DDL script)
  3. Create the table using the generated script
  4. Load all data from file to destination table

My main concern is STEP 2. How do I get the column names and their types (as suggested by SSIS) to generate a dynamic DDL for each file that comes in? A script task? If yes, then how exactly?

There is something similar to my concern mentioned here but I'm unable to drive my way towards a solution with what has been suggested there.

TIA!




↧
Search

File system task error on 'move file'

March 11, 2015, 9:50 am
$
0
0

I have a file system task where I'm trying to move a file from a folder to a subfolder. I get the following message:

-----------------------------------------------------------------------------------------------------------------------------------

SSIS package "MOVE_FILES.dtsx" starting.
Error: 0xC002F304 at File System Task, File System Task: An error occurred with the following error
message: "Could not find a part of the path.".
Warning: 0x80019002 at File System Task: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The
Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting
in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount.
Change the MaximumErrorCount or fix the errors.
Task failed: File System Task
Warning: 0x80019002 at MOVE_FILES: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The Execution
method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in
failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount.
Change the MaximumErrorCount or fix the errors.
SSIS package "MOVE_FILES.dtsx" finished: Failure.

-----------------------------------------------------------------------------------------------------------------------------------

I've cut and pasted the paths from the destination and source connections into the computer folder path and both are valid.   Any suggestions? 

↧
↧

SSIS Error : DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "OLEDB12dot0" failed with error code 0xC0209302.

March 11, 2015, 10:23 am
$
0
0

SSIS package is running successfully from command prompt - C:\Program
Files\Microsoft SQL Server\100\DTS\Binn.

It is also running successfully when executed through SQL Agent Job.

However, package fails to execute with above mentioned error when we connect Integration Services through SSMS.

SQL is 2008 R2 - SP3

↧

ETL with incrementing sequence number to prevent duplicates

March 10, 2015, 12:11 pm
$
0
0

I am trying to copy records from a table (table A) from one system (SQL Server) to a table (table B) in another system (Oracle).  Table B does not allow duplicates, has 20 fields, with two indices (one with 9 fields in the index and the other with 10 fields in the index; 9 of the fields in both indices are the same). One of the fields is named SequenceNumber which is included in both indices.

For Table A I use a query that uses the RowNumber function to increment the SequenceNumber field if there are duplicates.  For Table B I have a query that provides only the fields in the smaller index (9 fields) which I group together and use the MAX function on the SequenceNumber field so I will only have the records which have the greatest SequenceNumber for that index.  Both tables are sorted by the index fields.

In my SSIS package

  1. I first create a temp table which matches the Oracle table using an Execute SQL task. 
  2. Then I have a Data Flow task where i use a Merge Join transform to left outer join Table A with Table B based on the smaller index (9 field).  Then using a Conditional Split transform I separate New Records from Existing Records.  I am checking whether SequenceNumber is null from Table B to tell which type of record it is (New or Existing).  The New Records I connect to a Merge transform but the Existing Records I connect to a Derived Column transform to increment the SequenceNumber field depending on which SequenceNumber is greater from Table A or Table B.  If Table B SequenceNumber is greater than Table A SequenceNumber I add them together and make that the new SequenceNumber.  If not I add 1 to Table A SequenceNumber.  Then I connect it to the Merge transform.  This is inserted into the temp table.
  3. Then I connect this Data Flow task to another Data Flow Task where I copy the temp table records into Table B.
  4. I use another Data Flow task to update a Posted field in Table A to mark the record as posted.
  5. Finally I use an Execute SQL task to drop the temp table.

I am finding it is not increment the SequenceNumber so that it does not allow the records to be inserted.

Where am I going wrong in my logic?

Thanks for any help that can be provided.

Fred

↧

Orchestrating a Printing Process with SSIS

March 10, 2015, 2:37 pm
$
0
0

All, 

I have a solution designed in a combination of SSIS and SSRS where SSRS is used as a platform to render warehouse Pick Tickets into a PDF file and among other things, SSIS performs the orchestration being initiated once an order is submitted through the company's web site, rendering the order to a PDF and sending to the correct tray, on the correct printer in the warehouse. We have multiple printers. 

I'm looking for a new way to communicate between the rendered file and the printer. Today, I use a tool called Total PDF Print, but the tool hangs and throws a lot of exceptions leading us to believe this is not an enterprise ready tool. 

We need a tool that will be able to send a PDF file to an specific tray in a printer. 

Can you guys recommend anything to us?

↧
↧

Cannot change SQL command text in Data Flow Task

November 21, 2012, 8:24 am
$
0
0

I have an SSIS package that extracts data from Teradata into a SQL Server DB.

I'm using SQL Server 2008 R2 EE x64, and Visual Studio 2008 PE (BIDS) supplied with it, accessing Teradata v13.

In the Integration Services project, I have a Data Flow Task, it has an ADO .net source which has Data access mode set to “SQL Command”.

This worked for a while when I initially entered a SQL statement to select data.

But, when I change the existing SQL Command text and save the package, the changes are lost.
It keeps going back to the original SQL Statement.
It is currently “select col1, col2, … col10 from view1”.
When I change it to anything else, like “select col5 from view1”, and save, and then double click the ADO NET source again, I find that the SQL command text is still the old one. It goes back to the previous statement.

I’ve tried other statements like “exec macro” for Teradata, etc. but the same thing keeps happening - changes are not saved.

Does anyone have any ideas on this, or have you seen this before?


↧

For Each Loop: Enumerator Configuration In the For each loop Editor Help needed

March 11, 2015, 2:05 pm
$
0
0

Hi All,

I have a package that runs and updates a table with everytime someone adds a flat file in the source folder. One of the requirement is to upload files with the following

ACA Flat files will have a naming convention of “ACA_ccyy_<julian date>”  : How can I add this?

Under Files this is how I have setup the qualifies "ACA_*.*"

Is there anyway to pass the entire naming convention in the file section? If so how can I ?

Please help?

FM

↧

Ask about an Error in SQL 2014

March 9, 2015, 8:54 pm
$
0
0

Hi all,

I got an error in SSIS package when I upgraded from Sql server 2008 to 2014 as below

I have a format file with "Text qualifier" is "||" and "Column dilimiter" is "|" as below data example and it run well in SSIS 2008. But I got an error after I upgraded to Sql server to 2014 because SSIS 2014 did not know this kind of format. Is this a bug for new Sql 2014?

As I know, some first releases of Sql server 2008 also had this error and then it's fixed after that. Now, it happen again in new release of Sql server 2014. Is there any fix for that?

Error: "[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on MediaType returned error code 0xC0202091.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure."

Example data:

-----------------------------------------

||MediaTypeID|||||MediaTypeCode||

||xxx|||||yyy||

||ccc|||||zzz||

------------------------------------------

Thanks all.


↧
Search

Using Script Task to rename an excel sheet.

March 9, 2015, 8:15 am
$
0
0

I'm trying to use an SSIS Script Task to rename an excel sheet.

I've found several code samples on the net but most don't compile correctly and my knowledge of VB.net or C# isin't strong enough to fix the issues. I suspect they work but not for SSIS 2012 possibly.

Found the sample below which shows no errors at design time, but fails with an Invocation error so I'm unable to even trap what the error is to help in resolving the issue.

Does anyone have any sample code that they KNOW WORKS in SSIS 2012 or can see what the problem is ?

Dim oMissing As Object = System.Reflection.Missing.Value
        Dim xl As New Microsoft.Office.Interop.Excel.ApplicationClass()
        Dim xlBook As Microsoft.Office.Interop.Excel.Workbook
        Dim xlSheet As Microsoft.Office.Interop.Excel.Worksheet
        Dim laPath As String = "C:\Test.xlsx"

        Try
            xlBook = DirectCast(xl.Workbooks.Open(laPath, oMissing, oMissing, oMissing, oMissing, oMissing, _
            oMissing, oMissing, oMissing, oMissing, oMissing, oMissing, _
            oMissing, oMissing, oMissing), Workbook)

            xlSheet = DirectCast(xlBook.Worksheets.Item(1), Worksheet)
            xlSheet.Name = "Sheet1"
            xlBook.Save()
            xl.Application.Workbooks.Close()

            Dts.TaskResult = ScriptResults.Success

        Catch ex As Exception

            MessageBox.Show(ex.Message.ToString(), "Error", MessageBoxButtons.OK)
            Dts.Events.FireError(1, ex.TargetSite.ToString(), ex.Message.ToString(), "", 0)
            Dts.TaskResult = ScriptResults.Failure

        End Try

Thanks.

↧
↧

VS 2012 doesn't recognize SSDT installed

March 9, 2015, 11:11 am
$
0
0

When I try to open/edit SSIS packages in VS 2012 shell, I am getting the following message:

However SSDT is installed and shows up as an installed product:

I have uninstalled and reinstalled SSDT and SSDT-BI on this machine, but that doesn't seem to help. Any suggestions would be appreciated.

Thanks.

Tim Harding

↧

Loading Multiple csv files through SSIS pkg and the Impact when the ETL Server is down.

March 12, 2015, 3:59 am
$
0
0

The requirement is to load the multiple csv files in the FIFO order. For eg: one of my file has 1,00,000 records and the pkg fails after 30,000 record load, the my pkg has to start from 30,001th record and identifying the csv file which failed.

Please advice.

↧

Best way to Transform one XML to another XML using SSIS

March 12, 2015, 5:46 am
$
0
0

I am using Altova Mapforce tool to transform one XML into another XML, but now we are planning to use any other best way of using XSLT transformation. Could you guys shed me some light on different approaches and which one would be best based on time and cost constraint.

In Altova Map force Tool, due to below reasons we wanted to avoid that,

  1. Resource cannot work parallely in single XSLT
  2. Maintanence is very difficult
  3. Licensed version of tool
  4. When the XML size grows, new resourse are feeling difficult to edit the XSLT etc.,

Related to .net approaches would be best.

  1. Can we create XSLT by using SSIS to transfrom one xml into another xml format.
  2. In .net code we can do, but feel that it is time consuming due to large size of XML.
  3. Any other best way for handling XSLT transformation or any tool
↧

Capturing SSIS Error line in a mail

March 12, 2015, 7:23 am
$
0
0

Hi,

Can i capture an error message line in a mail instead of whole error message.

↧
↧

How to update an Oracle table from SSIS

March 12, 2015, 7:05 am
$
0
0

Hi there

I've been working in a SSIS project where I need to update a table in an Oracle database that is located in a remote server. Until now I have used the OLE DB COMMAND to do this in a sql server database. To access the data that belongs to the Oracle database I am using an OCDB task, but this component does not work with an OLE DB COMMAN and I also need to do an inner join with a table in SQL.

This is the schema of the query that I need to perform:

UPDATE OTSET
OT.COLUMN1= ST.COLUMN1,
OT.COLUMN2= ST.COLUMN2,
OT.COLUMN3= ST.COLUMN3,
OT.COLUMN4= ST.COLUMN4,FROM
ORACLE_TABLE AS OT INNERJOIN SQL_TABLE ST ON
OT.COLUMN5 = ST.COLUMN5WHERE(((OT.COLUMN1)IsNull)AND((ST.COLUMN2)In('A1','A2','A3')AND((ST.COLUMN3)<>'B1'Or(ST.COLUMN3)IsNull)AND((ST.COLUMN7)=1));

I am using Microsoft Visual Studio 2010.

↧

Capturing SSIS Error line in a mail

March 12, 2015, 7:38 am
$
0
0

Hi,

Can i capture an error message line in a mail instead of whole error message.

↧
Search

connection not found

March 12, 2015, 9:48 am
$
0
0

i deleted one connection which i am not using anymore in my package.

still i am getting error : The connection {sdfsdf} not found

what should i do

↧

Configuration file in ssis

March 12, 2015, 9:03 am
$
0
0

Can I have one configuration file for all the packages I have created, So that changing server name will change all packages server names?

If yes, help me out in doing it.

Thanks,

Sri

sree

↧
↧

SSIS error authenticating proxy with SQL Agent

March 12, 2015, 9:07 am
$
0
0

Hello,

I'm a BI Analyst (not DBA) and I've asked our IT department for access to SSIS and SQL Server Agent to deploy packages to import data from Oracle nightly and transform it into a dataset in SQL Server 2012 using SSIS.

I've written several packages, deployed these to a SQL Server. These run fine when ran locally through Object Explorer > Integration Services Catalog > SSIDB by right clicking/execute. 

However our DBA has set up an SSIS Proxy that I can use in conjunction with SQL Server Agent to run these packages. However, when I try to use this proxy I get the error message:

'Unable to start execution of step 1 (reason: error authoenticating proxy ad\[name removed], system error. The username or password is incorrect. The step failed.'

I've tried packages with security for 'dontsavesensitive' and 'encryptsensitivewithuserkey' but neither work. I'd like to rule anything out that I'm not doing, however I think it is a permissions issue (happy to be proven wrong though). 

Can anyone offer suggestions for next steps please?

Thanks, 

ZP


↧

Moving SSIS projects/packages from Dev to Prod on SQL 2012

March 12, 2015, 8:50 am
$
0
0
I have several SSIS packages. Not all the packages are part of one project. I have several projects and each project has several SSIS packages.  Developer created those projects/packages on Dev environment. Now I need to deploy all the projects/packages on Prod server. I created new environment as "Prod". I can change the environment from Dev to Prod for each project/package.  But it will take lots of times as I have several projects/packages. Is there any easy way i can deploy all projects and packages and then change the environment as well?
↧

Runtime error Exception has been thrown by the target of an invocation” from Script task

March 11, 2015, 2:37 pm
$
0
0

I have a SSIS package with a script task, I get the following error when i try to run it in my local system. It works fine for my collegues as well as in production. However, I am not able to run it locally, to test. I keep a debug point in the main method, but it is never reached, I get the error before it goes to main method.

I am using VS 2010, .Net framework 4.5.

The script task does compile. I get the following messages SSIS package "..\Test.dtsx" starting. Error: 0x1 at Test: Exception has been thrown by the target of an invocation. Task failed: Test SSIS package "..\Test.dtsx" finished: Success. The program '[2552] DtsDebugHost.exe: DTS' has exited with code 0 (0x0).

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>