Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

SSIS Scheduled Job with ODBC fails

$
0
0

I am running Sql Server 2008 R2 on a Windows Server 2008 R2 64 bit for both. I am importing data from an IBM universe database using the odbc driver supplied by the vendor. I can successfully run the package directly in SSMS but when I try and run it as a package I get the below error:

 

Executed as user: user. Microsoft (R) SQL Server Execute Package Utility  Version 10.50.2500.0 for 32-bit  Copyright (C) Microsoft Corporation 2010. All rights reserved.    Started:  2:43:55 PM  Error: 2012-01-09 14:43:56.52    Code: 0xC0047062     Source: Data Flow Task ADO NET Source [16]     Description: System.Data.Odbc.OdbcException: ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified    at System.Data.Odbc.OdbcConnection.HandleError(OdbcHandle hrHandle, RetCode retcode)     at System.Data.Odbc.OdbcConnectionHandle..ctor(OdbcConnection connection, OdbcConnectionString constr, OdbcEnvironmentHandle environmentHandle)    at System.Data.Odbc.OdbcConnectionOpen..ctor(OdbcConnection outerConnection, OdbcConnectionString connectionOptions)     at System.Data.Odbc.OdbcConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)     at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup)     at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)     at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)     at System.Data.Odbc.OdbcConnection.Open()     at Microsoft.SqlServer.Dts.Runtime.ManagedHelper.GetManagedConnection(String assemblyQualifiedName, String connStr, Object transaction)     at Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManager100.AcquireConnection(Object pTransaction)     at Microsoft.SqlServer.Dts.Pipeline.DataReaderSourceAdapter.AcquireConnections(Object transaction)     at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper100 wrapper, Object transaction)  End Error  Error: 2012-01-09 14:43:56.52     Code: 0xC0047017    Source: Data Flow Task SSIS.Pipeline     Description: component "ADO NET Source" (16) failed validation and returned error code 0x80131937.  End Error  Error: 2012-01-09 14:43:56.52     Code: 0xC004700C     Source: Data Flow Task SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2012-01-09 14:43:56.52     Code: 0xC0024107     Source: Data Flow Task      Description: There were errors during task validation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  2:43:55 PM  Finished: 2:43:56 PM  Elapsed:  1.031 seconds.  The package execution failed.  The step failed.

I assigned a proxy user to the step per a video I watched and the user associated the the credentials/proxy has sysadmin privileges and is also the logon used to connect through the ODBC manager. 
I am not finding any clear direction on where else to go for this problem. Any advise would be appreciated.
Thanks
Royal


Royal

The time has come! Step up SSIS Gurus! Your community needs heroes like you!

$
0
0

TechNet Gurus... we salute you!

You're awesome, and we know it!

Your knowledge uploads and nifty info nuggets are our life blood at TechNet Wiki.

Every awesome article that gets an award is just the start. We are building up the most sensational collection of gifts of knowledge from eminent community heavy weights and young guns alike. And we plan to promote you and your work wherever we can.

Reputations are being forged.

History is being made.

Generations will know your name.

Your children, grandchildren and great-grandchildren will marvel at your technical prowess.

And now, my mighty code warriors, cool consultants and platform specialists, now your chance is here again.

A new month of possibilities. Another chance to prove YOU are the ONE!

The mighty TechNet Guru medal winner for June!

Take up your mouse and keyboard!

Unleash your mighty words of wisdom and bask in the glory that we bestow upon you!

GO GO Gurus! Give, give, give!

All you have to do is add an article to TechNet Wiki from your own specialist field. Something that fits into one of the categories listed on the submissions page. Copy in your own blog posts, a forum solution, a white paper, or just something you had to solve for your own day's work today.

Drop us some nifty knowledge, or superb snippets, and become MICROSOFT TECHNOLOGY GURU OF THE MONTH!

This is an official Microsoft TechNet recognition, where people such as yourselves can truly get noticed!

HOW TO WIN

1) Please copy over your Microsoft technical solutions and revelations toTechNet Wiki.

2) Add a link to it on THIS WIKI COMPETITION PAGE (so we know you've contributed)

3) Every month, we will highlight your contributions, and select a "Guru of the Month" in each technology.

If you win, we will sing your praises in blogs and forums, similar to the weekly contributor awards. Once "on our radar" and making your mark, you will probably be interviewed for your greatness, and maybe eventually even invited into other inner TechNet/MSDN circles!

Winning this award in your favoured technology will help us learn the active members in each community.

Feel free to ask any questions below.

More about TechNet Guru Awards

Submit now : http://social.technet.microsoft.com/wiki/contents/articles/24692.technet-guru-contributions-for-june-2014.aspx

Thanks in advance!
Pete Laker


#PEJL

Got any nice code? If you invest time in coding an elegant, novel or impressive answer on MSDN forums, why not copy it over to the one and onlyTechNet Wiki, for future generations to benefit from! You'll never get archived again!

If you are a member of any user groups, please make sure you list them in the Microsoft User Groups Portal. Microsoft are trying to help promote your groups, and collating them here is the first step.


Dynamic excel name for Excel Destination in SSIS

$
0
0

I have a for each loop container which executes for each advertise rid.For each advertiser it runs a sql query and exports to an excel file.there are seven advertisers and hence there will be seven excel reports.But my requirement is to name them according to the advertisername.Can i do it dynamically from the parameter which is coming in to foreach loop.

Can i name it dynamically. Can anyone please help in doing this.

Thanks,

Abhinav.

SSIS and CDC - Incorrect state at end of “Mark Processed Range”

$
0
0

The Problem

I currently have CDC running on a table named subscription_events.  The corresponding CT table is being populated with new inserts, updates, and deletes.

I have two SSIS flows that move data from subscription_events into another table in a different database.  The first flow is the initial flow and has the following layout:

enter image description here

The Import Rows Into Vertica step simply has a source and a destination and copies every row into another table.  As a note, the source table is currently active and has new rows flowing into it every few minutes.  TheMark Initial Load Start/End steps store the current state in a variable and that is stored in a separate table meant for storing CDC names and states.

The second flow is the incremental flow and has the following layout:

enter image description here

The Import Rows Into Vertica step uses a CDC source and should pull the latest inserts, updates, and deletes from the CT table and these should be applied to the destination.  Here is where the problem resides; Inever receive anything from the CDC source, even though there are new rows being inserted into thesubscription_events table and the corresponding CT table is growing in size with new change data.

To my understanding, this is how things should work:

  1. Mark Initial Load Start
    1. CDC State should be ILSTART
  2. Data Flow
  3. Mark Initial Load End
    1. CDC State should be ILEND
  4. Get Processing Range (First Run)
    1. CDC State should be ILUPDATE
  5. Data Flow
  6. Mark Processed Range (First Run)
    1. CDC State should be TFEND
  7. Get Processing Range (Subsequent Runs)
    1. CDC State should be TFSTART
  8. Data Flow
  9. Mark Processed Range (Subsequent Runs)
    1. CDC State should be TFEND
  10. Repeat the last three steps

This is not how my CDC states are being set, though... Here are my states along the same process.

  1. Mark Initial Load Start
    1. CDC State is ILSTART
  2. Data Flow
  3. Mark Initial Load End
    1. CDC State is ILEND
  4. Get Processing Range (First Run)
    1. CDC State is ILUPDATE
  5. Data Flow
  6. Mark Processed Range (First Run)
    1. CDC State is ILEND
  7. Get Processing Range (Subsequent Runs)
    1. CDC State is ILUPDATE
  8. Data Flow
  9. Mark Processed Range (Subsequent Runs)
    1. CDC State is ILEND
  10. Repeat the last three steps

I am never able to get out of the ILUPDATE/ILEND loop, so I am never able to get any new data from the CT table.  Why is this happening and what can I do to fix this?

Thank you so much, in advance, for your help!  :)

Edit 1

Here are a couple of articles that sort of describe my situation, though not exactly.  They also did not help me resolve this issue, but it might help you think of something I can try.

The second article includes this image, which shows the ILUPDATE/ILEND loop I am trapped in.

enter image description here

Edit 2

Last week (May 26, 2014) I disabled then re-enabled CDC on the subscription_events table.  This didn't change anything, so I then disabled CDC on the entire database, re-enabled CDC on the database, and then enabled CDC on thesubscription_events table.  This did make CDC work for a few days (and I thought the problem had been resolved by going through this process).  However, at the end of last week (May 30, 2014) I needed to re-load the entire table via this process, and I ran into the same problem again.  I'm still stuck in this loop and I'm not sure why or how to get out of it.

Edit 3

Before I was having this problem, I was having a separate issue which I posted about here:

CDC is enabled, but cdc.dbo<table-name>_CT table is not being populated

Could running these commands have created this problem for me on SQL Server?

Identity or GUID

$
0
0

When we load data from a source to destination, assuming destination table has an identity or GUID column as a surrogate key there are certain articles (such as link below) taking about manually generating a surrogate key in spite of identity (or GUID) column. I am assuming when load data into a table with an identity key it generates the next id automatically ETL process need not look for that, please correct me if I am wrong.

Could you guys please give me scenarios where manual generation is mandatory. Thanks in advance for your suggestions.

http://svangasql.wordpress.com/2012/06/24/surrogate-key-ssis-typ1-type2-scd/


svk



Error with SSIS package running as SQL Agent job

$
0
0

I have a strange issue.

I have a SQL Agent job that execute 5 steps, each step is an SSIS package that imports a CSV file to a temp table, then executes a SQL script to update a production table. 

This job runs every 3 mins, every day, and has been for over a year.

Every once in a while it will start failing on step 2.  Sometime it only happens for a couple of runs, other times I have to restart the SQL server.  Once it corrects itself, it could be months before it happens again, but usually happens every few weeks.

The error in the job history is: "Unable to bulk copy data. You may need to run this package as an administrator"

Obviously I have all the permissions correct, as it was running successfully every 3 mins before this error suddenly starts. I'm at a loss as to what the problem is.  It is always on step 2, which is identical to step 1 except its a different CSV file going to a different table. As I typed this, it failed twice, then was successful afterwards.

Any ideas where to look for further information? I've checked the event viewer and looked in the SQL logs, nothing stands out at this time.

SSIS error Failed to start project...

$
0
0

Hi all,

I created a simple package in SSDT(SSIS) 2012 and configuring it with xml configuration.

i am getting the following error

TITLE: Microsoft Visual Studio
------------------------------

Failed to start project

------------------------------
ADDITIONAL INFORMATION:

The package has been modified since the last build. Build the project and try again.
 (Microsoft.DataTransformationServices.VsIntegration)

With out xml configuration,it is working fine.Do any 1 has idea about this ??

Regards

Shoeb

unpivot spreadsheet

$
0
0

I have following spreadsheet for My ID and Jan, Feb, Mar and Apr value

Type    JAN    FEB    MAR    APR    
1      8        1        7       3    
2      2        4       3        7

and would like to unpivot to

Type Year    Month  MyVlaue

1    2014      1         8

1     2014     2         1

2    2014     1          4

I have an Excel source and unpivot controls and have 'Type' pass through and input column 'JAN', 'FEB', 'MAR', 'APR'

Destination column 'MyActualData' which is my MS SQL server data field name to be inserted

pivot key value are '201401', '201402', '201403', '201404'

Pivot key value column name: 'YearMonth'

I enabled data view, but data view shows same data as original, I thought that data view should see the data after unpivot.

Please advise where I am missing here,

Your help and information is great appreciated,

Regards,

Souris,


SSIS- Reading Of File Name

$
0
0

Need to perform below steps while loading data into target table from access database file

  1. Check if file exists in Folder (particular folder say c:\bin\file...)  
    1. If file does not exist ... exit
    2. If file exists  read CSV File for Y/N ( CSV file has value y or n only one column one row)
  2. If CSV value = Y
    1. Read the file name, up till "."
    2. Load access data where entry type = AAA and populate an additional column of source type as the FileName before "."
  3. If CSV value = N
    1. Read the file name,up till "."
    2. Load access data where entry type <> AAA and populate an additional column of source typeas the FileName before "."


I can understand how to check the value of csv file for y or n

How can we check the presence of file, read the name uptill '.' store it in variable and populate it as a column value

Any help or idea on the above


aak


Adventureworkd 2012 data refresh SSIS package

$
0
0

there seems to be a example ssis package to populate the 2005 & 2008 adventureworks database for download but i can find the corresponding 2012 OR 2014 ones. There are 2012 AdventureworksDW databases so how do you run data into them without thes packages?

Thanks

How to load data into table Using Script Task

$
0
0

We have a directory/folder where we have a file. We need to insert the File Created Date , File Name , Extension into the database table of Sql server by using Script Task.

So could you please suggest , how to frame the connection string , fetch the file details and insert the data into the database table using Script Task of SSIS 2008

Exception from HRESULT: 0xC0048021

$
0
0

I am using the dll of sql 2012 components for dynamaically genearting the ssis package. The exception has raised when i used tried to create the package

Exception from HRESULT: 0xC0048021

at Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSDesigntimeComponent100.ProvideComponentProperties()
   at EY_CEDEX_LIB.SSIS.CedexPackage.AddOleDbDestination(Int32 StreamId, String StreamName, IDTSComponentMetaData100 OleDbDestination)

HTML table using script task in SSIS using c#

$
0
0

Hi,

I have a requirement to return a HTML table in email. I have used script task in SSIS and the programming language used is C#.

The problem I am facing is that <td width="100px"> is not working in the code and as a result the column name is wrapped up.

Ex: If the coloumn name is ExtractSourceSystem the result I am getting is ExtractSou

rceSystem. I want the cell width to increase to fit in EXTRACTSOURCESYSTEM in one line but its not working for me. I have included the part of the code here:

message += "<html><head>Audit Table"</head><body>";

message += "<br>";
message += "<table cellpadding=\"2\" cellspacing=\"2\" border=\"2\" >";
message += "<tr>";
message += "<td style=\"width:100px;font:Tahoma;font-size:18px;background-color:navy\">";
message += "ExtractSourceSystem";
message += "</td>";

Thanks

Ajit

SQL Server 2000 DTS Designer components are required to edit DTS packages - in SQL 2008

$
0
0

I m trying to open 2000 DTS package in SSMS of SQL 2008 and get the below error:

Any solution?

 

TITLE: Object Explorer
------------------------------

SQL Server 2000 DTS Designer components are required to edit DTS packages. Install the special Web download, "SQL Server 2000 DTS Designer Components" to use this feature. (Microsoft.SqlServer.DtsObjectExplorerUI)

------------------------------
BUTTONS:

OK
------------------------------

 

 

thanks

 

Problem with Connection Manager. Can't change connection string / Server name

$
0
0

Hi,


I have copied a number of SSID packages from a live server to my test server and am trying to get them up and running but I have a problem with the connection. The only thing that is different is the server name. I decided to keep the same connection in the connection manager as it is referenced in a dtsConfig file. I changed the dtsConfig file to reference the new servername [Data Source= new test server name]. I also check my paths and they are pointing to the correct config locations etc.


I found this post which was similar to my problem but I still get problems with the connection

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1319961&SiteID=1


I updated the connection by changing the server name in the actual connection. I did this from the data flow tab in the Connection Managers sections (MS Visual Studio). Right clicked - Edit - changed the name of the server name to my test server and selected the database. Test connection was fine, and saved this.

Once I saved this, the connection string changed to my current test server in the properties of the connection. However when I run the package I get the following error:


////////////////////////////////

Error: 0xC0202009 at ReportsImport, Connection manager "myconnectionstring": SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.

An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80004005  Description: "Login timeout expired".

An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80004005  Description: "An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections.".

An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80004005  Description: "Named Pipes Provider: Could not open a connection to SQL Server [53]. ".

Error: 0xC020801C at Myprocess import from file, BuyAtReportsImport [79]: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method call to the connection manager "myconnectionstring" failed with error code 0xC0202009.  There may be error messages posted before this with more information on why the AcquireConnection method call failed.

Error: 0xC0047017 at Myprocess import from file, DTS.Pipeline: component "ReportsImport" (79) failed validation and returned error code 0xC020801C.

Error: 0xC004700C at Myprocess import from file, DTS.Pipeline: One or more component failed validation.

Error: 0xC0024107 at Myprocess import from file: There were errors during task validation.

////////////////////////////////


So even if I change the existing connection properties and save and Build my package again, my connection properties (connectionstring, servername etc) all revert back to its original data source when I re-open the package. Simply it does not seem to be saving the connection properties i put in.


What am I doing wrong or how can I change the existing connection manager to look at the new server. The database is  the same and I have full admin privilege on the server.

I don't want to recreate my packages as there are alot and they are not simple.


After changing the connection properties, saving and reloading I get a connection error. But this is because it is not retaining the new connection manger setting i entered.

Incidentally - I created a new SSID to perform a simple connection and write to the the server and it was OK so there is no problem with actually seeing the my test server.


Thanks in advance

 


SSIS Data flow item does not show on the Tools Choose Tools Item

$
0
0

I would like to add share point to my data source for my SSIS project.

After I installed Share Point Source and Destination list, I went to Tools Choose Tools Item, but SSIS Data Flow tab does not show. I went to SSIS Tools box, but the components are not there.

I would like to know are there any way to add Share Point Data source using SSIS.

Your help and information is great appreciated,

I use SQL Server 2012 and VS 2010.

Regards,

Souris,

Get Excel File Name

$
0
0

I have an Excel Source and would like to use file name as a input column.

I noticed that I can use For Each Loop Container to get file name pass to the Excel Source, I just wanted to know does it work like flat file to get file name using advance edit?

Your help and information is great appreciated,

Regards,

Souris,

Can you change the value of a variable that is reset by a parameter?

$
0
0

Sorry if this is simple but I just crashed a package and can't do an autopsy. I suspect I sent it into an infinite loop. I know you can't change the value of any passed in parameters. What I tried to do was have the parameter set the value of a variable and then I tried to change the value of the variable. Is that even allowed?

I was trying to do this:

MyCheckVariable = MyParameter (which equals true)

(some cool stuff happens)

MyCheckVariable = False

I hit F5 and my package turned into the Energizer Bunny. Is the way I try to reset the variable the culprit?

When using Proxy account SSIS won't write to XL

$
0
0

I have an SSIS project that will write out 7200 records to an XL workbook in interactive move. When in the Job scheduler using a Proxy account, the job ends normally without writing any records to XL. Office is not and will not be installed on the Server where the job is running. I'm just writing to the XL spreadsheet no formatting of any sort. Permissions have been granted to the Proxy account to allow read/write access. The XL workbook is pre-existing. I am using the Excel Connection Manager and writing to an XLSX file.

Is the fact that Office is not installed on the server causing me to not be able to write records to XL.

Create an Excel sheet Dynamically in for each loop container

$
0
0

I have a for each loop(ADO Enumerator) container which executes for each Advertiserid which is coming from database.In for each loop I have to create a new excel file with the advertiser name.So if the loop executes 7 times there should be seven excel spreadsheets with seven advertiser names.

How can i create an excel dynamically in the foreach loop container.Can anyone please help!!!

Viewing all 24688 articles
Browse latest View live




Latest Images