Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Execute SSIS package in SSIS catalog cannot open flat file on network drive

November 2, 2016, 9:53 am
$
0
0

We have a simple import SSIS package that imports data from flat file on a network share to our SQL database table.

After we deploy the package into SSIS catalog, we execute from the SSIS catalog by right click execute, then if fails with message of cannot open the file, the file is on the network share, and I do have full permissions on it.

The package has no problem to execute either in Visual studio or SQL agent job, or if deployed to file system.

I understand this could be a Kerberos authentication issue.  But our server is not configured as Kerberos. And I don't want to require network admins to set up Kerberos authentication for just this small issue. Plus it worked in other places like I described above.

I thought SQL 2016 could fix this, but apparently not, Any other recommendations and suggestions?

Thanks

SQLFriend


↧
Search

Error with SSIS Connection Manager

November 2, 2016, 1:39 pm
$
0
0

I recently got a different pc with a fresh install of SSMS, SSDT and Visual Studio 2015 Professional. I am receiving errors related to opening the SSIS package in Visual Studio. My previous pc had the same applications and it worked fine. I'm really hoping for a quicker solution other than having IT reinstall for me. I've attached the error log below.

===================================

The new connection manager could not be created. (Microsoft Visual Studio)

===================================

Could not enumerate the registered connections. (Microsoft.DataTransformationServices.Design)

------------------------------
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%C2%AE%20Visual%20Studio%C2%AE%202015&ProdVer=14.0.25420.1&EvtSrc=Microsoft.DataTransformationServices.Design.SR&EvtID=CouldNotEnumerateConnectionInfos&LinkId=20476

------------------------------
Program Location:

   at Microsoft.DataTransformationServices.Design.DtsBasePackageDesigner.InitializeConnectionManagerInfos(IServiceProvider sp, Boolean forceRefresh)
   at Microsoft.DataTransformationServices.Design.DtsBasePackageDesigner.GetConnectionInfo(String creationName, IServiceProvider serviceProvider)
   at Microsoft.DataTransformationServices.Design.DtsConnectionService.CreateConnection(String connectionType, ConnectionManagerUIArgs connectionUIArg)

===================================

Unable to find the directories containing Tasks and Data Flow Task components. Check the integrity of your installation.


------------------------------
Program Location:

   at Microsoft.SqlServer.Dts.Runtime.Application.get_ConnectionInfos()
   at Microsoft.DataTransformationServices.Design.DtsBasePackageDesigner.InitializeConnectionManagerInfos(IServiceProvider sp, Boolean forceRefresh)

===================================

Unable to find the directories containing Tasks and Data Flow Task components. Check the integrity of your installation.


------------------------------
Program Location:

   at Microsoft.SqlServer.Dts.Runtime.Wrapper.ApplicationClass.get_ConnectionInfos()
   at Microsoft.SqlServer.Dts.Runtime.Application.get_ConnectionInfos()

↧
↧

SSIS Package Connection Manager issue when run from Integration Services

November 2, 2016, 2:48 pm
$
0
0

Hello,

I am new to SSIS and i have been trying to figure out what could be the issue with the connection manager ID as the Connection manager ID displayed in SSIS package is same as the one that throws error while running the package from SQL Server integration services from SSMS.

MASTER_Tables is the package imported


Error after running the package even though the ID is same as the SQL server connection manager

Oledb connection manager ID same as above

{8A6D415C-A395-4A76-A342-464BB97832A7}

please advice what could be the problem when the SSIS package runs okay but not from integerations services

Thanks

Senthil

↧

Update table column with value from select containing row_number

November 2, 2016, 12:00 pm
$
0
0

I have a table...

CREATE TABLE [dbo].[APAYsource] (
    [APAY_SRC_CD]    NVARCHAR (100) NULL,
    [TENDER_TYPE_CD] NVARCHAR (100) NULL,
    [EXT_SOURCE_ID]  NVARCHAR (100) NULL,
    [APAY_SRC_NAME]  NVARCHAR (100) NULL,
    [DESCR]          NVARCHAR (100) NULL,
    [BRANCHCOUNTER]  INT            NULL
);

...in which I want to  update the value of column BRANCHCOUNTER with the value from branchenumerator for the corresponding row in the select...

select *, row_number() over (PARTITION by descr order by descr) as branchenumerator from [APAYsource]

I tried to use the row_number() in an update but couldn't get it to work.  I also tried

update [APAYsource for banks] set BRANCHCOUNTER = (select row_number() over (PARTITION by descr order by descr) as branchcounter  from [APAYsource] as T1 where T1.EXT_SOURCE_ID = EXT_SOURCE_ID)

but doesn't work (syntax).

Could someone kindly show me how to do this?  Thanks tonnes for any help, Roscoe

↧

Best practice error handling

November 1, 2016, 11:42 pm
$
0
0

I’m re-writing our SQL 2014 SSIS packages in SQL 2016.  The main purpose of the re-write is to add additional error handling and intelligence to the packages. I’m looking for a little “best practice” guidance.

First I’ll give a brief explanation of our packages.  They are written to process files we receive on our FTP site. We have a main “wrapper” (parent) package that does things like: copies the package to a working directory, unzip the file if necessary and then copies it to an archive directory. When the file is ready for processing, the parent package calls a sub-package (child) that does the cleansing/importing of the file, then calls a stored proc to do the final handling of the data.

I want to record errors as they occur, at whatever point they occur at, and gracefully exit the package. This means I need to write a record to a table, handled with an Exec SQL Task that calls a sproc, and then delete the file from the working directory, then send an email to people based on where the package failed. I’m wondering if the parent package should have one OnError event at the main package level or if it should have a separate OnError event for each step. I will do the same thing no matter where the error occurs – call the sproc to insert a record into the table. If the error occurs in the child package I also need to call the same sproc to insert a record into the table, but I also need the error to bubble up to the parent package so I know an error occurred. If an error occurs in the parent or the child, as the very last step(s) I need to delete the file I was processing, then call another sproc that is going to determine where the error occurred and whom to email. I’m not sure of the best way to accomplish all of this so I’m looking for suggestions.

I was thinking of having one OnError event at the parent package, and one at the child package. I can handle all the errors that way just fine, but I can’t handle the final cleanup/email step.  I also don't know how to get the child package to tell the main package that an error has occurred.

Any help is appreciated.  Thanks in advance.

André

↧
↧

Script component task error

November 2, 2016, 9:38 am
$
0
0
Error: 0xC0047062 at Data Flow Task, Script Component 1 [1294]: Microsoft.SqlServer.Dts.Pipeline.CannotCreateUserComponentException: Cannot create user component class. Make sure there is one class marked with SSISScriptComponentEntryPointAttribute in your script.
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.CreateScriptObjectHelper(Assembly scriptAssembly)
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.GetScriptEntryPoint(String versionGuid)
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.CreateUserScriptInstance()
Error: 0xC0047062 at Data Flow Task, Script Component 1 [1294]: Microsoft.SqlServer.Dts.Pipeline.CannotCreateUserComponentException: Cannot create user component class. Make sure there is one class marked with SSISScriptComponentEntryPointAttribute in your script.
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.CreateScriptObjectHelper(Assembly scriptAssembly)
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.GetScriptEntryPoint(String versionGuid)
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.CreateUserScriptInstance()
Error: 0xC0047062 at Data Flow Task, Script Component 1 [1294]: Microsoft.SqlServer.Dts.Pipeline.CannotCreateUserComponentException: Cannot create user component class. Make sure there is one class marked with SSISScriptComponentEntryPointAttribute in your script.
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.CreateScriptObjectHelper(Assembly scriptAssembly)
   at Microsoft.SqlServer.Dts.Pipeline.VSTAComponentScriptingEngine.GetScriptEntryPoint(String versionGuid)
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.CreateUserScriptInstance()
Error: 0xC004706B at Data Flow Task, SSIS.Pipeline: "Script Component 1" failed validation and returned validation status "VS_ISBROKEN".
Error: 0xC004700C at Data Flow Task, SSIS.Pipeline: One or more component failed validation.
Error: 0xC0024107 at Data Flow Task: There were errors during task validation.
Warning: 0x80019002 at Package1: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The Execution method succeeded, but the number of errors raised (7) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
↧

Problem with variable (Excel tab name) in a Foreach Loop

October 3, 2013, 3:05 pm
$
0
0
Hello, folks.

I have to import a number of excel spreadsheets. I'm using the classic Foreach Loop inside another Foreach loop approach. The outside loop (Foreach File Enumerator) cycles through the Excel files, while the inside loop (Foreach ADO.NET Schema Rowset Enumerator - ExcelSchema - Tables) to cycle through the individual Excel sheets in each file.

Nothing special there; however, for some reason these excel files have some "phantom" tabs that should not be imported. Icall them phanton because they show up as an importable tab in a SSSIS import wizard but actually are not listed in the excel file structure (no, they are not hidden tabs, I checked).

My idea is to use a constraint to NOT import those phanton tabs. The name convention should allow me to do that because the normal tabs have the name'AAAAAAyyyymmdd$' and the phantom tabs show up as 'AAAAAAyyyymmdd$'_xlnm#_FilterDatabase (the line below was captured from the Local Variable window and show one of the phantom tabs name).

+ User::WorksheetName {'AAAAAAyyyymmdd$'_xlnm#_FilterDatabase} String

I tried using Len (@[User::WorksheetName]) == 17, which corresponds to the length of the normal tabs name ('AAAAAAyyyymmdd$'). However, it does not work. For some reason the portion of the phanton tab name after the ending single quote (_xlnm#_FilterDatabase)  appears to be ignored.

I tested with a number of different expressions, including reversing the variable, to no avail. It seems that internally just the standard name between quotes is what the constraint sees.

Ideas?
↧

How do I set Validation Timeout?

November 3, 2016, 4:33 am
$
0
0

I am experience problems with a package failing connections, I think because of timeouts.    I have already set the connection time out to be 600, without success.  So I'd like to try modifying the validation time out.  This is on SQL Server 2014.

Documentation here says this can be set from the properties dialog on the SSISDB catalog; https://msdn.microsoft.com/en-us/library/hh231008(v=sql.120).aspx

However when I look at this dialog there is no property listed for validation timeout.

Documentation here ; https://msdn.microsoft.com/en-us/library/ff878042.aspx

and here; https://msdn.microsoft.com/en-us/library/ff878147.aspx

says I can set the VALIDATION_TIMEOUT propery using catalog.configure_catalog

However when I try to do this I get the error "The value specified for the input parameter, 'property_name', is not valid."

Does anyone know if this VALIDATION_TIMEOUT even exists, and how do I set it?  Thanks.



↧
Search

Maintenance Plans get Catastrophic Failure

April 8, 2010, 1:50 am
$
0
0

Hello,

I'll hope anyone can help me!

OS: Windows 7 Enterprise x64
SQL: 2008 CTP

I cannot save or modify any maintenance plan. I'll get following message when I save the plan: Catastrophic Failure

Have you any idea why I'll get this error message?

Thank's!

↧
↧

SSIS Deployment using SQL Authentication

November 2, 2016, 9:16 pm
$
0
0
Can I use SQL Authentication from Visual Studio to deploy SSIS packages. I do not see any option to modify user credentials while deployoment.
↧

How to create a task in SSIS to "refresh" Report Model in SSRS

November 3, 2016, 10:59 am
$
0
0

There are Report Models in the SSRS and the source data feeding to it being refreshed daily. I'd like to create a task in SSIS to refresh/auto-generate the Report Model and deploy it to the SSRS with a scheduled SQL agent job. I am currently doing it manually. Any samples would be great. Thanks.

Kahlua

↧

Out of memory error when doing lookup

November 3, 2016, 10:21 am
$
0
0

What are some good strategies to use when running out of memory with a lookup?

When caching the data, the lookup task is using up available memory and throwing an error.

I'm comparing incoming business keys to existing business keys for a dimension table.

It appears to be caching abut 3G, which doesn't seem like all that much to me, but I get an out of memory error.

There's about 20 million rows in the dimension table that need to be cached in two lookups.

John Schroeder

↧

the version number cannot be greater than current version number

March 23, 2015, 6:07 am
$
0
0

i am deployingmy ssis package which is in 2012.(version 6)

and my ssis server also in 2012.

but i am getting error:

"the version number in package is not valid, the version number cannot be greater than current verison number"

and one more thing how to see ifmy package is in 2012,by looking it in notepad?

↧
↧

How to create SSIS package with PowerShell? I get ProvideComponentProperties() error

October 31, 2016, 6:56 am
$
0
0 
# SSIS package for Microsoft SQL Server 2014 (SP2) (KB3171021) - 12.0.5000.0 (X64)
# PSVersion 5.0.10586.117      32 bit

$ErrorActionPreference="Stop"
Set-PSDebug -Strict

Add-Type -AssemblyName "Microsoft.SQLServer.ManagedDTS, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91"
Add-Type -AssemblyName "Microsoft.SqlServer.DTSPipelineWrap, Version=12.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91"

[string]$srcConnStr = "Provider=SQLNCLI11;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=Test;Data Source=XYZ;"

[Microsoft.SqlServer.Dts.Runtime.Package]$Package = New-Object Microsoft.SqlServer.Dts.Runtime.Package
[Microsoft.SqlServer.Dts.Runtime.Application]$App = New-Object Microsoft.SqlServer.Dts.Runtime.Application
[Microsoft.SQLServer.Dts.Runtime.Executable]$exec = $Package.Executables.Add("STOCK:PipelineTask")
[Microsoft.SqlServer.Dts.Runtime.TaskHost]$th = [Microsoft.SqlServer.Dts.Runtime.TaskHost]$exec
$th.Name = "Data Flow Task"
[Microsoft.SqlServer.Dts.Pipeline.Wrapper.MainPipe]$DataPipe = [System.Runtime.InteropServices.Marshal]::CreateWrapperOfType($th.InnerObject, [Microsoft.SQLServer.DTS.pipeline.Wrapper.MainPipeClass])
[Microsoft.SqlServer.Dts.Runtime.ConnectionManager]$SourceConn = $package.Connections.Add("OLEDB") -As [Microsoft.SqlServer.Dts.Runtime.ConnectionManager]
$SourceConn.Name = "OLEDBConnectionSource"
$SourceConn.set_ConnectionString($srcConnStr)

$oledbSource = $DataPipe.ComponentMetaDataCollection.New()
$oledbSource.Name = "OLE DB Source"
$oledbSource.ComponentClassID = "DTSAdapter.OLEDBSource.1";

[Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapper]$InstanceSource = [System.Runtime.InteropServices.Marshal]::CreateWrapperOfType($oledbSource.Instantiate(),[Microsoft.SqlServer.Dts.Pipeline.Wrapper.CManagedComponentWrapperClass])
$instanceSource.ProvideComponentProperties()

Exception calling "ProvideComponentProperties" with "0" argument(s): "Exception from HRESULT: 0xC0048021"At N:\Psh\SSIS\LoadTest1.ps1:27 char:1
+ $instanceSource.ProvideComponentProperties()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : COMException

If I change DTSAdapter.OLEDBSource.1 -> DTSAdapter.OLEDBSource.4


Element not found. (Exception from HRESULT: 0x8002802B (TYPE_E_ELEMENTNOTFOUND))At N:\Psh\SSIS\LoadTest1.ps1:27 char:1
+ $instanceSource.ProvideComponentProperties()
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException

Thank you

↧

Getting errors in report while setting the connection string dynamically.

November 3, 2016, 7:24 am
$
0
0

As a requirement I need to encrypt the connection string for OLE DB connection for Oracle provider.
Since SSIS does not support encryption/decryption for Xml configuration file, I have followed the following steps:
1. Encrypt the connection string in SQL server using Self certificate and Symmetric key (Link)
2. Create variable X and assign the encrypted connection string value to it.
3. Create OLEDB connection manager and use the expression to set the connection string by using X variable.
4. As a first component in the package, use Script component, in which decrypt the connection string and set to the variable Xml

It is working fine from Visual Studio 2015.
When I deploy to SQL Server 2012 (SP3), even though package is running successfully I am getting error in the report log and it says the package failed.

Attached the screen shot for reference.

Is there any other best approach to encrypt/decrypt the connection string.

Please help for the same.

Syed

↧
Search

FLATFILE Connection Type not recognized

October 28, 2016, 5:01 pm
$
0
0


I am occasionally getting the following errors.  My searches have not yielded anything helpful yet.

I am using SQL Server 2014 Integration Services with SSISDB Catalog loading data to another SQL server.

The SSIS Package is executed in a loop from the parent package 10.000+ times during a typical load.

The error typically occurs early on, usually with in the 3-6 iteration through the loop. I have not seen it happen the on the first or second iteration.

Rerunning the package once or twice seems to resolve the issue for a while.

Error : -1073676264 : Error loading value "<DTS:ConnectionManagers xmlns:DTS="www.microsoft.com/SqlServer/Dts"><DTS:ConnectionManager DTS:refId="Package.ConnectionManagers[ABC Flat File]" DTS:CreationName="FLATFILE" DTS:DelayValidation="True" DTS:DTSID="{AC6FBA97-8F7F-410A-9A58-61A7BDF93566}" DTS" from node "DTS:ConnectionManagers".

Error : -1073659899 : The connection type "FLATFILE" specified for connection manager "ABC Flat File" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name.

The odd thing is it works 99% of the time.  When it does fail, it did work in the previous iteration just a few seconds earlier. 

At the same level there are 3 other FLATFILE connections defined the same way, they have not encountered this error, but I am assuming that may due to this connection comes first alphabetically.

The ConnectionString property is determined by a expression containing a variable @[User::ABCFlatFile]

Any thoughts on Troubleshooting this issue?

Thanks much,

↧

Read from XML file, match text with system variable and output value to user variable

October 28, 2016, 6:19 am
$
0
0

I am new with SSIS and struggling with Script task. Trying to figure out how to build script for following scenario.

Here is scenario

I have XML file has following data

  • <?xml version="1.0" encoding="utf-8"?>
    <environments>
      <environment>
        <name>DEV3WEB</name>
        <databaseservername>DEV3DB.abc.com</databaseservername>
        <webservername>http://DEV3WEB.abc.com/</webservername>
        <appservername>DEV3APP.abc.com</appservername>
      </environment>
      <environment>
        <name>DEV3APP</name>
        <databaseservername>DEV3DB.abc.com</databaseservername>
        <webservername>http://DEV3WEB.abc.com/</webservername>
        <appservername>DEV3APP.abc.com</appservername>
      </environment>

</environments>

I want script task read 'Name' from XML file - match with System variable 'Machine Name' and write output to user variable 'ServerNumber'.

If machine name is DEV3WEB - output will be number '1' and DEV3APP output will be number '2'.

Thanks in advance.

Thanks -Mehta


↧
↧

How to count and enumerate with a counter all records in table with specific field containing value

October 27, 2016, 12:43 pm
$
0
0

I have a table containing a couple of dozen fields the records of which will contain some 30 distinct values in one of the columns. I want to read through the table and enumerate each record with an integer counter to be placed in one of that records columns that indicates the count value of that record sharing one of the 30 distinct fields.

Each time I read a record I want to see the highest counter value for records with that distinct value in the distinct value column, add one to the count and put the new counter value in the "counter" field for that record.

Could someone kindly show me how to do this? I'm guessing its going to use an sql task in the control flow but if there is a better way would be happy to use that.  I've tried a couple of things but haven't yet got it right.

Thanks tonnes for any help, Roscoe

↧

How to email error messages?

October 24, 2012, 8:59 pm
$
0
0

Hi,

I am very new in SSIS and just have a very basic question: how to capture whatever error message in the whole package, and send it through an email?

All the postings I could find talks about how to fix a car engine or how to install LPG but no posting talks about how to start the car!

Thanks!

↧

Reconciling source and target row counts

November 4, 2016, 7:36 am
$
0
0

Hello,

What are some good ways to check that all the rows in a data transform made it from the source to the destination?

A recent audit of one of our systems pointed out that our SSIS jobs don't have any checks that ensure "all the rows got copied over". They didn't elaborate on how we're supposed to do this but I'm left with the problem regardless.

A simple source/destination row count comes to mind but the problem is that the source data in many cases can be changing on the fly so any before/after table count will be a false positive.

Any suggestions would be appreciated.

- Martin V.

- Martin V.

↧
Viewing all 24688 articles
Browse latest View live
Search