Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

execute permissions on integration tasks

June 6, 2016, 8:07 am
$
0
0

Hi,

I am executing an SSIS package using a c# application.  this SSIS packages extracts about 700K-2M records from a database and outputs to text or excel files.  the application allows the user to execute the package on demand.

This application uses these dlls:

Microsoft.SqlServer.ConnectionInfo

Microsoft.SqlServer.Management.IntegrationServices

Microsoft.SqlServer.Management.Sdk.Sfc

Microsoft.SqlServer.Smo

The package runs perfectly if the app is run under my login but fails when run with the user's login. (error message pasted below)

My hunch is that i need to grant execute permission on one or more of these dlls to the user account but i don't quite know how or where. Or is it some dcomcnfg permission?

Any help would be most welcome.

Regards

Habib




---------------------- Execution error ------------------

Failed to connect to server .\n   at Microsoft.SqlServer.Management.Common.ConnectionManager.Connect()

   at Microsoft.SqlServer.Management.Common.ConnectionManager.PoolConnect()

   at Microsoft.SqlServer.Management.Common.ConnectionManager.get_ServerVersion()

   at Microsoft.SqlServer.Management.Common.ServerConnection.get_TrueName()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices.Microsoft.SqlServer.Management.Sdk.Sfc.ISfcDomainLite.get_DomainInstanceName()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices.Key.GetUrnFragment()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcKeyChain.ToString()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcKeyChain.Microsoft.SqlServer.Management.Sdk.Sfc.IUrn.get_XPathExpression()

   at Microsoft.SqlServer.Management.Sdk.Sfc.Environment.GetObjectInfo(Object ci, RequestObjectInfo req)

   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.GetObjectInfo(Object connectionInfo, RequestObjectInfo requestObjectInfo)

   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.Process(Object connectionInfo, RequestObjectInfo requestObjectInfo)

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.GetSupportedResultTypes(ISfcConnection connection, Urn urn)

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.Initialize()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.MarkRootAsConnected()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices..ctor(SqlConnection sqlConnection)

   at HSalimCo.Pivots.DataExport.bkworker_DoWork(Object sender, DoWorkEventArgs e)

----------------------------------------


↧
Search

SSIS Package running slow with no reason

June 6, 2016, 9:13 pm
$
0
0

Hi All,

I have 169 of 355 packages are running slow than usual. The gap increased from 12 seconds to 110 seconds. I could not be able to find any solution. 

SQL Server 2012 SP3.

select 
  e.project_name,  

e.package_name,

e.start_time,cast(e.end_time as  datetime) as SSISDbcompletedtime

,ssisdbfinishedinss = DATEDIFF(second,e.start_time,e.end_time),

m.DateLog,gap = datediff(SECOND,cast(e.end_time as  datetime),m.DateLog)

  ,e.execution_id

  from SSISDB.catalog.executions e

  join xxxx.xxxx.SSISDBLog m on e.package_name = m.PackageName collate Latin1_General_CI_AS and e.execution_id = m.ExecutionId
  where cast(e.start_time as date) > '2016-5-5'
  and e.folder_name = 'xxxx'
  and e.package_name = 'xxxx.dtsx'

 xxxx.xxxx.SSISDBLog is our log table. we use below code to get package finish time while SP running package. the code as followed

-- Create Instance of package
 EXEC [SSISDB].[catalog].[Create_execution] 
            @package_name=@packageName, 
            @execution_id=@executionId output, 
            @folder_name=@folderName, 
            @project_name=@projectName, 
            @use32bitruntime=false 

EXEC [SSISDB].[catalog].[set_execution_parameter_value] 
        @executionid,  
        @object_type=50, 
        @parameter_name=N'SYNCHRONIZED', 
        @parameter_value=1; -- true

-- Set our package parameters
EXEC [SSISDB].[catalog].[set_execution_parameter_value] 
        @executionid,  
        @object_type=50, 
        @parameter_name=N'DUMP_ON_EVENT', 
        @parameter_value=1; -- true

EXEC [SSISDB].[catalog].[set_execution_parameter_value] 
        @executionid,  
        @object_type=50, 
        @parameter_name=N'DUMP_EVENT_CODE', 
        @parameter_value=N'0x80040E4D;0x80004005';

EXEC [SSISDB].[catalog].[set_execution_parameter_value] 
        @executionid,  
        @object_type=50, 
        @parameter_name=N'LOGGING_LEVEL', 
        @parameter_value= 1; -- Basic

EXEC [SSISDB].[catalog].[set_execution_parameter_value] 
        @executionid,  
        @object_type=50, 
        @parameter_name=N'DUMP_ON_ERROR', 
        @parameter_value=1; -- true


      -- Assign parameters to the instance of package
          EXEC [SSISDB].[catalog].[Set_execution_parameter_value] 
            @executionId, 
            @object_type=30, -- package parameter 
            @parameter_name= 'AuditKey', 
            @parameter_value= @auditKey


EXEC [SSISDB].[catalog].[Start_execution]

            @executionId

             SELECT @validationMessage = CASE  WHEN [status] = 1 THEN 'created'

             WHEN [status] = 2 THEN 'running'

             WHEN [status] = 3 THEN 'canceled'

             WHEN [status] = 4 THEN 'failed'

             WHEN [status] = 5 THEN 'pending'

             WHEN [status] = 6 THEN 'ended unexpectedly'

             WHEN [status] = 7 THEN 'succeeded'

             WHEN [status] = 8 THEN 'stopping'

             WHEN [status] = 9 THEN 'completed'

             END

             FROM [SSISDB].[catalog].[executions]

             WHERE execution_id = @executionId

INSERT INTO [Control].[SSISDBLog]([AuditKey],[ProjectName]     ,[FolderName],[PackageName],[Message],[DateLog],[ExecutionId])

SELECT @AuditKey,@projectName, @folderName, @packageName, @validationMessage,Getdate(),@executionId


Shawn



↧
↧

Using Attunity Drivers 3.0 in Visual Studio 2015 Preview

February 2, 2016, 2:43 am
$
0
0

Hi,

We currently use Attunity SSIS Connectors v2.0 for Oracle with SQL Server 2012 Integration services within the Visual Studio 2012 IDE.

We are moving to SQL 2014 and want to leverage Visual Studio 2015 SSDT as this now includes all the BI projects (SSIS, SSAS, SSRS) + SSDT and integrated with Visual Team Services. We are testing the dev tools prior to the upgrade.

I have installed the version 3.0 Attunity SSIS Connectors for Oracle (both 32bit and 64bit) which states that is works for VS 2013+, but the Connection type is not available within my Visual Studio 2015 SSIS project.

So my question is - are the Attunity SSIS Connectors v3.0 for Oracle supposed to be compatible with VS 2015, and if not, is there expected to be an update in the future?

Kind Regards,

James

↧

SSIS File System Task Rename File Could not find a part of the path error

June 6, 2016, 5:50 am
$
0
0

I have created a SSIS package that uses a File System Task that moves and renames a file from one folder
to another in one step. The package runs fine and it works a great (it moved and renamed the file) when I use the local drive file paths (I'm using variables) for source(C:\temp\mysourcefile\abc.txt) and
destination (C:\temp\myarchivefile\abc20160606.txt) but when I use the network drive file paths(variables):
source(\\Geneva(machine)\hsteam(share)\it\projects\provider\mysourcefiles\abc.txt)
and destination(\\Geneva(machine)\hsteam(share)\it\projects\provider\myarchivefiles\), the
package runs successfully but it didn’t move and rename the file. And I also
see the following error in the log:Error: 0xC002F304 at Rename file, File System Task:
An error occurred with the following error message: "Could not find a part
of the path.".<o:p></o:p>

SSIS package "myTest.dtsx" finished:
Success.<o:p></o:p>

The program '[3952]
myTest.dtsx: DTS' has exited with code 0 (0x0).<o:p></o:p>


↧

Why a statement can fire twice

June 7, 2016, 3:46 am
$
0
0

Hi Guys,

I have SSIS package 1.truncate tbl1 

                                 2.load Tbl1

3.truncate tbl2

                                4.Load tbl2

then exit

but when package execution got completed i cant able to see the data in first table,I ran sql server profiler to see what is actually db,

I found and gone crazy ,1 step is fired at first then it also firing with 3 step and all those are connected and execution squentially

Can any one face the same before

Thanks in advance

ADKR

↧
↧

Can Any One tell Me How to Integrate MSSQL server and Kafka

June 6, 2016, 7:04 am
$
0
0

Hi

I need to integrate Sql server with kafka. How can i integrate both. is there any way?

can anyone help me to solve this problem

Thanks,

Chinnarao. 

↧

DTS Runtime Error

June 7, 2016, 3:53 am
$
0
0

I trying to capture one system variable like below:

 Public Sub Main()
        '
        ' Add your code here
        '

        MsgBox(Dts.Variables("System::UserName").Value, MsgBoxStyle.Information, "System::UserName")




        Dts.TaskResult = ScriptResults.Success
    End Sub

biut it retrieves an error. Any idea why? Thanks guys.

↧

Visual Studio Development to Extract Data From Sharepoint

May 22, 2016, 7:00 am
$
0
0

Hi Guys,

We have a client who would like to extract data from sharepoint and send it into their data warehouse. Small volumes.

I found an adapter that should be able to fit into SSIS properly on this link.

http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652

I have installed the adapters on a 2012 dev server and started up Visual Studio 2010 and tried to create a new SSIS project as per the documentation.

When I do that I get the message. 

"The following assemblies are installed SDK assembly but could not be shown in the customize tools dialog box because they are missing one or more components. 

Please make sure that all necessary libraries are available."

MySQL.Data.CF.dll
AspNetMMCxt.dll

I have been looking around to see if this message tells me to do anything specific but so far no luck.

So a couple of questions really.

1. Has anyone else tried this here and is there an easy fix for these sorts of messages? I am sure it is just installing something else that Visual Studio wants.

2. We have our own ETL tool that can execute anything that can run in batch mode on the ETL machine.

So I was wondering. Apart from going to the trouble of using this tool and creating an SSIS package and trying to integrate the execution of the package into our scheduler...someone must have used the APIs necessary to create a batch command that dumps data out of SSIS by now. I mean, surely I can not be the first man to want to do that a better way than unloading via spreadsheets etc. I have seen a few products on the market that are GUIs that replicate data from Sharepoint to SQL Server so it can't be too hard to do. 

If there is not something readily available it looks like us creating a tool to do this might be useful more widely.....

Thank you in advance for any suggestions or tips you have.

Best Regards 

Peter 

Peter Nolan

↧
Search

Derived column implicit DT_STR to DT_WSTR conversion question (SSIS 2008 SP1)

November 18, 2009, 6:44 am
$
0
0
Hello all,

in my dataflow i'm using an OleDB data source which points to a SQL Server 2008 table with some varchar columns. Connected to this datasource is a derived column component. The metadata of the green connection arrow shows all varchar columns as DT_STR.

Creating a derived column based on one of the varchar columns will result in a new column of type DT_WSTR. In SSIS 2005 there was a combobox for changing the datatype back to DT_STR (preserving the string length from the datasource), but in SSIS 2208 unfortunately it is missing (or hidden very well from me).

Do now (in SSIS 2008) I really have to do this step manually? e.g. casting to DT_STR in the derived column or adding a data conversion task? Both cases are much more manual work (compared to 2005) and in the first option I even have to take care of the string length myself.

What about upgrading existing packages - will they break?

thanks in advance and best regards,
Gerald
↧
↧

database manager is not able to accept new requests

July 31, 2013, 9:45 am
$
0
0

Working on this issue ...

SSIS package trying to load data from DB2 to SQL ....

IBM OLE DB Provider for DB2"

Hresult: 0x8004D01CDescription:" SQL1224N The database manager is not able to accept new requests, has terminated all requests in progress, or has terminated the specified request because of an error or a forced interrupt. SQLSTATE=55032

SSIS Error Code DTS_E_PRIMEOUTPUTFAILED -

The PrimeOutput method on component "OLE DB Source"(1) returned error code 0xC0202009. The component returned a failure codewhen the pipeline engine called PrimeOutput(). The meaning of the failure codeis definedby the component, but the error is fataland the pipelinestopped executing. There may be error messages posted before thiswith more information about the failure.

Please Mark As Answer if it is helpful. \\Aim To Inspire Rather to Teach A.Shah

↧

Export data from SQL Server to SFTP

June 7, 2016, 8:12 am
$
0
0

Hi 

Is there a way that we export data from SQL Server Table to SFTP in the form of XML ?

Thanks

↧

Element and child element with the same name

June 7, 2016, 8:58 am
$
0
0

Here is code section from my .xsd file:

              <xs:element minOccurs="0" maxOccurs="unbounded" name="AwardOrRecognition">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element minOccurs="0" name="AwardOrRecognition" type="xs:string" />
                    <xs:element minOccurs="0" name="OrganizationOrSponsor" type="xs:string" />
                    <xs:element minOccurs="0" name="DateReceivedStart" type="xs:string" />
                    <xs:element minOccurs="0" name="DateReceivedEnd" type="xs:string" />
                    <xs:element minOccurs="0" name="SortOrder" type="xs:unsignedByte" />
                  </xs:sequence>
                </xs:complexType>

As you can see, I have an element, "AwardOrRecognition" that has a child element with the same name.  Is there any way to edit the .xsd to get the child element to be output without resorting to .xslt (assuming the source .xml file can't be edited)?  I've tried changing the child element name but that obviously doesn't work.

So Long And Thanks For All The Fish!

↧

Configure SSIS Package with Environment variables

April 29, 2016, 8:23 am
$
0
0

While Configuring SSIS Project on Environment Variable match, the drop down list is a mess, especially when you have a long list of vaiables, it is very hard to locate the one you need.

Anyone knows any solution?

It won't allow me to post screen shot, not sure how to validate my account.

↧
↧

SSIS Task Required to Download the encrypted file using azure key Vault from Blob and load data into SQL table

June 7, 2016, 10:47 am
$
0
0

Hi All 

we have a requirement to download the encrypted file using Azure key vault  stored in Blob  and load the data into SQL table .

we are able to decrypt and downlaod using Cloudblob class . ie  blob.DownloadToStream(stream, null, options, null);

The challenge here is  we are not allowed download decrypted file on sql server as per our security policy , 
we need to download , decrypt and load directly to sql server without keeping on sql server physical drive.

Please suggest us good way to achive that 

Surendra Thota

↧

Trouble using an or statement with a boolean value

June 7, 2016, 1:11 pm
$
0
0

I am trying to create a derived column where if

A = 1  or B = 1 or C = 1 then D will = 1

(DT_I4) A == 1  ?  1  :  0   -  this will work for just one column however as soon as I add a new column with an | | statement it errors

(DT_I4)([A== 1 | | B == 1)  ? 1  :  0

↧
Search

delete duplicate entries automatically

June 6, 2016, 6:50 am
$
0
0

Hi,

I have a script which create a csv of Windows eventrecord. Then SSIS format this file and then integrate it in a DB SQL.

I would like to know how I have to do to eliminate duplicate entries automatically. This script will execute everyday. I would like to delete same lines.

For example I have :

Date        Name      ID      

02/06      X              4624

02/06      X              4624

03/06      X              4624

Here for example I would like to delete line 2

I don't know really SQL and SSIS, pls give me detailled explanation :)

Thanks for your help

↧

find all current files in directory

June 7, 2016, 8:51 pm
$
0
0
Directory holds multiples of many distinct files I want to find the current/latest file of each distinct file.
↧
↧

Can not open VSTA through Script task Editor in SSIS 2012

June 8, 2016, 12:45 am
$
0
0

Hi I  am working with script task to write C# logic for one my SSIS package that is created using SSDT 2012 on windows 7
ultimate edition but getting error message about dotnet framework v4.0 Target issue.I have my feature of .net frmwrk 3.5 on in windows 7.What it is?Can not figure out.I tried downloading 4.0 frmwrk from Microsoft site but no use....
Pls if anyone can guide me on it.

Thanx in advance.

srm_2010

↧

Check for number of rows returned and do not create empty destination file

June 7, 2016, 8:58 pm
$
0
0

Hi,

I checked various forums for this issue and found nothing that suits my scenario. One of the sites I saw is:

https://munishbansal.wordpress.com/2014/10/22/how-to-prevent-ssis-package-creating-empty-flat-file-at-the-destination/

My scenario is like this:

I get all the generic data from database using OLEDB Source. Then I use Conditional Split to split output based on certain conditions. So, 2 splits and 2 files gets generated. So, I don't know which split will have data and which doesn't have. I tried 2nd approach given in above article. But still empty file gets created.

In their third approcach, they tell to get record count in DFT and use it in control flow to precedence constraint. But for me I will have 2 variables to check for record count for 2 files.

What is the best way to do it.

Thanks,

↧

How to consume survey data with ssis

June 8, 2016, 1:18 am
$
0
0

I am planing an ETL with SSIS and need some help. I have the following basics:

I have a dataprovider who stores data from different datasets. Let us say Survey 2012, Survey 2013 and so on.

  1. I get a Token from a Rest Service
  2. Using this token in a query I get a list of Keys (each key for a dataset)
  3. Now I can query with the token and the Key the columnnames for each dataset and the data. So to get all data I have to implement a Loop. (All in Json)

My ideas:

  1. I write a script task to get the token and store this token in a variable
  2. I write another script task to get the dataset-keys ans store these onse in a text file
  3. I write a script task to query the columns looping the text file and store the columnnames
  4. I loop though the Datasets and store these data as well.

My second idea is to write only one script task but I am not sure what to return (datasets? or can I write the output direct into a database?)

I am also not sure which approach would be better.

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>