Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Execute powershell script from ssis?

$
0
0

Hi,

I was trying to use the execute process task to kick off a powershell script.  However, nothing happens when I run in debug (the component turns yellow and stays yellow).  Any idea if what I am trying to do is possible and the proper way to configure it?

btw, I am using powershell for the remoting capabilites.  I need to execute a bat file on a remote server which runs a process in a legacy program. 

Update:  When I name the ps1 script file in the executable window, it opens it in notepad.  This would be like the default if you double clicked the file.

Mark


Script Task Error

$
0
0

Hello guru's - I am working on a ssis package and ran into a issue with a simple script task uses c# to check a file to make sure the size is bigger than 0 and if it is it moves it to a different folder. When i run the script task i get the error in screen shot. I created a console app in visual studio running the same code and it works fine so i know the code works. Im not really sure what this error message means or what i need to do to get this to work. Any help would be greatly appreciated!!

temp table not able to be recognized in SSIS Package

$
0
0

Issue Description:

[SQL Server Version:SQL Server 2012]

Data flow task always show me the temp table not be recognized as it not existed in source DB . 

Component flow :

Step #1. [Execute SQL Task] - it used to create the temp table ahead (Connection -> Source)

Step #2. [Data Flow Task] - [OLE DB Source] prepare one variable to store the source query which need to join with temp table to do the filter . e.x: select .. from tableA join #tempB on ... . One more thing need to highlight: To make sure the [OLE DB Source] can get the metadata for input column and output column , it need to create one real table which has the same schema with the temp table . And then update the variable to change back to temp table . (Connection -> Source)

      [Data Flow Task] - [OLE DB Destination] populate the data to target table (Connection -> Destination)

NOTE:

a. Set the property [RetainSameConnection] = TRUE for all the connections . 

b. Set the property [DelayValiation] = TRUE for data flow task.

But if i replace with global temp table , the issue will be gone . In other words, the two components [Execute SQL Task] &[OLE DB Source] point to the same connection , but they are locate in the different sessions . WHO CAN HELP ME TO FIND THE ROOT CAUSE ? 

Thanks in advance!


Lookup performance issue

$
0
0

Previously the staging data was getting lookup with full cache to find the Match and No Match for Insert and Update Operation.

Now the Lookup data holds around 1 Million records and Data Flow Task failed to execute .

Since with Full cache  is using  Memory Operation it gives the error.

Decided to load the data in Cache File and use in Lookup to do minimum changes but that too fails

The buffer Manager failed a memory allocation call for 1638000 bytes , but was unable to swap any buffer to remove memory pressure . 2290 buffer were considered and 2 were locked 

tried to change the Setting for the data Flow task DefaultmaxBufferRow -500 and Defaultbuffersize value is as it is but it failed

Decide to divide the data into series and tried to load in cache file but cache file can be used once . please suggest

Database Mirroring between SQL and DB2

$
0
0

Is there a way to mirror database tables from SQL Server to IBM DB2 and vice versa using Integration Services or in another method.

Thank you.

Split

$
0
0

How does Split treat Input flow when several conditions are set? Does a condition reduce the flow?

For example input flow field have a number set from 1 to 10, conditions are:

1. A>7

2. A<3

3. A>0

Will 3rd flow have all 10 rows or only 4,5,6?

catalog.start_execution SYNCHRONIZED in Transaction

$
0
0

Hi, I am trying to execute a SSIS Package from a stored procedure within a transaction. I also want the execution to be in Synchronized mode. This is an example of my stored procedure code:

begin try
begin transaction

	/*
	inserts, updates
	*/

	declare @execution_id bigint
	EXEC [SSISDB].[catalog].[create_execution]	@package_name=N'My Package.dtsx',
												@execution_id=@execution_id OUTPUT,
												@folder_name=N'MyFolder',
												@project_name=N'MyProject',
												@use32bitruntime=False,
												@reference_id=Null
	EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,
															@object_type=50,
															@parameter_name=N'LOGGING_LEVEL',
															@parameter_value= 1
	EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,
															@object_type=50,
															@parameter_name=N'SYNCHRONIZED',
															@parameter_value= 1
	EXEC [SSISDB].[catalog].[start_execution] @execution_id;

	/*
	throw error if execution status is not = 7 ("Success")
	*/

commit transaction
end try
begin catch
	if @@Trancount > 0
		rollback transaction;
end catch

The problem is that the package reaches a timeout and gets a status = 5 ("Pending").

If I remove the Synchronized setting, it works. If I remove the transaction, it works. But a combination of both will not work. Can anyone think of a simple workaround for this?

Thank you,

Filip

Staging tables

$
0
0

Hi All,

We are in the process of creating a data warehouse using SSIS and I was wondering if someone could provide their thoughts on staging tables. How often do you bring table from source system to the staging database? Do you guys truncate all the tables first in the staging database before inserting data from source tables? We are thinking of truncating all the table sin the staging database before loading them with fresh data, is that a right approach?

Thanks in advance for your help.


C# in SSIS not picking the file

$
0
0

Hello,

I am a sql dba/developer working with SSIS packages that is using the C# code and I am new to C# . All I am trying to do is in a folder with a bunch of csv files I need to identify only specific csv files that start with the string connect and move data from that csv file to a sql database table. Here is my code:

if (file.LastWriteTime > lastModified && Path.GetFileName(file.FullName).StartsWith("connect*", StringComparison.InvariantCultureIgnoreCase) == true && file.Extension == "*.csv")

I am trying to look for the file that starts with the string connect. The name of file looks like connectwithme_04052016_0910AM.csv for instance. The above code doesnt error out but does nothing.

Can someone tell me where am I doing woring?

VS2013 SSIS DataFlow not updating

$
0
0

Hi,

I have a really annoying problem with VS2013 data flows. When I run the packages, the data flows complete successfully but when I look at them in the designer, they have "0 rows" on the links and none of the items have changed colour. If I add a row count item, the variable within it does get updated, but again the UI shows the data flow as if it's in design mode. I've also tried the following:

- Add a script task with a breakpoint. The breakpoint is hit but UI still looks as if in design mode

- Reinstall SQL 2014

- Reinstall VS 2013

- Reinstall SSDT

- Checked I am local admin

- Tried using OLEDB & Flat file source & destinations

- Someone else running my package on another machine (which works fine)

- Someone else running my package on my machine (which doesn't work)

- Checked windows & SQL Logs for errors, but can't find anything.

- Numerous reboots

Has anyone else experienced this, or have any advice on how to solve it?

Thanks in advance

Bobby

VSTA Script Support Error

$
0
0
I published a package using visual studio 2015 to A SQL Server 2014 (12.0.2269.0). When the package is executed it errors out on the script task with a message indicating that the script version 14.0 is not supported on this version if SSIS. Questions is where do I get the latest version/update to allow these to run?

Lee

SSIS TextQualifier Issue _x0022_

$
0
0

Hi,

My SSIS package has the TextQualifier as:

<DTS:Property DTS:Name="TextQualifier" xml:space="preserve">_x0022_</DTS:Property>

I manually update it to:

<DTS:Property DTS:Name="TextQualifier" xml:space="preserve">"</DTS:Property>

However, when I test the package it reverts to using _x0022_ and when I export it and view the dtsx code it has also reverted to using _x0022_

Why is this?

Basically, when I run the SSIS package in Visual Studio or from the SSIS server, it runs fine.

When run from the calling server, it adds _x0022_ to each cell value!

Thanks!


Script components not working with VS2015 SSDT and SQL Server 2012

$
0
0

Downloaded the latest preview of SSDT for VS 2015 today. SSIS works fine when TargetServerVersion is set to 2014 or 2016. When I change the TargetServerVersion to 2012 to publish to my DB, the script components stop working.

Specifically, I get null reference exceptions from the row buffer.

Noticed that SQL Server 2014-2016 use TxScript version 12 and 13 respectively. Setting TargetServerVersion = SqlServer2012 drops TxScript to version 11. I have a feeling the auto generated code when Script Components are created in SSIS is not compatible with version 11.

Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost, Microsoft.SqlServer.TxScript, Version=11.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91

I'm kind of stuck now since I have my SSIS packages built, but cannot deploy them to my SQL Server 2012 instance.


How to stop excel or csv shifting data

$
0
0

I'm trying to pull the data using SSIS and save into the CSv file but when i open it the data is already shifted to other column.

I use simple process in the SSIS. 1> data pull it from using OLE Db Source.

2> flat file connection with delimited option. I've some data like(i.e Sony, inc)

Can someone help to resolve this issue. 

Microsoft DB2 OLE DB Provider Error 0x80040E00

$
0
0

Hello,

i have a strange issue using Microsoft OLEDB provider for DB2, I was migrating SSIS 2008 to SSIS 2012 the dataflow task with OLEDB Source which have error output configured are throwing error in 2012 where as works fine in 2008.

Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E00.
An OLE DB record is available.  Source: "Microsoft DB2 OLE DB Provider"  Hresult: 0x80040E00  Description: "Accessor is invalid.".

When i remove the error output everything executes fine, does anyone encountered this before? 

Thanks,

naveej


SQL job using Map network drive

$
0
0

I've SSIS package is created, to pull the data and save it to Map network drive.

When I deployed manually package to create CSV from visual studio it's work but When I schedule a job it's getting failed.

Is there any solution for this or I cannot use MAp network drive for SSIS?

Here is below my error;

PSP\SQLAdmin. Microsoft (R) SQL Server Execute Package Utility  Version 9.00.5324.00 for 64-bit  Copyright (C) Microsoft Corp 1984-2005. All rights reserved.    Started:  2:17:57 PM Error: 2016-04-06 14:17:58.26     Code: 0xC020200E     Source: Data Flow Task Flat File Destination [64]     Description: Cannot open the datafile "Y:\Name_List_Process.csv".  End Error  Error: 2016-04-06 14:17:58.26     Code: 0xC004701A     Source: Data Flow Task DTS.Pipeline     Description: component "Flat File Destination" (64) failed the pre-execute phase and returned error code 0xC020200E.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  2:17:57 PM  Finished: 2:17:58 PM  Elapsed:  0.359 seconds.  The package execution failed.  The step failed.

Custom logging in Active task SSIS

$
0
0

I am facing two issues in my SSIS package. The SSIS package I have has an active task that formats an excel sheet as per my requirement and saves it as a different file, modified.xlsx. This file is then used in my data flow task to process and upload data to database tables. This package works perfectly in my local system, but when I created a scheduled job on my SQL server to run this package it fails with the generic error message "Microsoft (R) SQL Server Execute Package Utility Version 11.0.5058.0 for 64-bit Copyright (C) Microsoft Corporation. All rights reserved. Started: 12:06:55 PM Error: 2016-04-01 12:06:57.06 Code: 0x00000001 Source: Script Task Description: Exception has been thrown by the target of an invocation. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 12:06:55 PM Finished: 12:06:57 PM Elapsed: 1.563 seconds. The package execution failed. The step failed."
I have set the steps to use 32 bit runtime.

To get a more detailed error message i tried to set up logging for the active task. I configured logging to write log entries for events to a CSV file as in the below screen shot. enter image description here

I enabled logging for package and checked individuals tasks as well. In active tasks, I added Dts.Log("",0,bytes); to keep track of any exception if any also to log each steps.

public void Main(){
            LogMessages("");

            LogMessages("Update Bug package execution started at :: "+ DateTime.Now.ToLongTimeString());
            LogMessages("Loading package configuration values to local variables.");

            FileName =Convert.ToString(Dts.Variables["User::ExcelFileName"].Value);
            SourceFileLocation =Convert.ToString(Dts.Variables["User::SourceFileLoc"].Value);

            SourceFileName = Path.Combine(SourceFileLocation, FileName);
            saveLoc = Path.Combine(SourceFileLocation,"ModifiedExcel.xlsx");


            var excel = new Excel.Application();
            var workbook = excel.Workbooks.Open(SourceFileName);
            try{

                foreach (Excel.Worksheet tempSheet in workbook.Worksheets){
                    LogMessages("For loop to check sheet names");if(((Excel.Worksheet)(tempSheet)).Name.Contains("Test")){if(File.Exists(saveLoc)){File.Delete(saveLoc);}//File.Create(saveLoc);
                        tempSheet.Select();
                        workbook.SaveAs(saveLoc);}

                    System.Runtime.InteropServices.Marshal.ReleaseComObject(tempSheet);}

                workbook.Save();
                workbook.Close();
                excel.Quit();
                LogMessages("Quit Excel sheet");

                System.Runtime.InteropServices.Marshal.ReleaseComObject(workbook);
                System.Runtime.InteropServices.Marshal.ReleaseComObject(excel);
                LogMessages("Release excel objects");}
            catch(Exception ex){
                LogMessages("Exception: "+ ex.InnerException);
                System.Runtime.InteropServices.Marshal.ReleaseComObject(workbook);
                System.Runtime.InteropServices.Marshal.ReleaseComObject(excel);}


            Dts.TaskResult =(int)ScriptResults.Success;}#region ScriptResults declaration///<summary>/// This enum provides a convenient shorthand within the scope of this class for setting the/// result of the script.////// This code was generated automatically.///</summary>
        enum ScriptResults{
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure};#endregion#region Log messages to package log files/table.public void LogMessages(string strLogMsg){
            Dts.Log(strLogMsg,0, bytes);}#endregion

But when i run the package the log file is not updated. The log file just contains the following :

Fields: event,computer,operator,source,sourceid,executionid,starttime,endtime,datacode,databytes,message

Can someone please help me understand what I am missing here for logging? Also, what could be the issue for the job to fail in SQL server?

work with Custom control

$
0
0
Hello Friends

 I need to work on the Custom Control .Till now i have not worked on the custom controls in SSIS.I tried to search on Net regarding this but unable to find any good link .

Will anyone let know any good references regarding the same .I need to make some changes in the existing custom control ,as well create a new one .

please let know good web links /PDF regarding this

Failed to acquire connection. Connection may not be configured correctly or you may not have the right permissions on this connection.

$
0
0

Hi,

We deployed SSIS ETL packages in SSIS catalog DB- Sql Server 2012. BUT Sometimes we got an below error:-  "Failed to acquire connection. Connection may not be configured correctly or you may not have the right permissions on this connection." while connecting to sql server through my ETL SSIS package.

This is an random behaviour where error occurs randomly to any ETL SSIS package. Even sometimes it does not show any error and working fine. So can anyone let me know how to overcome thisERROR.

Any help would be highly appreciated!!


Insert to same tabe from mutiple source

$
0
0

Hi

We use ssis to load data in target table. The same target table will also get inserted by application team using front end UI. we used table lock in ssis which was blocking insert from UI. we have unchecked table lock option to allow insert in parallel.

Is there any other best approach available to allow insert using ssis and UI to singe tabe in parallel

Thank you


Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>