Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Connection Manager Works Locally Not on Stage Server

$
0
0

I have an SSIS package which makes an Oracle OLEDB connection using MSDAORA. The connection uses a specific user name and password. The SSIS package is triggered from a SQL Agent Job.  The job uses a proxy account as DOMAIN\MYUSER in order to give network path access to a stored procedure called from the package. When executing the job on my local machine it execute perfectly.  However, when I run the SQL Agent Job on the stage server it errors.  The package is exactly the same and the setup of the job is exactly the same.  Here is the error.  Any idea?

Executed as user: DOMAIN\MYUSER. Microsoft (R) SQL Server Execute Package Utility  Version 10.0.5500.0 for 32-bit  Copyright (C) Microsoft Corp 1984-2005. All rights reserved.    Started:  10:35:19 AM  Error: 2016-07-05 10:35:22.21     Code: 0xC0202009     Source: Report  IDV Connection manager "CCI"     Description: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80070005.  An OLE DB record is available.  Source: "Microsoft OLE DB Service Components"  Hresult: 0x80070005  Description: "Access is denied.".  End Error  Error: 2016-07-05 10:35:22.23     Code: 0xC020801C     Source: CCI Contact Information OLE DB Source [1]     Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method call to the connection manager"CCI" failed with error code 0xC0202009.  There may be error messages posted before this with more information on why the AcquireConnection method call failed.  End Error  Error: 2016-07-05 10:35:22.24     Code: 0xC0047017     Source: CCI Contact Information SSIS.Pipeline     Description: component "OLE DB Source" (1) failed validation and returned error code 0xC020801C.  End Error  Error: 2016-07-05 10:35:22.24     Code: 0xC004700C     Source: CCI Contact Information SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2016-07-05 10:35:22.24     Code: 0xC0024107     Source: CCI Contact Information      Description: There were errors during task validation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  10:35:19 AM  Finished: 10:35:22 AM  Elapsed:  2.469 seconds.  The package execution failed.  The step failed.


Ragged right with [CR] [LF] export column position help needed

$
0
0

Hi All,

This is my first time with ragged right /fized width file export. I have an issue where data is not present in the desired position and our external system is not accepting the file.

On a high level, the flat file has data, however the columns are one or more tabs away from desired positions.. how can i trouble shoot the issue?

where in SSIS workflow can I verify the column positions? can someone guide me through on this ??? I dont even know where to start ??????

Your help is much appreciated


FM

Breakpoints in Script Component Task as Source

$
0
0

I'm trying to debug a C#-script that was written in a Script Component in a dataflow. I've created some breakpoints but in contrary to what the documentation says, the debug-process doesn't halt at my breakpoints. Why is that and what can I do about it? I really need to debug this step... No, messageboxes etc. are not the solution I'm looking for...

SSIS package take more time than usual

$
0
0

Hello everybody
I am new in sql server and the task assigned to me is to follow the SSIS jobs launched by SQL Server Agent.

Today I had a run-time problem of an SSIS package, it makes more time than usual.

the package is related to another master package
Usually , the execution is done one after the other but in my case the second package run 3 times before than the other starts.
Master.dtsx package ( allows to load a  tabular
cube and should run every 15 minutes but in my case the because of  the package "Alimentation TotaleDWHTotale.dtsx " how runs 3 times successively, master takes more than 40 minutes and directly affects the cube .
What can I do to find  why the job has run 3 times ?and what can I do to avoid it is still happening at other times .
For information, the afternoon the problem disappeared and the package are executed in the order

SSIS recursive update failure and error handling for the exception

$
0
0

I have a SSIS package which contains a sequence container holding 2 tasks

1. Execute sql task 

Which executes a sql script loading child and parent in a o/p table .Whenever child =parent it goes up and gets the next parent using recursive update

2. Create csv file task using the o/p table created in the above task.The o/p table is truncated before every load 

In the Execute sql task ,I have a recursive update .

3.There is no file archiving in place for the CSV file

4.There is no audit history of the output table

5.No error notification

The problem right now I am having is the recursive update is failing and we are not getting notified.When we have looked into the issue with more detail we have found the issue happens when the lookup condition in recursive update doesnt get any value.

i.e when it expected a value during lookup it didnt get a value and it completes  with a error message .

(136 row(s) affected)

Msg 245, Level 16, State 1, Line 333
Conversion failed when converting the varchar value 'kv1234' to data type int.

When we digged furhter in we found the source table for the look up condition was not having the required value.

The execute sql task completes with no failures recorded in the job and the output table and csv file are populated with whatever data it got till the failure.

So my expert friends please suggest me a solution which  you think I should implement.

The solution should notify the failure and pick the column or record which is the cause for the failure.

The file which generated with no issues must be archived at the end of the load.







Development test practices

$
0
0

In a theoretical case that we have deployed an project on SSIS 2012 and we do some changes on the development part, what would be a good practice in order to test the development project without messing with the existing deployed project on production?

Is there any option, except of creating staging tables that would required effort and time?

The answer to my question would be to create 2 enviroments at SSIS catalog? (https://www.simple-talk.com/sql/ssis/ssis-2012-projects-deployment-configurations-and-monitoring/)

The next part of this procedure, if the Dev. project have met our expectations, would be to deployed to production.



Populate Fact Table with Surrogate keys from DimCustomer Table

$
0
0

Hi,

While populating the fact tables with surrogate keys coming from dimension tables, if from source, the natural key of Customer Table (say CustID) matches with all SCD records for that CustID available in DimCustomer Table (current and history records), shall we load all matching Surrogate keys into Fact Table?

Please explain me with the same example.

__

Thanks

Hemanth

Remove DTD from XML file using SSIS script Task

$
0
0

I have to remove DTD from 2 GB XML file , i have used below code for that , but i have getting system out of memory exception. even though there 16 GB RAM on server /



XmlDocument XDoc = new XmlDocument();
   xmlDoc.XmlResolver = null ;
                XDoc.Load(Dts.Variables["D:\\xmlfiles\\old.xml"].Value.ToString());
                XmlDocumentType XDType = XDoc.DocumentType;
                XDoc.RemoveChild(XDType);
                XDoc.Save(Dts.Variables["D:\\xmlfiles\\new.xml"].Value.ToString());

Is there other way to remove DTD, pls help on this ..

Looping through the files in SSIS

$
0
0

Hi 

i have a scenario that requires 

How to load multiple excel files with different names into corresponding tables in SSIS

Please some one help me with regarding this 

Thanks 

Check Points issue with sequence container

$
0
0

Hi i am new to ssis tool, i am working with small control flow task(cft) where it contain 5 data flow task(dft) in sequence container i have mapped in a way that dft2 should run after dft1 run successfully and dft4 should run after dft3 run's successfully all these 5 dft's are in sequence container. I have applied check point property (xml file created, save check point:true, checkpoits usage:if exist). when i ran the package for first time dft1 got successful dft5 got successful but dft3 got failed. i have checked for xml file its present, but when i ran the same package for 2nd time again then execution should start from dft3-->dft4, dft2 only should run but in my senario all the dft1 dft5 are also executing which should not happen. 

Please help me what is the problem, where i am going wrong, if?

Please suggest me some solution for this.I will look forward for help.

Thanks,

YaduVamsi

SSIS 2012 OData Error - Cannot acquire a managed connection from the run-time connection manager

$
0
0

Hi,

I have an SSIS package that uses the OData connector to retrieve list data from a Sharepoint 365 site. Running the package locally in SSDT or executing the package locally works fine but once deployed to the server it gets this error: "Cannot acquire a managed connection from the run-time connection manager"

The package is running with the use 32 bit runtime option in a SQL Agent job

The OData connection settings set windows auth to false, online auth to true. I am using a named user id and password with site admin permissions in sharepoint

Any help would be appreciated

Thanks

Error while parsing deployment manifest: The element DTSDeploymentManifest has invalid child element ConfigurationFile.

$
0
0

Hi

I am am running VS2012 on a 2012 windows server

I have designed a package in VS2012 and it uses a config file to store password of oracle schema to which one of the connection managers refers to. I build the package, and when I run the deployment manifest file (so I can access the package from sql server and run it with an sql agent job) I get error:

Error while parsing deployment manifest: The element DTSDeploymentManifest has invalid child element ConfigurationFile.

I have not found this exact error on a google search, and to my knowledge I have not done anything different with this deployment........Any ideas?

OData - Service Document Location

$
0
0

Hi,

This my attempt at trying to use the Odata source in SSIS and need to specify the service documentation location. How do I create this from my SharePoint URL? There doesn't seem to be detailed anywhere in a clear fashion.

https://mycompany.sharepoint.com/sites/Lists/Portfolio%20Management%20Master/Portfolio%20Management%20Master%20%20Submitted%20By.aspx

Thanks

SSIS Newbie - How to send records from a table in an e-mail attachment using SSIS package

$
0
0

Hi dear all,

I am  kind of desperate, please could anyone help me? I am very new into SSIS.

My task is to send the records from a SQL table as an HTML email attachment using SSIS ?

Following steps so far:

In the SSIS package I have placed the following tasks:

Execute SQL task, Foreach loop container, Script task within the Foreach loop container and Send Email task.

I put the result set from SQL get data in a systemobject Variable ( NewVariableOject)


In the next step I have configured the Foreach Loop Container and set the needed variable mappings.



Further, I have put the C# Code into the Scripttask which is located in the ForEachLoop Container.

namespace ST_81f0d3597e7b418d851f1a7cdb0cc0c6
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
	[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
	public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
	{
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to
         * either the list contained in the ReadOnlyVariables property or the list contained in
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         *
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         *
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         *
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         *
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         *
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         *
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         *
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         *
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic
         * "Working with Connection Managers Programatically" for details.
         *
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion


		/// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
		public void Main()
		{
            Variables varCollection = null;
            string header = string.Empty;
            string message = string.Empty;

            Dts.VariableDispenser.LockForWrite("User::EmailMessage");
            Dts.VariableDispenser.LockForWrite("User::LogID");
            Dts.VariableDispenser.LockForWrite("User::PackageName");
            Dts.VariableDispenser.LockForWrite("User::ComponentName");
            Dts.VariableDispenser.LockForWrite("User::StartTime");
            Dts.VariableDispenser.LockForWrite("User::Status");

            Dts.VariableDispenser.GetVariables(ref varCollection);

            //Set the header message for the query result
            if (varCollection["User::EmailMessage"].Value == string.Empty)
            {
                header = "Execute SQL task output sent using Send Email Task in SSIS:\n\n";
                header += string.Format("{0}\t{1}\t{2}\t{3}\n\n", "LogID", "Package Name","Start Time", "Status", "Component Name");
                varCollection["User::EmailMessage"].Value = header;
            }

            //Format the query result with tab delimiters
            message = string.Format("{0} \t{1}\t{2}\t{3}\n\n",
                                        varCollection["User::LogID"].Value,
                                        varCollection["User::PackageName"].Value,
                                        varCollection["User::ComponentName"].Value,
                                        varCollection["User::StartTime"].Value,

                                        varCollection["User::Status"].Value);


            varCollection["User::EmailMessage"].Value = varCollection["User::EmailMessage"].Value + message;

			Dts.TaskResult = (int)ScriptResults.Success;
		}

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        ///
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

	}
}

As a last step I send the E-Mail via Send Mail Task and it really works, meaning it displays the table data in the mail body.

However, I need the results as a table in an attachment in the mail. What do I need to change in my Visual C# 2012 Code?

Please explain for beginners ;-)

I am using Visual Studio 2014.

Please help me out. Thanks in advance

ssis c# script to process data

$
0
0

Hello,
The varchar(max) column of tbl1 is populated with data from a comma delimited .csv file data...
How can C# process each line at a time inside the varchar(max) column.
Thank you


unable to extend temp segment by 64 in tablespace TEMP (SSIS error while copying data from oracle)

$
0
0

I am transferring data from oracle and getting below error message.

 

I using 4 data flow tasks with in a single control flow and all the 4 tasks quueries same table but populates data in to different sql tables based on the where contidion

 

 

[OLE DB Source 1 [853]] Error: An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available.  Source: "Microsoft OLE DB Provider for Oracle"  Hresult: 0x80004005  Description: "ORA-01652: unable to extend temp segment by 64 in tablespace TEMP ".

Starting DFT source components in parallel

$
0
0

I have 1 Data Flow task, which contains 3 OLEDB source components.  These are unioned together into one destination. The issue is that the DFT is not starting all the source components at the same time.  Each source component has to be returning rows before the next one will be started. (And these queries run for 5 mins before they return data)

I know that the data flow task will only run after the pre-execute phase is complete, but I thought that it would at least start all the source components in parallel as part of the pre-execute, rather than run them one by one.

You can simulate this by using a SQL query like the following in each source component:

WAITFOR DELAY '00:00:05';
SELECT GETDATE() as RunDate;

You'll see that the package will take ~15 seconds to run instead of ~5. Has anyone seen anything like this before, and is there a way around it?

Thanks.

Converting Data Types of Output Columns

$
0
0

I have a package that imports data from a Flat file. It has about 700 columns in it. The flat file connection takes the column data type as string of length 50 by default. I get frustrated changing the data types for each and every column to the required data type and it takes a whole lot of time.

Anyway I can change the data type of all the columns at once in the flat file source?

Thanks in advance.


Shanmukha

Unzipping file from script task not working

$
0
0

Hi experts,
  I am unzipping the file. The file is gzip. I am using GZipStream with CompressionMode.Decompress

It is working means extracting the file but only top 10 lines only. If I extract manually with 7gip, the file is extracted full with the size 58MB. Using the code, it is extracting only 10KB or like that.

 It's not giving any error but just extracting only small portion of the file.

 How can I debug? What could be the issue? It is working well previously. I don't know what is the difference between old and new files. It's not generating any errors.

 If I manually unzip and zip again, the code working well.

 My code as below

 Dim DecompressedFile As String = NewZipFile.Substring(0, NewZipFile.Length - 3)
        Dim working(2048) As Byte
        Dim n As Integer = 1
        Using input As Stream = File.OpenRead(NewZipFile)
            Using decompressor As Stream = New Ionic.Zlib.GZipStream(input, CompressionMode.Decompress, True)

                Using output As Stream = File.Create(DecompressedFile)
                    Do
                        n = decompressor.Read(working, 0, working.Length)

                        If n > 0 Then
                            output.Write(working, 0, n)
                        End If
                    Loop While (n > 0)
                End Using
            End Using
        End Using

Execute process arguments supplied dynamically

$
0
0

Hi mates,

I am using ExecuteProcess task to unzip a compressed file(.gz)

I am using 7z.exe but my arguments need to be UNC path(\\servername\shared\zipfile.gz)

Settings as below

Command: c:\program files\7-zip\7z.exe

Arguments e \\\\servername\shared\zipfile.gz  (This is supplied through an expression)

When I run, I am getting error

[Execute Process Task]

Error:InExecuting"C:\Program Files\7-Zip\7z.exe" "e \\servername\Datawarehouse\shared\mydata.txt.gz" at "",The process exitcode was "2"whilethe expected was "0".

What is the issue?

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>