Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Calling SSIS packge from C# application

$
0
0

Hi,

   One of the client production deployment we have SSIS packages to Import the Data from excel(Sharepoint Source) to SQL DB.

  Its working fine in Dev enviornment but in production its Throwing Error.

 

Background of Application:

 

* From Application we are uploading Excel sheet to Sharepoint Library.

* SSIS Package deployed in SQL server will retrieve this Excel file from sharepoint physicle path  and load the data to SQL table

* This is working fine if sharepoint,Application, SQL server is in the same Server.

* In Production we have 2 server :Application and Sharepoint is in one server ,DB in second server.

* We have installed SSIS services in Apllication server.

 

Below is the error we are recieving while Uploding the Excel from Application to Sharepoint.Once we click the Upload button,Application will upload Excel to Sharepoint and Package will Exceute for the same Excel.

 

Please suggest Us on this, Below is the error that we are getting.


Component "component "DRV- Variables" (394)" could not be created and returned error code 0x80070005.
 Make sure that the component is registered correctly. The version of component "LKP-INS_FS_Account" (417)
is not compatible with this version of the DataFlow. The component is missing, not registered,
not upgradeable, or missing required interfaces.
The contact information for this component is "Derived Column;Microsoft Corporation; Microsoft SqlServer v10; (C)
Microsoft Corporation; All Rights Reserved; http://www.microsoft.com/sql/support;0".
component "DRV- Variables" (394) failed validation and returned error code 0xC0048021.
One or more component failed validation. There were errors during task validation.

 

Thanks

chandra



ExecuteProcessTask is not calling bat file?

$
0
0

Hi All,

I have SSIS package which contains execute process task which calls bat file. The bat file contains the below code. In execute process task, i have selected this filename(C:\StartStopPublication\ICRV_STOP.bat) as executable.

@echo off
:Begin
setlocal

SET CLASSPATH=C:\Program Files\Java\jdk1.7.0_03\lib;C:\StartStopPublication\lib;

@echo off
java -Djava.ext.dirs=./lib/ StartStopPublication ogrdsdevuser ogrdsdevpwd Stop CA1 ICRV >MessageID.txt
exit 0

Iam trying to call batch(.bat) file from SSIS of executeProcess task. But The batch file is not triggered when i tried to execute this package. If SSIS execute process task calls this batch file which can write a messageID into text file, it is not happening.

if i run this batch file from command prompt, it is working fine.

Could you please suggest me what i did wrong here.


Thanks & Regards, Anil

Sharepoint list control issue

$
0
0

Hi all,


I am trying to add existing package to the project which has shrepoint list control source and SQL destination. But When i am trying to add it i got below error i said Ok..but when i try to open list control it looks like corrupted. Already i installed Sharepoint list source and destination to the toollist. But still i have same issue. 

Error at Uverse T2 Sharepoint [Uverse T2 Sharepoint (SharePoint List Source -  Uverse T2 Daily Metric Input [22695])]: The managed pipeline component "Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource, SharePointListAdapters, Version=1.0.0.0, Culture=neutral, PublicKeyToken=933a2c7edf82ac1f" could not be loaded.  The exception was: Could not load file or assembly 'SharePointListAdapters, Version=1.0.0.0, Culture=neutral, PublicKeyToken=933a2c7edf82ac1f' or one of its dependencies. The system cannot find the file specified..

Error at AAA Sharepoint [AAA (SSIS.Pipeline)]: The "component "SharePoint List Source -  AAA Daily Metric Input" (22695)" failed to cache the component metadata object and returned error code 0x80131600.

Can anyone please let me know what might be the issue.

The is what i installed

http://sqlsrvintegrationsrv.codeplex.com/releases/view/17652

Thanks in advance,

Jyo

SSIS is not using the provided custom Sharepoint credentials

$
0
0

Hello,

I have multiple packages that need to access a Sharepoint server. So I am using the popular package from here:
http://sqlsrvintegrationsrv.codeplex.com/releases

The way I have it set up, is that there is a job in SQL Server that runs multiple packages every day, however when it tries to run them, it does not work. I am getting authentication and validation errors.

When I run the packages locally, it works fine.

After investigating, I found out that the credentials I provide to connect to Sharepoint are not used. The package uses my windows account even if I specifically set it to use custom credentials. To test this, I simply created a new account on my computer and ran the package with correct custom credentials and found that it didn't work. If I do the same on the account that has access to Sharepoint, it works.

I would like to know if there is a fix or a workaround for this ? 

Thank you,

Yan

Error deploying SSIS 2012 to Catalog DB

$
0
0

Hi,

Am having an intermittent issue that happens when deploying to an SSIS Catalog (in the "Deploying project" step). basically sometimes when deploying an SSIS project to a Catalog that I administer, I receive an error telling me to lookup a particular operation error message in the [catalog].[operation_messages] VIEW in the SSIS Catalog DB (and gives me an operation ID), the error in the view above is:

The project or operation records do not exist or you have not been granted the appropriate permissions to view them.

Now I am sure there is no rights or permission issues, and if I try deploying the project again (without changing any deployment parameters) there is no issue (albiet it might take a couple of tries). There is no SSIS packages running during the time I am deploying and all server resources seems to be fine.

Am not entirely sure what's going on here.

SSIS script component endofrowset not evaluated to true

$
0
0

Hi,

I am not skilled vb.net programmer, but I am able to code transformations. I need one in my asynchronous script component. I dont understand,why my code shows always "false". I use SSIS 2008.

Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)

MsgBox(Row.EndOfRowset)

End Sub

ssis 2008r2 dataflow freeze

$
0
0

Hi,

today my dts package freeze during the loading of my data.

I have multiple dataflows in my package, and the freeze occurs in the dataflow 5 or 6, never on the first dataflows.

the 5 is pretty simple, I get data from SQL server to fill a cache transform.

but the freeze could occurs here, even if the execution normaly takes 5 to 10 sec.

and if the freeze is not in the DF 5, its in the 6 which is the biggest one.

yesterday everything works fine!!!

I try to execute inside VS and using DTExec, same behavior.

on the SQL Server side, I see that the network IO wait task is at 1000ms and no more activity.

so why this could occurs?

why today and not yesterday? (I try this 10 times and always the same behavior occurs at different point in time)

why never on my first dataflows and only the DF 5 or 6?

Dev and Live configurations - how to change data sources automatically

$
0
0

Hi all,

I'm fairly new to SSIS. I have a group of packages (one parent, four children) which mostly call sprocs so that they run in parallel. I have a dev server and a live server and their structures are identical. I have used shared data sources, so that I can redirect all the packages from one source to another easily (I hope).

I've used the Configuration Manager to add a Live configuration as well as a Development one. When using the Live configuration I want the data sources to point at my Live server (with the same DB names). When using Dev I want them to point at my dev server. When deploying, I want the files to deploy to \\<servername>\SSIS.

Is there a way to set this up. I really don't want to have to change the data sources each time I repoint from dev to live or vice versa.

Lastly, is there a way to simply deploy directly into the server instead of having to create dtsx files? I don't have to create additional files to deploy tables or functions, stored procedures or reports. Surely IS works the same way, although I cannot figure out how.

I'm sure other people will have asked the same questions. If someone could point me to the existing answers, that'd be great.

Many thanks,

Mark


Mark Roworth


error in SSIS script component

$
0
0

Hello,

I'm using SQL sder er 2008 Standard/

 

I am trying to use a script component to do a lookup, but when I execute the package (in debugging mode), In get an error screen.

Part of the error description is in Dutch, but it says "Object reference not set to an instance of an object "

More detailed, from the debug output screen:

Error: 0xC0047062 at Data Flow Task, Script Component 1 [1518]: System.NullReferenceException: Object reference not set to an instance of an object .

at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception e)

at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.PreExecute()

at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPreExecute(IDTSManagedComponentWrapper100 wrapper)

Error: 0xC004701A at Data Flow Task, SSIS.Pipeline: component "Script Component 1" (1518) failed the pre-execute phase and returned error code 0x80004003.

 

This is the script:

using System;

using System.Data;

using Microsoft.SqlServer.Dts.Pipeline.Wrapper;

using Microsoft.SqlServer.Dts.Runtime.Wrapper;

using System.Data.SqlClient;

using System.Collections.Generic;

public struct RangeItem : IComparable<RangeItem>, IComparable<DateTime>

{

    public Int32 Key;

    public Int32 Value;

    public RangeItem(Int32 key, Int32 value)

    {

        Key = key;

        Value = value;

    }

    #region IComparable Members

    public int CompareTo(RangeItem other)

    {

        return Key.CompareTo(other.Key);

    }

    public int CompareTo(DateTime other)

    {

        return Key.CompareTo(other);

    }

    #endregion

}

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]

public class ScriptMain : UserComponent

{

    // Variables

    SqlConnection connection = null;

    Dictionary<String, List<RangeItem>> Cache = new Dictionary<String, List<RangeItem>>();

    const string query = "select WerkID, MedewerkerID, Werk_eind_KEY from DimMedewerker";

    public override void PreExecute()

    {

        base.PreExecute();

        connection = (SqlConnection)this.Connections.Connection.AcquireConnection(null);

        if (this.Variables.PreCache)

        {

            FillCache();

        }

    }

    private void FillCache()

    {

        FillCache(null);

    }

    private List<RangeItem> FillCache(string key)

    {

        string sql = query;

        if (!string.IsNullOrEmpty(key))

        {

            sql += string.Format(" where MedewerkerID = '{0}'", key);

        }

        sql += " Order By MedewerkerID, Werk_eind_KEY";

        SqlCommand command = new SqlCommand(sql, connection);

        SqlDataReader reader = command.ExecuteReader();

        List<RangeItem> times = null;

        try

        {

            Int32 lastKey = 0;

            Int32 endKey = 0;

            while (reader.Read())

            {

                Int32 WerkID = reader.GetInt32(0);

                Int32 MedewerkerID = reader.GetInt32(1);

                Int32 Werk_eind_KEY;

                if (reader.IsDBNull(2))

                {

                    Werk_eind_KEY = Int32.MaxValue;

                }

                else

                {

                    Werk_eind_KEY = reader.GetInt32(2);

                }

               

                if (lastKey != MedewerkerID)

                {

                    if (!string.IsNullOrEmpty(Convert.ToString(lastKey)))

                    {

                        // cache everything up until now

                        times.Add(new RangeItem(Int32.MaxValue, endKey));

                        Cache.Add(Convert.ToString(lastKey), times);

                    }

                    lastKey = MedewerkerID;

                    times = new List<RangeItem>();

                }

                if (Werk_eind_KEY < Int32.MaxValue)

                {

                    times.Add(new RangeItem(Werk_eind_KEY, WerkID));

                }

                else

                {

                    endKey = WerkID;

                }

            }

            times.Add(new RangeItem(Int32.MaxValue, endKey));

            if (!string.IsNullOrEmpty(Convert.ToString(lastKey)) && times != null && times.Count > 0)

            {

                try

                {

                    Cache.Add(Convert.ToString(lastKey), times);

                }

                catch (Exception e)

                {

                    throw new Exception(lastKey + " " + times[0].Key + " " + times[0].Value, e);

                }

            }

        }

        finally

        {

            reader.Close();

        }

        return times;

    }

    public override void PostExecute()

    {

        base.PostExecute();

    }

    public override void Input0_ProcessInputRow(Input0Buffer Row)

    {

        Int32 MedewerkerID = Row.MedewerkerID;

        List<RangeItem> ranges = null;

        if (!Cache.TryGetValue(Convert.ToString(MedewerkerID), out ranges))

        {

            // We don't have this value yet

            ranges = FillCache(Convert.ToString(MedewerkerID));

        }

       

        if (ranges == null)

        {

            throw new Exception("Couldn't find value for MedewerkerID " + MedewerkerID);

        }

       

        int index = 0;

        if (ranges.Count > 1)

        {

            index = ranges.BinarySearch(new RangeItem(Row.ZiekDagKEY, 0));

            if (index < 0)

            {

                index = ~index;

            }

        }

        try

        {

            Row.WerkID = ranges[index].Value;

        }

        catch (Exception e)

        {

            throw new Exception(string.Format("size: {0} index: {1} key: {2}", ranges.Count, index, Row.MedewerkerID), e);

        }

    }

}

 

What am I doing wrong?

 

Regards, Hennie

Mdw Problem

$
0
0

Hi, Guys..

When i finish to configure MDW, I saw in the log of the collectors, the message appears = Error 0xC0014062. Failed to load the package "\\Data Collector\\TSQLQueryCollect" from SQL Server "SIDESHOWBOB". Make sure the package exist on the SQL Server and you have the correct credential to access it. 

I want to collect data in a LOCAL Machine. The account of SQL Server Agent is Network Services, i don't understand that.


/* Se útil por favor classifique a resposta */

Thread local storage failure when executing a SSIS package from C# code

$
0
0

Hi everyone,

We have made an application on .NET 3.5 using C# to run some SSIS (*.dstx package) we have built.

We are working with:

 

  • Intel Xeon Processor E5620
  • 12 GB Ram
  • SQL Server 2008
  • Windows Server 2008 R2 standard (64x)

 

There is a thread for SSIS package. This packages are quite heavy, so we have limited from code to 5 as the max number that can be run at the same time.

We are using this code:

 

//This saves the start log
new TaskLog(IdTask, TaskLog.LogType.TaskAcction, DateTime.Now, " Executing SSIS Package : " + packageName).Save();
Package pkg = app.LoadPackage(this.PackagePath, eventListener);
DTSExecResult pkgResults = pkg.Execute(null, null, eventListener, null, null);
// this saves the end log
new TaskLog(IdTask, TaskLog.LogType.TaskAcction, DateTime.Now, " SSIS Package Executed : " + packageName + ". Result : " + pkgResults.ToString()).Save();

 

After few days of running (our app runs like a windows service):

 

SOURCE: Microsoft.SqlServer.DTSRuntimeWrap

MESSAGE: Thread local storage failure (Exception from HRESULT: 0x80004006 (CO_E_INIT_TLS))

TARGETSITE: Microsoft.SqlServer.Dts.Runtime.DTSExecResult Execute(Microsoft.SqlServer.Dts.Runtime.Connections, Microsoft.SqlServer.Dts.Runtime.Variables, Microsoft.SqlServer.Dts.Runtime.IDTSEvents, Microsoft.SqlServer.Dts.Runtime.IDTSLogging, System.Object)

STACKTRACE:    at Microsoft.SqlServer.Dts.Runtime.DtsContainer.Execute(Connections connections, Variables variables, IDTSEvents events, IDTSLogging log, Object transaction) at BizLayer.ExecutionStgys.SSISExecutor.Execute() in SSISExecutor.cs:line 126

 

Line 126 is :

DTSExecResult pkgResults = pkg.Execute(null, null, eventListener, null, null);

The main problem is that the application can't run any other package until we restart the windows service.

Do you have any clue of what's going on in here? We have tried some workarounds and nothing is working.

Thanks in advance,

Alan.-

 

 

 

 

IF condition in Derived Column

$
0
0

Hi All,

I need to use IF statment in DErived COulmn.

If Product_Type = 'PL' then the New column shld be populated with 'SD' else if it is null it shld be populated with 'LD'. Kindly help.

REgards,

James.

How to call batch file (.bat) in SSIS?

$
0
0

Hi, I have created a test package to load data into 2 tables. so created two simple insert scripts and created a batch file and calling batch file in execute process task. but i am getting error :

Error: 0xC0029151 at Execute Process Task, Execute Process Task: In Executing "C:\sql.bat" "" at "", The process exit code was "1" while the expected was "0".

and if i check in dos prompt, i am getting error Login Failed.

Can somebody tell me how to call a batch file in ssis?

the script i have used in batch file is

SQLCMD -E -dHRMS_SITE -iC:\aTest1.sql

SQLCMD -E -dHRMS_SITE -iC:\aTest2.sql

how to send mail from SSIS Package using Send Mail Task

$
0
0

Hi all

The client has given as ABC_SMTP  as the SMTP Server details... and we are working on their computer network domain

So they told us use WINDOWS AUTHENTICATION setting in Send mail task while configuring the send mail task

and they have give xyz@ABCGroup.com for From email address and efg@ABCGroup.com for to email address

i have configure the about information in SSIS Send email task and when i exeucte the package i got errror saying

[Send Mail Task] Error: An error occurred with the following error message: "Failure sending mail.  System.Net.WebException: The remote name could not be resolved: 'ABC_SMTP' ".

Please help me what to do....


ilikemicrosoft

Update-Insert while looping through Excels

$
0
0
 

Hi gurus,

I have scenario that I loop through set of files and load them into db and then archive those files into Archive folder., which I did fine. Now, there is a chance that there may be same filename with (same rows + some additional rows) added to it and loaded into the folder. so, I dont want to insert those same rows into my db., I just want to insert new rows. How can I do that.Can anyone explain me in detail.

FYI: I also filename into my db for some additional purpose.

Please help me with this.



common cartridges (IMS) and siss

Access to path denied in SSIS 2012 script task

$
0
0

Note: this applies to SQL 2012 only.

I'm using the project deployment model in SQL SSIS 2012

I have a script task inside the SSIS package that renames some files on a domain share. (\\mydomain\myfileshare)

The database owner is a domain user (mydomain\svclogin)

I start the package from a stored procedure like so:

EXEC [SSISDB].[catalog].[start_execution] @execution_id

When the package is deployed to my local machine, it executes without error.

When it is deployed to a remote server on the same domain (2012), I get an error:

Access to the path \\mydomain\myfileshare is denied.

The report log show that the caller is mydomain\svclogin. I added some informational logging of my own and I see that the variable System::UserName is also mydomain\svclogin

I logged on to windows as mydomain\svclogin and verified that I can modify/delete/change files on  \\mydomain\myfileshare

On older packages, I used credentials and proxy accounts to set up access to domain resources.

1. How do you specify the account (a domain account) that will access domain resources in an SSIS 2012 package ?
2. How do I fix the error?


John Lucas - Code Monkey

Error Updating NTEXT Column in SharePoint List Using SP List Destination

$
0
0

I am getting an error when I try to update (or insert) data into a SharePoint List Multi-Row field. The field's property is set to NTEXT. I have tried changing the datatype of the data being written to that field from WSTR to NTEXT and everything in between, but it always throws an error. If I omit sending data to those columns, my task runs fine.

[SharePoint List Destination [653]] Error: System.ServiceModel.ProtocolException: The remote server returned an unexpected response: (400) Bad Request. ---> System.Net.WebException: The remote server returned an error: (400) Bad Request. at System.Net.HttpWebRequest.GetResponse() at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) --- End of inner exception stack trace --- Server stack trace: at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException, ChannelBinding channelBinding) at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoap.UpdateListItems(UpdateListItemsRequest request) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.ListsService_ListsSoap_UpdateListItems(UpdateListItemsRequest request) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.UpdateListItems(String listName, XElement updates) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.ExecuteSharePointUpdateBatch(String listName, String viewId, XElement batchXml, Int16 batchSize) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.UpdateListItems(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName, IEnumerable`1 fieldValueList, Int16 batchSize) at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListDestination.ProcessInput(Int32 inputID, PipelineBuffer buffer) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)

How do you update SharePoint Lists NTEXT Columns using SSIS?

Thanks,

Lee

Calculate Running Count during data import

$
0
0

We have a flat file data source with Invoice records.

Each invoice is bounded by two key "START" and "END". Each Invoice has multiple transactions

================ START ...

TXN1

TXN2 ... END START ...

TXN1

TXN2

TXN3 ... END ================

In the example above, there are 2 invoices IN_A and IN_B. It is targeted to import those records into single DB table.

The problem is the additional requirement that we need a running count as a "Key" in each row to group the invoice. As shown in above example, the table result will be

K1, TXN1

K1, TXN2

K2, TXN1

K2, TXN2

K2, TXN3

where K1 and K2 are the running keys

I know how to do that in .NET. I am not familiar in SSIS and not sure whether it is possible to do the stuff.

If it is possible to do that, please let us know. Thank you very much for your help.

Error on Execute Process Task : "The data area passed to a system call is too small"

$
0
0

When I try to run the process execute task I get the following error

Execute Process Task] Error: An error occurred with the following error message: "The data area passed to a system call is too small".

I am calling an exe and the arguments passed to the exe is quite large, is this the cause ?  The same exe runs fine when run from command prompt

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>