I see a lot of information on using web services task to http, but it doesn't look like everyone is in agreement on using this task to write to a https site.

I'm being tasked with a uploading a file daily to a https site and wanted to see what others are using? I am not much of a programmer, but if the consensus is to script this, does anyone have some examples they could share? We are using ssis 2012

thanks

kam

Hi All

I'm new in SSIS and reading a book called professional_microsoft_sql_server_2012_integration_services and I'm stuck on one topic. Details are:

I did the following steps:

Steps

1- Create a directory with an easy name like c:\ProSSIS\Chapter5\Export that you can use when exporting these pictures.

Create a directory with an easy name likec:\ProSSIS\Chapter5\Exportthat you can use

Create a new SSIS package named Export Column Example.dtsx. Add a Data Flow Task to

On the Data Flow design surface, add an OLE DB Data Source confi gured to the

Add a Derived Column Transformation Task to the Data Flow design surface. Connect the

Create a Derived Column Name named FilePath. Use the Derived Column setting of

I am importing data from CSV file to SQL using SSIS. The SSIS job has been running perfectly fine for last few months but suddenly it gives error message while running a script. The SSIS job is not changed at all.

It gives error while running a script which calculates maximum date from three date.

I am not sure what is causing the error as it's been running fine always. Error is as below.

[Script Component [7437]] Error: System.FormatException: String was not recognized as a valid DateTime.
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception e)
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.ProcessInput(Int32 inputID, PipelineBuffer buffer)
   at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)

My SSIS package is running from a parent package. Child package is accessing connection values like DB Server name and DB name from configuration file. When I run package by SQL agent job in test environment it runs well. But when I run the same package in QA (ofcourse with QA server and db name in config file) it gives me error that it does not have execute permission on sp_SSIS_ADDlog. When I run the same package in my SSMS with QA server and DB and then run it from parent in QA it works. This is strange behavior.

   Can anyone please help in this ? Thanks in advance.

Hi in visual studio 2012, I had a derived column with the expression: 

"(" + SUBSTRING(Ethnicity,(FINDSTRING(Ethnicity,"(",1) + 1),(FINDSTRING(Ethnicity,")",1) - 2)) + ")"

It was working fine, and esentally it was striping out every think that was not words in parentheses to leave (xxxx). however i noticed that the length of the columns was not correct so i repointed the flat file connection manager to the csv data source and it now showed that it had pulled 5170 rows instead of the 2700 out from the csv, however a error is now occurring at derived column 2, and the error message is not that helpful when your a noob at SSIS

[Derived Column 2 [11]] Error: An error occurred while evaluating the function.

[Derived Column 2 [11]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "Derived Column 2" failed because error code 0xC0049067 occurred, and the error row disposition on "Derived Column 2.Inputs[Derived Column Input].Columns[ReadCode]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Derived Column 2" (11) failed with error code 0xC0209029 while processing input "Derived Column Input" (12). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

Can anyone help?

Hello,

For a particular SSIS 2008 R2 ETL solution, we distinguish four environments:

1. Development
2. Test
3. User Acceptance Test
4. Production

We use the configuration approach described in 'Defining a Configuration Approach for Integration Services Packages' on MSDN, in particular theSQL Server Approach: At the server level an environment variable is defined which contains the connection string to the SQL Server configuration database. Every package configuration starts with the setting of the connectionstring of the connection manager of the configuration database using the environment variable followed by the necessary SQL Server Table configurations.

Now this approach works fine, except for the User Acceptance Test- and Production-environment. Point is thatthese environments reside on the same Windows server, i.e. share the same environment variable. Both environments have their own databasebase-server with an own configuration database. Packages reside on the file system and both environments have their own package location and SQL Server Agent Job which kicks off the main package with the dtexec utility. I quess you can now anticipate my problem/question: At startup (i.e. configuration phase) the main package references the environment variable which contains of course the connection string of just 1 configuration database (the one for User Acceptance Test or Production): the main package must decide which configuration database to use and the only clue it has is it's own execution location.  I can think of some workarounds and in practice our DBA has defined two environment variables (one for each environment/configuration database) and after each deployment of a new release manually update the XML of the packages (i.e. search/replace the name of the environment variable), not very nice. I'm a little bit puzzled because this seems to me a pretty common scenario (i.e. two ETL environments on the same machine) but I cannot find a clear solution
for the configuration of these environment.

Any suggestions?

Thanks very much,

Gert Broekhaar
RIVM, The Netherlands

<account>123456
<name>John SMITH</name>
<adress>Something
<street>Dummy street
<number>5</number>
</street>
<city> New York
<postCode>12345</postCode>
</city>
</adress>
</account>

Hi All,

I have migrated few packages which contains script component from 2005 to 2008 .

Everything else is working just fine except the script component.

Following is the code and error from one of the script component.

read olny var : User::ODSConnectionString,System::PackageName

read write :User::loadExecutionID,User::SQLConnectionString

CODE

------

' Microsoft SQL Server Integration Services user script component
 ' This is your new script component in Microsoft Visual Basic .NET
 ' ScriptMain is the entrypoint class for script components

 Imports System
 Imports System.Data
 Imports System.Math
 Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
 Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
 Imports System.Data.SqlClient


 <Microsoft.SqlServer.Dts.Pipeline.DtsPipelineComponent()> _<CLSCompliant(False)> _ <Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute> _<CLSCompliant(False)> _
 Public Class ScriptMain
     Inherits UserComponent

     Dim pckName As String
     Dim ODSConnectionString, SQLConnectionString As String
     Dim sqlConn As SqlConnection
     Dim sqlReader As SqlDataReader
     Dim sqlComm, updComm As SqlCommand
     Dim updParam As SqlParameter
     Dim loadExecID As Integer
     Dim logMsg As String

     Public Overrides Sub PreExecute()
         Try
             pckName = Me.Variables.PackageName
             ODSConnectionString = Me.Variables.ODSConnectionString
             Dim connStr() As String = ODSConnectionString.Split(CType(";", Char))
             Dim i, cnt As Integer
             Dim chkQuery, updQuery, selQuery As String

             'Removing the "Provider=SQLOLEDB" clause as it is not required in 
             'SqlClient connection strings
             cnt = 0
             For i = 0 To connStr.Length - 1
                 Dim elements() As String = connStr(i).Split(CType("=", Char))
                 If elements(0) <> "Provider" Then
                     If cnt <> 0 Then
                         SQLConnectionString += ";"
                     End If
                     SQLConnectionString += connStr(i)
                     cnt = cnt + 1
                 End If
             Next

             sqlConn = New SqlConnection(SQLConnectionString)
             sqlConn.Open()

             chkQuery = "SELECT LoadExecutionID, Status from LoadExecutionDetail " & _" WHERE LoadExecutionID = " & _" (SELECT MAX(LoadExecutionID) from LoadExecutionDetail)"

             updQuery = "UPDATE LoadExecutionDetail SET STATUS = 'EXECUTING' " & _" WHERE LoadExecutionID = @LoadexecutionID"

             sqlComm = New SqlCommand(chkQuery, sqlConn)
             sqlReader = sqlComm.ExecuteReader
             sqlReader.Read()

             loadExecID = CType(sqlReader(0).ToString, Integer)
             If sqlReader(1).ToString.ToUpper = "FAILURE" Then
                 sqlReader.Close()
                 updComm = New SqlCommand(updQuery, sqlConn)
                 updParam = New SqlParameter("@LoadExecutionID", SqlDbType.Int)
                 updComm.Parameters.Add(updParam)

                 updComm.Parameters("@LoadExecutionID").Value = loadExecID
                 updComm.ExecuteNonQuery()
             ElseIf sqlReader(1).ToString.ToUpper = "EXECUTING" Then
                 'No need to do anything as loadExecID has been assigned with the correct value
             Else
                 'Fire an error as the status for the current run is not EXECUTING or FAILURE
                 Dim specMsg As String = "INVALID Status FOR MAX(LoadExecutionID) in LoadExecutionDetail"
                 Dim locStr1 As String = Me.ComponentMetaData.Name & " => " & Me.GetType.Name & " => " _& System.Reflection.MethodInfo.GetCurrentMethod.ToString
                 logMsg = pckName & ":Failed in " & locStr1 & ":" & specMsg

                 Me.ComponentMetaData.FireError(0, "", logMsg, "", 0, True)
             End If
         Catch ex As Exception
             Dim locStr1 As String = Me.ComponentMetaData.Name & " => " & Me.GetType.Name & " => " _& System.Reflection.MethodInfo.GetCurrentMethod.ToString
             logMsg = pckName & ":Failed in " & locStr1 & ":" & ex.Message

             Me.ComponentMetaData.FireError(0, "", logMsg, "", 0, True)
         End Try
     End Sub

     Public Overrides Sub myInput_ProcessInputRow(ByVal Row As MyInputBuffer)
         '
         ' Add your code here
         '

         Row.LoadExecutionID = loadExecID
     End Sub

     Public Overrides Sub PostExecute()
         Me.Variables.loadExecutionID = CStr(loadExecID)
         Me.Variables.SQLConnectionString = SQLConnectionString
     End Sub

     Public Overrides Sub ReleaseConnections()
         sqlConn.Close()
     End Sub

 End Class

----------------------------------------------------------------------- end of code

The error is pointed to 

<Microsoft.SqlServer.Dts.Pipeline.DtsPipelineComponent()> _
<CLSCompliant(False)> _
<Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute> _
<CLSCompliant(False)> _ 

which says   "Attribute 'CLSCompliantAttribute' cannot be applied multiple times'

When I remove one of the above  <CLSCompliant(False)> _ the code build shows success but once I save it and reopen script component its not even opening throwing an error "object reference not set to an instance of an object"

Full Error Message

------------------------

Package Validation Error (Package Validation Error)

===================================

Error at df_DepositStage [sc_get_LoadExecutionID [16]]: Microsoft.SqlServer.Dts.Pipeline.CannotCreateUserComponentException: Cannot create user component class. Make sure there is one class marked with SSISScriptComponentEntryPointAttribute in your script.
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.CreateUserScriptInstance()
Error at df_DepositStage [sc_get_LoadExecutionID [16]]: Microsoft.SqlServer.Dts.Pipeline.CannotCreateUserComponentException: Cannot create user component class. Make sure there is one class marked with SSISScriptComponentEntryPointAttribute in your script.
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.CreateUserScriptInstance()
Error at df_DepositStage [sc_get_LoadExecutionID [16]]: The binary code for the script is not found. Please open the script in the designer by clicking Edit Script button and make sure it builds successfully.
Error at df_DepositStage [SSIS.Pipeline]: "component "sc_get_LoadExecutionID" (16)" failed validation and returned validation status "VS_ISBROKEN".

Error at df_DepositStage [SSIS.Pipeline]: One or more component failed validation.

Error at df_DepositStage: There were errors during task validation.

 (Microsoft.DataTransformationServices.VsIntegration)

------------------------------
Program Location:

   at Microsoft.DataTransformationServices.Project.DataTransformationsPackageDebugger.ValidateAndRunDebugger(Int32 flags, DataWarehouseProjectManager manager, IOutputWindow outputWindow, DataTransformationsProjectConfigurationOptions options)
   at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchDtsPackage(Int32 launchOptions, ProjectItem startupProjItem, DataTransformationsProjectConfigurationOptions options)
   at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchActivePackage(Int32 launchOptions)
   at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.LaunchDtsPackage(Int32 launchOptions, DataTransformationsProjectConfigurationOptions options)
   at Microsoft.DataTransformationServices.Project.DataTransformationsProjectDebugger.Launch(Int32 launchOptions, DataTransformationsProjectConfigurationOptions options)

I am not great with vb script ... ANY HELP IS HIGHLY APPRECIATED ..stuck big time on this .

Please let me know in case any further detail is required.

Thanks and Regards

Praveen Uppath

Hi !

We have develop and deployed proj using Project Deployment which has 100 packages.

After changes to one of the package , i need to deploy that particular package alone to the Folder in Project deployment model?

Is it possible to do this ?

Regards,

Hi

I need to export data from sql server table to excel sheet using SSIS. But my package stops because the excel sheet gets full. And I need to use another excel sheet for rest of the data. 

How can I do that? or Is there work around to achieve that goal.

Hi,

I am trying to inserting rows into a table based on Column.

Id     Name       Data

N105.1.1 http://www.example.com/New/105.1.1

U105.1.2               http://www.example.com/Used/105.1.2

S105.1.3               http://www.example.com/Sold/105.1.3

I want a table like this. 

I want to insert Data column value based on Id and Name Columns

If Id = N and Name = 105.1.1 then Data value should be insert as   http://www.example.com/New/105.1.1

If Id = U and Name = 105.1.2 then Data value should be  insert as  http://www.example.com/Used/105.1.2

If Id = S and Name = 105.1.3 then Data value should be insert as   http://www.example.com/Sold/105.1.3

Can you please help us to write query to insert above Data Column values based on Id and Name Columns.

I am using the following query in Execute SQL Task

SELECT CASE(SELECT COUNT(*) FROM STG_APJ_APIW_DETAILS
WHERE source_filename = ?
AND lTRIM(RTRIM([BILLING DATE])) <> '') WHEN 0 THEN (SELECT CONVERT(VARCHAR(8), GETDATE(), 112) AS [YYYYMMDD])
ELSE
(SELECT TOP 1 [BILLING DATE] FROM STG_APJ_APIW_DETAILS
WHERE source_filename = ?
AND lTRIM(RTRIM([BILLING DATE])) <> '')
END

In parameter Mapping i have created two parameters and in result set i choose my variable.

Result set property is Single Row.

Following is the error message i received,

[Execute SQL Task] Error: Executing the query "SELECT CASE(SELECT COUNT(*) FROM STG_APJ_APIW_DETA..." failed with the following error: "Syntax error, permission violation, or other nonspecific error". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

Sarvan

Does anyone know of a utility or process in SSIS Data Tools using Visual Studio 2012 when working with .csv flat file data source where I can filter out rows based on columns that have invalid characters? So, basically I want to skip rows that  would otherwise make the load stop or bomb and then send the physical data that is incorrect to a file location  that the source developer can troubleshoot and correct?

Another way to say this is;

I’m loading data into a SQL Server table from multiple .csv flat file sources and the problem is that there are handfuls of rows where the delimiter is getting thrown off due to strings that have invalid characters. Can I skip those, throw them into another .txt file and continue only loading the rows that are correct?

I know with a fuzzy look-up I could work with tables but not sure how to do this using a flat file .csv source? Any ideas? 

Thank you

I've searched through the related posts but cannot find exactly what I'm looking for.

I have an OLE DB Source that produces multiple rows (around 200) through a stored procedure. I need to take 3 values in each row and pass them to an OLE DB Command that runs another stored procedure for each row.

What I need is to get the output of the second stored procedure. I've read posts regarding using lookup, merge join, derived column but I still don't understand exactly how to put it all together. My destination is an excel file. Once I've obtained this output, I believe I can figure out how to merge the OLE DB Source results and the OLE DB Command results into one table.  The output columns of the second SP are not available to be mapped to my destination file.

I've also explored running an Execute SQL task and passing the rowset to a variable but don't know how to pass that variable into the next stored procedure and then capture the second stored procedure results.

OLE DB Source contains Stored Procedure X:

DECLARE@return_value int
declare @newdate smalldatetime
set @newdate = convert(smalldatetime,convert(varchar(10),getdate(),101))

EXEC@return_value = [dbo].[getnumbers]
@dtAfterDate = @newdate,
@dtBeforeDate = '2099-12-31 5:00:00',
@Status = 'Any'

NEXT IN THE PIPELINE

OLE DB Command contains Stored Procedure Y:

EXEC[dbo].[getaddress] ?,?,?

One more thing, CREATING A TEMP TABLE ISN'T AN OPTION!

Thanks!

I'm trying to create a SSIS package to connect to an IBM iSeries DB2 database.  After running into all sorts of issues seemingly related to data type (issues with packed decimal fields actually having places to the right of the decimal) I decided to try creating an ODBC data source on my windows server 2008 r2 Administrative Tools - Data Sources ODBC to connect to the iSeries and then create an SSIS connection manager to use the windows ODBC connection.  Successfully created the ODBC connection in windows then successfully added a SSIS connection manger to my project using the windows ODBC connection and tested the connection manager successfully.  To create the connection manager I RMB in the connection manager section - select new connection - select ODBC - select the newly created ODBC connection from the list and operate the ok button.  Edited the connection manager specified the userid and pw and connected succcessfully.

Problem is I don't seem to be able to find a Data Flow Source in the SSIS tool box that will let me specify the connection manager using the ODBC connection.  The Data Flow Sources in my tool box are "Pointer", "ADO NET Source","Excel Source", "Flat File Source", "OLE DB Source", "Raw File Source" and "XML Source".

Can someone kindly show me what Data Flow Source I should be using and how to specify my SSIS connection manager that uses the windows ODBC connection?

Thanks Much in advance for any help, Roscoe

Hi

I use a parameter P1 in a package

This package contains also a foreach loop container including a parameter P2 (P1 is on outside of the foreach loop container)

In this foreach loop container, I need to insert into a table T1 the parameter P1 and P2 from an execute sql task :

INSERT INTO T1 (F1, F2) VALUES (?,?)

My question : how to access to the P1 parameter in the container ?

Thx for your help

I have an SSIS package I created and ran successfully, locally on my desktop.  I then copied the package to the Integration Server, running SQL 2012 and imported the package.  The import worked fine but when I go to execute the package I get the following error:

Failed to acquire connection "MyConnection".  Connection may not be configured correctly or you may not have the right permissions on this connection.

The Package Execution Progress window says that the package is attempting to configure from the XML file "MyConfig.dtsConfig" 

I attempted to access the database server that the SSIS Package is trying to access from my integration server, with the credentials that are in my config file and I was able to successfully connect.  

Can someone point me in the right direction on how to start trying to solve this issue?

Thanks

Bob

I am working in an environment where I have limited permissions in production (no complaints).  I was poking around the SSISDB and was able to view the list of views, which surprised me.  I think what surprised me more is that when I run a query against catalog.executions, I don't get any errors.  But I don't get any data returned either.

It appears to me that I do have access to SSISDB and select permission.  I suspect that they have set the logging parameter to no logging, which is not a good idea, especially not in our specific situation (long story).

Before I bug my dbas for job scripts and all, I wanted to check whether there would be a situation where I would have permissions to the SSISDB and to select on the catalog.executions view and not see information about the executions.

I do have one additional theory: if I don't have permissions to the project, maybe I won't have permission to see the execution log for the project.

Russel Loski, MCT, MCSE Data Platform/Business Intelligence. Twitter: @sqlmovers; blog: www.sqlmovers.com