Hi all, 

I am trying to deploy an SSIS project with SQL server authentication, using powershell to a virtual machine.

As far as I am aware, SSIS deployment is possible with SQL Server authentication. Do I need to change something on the server side?

Script:

$ServerName = "XX.XX.XX.XX"
$UserName = "test"
$UserPassword = "test"

$SSISCatalog = "SSISDB"
$ProjectFilePath = "C:\ReleaseProcess\Test.ispac"
$ProjectName = "Test"
$FolderName = "Test"

# Load the IntegrationServices Assembly
[Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.IntegrationServices")

# Store the IntegrationServices Assembly namespace to avoid typing it every time
$ISNamespace = "Microsoft.SqlServer.Management.IntegrationServices"

Write-Host "Connecting to server ..."

# Create a connection to the server

$sqlConnectionString = "Data Source=$ServerName;Initial Catalog=master;User ID=$UserName;Password=$UserPassword;TrustServerCertificate=True;"
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection $sqlConnectionString

Write-Host "Connected"
$integrationServices = New-Object "$ISNamespace.IntegrationServices" $sqlConnection
$catalog = $integrationServices.Catalogs["SSISDB"]
$folder = $catalog.Folders[$FolderName]

# Read the project file, and deploy it to the folder
Write-Host "Deploying Project ..."
[byte[]] $projectFile = [System.IO.File]::ReadAllBytes($ProjectFilePath)
$folder.DeployProject($ProjectName, $projectFile)

So I get the below error: 

Exception calling "DeployProject" with "2" argument(s): "The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an 
account that uses Integrated Authentication."
At line:38 char:1+ $folder.DeployProject($ProjectName, $projectFile)+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+ CategoryInfo          : NotSpecified: (:) [], MethodInvocationException+ FullyQualifiedErrorId : SqlException

Carrier

Project Overview

Loading data from Oracle Database to Datawarehouse Staging Database(SQL Server). Using ETL process we are loading data from Oracle to SQL Server. Staging Database in SQL server is an exact replica of Source Oracle schema.

We have developed an SSIS package to initially load complete data into Staging Database and then apply data cleaning and load the data to Data Warehouse(SQL Server). In SSIS Package for migrating data from Source to Staging, we are using "Data Flow Task" in SSIS package and we are splitting the table data into 20 batches and loading batch-wise into Stage database in a sequential order

Issue:

We will build the .ispac file and deploy in PROD Server scheduled a package execution in SQL server agent. After package complete execution. The count of the source tables is not matching with the stage tables.

But when we are running the same package in the local machine the count of source tables are matching with stage tables.

As we are loading the data in Batch-Wise. In some cases, we are missing a few records in every batch or only a few batches. But we are not sure why the records are missing while transferring the data only in PROD Server

Hi we run 2017 standard. I see this more and more often and wonder.

Lets say that for a given pkg max parallel dft's is 7 either by default or explicit setting.  Not worrying about engine threads right now.   

what I see people doing is layering containers such that no container runs more than 7 dfts in parallel.  And only after a container is done can the next container start running.

I wonder if layering is really necessary given that ssis will do what it can when it can and therefore its overkill trying to help it decide.  Perhaps just putting 14 things in parallel without extra containers is fine since ssis knows already what it can and cant do. 

Hello,

I would like to import an excel spreadsheet into an existing table 'a' in SSMS using SSIS.

But I need to firstly copy the current contents of 'a' an append to an archive table 'b' with the current date in the date field, then clear the contents of 'a' before importing the contents of the spreadsheet into 'a'.

Could anyone help?

Thank you.

Andy.

Hi we run 2017 standard and are at a crossroads where an older etl product will be replaced by ssis and we want to set strategic direction.

what are we giving up if we opt to use the file system rather than the catalog as our package repository?

Hi,

I have the SSRS report and the subscription runs daily @ 1 am US = 2 PM China and 7PM China = 8 am US.

I have the problem with the subscription runs @ 7PM because the today's order show the current day here, but next day there.

I am thinking creating the another rdl file to get the reportdate = getdate() +1, but I don't want to do this way.  I would like to know if any other way to set up from the subscription to let the report runs one date a head from here?

Thanks

Hello,

We have an SSIS 2012 package that imports flat files in a ForEach loop.  Occasionally, the flat file connection fails because the text qualifier--a double-quote character--appears inside a text value on some rows.  How do I handle this error so that the package doesn't fail and other files present can be imported?

I already have an OnError event handler for the entire package, but I don't want it to execute.  Do I use the error precedence constraint (the red arrow) from the Flat File Source to...do something (I don't know what)?  Or do I create an OnError event handler for the data flow task that calls the Email task?

How do I keep the flat file read failure from causing the entire package to fail?

Thanks,
Eric B.

Don't understand point 5, as lesson 3 weas copied and pasted in the previous section?  When I did that a file Lesson 4.dtsx was created??

Add the completed Lesson 2 package

Open SQL Server Data Tools and open the SSIS Tutorial project.

In Solution Explorer, right-click SSIS Packages and selectAdd Existing Package.

In the Add Copy of Existing Package dialog, in Package location, selectFile system.

Select the browse (...) button, navigate to Lesson 2.dtsx on your machine, and then selectOpen.

Copy and paste the Lesson 3 package as described in steps 3-8 in the previous section.

Also, Lesson 3-2 point 9,  Containers pane??

In the Containers pane, expand all nodes of the package container hierarchy, and then clear all check boxes, including theExtract Sample Currency Data check box. Now select the check box forExtract Sample Currency Data to get only the events for this node.

What Containers pane, There isn't one.  Thought they were refering to ForEach container, but can't see anything with check boxes.

When I write instructions, which I have had to during my career, I write instructions for IDIOTS, how else can idiots become experts?  I also believe that a picture saves a thousand words.

TEH

Using SQL 2016 with projects deployed to the Catalog. We have many projects that work fine and one new one that doesn't catalog any messages except errors. The logging level on the server is set as Performance.

The package is run as a SQL Agent job step and the logging level set there is Performance.

Nick Ryan MIS Programmer Analyst, ANZ Bank

Hi All,

 I need to pull data from OAUTH2.0 using Script task and script component.I have referred many articles for the same and not able to get any help with them. if someone already worked on it could help me please.

we cannot use 3rd party paid tools

SSIS version:2014

Thanks in advance.

Hi Guys,

I'm very new to SSIS. I have one package and have an Execute SQL Task named asCall Procedure in which i'm calling a procedure. I want to log data if any error occurs. So I created an Execute SQL Task named asLogging Error SQL Task in Event Handlers for package and added expression 

"INSERT INTO [LabourFlash_OLAP].[dbo].[LoggingError]
([Package_Name]
,[Task_Name]
,[Error_Code]
,[Error_Description])
VALUES
(
'" + @[System::PackageName] + "'
,'" + @[System::TaskName] + "'
," + (DT_WSTR,12) @[System::ErrorCode] + "
,'" + @[System::ErrorDescription] + "'
)"

In Call Procedure I gave wrong procedure name so its failing this task now I want Logging Error SQL Task to execute successfully inEvent Handlers but its failing.

When I right click on Logging Error SQL Task and select Execute Task its working fine and data is inserting. 

But when I run the package then Its failing.

Can any one help me on this. 

Thanks in advance.

First some background info - we have a brand-new server setup with Win 2008 R2 and SQL 2012 Enterprise, both 64 bit.  We've upgraded all of our SQL 2008 R2 SSIS packages into SQL 2012 without any issues. 

Now we're trying to run a package and we're getting all kinds of errors. The package uses parameters in the OLE DB Source.  These packages currently run flawlessly every day in 2008 R2 Production.

When I open the package in SQL Server Data Tools I get the following errors:

Error    2    Validation error. Produce Output: Produce Output: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.  An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80004005  Description: "The metadata could not be determined because statement 'REVERT        --Check if SSB is enabled in this database' in procedure 'sp_send_dbmail' does not support metadata discovery.".      MyPackage.dtsx    0    0    

Error    3    Validation error. Produce Output: Produce Output: Unable to retrieve column information from the data source. Make sure your target table in the database is available.      MyPackage.dtsx   0    0    

Error    4    Validation error. Produce Output Data Feed Result Set [2]: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.  An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80004005  Description: "The metadata could not be determined because statement 'REVERT        --Check if SSB is enabled in this database' in procedure 'sp_send_dbmail' does not support metadata discovery.".      MyPackage.dtsx    0    0    

Error    5    Validation error. Produce Output Data Feed Result Set [2]: Unable to retrieve column information from the data source. Make sure your target table in the database is available.      MyPackage.dtsx   0    0    

I've searched and searched and can't find anything of value that will help with this issue. I created a new package and it works just fine, so it has to be something in our upgraded package.  I've confirmed the Connection Manager connection is setup properly, and the OLE DB Source returns success when I click "Parse Query".

Any help is greatly appreciated as we are at a complete halt with our SQL 2012 migration.

Thank you,

Andre

André

Hi,

I have a weird case in SSIS with an aggregation. I have a column which I aggregate on (I have sorted the data), and the aggregator removes rows that are not equal. Here you probably say - you have a case-sensitive problem...Well I guess so, but I cannot fathom how.

The column that I aggregate on all comprise of the following pattern: <6 numbers>¤<4 numbers> (Ex. 152733¤8440). So, no characters, at all. I have over 7 mio. rows in my source, and of those I 'lose' around 2.300 unique combinations, that should not have been aggregated. 

The super-weird thing is that if I tell the aggregator to be case-sensitive, then it does not remove the rows. So - how can a number + ¤ + number be affected by case? The above example (152733¤8440) is one of the rows that's remove in a non-case sensitive setup. This makes no sense to me - and I have a very hard time coming up with a way to debug which other rows cause the actual 'wrong' aggregate. 

Does anyone have suggestion as to what might be 'hitting' me here? I can, as a workaround, continue with the case-sensitive comparison. I just don't understand what happens - and I don't like the solution, as my database is case insensitive - so that should be the correct comparison. (Note: The source has the column in question as an unique key, so the SQL server does not see the rows as equal). The datatype in SSIS is (DT_STR, 100, 1252) - it's varchar(100) in the source.

Thanks in advance

I am trying to transform the existing table to different table. I am getting multiple rows as string and  want to store it under one row using comma.

I have creating a data flow task and added a script component that will take the string and convert to blob data. But I get an error for output column which is nvarchar(max) in SQL Server

 using System;
 using System.Data;
 using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
 using Microsoft.SqlServer.Dts.Runtime.Wrapper;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
 public class ScriptMain : UserComponent
{
        private string ComputerName;
        private string TimeofLastScanAttempt;
        private string CopyofRPMCustom;
        Byte buffer;
        private bool FirstRow = true;


       public override void Input0_ProcessInputRow(Input0Buffer Row)
      {

             if ((Row.ComputerName == ComputerName && 
             Row.TimeofLastScanAttempt == TimeofLastScanAttempt) || 
            FirstRow)
     {
            CopyofRPMCustom += (Row.CopyofRPMCustom_IsNull? "," : 
            Row.CopyofRPMCustom);
            ComputerName = Row.ComputerName;
            TimeofLastScanAttempt = Row.TimeofLastScanAttempt;
            FirstRow = false;
      }
       else
      {
           Output0Buffer.AddRow();
           Output0Buffer.TimeofLastScanAttempt = TimeofLastScanAttempt;
           Output0Buffer.ComputerName = ComputerName;

           Output0Buffer.RPM.AddBlobData(
           System.Text.Encoding.Unicode.GetBytes(CopyofRPMCustom ?? ""));

            CopyofRPMCustom += (Row.CopyofRPMCustom_IsNull ? "," : 
           Row.CopyofRPMCustom);
            ComputerName = Row.ComputerName;
            TimeofLastScanAttempt = Row.TimeofLastScanAttempt;

          }
       }


      public override void FinishOutputs()
    {



         Output0Buffer.AddRow();
         Output0Buffer.TimeofLastScanAttempt = TimeofLastScanAttempt;
        Output0Buffer.ComputerName = ComputerName;

         Output0Buffer.RPM.AddBlobData(
         System.Text.Encoding.Unicode.GetBytes(CopyofRPMCustom ?? ""));
        base.FinishOutputs();

    }

When I debug this code I get the error- 'Output0Buffer.RPM' threw an exception of type 'Microsoft.SqlServer.Dts.Pipeline.NoCurrentRowException' 'Output0Buffer.RPM_IsNull' threw an exception of type 'Microsoft.SqlServer.Dts.Pipeline.NoCurrentRowException'

The tasks run  successfully but the RPM column is empty. Everything else is populated.

Not sure whats going on, but every time I get to test lesson 2 in the 'Integration Services project' tutorial the program restarts.
(https://docs.microsoft.com/en-us/sql/integration-services/ssis-how-to-create-an-etl-package?view=sql-server-2017)

When it restarts it changes the ID so causes an exception. Here is the message:

TITLE: Package Validation Error
------------------------------
Package Validation Error
------------------------------
ADDITIONAL INFORMATION:
Error at Lesson 1: The connection "{991203A2-2BF5-4476-AB0D-B00A2842FCFD}" is not found. This error is thrown by Connections collection when the specific connection element is not found.
Error at ExtractSample Currency Data [SSIS.Pipeline]: Cannot find the connection manager with ID "{991203A2-2BF5-4476-AB0D-B00A2842FCFD}" in the connection manager collection due to error code 0xC0010009. That connection manager is needed by "Extract Sample Currency Data.Connections[FlatFileConnection]" in the connection manager collection of "Extract Sample Currency Data". Verify that a connection manager in the connection manager collection, Connections, has been created with that ID.
Error at ExtractSample Currency Data [SSIS.Pipeline]: Extract Sample Currency Data failed validation and returned error code 0xC004800B.
Error at ExtractSample Currency Data [SSIS.Pipeline]: One or more component failed validation.
Error at ExtractSample Currency Data: There were errors during task validation.
 (Microsoft.DataTransformationServices.VsIntegration)

I generate a new key when asked to in the instructions!

I have now restarted this project numerous times, deleting the project folder before starting again.  Is there something retained in the background that is causing this?  I am using the same project name each time, should I use a different name?

TEH

CARRIER

Good morning, all -

Sometimes, when I open up an SSIS package - admittedly, a large one with a coupla dozen tasks - it takes forever for it to validate upon opening. But, it's not consistent. There are times when it opens up in about five seconds.

Further, there are times when I've sat five or more minutes waiting for validation to finish and I've simply thrown up my hands, hit Ctrl-Alt-Del and ended the Visual Studio task. Then, as soon as it's gone, I reopen the package and it blasts through validation like it wasn't there.

What on earth may be going on?

I mean, I could understand if it always took a long time to validate - there'd be consistency. But, to vary from less than ten seconds to over five minutes? Seems like to much variance to me.

Any ideas?

Thanx in advance for any info!

Carrier