Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

How to use a script transformation component to divide buffer data into multiple tabs of the same excel file and download it

November 8, 2019, 4:40 pm
$
0
0

I am trying to get the buffer data into an excel file which will multiple tabs. So for example, if the buffer day has employee data with one of the field/column being the city, I would want individual tabs for each city and the employee information in it. So the tab Boston would have all the employees living in Boston. The column "City" can have any number of cities, so this is a dynamic field.

The approach I want to take is first all loop through the values in the "City" column and save the unique values to an array. Once we get the unique values loop through the data again and save the rows to its corresponding cites. I am new to C# language, but any pointers would be greatly appreciated.

Subham

↧
Search

How to execute data flow task based on multiple input data flow from Expression Task.

November 8, 2019, 4:47 pm
$
0
0

Hi

Is it possible to execute the data flow task based on multiple Expression task. The expression task contains the source query variablevar_srcquery and a dynamic query assigned here  at runtime. There is 4 Expression Task all assigning to the same variable var_srcquery based on the condition in the precedence constraint. At a time only one Expression task will execute and go to the data flow task as shown in the picture.

When i execute the package it correctly building the dynamic query and getting assigned to the variable var_srcquery but it is not going to the Data flow task.

Is the Data flow task accepts only 1 input flow.

is there any other way to handle this, please let me know.

Thanks


DD

↧
↧

load Dimesion having auto generated key and fact table in a single flow without lookup in ssis package

November 9, 2019, 9:38 am
$
0
0

I have a requirement like this:Needs to load 3 dimension table  & 1 fact table from a single flat file.Dimension table contain auto generated surrogate key in sql server. How to load all 3  dimension table  and then save their auto generated key into variables.These key will use while loading fact table.Can we do all these operation in single package without doing lookup ?

many many thanks in advance.

↧

Importing a sensitive data

November 10, 2019, 5:40 am
$
0
0
Hello, I am new to SSIS and I  want to import data from the SQL database to a flat-file.
I have a column name named 'SecurityNumber' which contains sensitive data, so using a derived column I want to use a symbol * to hide some numbers.
For example, if the column has a value 1983121802 and
if substring 3 plus substring 4( in this case 8+3) is less than 13 display the column as  **83****** or else display the whole value(1983121802) 
how can I do that? 
and the column SecurityNumber has a string data type, How can I change it to another data type to make the comparison?
Thank you for your help!
↧

VS2019 SSIS TargetVersion SQL 2019 - ADF IR Runtime Error Data Flow Script Component Not Compatible Mismatch

November 10, 2019, 9:58 am
$
0
0

I've created an SSIS package in VS2019 with both SSIS Azure Feature Packs installed for 2017 and 2019. I initially used TargetVersion = SQL Server 2017 in the project build properties and was able to deploy the package to the Azure Data Factory and run it successfully with a Data Flow task that has a Script Component in it. I also had various tasks included for file upload/download from an Azure Blob Store that were running successfully. I changed the TargetVersion = SQL Server 2019 on the project so that I could use the new Delete option on the Flexible File Task with the Azure Blob Store. The package runs successfully when executing locally on my machine in VS2019, however, when I deploy to the ADF IR, I get the following runtime error message. If I change TargetVersion back to SQL Server 2017 then the package will run again in the ADF IR but I can no longer use the Delete option that is needed. Is there anything I can do to resolve the Script Component error?

OperationErrorMessages": "11/10/2019 2:10:42 AM +00:00 : Data Flow Task Processing:Error: Microsoft.SqlServer.Dts.Pipeline.ComponentVersionMismatchException: The version of Script Component is not compatible with this version of the DataFlow.  [[The version or pipeline version or both for the specified component is higher than the current version.  This package was probably created on a new version of DTS or the component than is installed on the current PC.]]\r\n   at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostCheckAndPerformUpgrade(IDTSManagedComponentWrapper100 wrapper, Int32 lPipelineVersion)\n11/10/2019 2:10:42 AM +00:00 : Data Flow Task Processing:Error: The component metadata for \"Script Component\" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.\r\n\n11/10/2019 2:10:43 AM +00:00 : Data Flow Task Processing:Error: The component metadata for \"Script Component, clsid {8DC69D45-2AD5-40C6-AAEC-25722F92D6FC}\" could not be upgraded to the newer version of the component. The PerformUpgrade method failed.\r\n\n"

↧
↧

For Each Loop

November 10, 2019, 1:38 pm
$
0
0
I am new to SSIS and I am trying to import data from SQL database table to a flat-file, I want to use For each loop container.
I have a column named Values which contains,
20
21
22
23
24
29
20
21
22
23
24
27
20
23
24
29
and what I wanted to do is, using the  For each loop container,  to loop through the table(column values), and from the first value 20 till the next 20 to rename the values to 1, and from the second 20 to the third 20 to rename the values to 2, from the third 20 to the next 20 to rename the values to 3...
How can I do that?
Thank you for your help!
↧

SSIS Log Provider for SQL Server Logging Enabled on a Package using a SQL Server Connection. when I convert this Package connection to Project connection, throws an Error

November 7, 2019, 4:44 pm
$
0
0

Hi All,

Today, I faced an issue with existing project and Packages. 

  1. Have a Connection Called DW_Manager to Connect to SQL Server DB: DWMANAGER to capture log using SSIS Logging (Basically It will create a table [dbo].[sysssislog] under System Tables in Database: DWMANAGER ).Note: This is a package Connection and not a Project Connection. 
  2. Logging Enabled for a package(Go to Menu--> SSIS-->Logging -->Providers and Logs-->Provider Type-->SSIS Log Provider for SQL Server-->Name: SSIS Log Provider for SQL Server1-->Description: Writes log entries for events to a SQL Server database-->Configuration: DW_Manager) https://www.sqlshack.com/overview-of-ssis-package-logging/
  3. Have a Project Parameter called PP_DW_Manager and provide connection details here: Data Source=Server1;Initial Catalog=DWMANAGER;Provider=SQLNCLI11.1;Integrated Security=SSPI;Auto Translate=False;
  4. Open a Package-->Select Connection DW_Manager-->Properties-->Expression-->False (This means It is not taking value from Project parameter)

If I deploy this package, It works fine (loads data). 

  1. The Time, I convert DW_MANAGER from Package connection to Project Connection, My Package starts throwing below error message but it loads data
  2. The Time, I convert DW_MANAGER, to use Project Parameter (Open a Package-->Select Connection DW_Manager-->Properties-->Expression-->True-->Property Expression Editor-->Property: ConnectionString-->Expression: @[$Project::PP_DW_Manager]), It starts failing (Don't Load any data. It fails at validation If a run job through SQL Agent JOB. If I run this package from BIDS It works with below error message). 

If I convert connection DW_MANAGER to Project Connection still It works and loads data though it throws and Error Message: 

Error: The connection "DW_MANAGER" is not found. This error is thrown by Connections collection when the specific connection element is not found.


Conclusion: If a "SSIS Log Provider for SQL Server logging" is enabled on a package then connection used under configuration should be a package connection and can't be a Project Connection. The time We convert connection DW_Manager from package connection to Project Connection We get above error message. and Time we use Project parameter for DW_Manager connection, then it fails validation SQL Server Agent Job But If same package I Run on BIDS, It loads data with above error Message. 

Not sure this is expected behavior ? Or this is a bug?

Guys, Please correct me If am wrong. We are using SSIS 2012.

I found couple of threads but things are not clear.

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/91394766-c56f-492f-b62a-40aeeb8bdfeb/sql-2012-ssis-log-provider-the-connection-quotquot-is-not-found-this-error-is-thrown-by?forum=sqlintegrationservices

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/2af470ac-d76d-4162-849f-a5e0411aa053/the-connection-quotquot-is-not-found-this-error-is-thrown-by-connections-collection-when-the?forum=sqlintegrationservices

  Logging:

https://www.sqlshack.com/overview-of-ssis-package-logging/

Thanks Shiven:) If Answer is Helpful, Please Vote





↧

SSDT install issue with SSIS featured enabled on vc2019 version.

November 8, 2019, 12:43 am
$
0
0
SSDT-setup-ENU .exe   installation  is not successful for  SSIS option checked  for  - 2019  visual studio tools  on windows 10 OS,   How  SSIS  feature got installed  in this scenario?
↧
Search

Visual Studio 2019 nothing happens when I hit "Edit Script..."

November 3, 2019, 8:37 pm
$
0
0

Hello, 

I am using Visual Studio 2019 with new SSIS addon.  I cannot launch a new instance of Visual Studio when I hit "Edit Script..."  in Script Task.
ScriptLanguage shows Microsoft Visual C# 2017 which I do not have installed and it's strange because in Tools -> Options -...  Integration Services Designer under Script language shows Microsoft Visual C# 2019 . 

Anyone else figured this out or it's official you cannot use  Script Tasks in Visual Studio 2019  just yet. 

Thanks

↧
↧

Format a written Variable from a script task SSIS 2010

November 8, 2019, 12:35 am
$
0
0

Hi

I have a script task in SSIS 2010 - Not very good at c# script task

This will write the modified date of the excel file into a variable - [User::LastFileDate]

This is the line in the script task

 Dts.Variables["User::LastFileDate"].Value = theFile.LastWriteTime;

I just want the format to be like with a rounded timestamp 06/11/2019 00:00:00 and not 06/11/2019 09:45:01 - I would like please

06/11/2019 00:00:00

I have tried to change it to something like below with no joy, I get an error where the ; =

((

DateTime)Dts.Variables["User::LastFileDate"].Value).ToString("dd/mm/yyyy"); = theFile.LastWriteTime;

I have also tried with no joy

theFile.LastWriteTime = (

DateTime) Dts.Variables["User::LastFileDate"].Value

this is a copy of the script so far 

 

publicvoidMain()

        {

            System.IO.

FileInfotheFile =

                 

newSystem.IO.FileInfo(Dts.Variables["User::FileExcel"].Value.ToString());

           

if(theFile.Exists)

            {

               

//Dts.Variables["User::LastFileDate"].Value = theFile.LastWriteTime;


            theFile.LastWriteTime = (

DateTime) Dts.Variables["User::LastFileDate"].Value ;

string

strMessage = Dts.Variables["User::LastFileDate"].Value.ToString();

           

MessageBox.Show(strMessage);

// ((DateTime)Dts.Variables["User::LastFileDate"].Value).ToString("dd/mm/yyyy");


            }

            Dts.TaskResult = (

int)ScriptResults.Success;

}

I have also got the script task to ouput a message box and it says file was modified 30/12/1899 00:00:00 ?? when it should be 06/11/2019 14:30:21

Please help team




↧

Use Environment variable in the package path of 'Execute Package Task'

November 8, 2019, 5:01 am
$
0
0

Hi,

I am with a problem, in the development environment my packages are stored in one path, and in the production environment, my packages are stored in another path. My project is configured as 'Package Deployment Model' instead of 'Project Deployment Model'.
I want to simulate a relative path for my problem, so I want to use an environment variable that will have different values for package path in development/production.

path: VARIABLE/PackageName.dtsx

Is it possible what I want? If not possible, what I can do in Package Deployment Model to solve this problem?
I know that the environment variable isn't the best approach, because I will not be able to have two versions of SSIS configured at the same time in my machine. The best thing to me would be to be independent of path configuration/customization; a default approach. But the Environment variable was the only thing that I thought.

Best Regards,
Luis.


↧

SSIS - SFTP in-built task

November 11, 2019, 5:25 am
$
0
0

Hi,

I have been using a ZappySys third-party functionality to connect to SFTP. However, the ZappySys licensing is not being renewed and hence we are replacing the ZappySys components with native SSIS components.

I have read a lot of articles where there is no mention of native SFTP component in SSIS but all these articles are kind of old. Hence, wanted to check if this feature has been introduced or if there is any alternate method to achieve this without the use of a third-party functionality.

Thanks,

Dev

↧

can i read a git repository file in an ssis pkg

November 11, 2019, 5:42 am
$
0
0

Hi we run 2017 std. We may have a need to read the latest version of a file from a remote git repo.

The best I can tell, a file task cant open such a repo as a source in a file copy operation.   For grins I entered much of the url path in a web service task's connection "server url" box but got a 401 (unauthorized) error return code when trying to connect.

If memory serves, one could retrieve a file without coding a .net script in ssis from tfs.  But maybe I'm wrong.  Is there a way to read a file into the ssis buffer from a git repository?

   
↧
↧

SSDT 2017 - Odata Source - Cannot acquire a managed connection

November 11, 2019, 6:32 am
$
0
0

Hello everyone.

I am at the end of my tether trying to get this to work.

I require to connect to an external OData source from SSDT for Visual Studio 2017 and SQL Server 2016

The OData connector comes pre-installed with this version.

No matter what I do, I get the dreaded

Error: 0xC020801F at Data Flow Task, OData Source [2]: Cannot acquire a managed connection from the run-time connection manager.
Error: 0xC0047017 at Data Flow Task, SSIS.Pipeline: OData Source failed validation and returned error code 0xC020801F.

I have scoured I think every website available and still I cannot get the answer.

This is to execute the task through SSDT.  I have tried against the OData source I need to connect to and to a number of Northwind Test sources from Microsoft to ensure that it is not specifically my source.

As you can see, the data previews ok but when I execute it...


I have read that should install 32bit Odata Driver.  I found this from SQL Server 2014 Feature pack but although installing it makes no difference.  There is no way to tell the OData source component in SSDT to use it as the UserComponentTypeName is Microsoft.SqlServer.IntegrationServices.OData.ODataSource, Microsoft.SqlServer.IntegrationServices.ODataSrc, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 
This is the inbuilt version I think, does not appear to be any way to change this.

I have tried this from home so as to rule out corporate firewall issues, still the same.

I have tried installing earlier versions of SSDT for VS (2015 and 2013) still with no luck  (Same error on 2015 and some other error on 2013).

I really need to be able to use OData as a source for populating a Data Warehouse.  

Can anyone suggest something please, I have spent hours trying to get this working.  I'll be your best friend if you can figure out the solution! :-)

Thanks, 

Phil

↧

Abnormal Database Size Increase when SSIS Runs

November 11, 2019, 7:11 am
$
0
0

Guys,

I needed to convert a working DTS package from SQL Server 2008 R2 into SSIS SQL Server 2014 R2, the process goes through some tables and creates, updates and deletes some records. In SQL Server 2008 R2 was running without issues.

once brought over and updated to Server 2014, I noticed every 10 days the process of my Windows Server 2012 R2 would hang at 100% cpu utilization every time the job runs and it goes back down to 5% once done.

Today, I found the database size was 1.3GB! and the CPU maxed out at 100% when the process runs (for about 10 minutes) . I did DB shrink and the database size went to 93MB from 1.3GB and the CPU doesn't even go beyond 30% when the process runs and it takes less than 2 minutes !

very weird, I believe its some settings in the SQL Server since this process was working without issues in the previous server.

The process itself reads a text file and import those records from the text file into a database table then it does all the functions mentioned bove.

Any idea as where i should be looking? I don't want to keep shrinking the database. Thanks

↧
Search

Load logo form excel sheet

November 11, 2019, 11:13 am
$
0
0

Hello ,

I am not sure if this can be done or not. Is there a way we can load a logo from one excel source to an excel destination. The logo changes each time. The top portion is a logo with data following the consequent cells. Thanks in advance.

↧

TargetServerVersion node cannot be found in custom default persistence.

November 11, 2019, 12:22 pm
$
0
0

Hey folks,

I am trying to diagnose a SSIS 2012 package that my predecessor built and has running on a SQL 2016 instance. I'm starting to see the following errors starting a few weeks ago:

TargetServerVersion node cannot be found in custom default persistence.

I can run this project locally off my desktop without issue but whenever I deploy to the  SQL Server instance it errors out. What am I missing here? I've tried rebuilding the package and project from scratch but I keep encountering the same error. 

SQL Instance Version: 13.05081.1

SSIS Package Version: Created with visual studio 2012. 

↧
↧

Increase length in Azure Blob Source column

November 11, 2019, 3:33 pm
$
0
0

I stored a CSV file in an Azure blob that contains 30 columns, 5 of them are 600 length and I need to upload them into a SQL Database.  Is it possible to change the Length +255 of a column in an Azure Blob Source?

When I go into Advanced Editor \ Input and output Properties and update the Length in Data type Properties I get the following error:

Error at Data Flow Task [Azure Blob Source [1]]: The mycolumnname does not allow setting external metadata column datatype properties.

If its not possible, whats the work around?

↧

SSIS Dataviewer in VS2019 is not working

October 25, 2019, 12:11 am
$
0
0

Hi all,

the Data Viewer in SSIS with Visial Studio 2019 is not working as used. You can see it, but the content of data viewer window is empty...

olaf870

↧

Getting an Error like Entity Id must be specified for Update ?

October 14, 2014, 4:02 am
$
0
0

Hi all,

I am getting an error like "Entity Id must be specified for Update" when try to update the entity in CRM 2013 from SSIS Script Transformation. Could you please help us in this regards.

Thanks & Regards, Anil

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>