Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Slowly Changing Dimension Table in SSIS - Set End Date based on Inactive Field

June 4, 2020, 9:22 am
$
0
0

In our Source table we have an Inactive field that will dictate if the record is active or not, and I'd like to update the EndDate based on this and not based on whether or not the record exists.  I created a Data Flow with Slowly Changing Dimension with the appropriate values and modified the Derived Column for EndDate to have this:

CurrentStatus == "Inactive" ? (DT_DBTIMESTAMP)(@[System::StartTime]) : NULL(DT_DBDATE)

I'm not seeing that this is being picked up.  First run has each item going through the New Output which was to be expected, but subsequent runs aren't updating the End Date using this.  Any suggestions?  Thanks.

↧
Search

Insert and Update SSIS job now only inserts NULLS

June 4, 2020, 9:47 am
$
0
0

Hi,

   A few months ago, I created an SSIS job that takes data from an excel sheet (which is the user input sheet) and loads it into a table. I also created the job to make updates to the table whenever an existing value was changed in the excel sheet. And if any new rows were added, it would add those as well. I followed this tutorial http://www.learnmsbitutorials.net/insert-update-data-using-merge-join-ssis.php to do it. It worked very well, and the job was published to the server. Its been working fine up until a couple of days ago, where it will make updates, but any new rows inserted, only show as NULL. All the columns display NULLs. 

I tried recreating the excel sheet, thinking maybe it's to do with formatting, but that didn't work either. Not sure what to do at this point. Any ideas would be very helpful.

UMAIR AHMED

↧
↧

SSIS : Creating Variable Expressions

June 4, 2020, 12:26 pm
$
0
0

Hi Everyone,

Can you please help me in the below situation.

How can we create Variables for PreviousMonth Firstday and PreviousMonth LastDay expressions in SSIS.

If we pass the Date as 2020-02-01, variable should return 2020-01-01  as firstday and 2020-01-31 as lastday

                                 2020-01-01, variable should return 2019-12-01 as firstday and 2020-12-31 as lastday

If we dont pass any date, Default it should return the Getdate()-1 month firstday and lastday dates.

I have tried the below code but it is working only for Getdate() scenario, not for the others.

REPLACE(LEFT((DT_WSTR,20)DATEADD("d",-1,DATEADD("m",DATEDIFF("m",(DT_DATE)"1901-01-01",GETDATE()),(DT_DATE)"1901-01-01")),10),"-","-")

Thanks in advance.

↧

SSIS 2008:- Visual Studio 2008 dependency on SQL SSIS package execution

June 4, 2020, 1:18 pm
$
0
0

Hello Team,

Good afternoon. Because of security challenges, our IT team has decided to uninstall the visual studio 2008 on our server. If we uninstall the Visual studio 2008, Do we see any impact to the SSIS packages/ SSAS cube processing which are running on the server?. As per my understanding, Both are independent and it shouldnt impact each other.

Thank you,

Pradeep.

↧

How Do I See When Package Last Ran in SSIS?

March 12, 2015, 12:09 pm
$
0
0

I've searched for possible solutions, however none had worked for me (mostly queries for SQL Server). My problem is this:

I have a MAIN package that runs about 100 little packages in 1 main process.  A lot of times on the first load the packages are disabled are kept enabled.  I found some missing data that a package had inserted into SQL Server.  It could be a few different packages that did this.

What I need is to see the time and date when the packages I have pinpointed as problematic have ran.  They could be enabled or disabled.  

Is there a simple way to look somewhere in SSIS to see the last date and time these packages have ran so I can compare to the recorded creation date in one of my tables?

Thanks.

↧
↧

How to import azure table storage data to SQL database

June 4, 2020, 9:02 pm
$
0
0

Hi Team,

My team storing some transnational data in to azure table. Want to import those data into local SQL database. Could you please help on to import data to local from azure table storage.

Thanks Bala Narasimha

↧

What SSIS Task can I use in an event handler that will log to a text file - How To

June 5, 2020, 12:21 am
$
0
0

Hi

I am creating a SSIS package that will run every 3 minutes, my log files are created daily using date as its filename. Now, I would like to use event handlers such as OnError to log a message when it failed to this text file created. 

The thing, I am not sure if I should be using Script Task ( open connection to the log file..), or Execute SQL Task, write a output command to the text file or is there a specific task that I can use in SSIS that I am missing out.

Execute SQL task makes more sense if I am logging to a table inside SQL. Script task would definitely do the work , but is this the right approach. Is there some other task that I can use.

A beginner seeking help. 

↧

Import excel files in a SQL server from TEAMS

June 5, 2020, 1:09 am
$
0
0

Hi,

I need to import the excel files saved in a teams site in some sql server table (sql is on a VM on Azure, it is not a SQL Azure) with SSIS.

The teams site is something like:

https://sharepoint.com/teams/TEST/Shared%20Documents/Forms/AllItems.aspx

How can I reach the goal without using third party objects?

Many Thanks,

Simona


↧
Search

Obtain the number of columns in a flatfile connection manager

June 5, 2020, 8:56 am
$
0
0

I have an issue with importing a CSV file, whereby processing continues even if the number of input columns is incorrect. So my basic question is, how can I do that? At the moment, all that happens is the target table in the database ends up with no values but that could be a valid result so long as the number of columns is valid.

For example:

A,B,C,D,E{LF} -- valid
A,B,C,D,{LF} -- valid

A,B,C,D{LF} -- invalid

My thinking was to write a script task which reads the first line of the input file, counts the columns, compares with the number against those in the connection manager (I want the script to be dynamic) and if they don't match, throw an error. I've managed to get as far as:

DtsProperty d = Dts.Connections["flatfile"].Properties["Columns"];

but I can't work out how to get the details of the "d" object. I can see from the properties pane that it's a "System.__ComObject" but can't work out how to explore/iterate it.

So, is it possible to get the columns count from a flatfile connection manager and if so how?

If not, how can I verify that my input file has the right number of columns and, if it doesn't, error?

Thanks.

↧
↧

SSIS package getting failed when executed from batch file

June 6, 2020, 4:25 am
$
0
0

Hi, I am trying to execute a SSIS package as below content in batch file on my local computer, SQL Server 2016-

"C:\Program Files\Microsoft SQL Server\130\DTS\Binn\DTExec.exe" /F "C:\Users\Vikas\source\repos\SSIS_First_project\createNloadNewTable.dtsx" > log.txt

but getting error as logged on above log.txt file as-

=======Error: 2020-06-06 00:28:44.86
   Code: 0xC001000E
   Source: createNloadNewTable 
   Description: The connection "{523412D2-FE6B-497C-8A69-DE7B732E253F}" is not found. This error is thrown by Connections collection when the specific connection element is not found.
End Error
Error: 2020-06-06 00:28:44.86
   Code: 0xC004800B
   Source: Data Flow Task SSIS.Pipeline
   Description: Cannot find the connection manager with ID "{523412D2-FE6B-497C-8A69-DE7B732E253F}" in the connection manager collection due to error code 0xC0010009. That connection manager is needed by "OLE DB Destination.Connections[OleDbConnection]" in the connection manager collection of "OLE DB Destination". Verify that a connection manager in the connection manager collection, Connections, has been created with that ID.
End Error======

The package gets execute properly from SSIS and SQL Server catalog so not sure whats the issue, i tried changing the DTexec.exe path to different values with "Program Files" and "Program Files (x86)" for different values of versions (110, 120, 130, etc) but couldnt get any success.

Really appreciate any guidance on this issue.

↧

Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper100 wrapper, Object transaction)

June 6, 2020, 9:09 pm
$
0
0

Hi everyone,

I am having a unexpected error after I migrated my SSIS project to visual studio 2017.MY SSIS is using some DTS pipeline  components. There are some file reader user component written in C# which are used in SSIS as componenets. It was working fine with ssis 2012 until I moved the code to SSIS 2017. I get following error when i click on the component

Error at Update Equity Recap Pricing [BloombergFileReader [20]]: System.MissingMethodException: Method not found: 'Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSConnectionManager100 Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSRuntimeConnection100.get_ConnectionManager()'.

at BloombergFileReader.BloombergFileReader.AcquireConnections(Object transaction)

at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper100 wrapper, Object transaction)

Error at Update Equity Recap Pricing [SSIS.Pipeline]: BloombergFileReader failed validation and returned error code 0x80131513.

I read on some forums to set  embedInteropfalse will fix this. I right clicked the dll "Microsoft.SQLServer.DTSPipelineWrap.dll sectio" in C# and changed embedInteropfalse to false. I couldn't find this on csproj file. is this the solution? What am i misisng?  I am still having issue loading the components in my SSIS project. 

Any help would be highly appreciated.


↧

Flexible File Destination filename from variable

December 17, 2019, 2:31 am
$
0
0

Hi All,

I'm using Flexible File Destination to insert records to Data Lake Gen2 files, my question is how can i pass variable to the filename property?

Everyday running the package, it created file with today name "20191212.txt", "20191213", "20191214",... ?

Thanks,

Mo Ghaffar


Regards, Mohamed Abdel Ghaffar | http://sharepointfoundation2010.blogspot.com

↧

error in download json from api - System.Net.WebException: 'The request was aborted: Could not create SSL/TLS secure channel.'

May 24, 2020, 3:23 am
$
0
0

Hi All,

i'm trying to get data from rest api with token with "script component" in ssis.

 when the code is reaching the part of   return client.DownloadString(downloadURL) it fails

and return the error: System.Net.WebException: 'The request was aborted: Could not create SSL/TLS secure channel.'

this is what i'm getting in the script component: runtime error:

   at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
   at System.Net.WebClient.DownloadString(Uri address)
   at System.Net.WebClient.DownloadString(String address)
   at ScriptMain.DownloadJson(String downloadURL) in c:\Users\320049665\AppData\Local\Temp\Vsta\624378d242724fcc9e5bf1dbf2294601\main.cs:line 26
   at ScriptMain.PreExecute() in c:\Users\320049665\AppData\Local\Temp\Vsta\624378d242724fcc9e5bf1dbf2294601\main.cs:line 17
   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.PreExecute()

this is the code:

using System;
using System.Net;
using System.Collections.Generic;
using System.Web.Script.Serialization;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    string json;

    public override void PreExecute()
    {
        base.PreExecute();
        string url = "https://www.xxx?Authrization=Bearer eyJhbGciOiJIUzxxxv4_NP7wg";
        json = DownloadJson(url);
    }

}

    public static string DownloadJson(string downloadURL)
    {
        using (WebClient client = new WebClient())
        {
            return client.DownloadString(downloadURL);
        }
    }

please help

↧
↧

getting error when delete the able donot have data

April 22, 2020, 10:02 pm
$
0
0
Hi I have one doubt in ssis

I want delete the data in the target server(postgres) tables data using ssis package.

database : postgres server 
Table : emp
in execute sql task :scriptis:  delete from emp and connection used odbc

when i run the executesql task in ssis package
if emp table have data then its working fine and i am getting the error when emp table donot have data.

[Execute SQL Task] Error: Executing the query "delete from  emp
usin..." failed with the following error: "Error HRESULT E_FAIL has been returned from a call to a COM component.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

suppose if i run same query in pgadmin tool  that time it is not getting any error even though emp table donot have data.

could you please tell me how to avoid this issue in ssis package.
↧

Conditional statement with a DATEADD as the THEN clause

June 7, 2020, 1:14 pm
$
0
0

Hi,

I have some records that have a residual amount (46000)

and what I want is that if that residual amount is >0 ? DATEADD("dd",1,(DateTimestamp):0

How do I get this to work?

Do I have to cast both the integers?

This is what I've got… sorry (feeling embarrassed lol)

Startdate is a database timestamp

Residual is an int.

(DT_WSTR, 15)[Residual]!= "0" ?DATEADD("dd",1,(startdate): (DT_WSTR, 1)"0"

I am a little confused with the DATEADD even after reading quite a few web articles so if there's a super simple page with explanations that would be helpful too.

Thank you 

↧
Search

Create XML DataSet Error

June 7, 2020, 6:22 pm
$
0
0

I'm trying to create XML data set using ADO.Net connection and i'm getting below error when i try to run SQL Agent job. Below is the error. It works fine when i run the package locally but doesnt work on the server. I;m using Visual Studio 2015.

"Create XML DataSet Error: Failed to acquire connection ADo.Net. Connection may not be configured correctly or you may not have the right permissions on this connection"

ANy help much appreciated.


↧

Could not load file or assembly 'microsoft.sqlserver.integrationservices.server

September 18, 2019, 1:22 pm
$
0
0

I'm trying to deploy a SSIS package from Visual Studio to my secondary server.  The db owner is the same on server 2 as it is on server 1.  This is the error message that I get.

An error occurred in the Microsoft .NET Framework while trying to load assembly id 65536. The server may be running out of resources, or the assembly may not be trusted with PERMISSION_SET = EXTERNAL_ACCESS or UNSAFE. Run the query again, or check documentation to see how to solve the assembly trust issues. For more information about this error: 
System.IO.FileLoadException: Could not load file or assembly 'microsoft.sqlserver.integrationservices.server, Version=13.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. An error relating to security occurred. (Exception from HRESULT: 0x8013150A)
System.IO.FileLoadException: 
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean forIntrospection)
   at System.Reflection.RuntimeAssembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.Load(String assemblyString)
 (Microsoft SQL Server, Error: 10314)


↧
↧

Calling a child package using a variable

June 8, 2020, 6:37 am
$
0
0

Hello,

I have a solution that has on parent package and 5 child packages.

I want to do it in a way that I will tell it what packages to call with a variable.

For example

I have parent package

package_one

package_two

package_three

package_four

Parent package will start and it will call the first two packages on the variable object, when one package is finished it will keep going down the list.

can it be done? 

thanks

↧

SSIS server to connect Different Server database

June 8, 2020, 2:49 pm
$
0
0

Hello,

I have configured SSIS server on Sql Server 2017 and developer have created the SSIS package on this new SSIS server and loading data from file to another Server database.

My SSIS Server name is SSISSQL and running package from SSISSQL and loading data into another server SVR1 of database called stage.

I ma having connectivity issue error and package getting failed. I am doing first time and don't know my configuration string is proper or not.

Do i need to create linked server?

Service account is MyUser with SysAdmin role. MyUser is exist in both the server (SSISSQL and SVR1) as SysAdmin Role.

I am running package through SSMS ==> Integration Service Catalog ==> SSISDB

Thanks for your help!

↧

Problem to load collection from odata in SSIS

June 9, 2020, 6:35 am
$
0
0

Hello, 

It's my first time with SSIS loading Odata source.

I've added a dataflow, with a odata source component.

I have created my connection manager, connection runs successfully.

But when I'm back in the odata source editor, I got this message and it doesn't load the collections : 

Unable to load collection for the connection OData Source, reason: Cannot open this connection.

I would be very grateful if you had an idea on how to solve this issue.

Thanks a lot, AnneSo

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>