Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

dynamically loading multiple files( different columns in sql tables)

March 30, 2020, 7:26 am
$
0
0

Hi Everyone,

Hope everyone is safe and healthy.

I have a situation where i need to load multiple files in Sql table. I cant create manual connections in SSIS because due to some business need, my client will send me different files every week ( more /Fewer) columns so i am looking for a way where i can dynamically load files every week.

In SSIS, i know i can create 1 connection per file but that is not what is needed. 

E.g  

File A(week 10)File B(week10)

IDName DeptSalary
1  ABC        H.R50000
     

File A(week 11)File B(week11)

IDName AgeDept SalaryID
1ABC         23H.R500003

I am not good with C# so would appreciate your help. Van itg be done in ssis? Thanks

Shakky

↧
Search

Import from Excel 2016 and Add Columns?

March 30, 2020, 8:27 am
$
0
0

I have an Excel 2016 data file that I want to set up for import into a SQL Server 2016 table using SSIS. Visual Studio version is 2012. The Excel file has two columns. There are three additional columns in the destination table that can't be null, and I'd like to know if it's possible to populate them during the Excel import and if so, how. Two of the three columns are text and would always be populated with the same value. The third is a Date Stamp that needs to be populated with the current date and time (smalldatetime format).

I'm hoping to avoid having to create a new SQL Server table as this requires a change approval process at my company.

↧
↧

How do I convert date in SSIS?

March 30, 2020, 2:09 pm
$
0
0

Hello,

I'm new to SSIS and having issue when trying to load data from Sybase to Sql Server. 

I need to convert a date field from 6/30/2006 8:33:19.000 AM to 2006-06-30 08:33:19.000.

I've read to use Data Conversion and other suggestions to use derived column. I'm lost at this point and would greatly appreciate help with appropriate syntax to get the date field formatted correctly.

Thank You,

Tracy

↧

Value cannot be null. Parameter name: value (mscorlib)

March 30, 2020, 5:35 am
$
0
0

Hello,

   I'll try to start PowerShell for Integration services Catalog and i'll get this error :

===================================

Value cannot be null.
Parameter name: value (mscorlib)

------------------------------
Program Location:

   at System.String.IndexOf(String value, Int32 startIndex, Int32 count, StringComparison comparisonType)
   at Microsoft.SqlServer.Management.PowerShellTasks.StartPowerShellActionHandler.ReplaceServerNameWithServerNameAndPortInUrn(Urn urn, String serverNameWithPort)
   at Microsoft.SqlServer.Management.PowerShellTasks.StartPowerShellActionHandler.PerformAction(String actionMoniker, IContext context)
   at Microsoft.SqlServer.Management.TaskManagementService.Microsoft.SqlServer.Management.ITaskLauncherService.PerformAction(String actionMoniker, IContext context)
   at Microsoft.SqlServer.Management.UI.VSIntegration.ObjectExplorer.ActionMenuItem.Invoke()
   at Microsoft.SqlServer.Management.UI.VSIntegration.ObjectExplorer.ToolsMenuItemBase.MenuInvokedHandler(Object sender, EventArgs args)

Can someone help me please?

Regards,

Amar ADGHAR

 

Reponse

↧

Error: 0xC001000E. Connection Element not found

March 24, 2020, 11:57 pm
$
0
0

Hi,

I created incremental upload ssis package using visual studio which was intended to use with batch file. Package runs fine in visual studio, but when i try to run the same package through the use of bat file throws me an missing conection element error message.

Error as follows;

Started:  2:55:47 PM
Error: 2020-03-25 14:55:47.19
   Code: 0xC001000E
   Source: Package
   Description: The connection "SourceConnectionFlatFile" is not found. This error is thrown by Connections collection when the specific connection element is not found.
End Error
DTExec: Could not find the SourceConnectionFlatFile connection in the package.

I don't know how to fix this error this makes no sense. Maybe i am not properly utilizing the package file i just copy pasted the package file which is .dtsx extension to different folder to use it with that bat file. Looking forward for help.

TIA.

↧
↧

GNU PG Encryption Key

March 31, 2020, 2:22 am
$
0
0

Hi Guys!!

Because of some data security, we have to re-visit encryption key implementation via GNU PG.

What i want to do is, i have to import key via GNU PG tool by say service account 1, this encryption key is used in SSIS.

Now, i have to execute this SSIS via job however job is invoked by another service account 2 which do not have access to encryption key generated by service account 1, so job failing.

Can both Service accounts will have access to encryption key generated via GNU PG tool which is created by using Service account 1.

Thanks

Amit

Amit Srivastava,

NIIT Technologies

Please mark as answer if helpful

Blog:fascinatingsql.blogspot.in

↧

Power query source, change folder path

March 24, 2020, 1:21 am
$
0
0

For a data flow, I use the power query source for getting json data in a table format.

That works like a charm if i don't change the folder path on the connection manager. If I change the folder in the connection manager to another folder, I getting always an error with "Credentials are required to connect to the Folder source: 'old path to folder'".

That's probably because in the power query is still the old path. e.g. the query looks like this:

let
    Source = Folder.Files("D:\StageTransfer\MedicationBilling\Data")

....

If I change the path also in the Power Query Source, it will work again without an error.
But I want to have a Project Connection Manager, where I only have to change the project connection manager. So I have always to change the path also in the corresponding SSIS Package.

Probably I could do a workaround, if I change the query mode from "single query" to "single query from variable", but then, I have to escape every quotation mark in the query, making the query unmaintainable.

Does anybody have the same problem and has another solution for that?


↧

Use jet oledb 4.0 or ace oledb 12 or 16 for export to .xls or .xlsx

March 24, 2020, 2:11 pm
$
0
0

Greetings, I have had some issues moving to a new server and porting my existing ssis packages that work fine on a windows 2008 r2 SQL 2008 r2 server running file based packages and the packages through agent jobs creating .xls and .xlsx files with no issues. The old server odbc and oledb drivers are below under serveradrivers. New server is windows 2016 running sql server 2014, and I am using the same approach its odbc and oledb driver below on serverbdrivers. sql agent jobs running file based packages but only the packages that are writing out the .xls files using the jet oledb 4.0 provider are working. any that try to write out a .xlsx file using the ace oledb 12 provider fail with a "Connection may not be configured correctly or you may not have the right permissions on this connection" and the agent job continues to run, i have to cancel/stop the job. I have spent quite a bit of time trying to resolve this, here is another question I asked last week and a very helpful guy named Yitzhak Khabinsky, he worked through it with me to the conclusion that maybe the .dlls could be corrupted. I have uninstalled/reinstalled, only installed 2010, only installed 2016 and nothing works here. My question now is should I simply move ahead using the jet oledb 4.0 provider on all of the packages since it works or must I upgrade this? At some point I will be replacing the new 2016 server where I am having the issues with a server with SQL 2016 and be using SSISDB, and at that time hope to remove a lot of these spreadsheets moving to SSRS. What is the downside of leaving the jet 4.0 provider in these packages for now? I know there is a 65536 row limitation in the .xls files so if we can live with that is there any other gotcha's or issues with this running on 2016 and newer?

 Serveradrivers

Serverbdrivers

↧
Search

Using Global ##Temporary Tables across SSIS Packages

March 31, 2020, 1:04 pm
$
0
0

So we were hoping not to create a "waste" Table to store an aggregate file list that our client is asking for from the 3 or 4 SSIS Processes that we have built. As each SSIS Package that runs and produces a Pre File...a Post file...a Lead File...a Shipment File...and stores the file name and its size so it produces what they are calling a "Control File" which is nothing more than the list of files and their sizes. And we were we were hoping to do so by using a Global ##Temporary Table. However, when our first SSIS Package is complete, it does not seem to retain the Global ##Temporary Table.

Is there any way around having to actually create a static SQL Server Table that will only be used for this purpose?

Thanks for your review and am hopeful for a reply.

 
↧
↧

simple file movement pkg

March 17, 2020, 8:28 pm
$
0
0

Hello,

I have to create a simple pkg to load one table from .txt/.csv file and once loaded archive the file to diff archive folder.

the .csv file will be coming every hour or so, with file containing time stamp (e.g. FileName20200317.1830.csv, FileName20200317.1930.csv... )

I need to update a table in a db by copying the file name and flag as completed once ingested and archived.

eg. Control_table (Id, FileName, Status) values (1,FileName20200317.1830.csv, completed)... I need to add this pkg to job which runs every hour to check if any file has arrived.

If there is any link to such the sample pkg, I will be using 2015 SSDT and 2016SqlServer for this. I can store the config info in xml file and pkg will be stored on file system.

Thanks,

Neil

Neil

↧

Package configuration xml file

March 31, 2020, 1:18 pm
$
0
0

Hello,

When I am adding package config file through SQL Server Agent Job after adding package to the job, will this override the config file I have attached during the development of package? or I have to remove the config file from the package and deploy using agent job and attached config file using sql agent job?  My SQL Server is 2016 and SSTD I have used 2015 Visual studio. 


Neil


↧

The incoming tabular data stream (TDS) protocol stream is incorrect

April 1, 2020, 12:54 am
$
0
0

We have several dtsx working properly on SQL SERVER 2017 but if we run on 2019 we have the error in the subject.

We discovered it happens during insert into oledb command while if we change in destination oledb it runs properly.

We don't want to change any packages to substitute oledb command because our dtsx have to work on 2017 too.

Any idea?

Simone

↧

SSIS 2016 - System.Runtime.InteropServices.COMException (0xC0048021): Exception from HRESULT: 0xC0048021 at Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSDesigntimeComponent100.ProvideComponentProperties()

September 22, 2016, 9:07 am
$
0
0

We are testing our SSIS migration tool which use dynamic package creation and execution. The tool is working find for SQL Server 2012 and SQL Server 2014. When we run the tool on SQL Server 2016, we got the above error.

The code sample:

    IDTSComponentMetaData100 source = dataFlowTask.ComponentMetaDataCollection.New();
            source.ComponentClassID = "DTSAdapter.OleDbSource";
            source.ValidateExternalMetadata = true;
            source.UsesDispositions = true; 
            CManagedComponentWrapper srcDesignTime = source.Instantiate();
            srcDesignTime.ProvideComponentProperties();




↧
↧

SSIS Deployment Issue

April 1, 2020, 4:52 am
$
0
0

Problem:  Cannot deploy SSIS packages to AZURE VM/SQL Server Instance (or execute them via SSDT/VS) from a local laptop using a SQL Login. FYI: Project Deployment Model is used.

Background: 

We moved our development SQL Server instance to an AZURE VM, and would like to avoid giving every developer RDP permissions access to the server.

It seems that SSIS Projects can be only be deployed via a WINDOWS AD account while logged onto a server using an ISPAC file, or opening an SSIS Project on the server itself and deploying from there.

See error specifics below but my question is this: is there a work-around so that we can connect to AZURE SQL SERVER instance using SQL LOGIN’s and deploy/run package from our development laptop, or must every developer have RDP/Windows AD access to the new development server?

When we try to deploy SSIS Projects to the Integration Services Catalog from SSDT or VS, using a SQL Login we receive the following error message:

------------------------------

TITLE: SQL Server Integration Services

------------------------------

Operation 'Create' on object 'CatalogFolder[@Name='DNA_ODS']' failed during execution. (Microsoft.SqlServer.Management.Sdk.Sfc)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=15.0.17243.0+((SSMS_Rel).180502-2324)&LinkId=20476

------------------------------

ADDITIONAL INFORMATION:

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

------------------------------

The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an account that uses Integrated Authentication.

The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an account that uses Integrated Authentication. (Microsoft SQL Server, Error: 27123)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=14.00.3281&EvtSrc=MSSQLServer&EvtID=27123&LinkId=20476

------------------------------

Also tried to change connection type to Windows Authentication (in my case have more than adequate permissions) and received the following error message:

------------------------------

TITLE: SQL Server Integration Services

------------------------------

Failed to connect to server 172.21.2.5. (Microsoft.SqlServer.ConnectionInfo)

------------------------------

ADDITIONAL INFORMATION:

The target principal name is incorrect.  Cannot generate SSPI context. (Microsoft SQL Server, Error: 0)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&EvtSrc=MSSQLServer&EvtID=0&LinkId=20476

------------------------------

↧

Derived Column Expression - How to specify a Unicode character without using a literal?

April 1, 2020, 10:14 am
$
0
0

Is there a way to specify a Unicode character in a Derived Column transformation Expression, using its Dec, Oct or Hex value instead of the literal?

Specifically, I'm conditionally testing for a "fullwidth right parenthesis" (Dec 65289, Oct 0177411, Hex 0xFF09) at the end of a column:

  • RIGHT([column_name],1) == "）" ? «when_true» : «when_false»


Although the Expression works well with the literal "）", the problem is when I copy and paste the Expression into a text editor, the literal becomes a normal right parenthesis so that, if I were to copy it from the text editor and paste back into SSIS, the special character is lost.

I tried RIGHT([column_name],1) == nchar(65289) & RIGHT([column_name],1) == (DT_WSTR,5)65289 but neither worked.

Any help would be greatly appreciated.


↧
Search

can a script convert 2014 pkgs to 2019?

April 1, 2020, 10:44 am
$
0
0

Hi , one option that would be nice in our 2019 upgrade from 2014 is for our change management group to write a script that automatically reads 2014 pkgs , somehow automatically changes their deprecated native sql provider (which seems easy) but also converts them to 2019 as if someone opened them manually in vs 2019 before landing them in their new tfs folders and subsequently deploys them to their new catalog home as the 2019 version. 

is the "automatically converting to 2019" part (not the provider part) doable?  from a script?  it would be like the script is running a wizard on them in real time. 

↧

Child package executions not show in "Reports -> All Executions"

April 1, 2020, 10:47 am
$
0
0

Is there a reason why "Reports -> All Executions" show no entries for child packages even though I know they've been executed? If I wade through the reports for the parent package, I can find the entries associated with the child package. But as there are so many I need to export them to Excel simply to find the ones I need.

If this is a setting, can you advise where to change it?

↧
↧

I don't see in the connector mapping the new fields that I have created

April 1, 2020, 5:36 pm
$
0
0

Hello,

I have made some modifications in the SQL command text of my OLE DB Source Connector but I don't see those new fields mapping into OEL DB Destination connector.

Did I forget a step.

Does someone can help me with that issue.

Regards,

↧

VS 2012 - Can Not Load Existing Projects or Create New Project

April 1, 2020, 6:40 pm
$
0
0
 I am running VS 2012 on a virtual machine under Windows 7 Enterprise. I have been creating new projects and have been able to save and reload them. A few days ago I tried to open a saved project and received the error "One or more projects in the solution were not loaded correctly. Please see the output window for details". I also tried to create a new project and got the error "Operation is not valid due to the current state of the object". I have not been able to open existing projects or create new projects since. I check the amount of RAM and free disk space and both exceed system requirements. We reinstalled VS 2012 and the same errors are appearing. Any ideas about what is causing the errors and how to resolve would be appreciated.
↧

common CM for multiple file movement

April 1, 2020, 10:01 pm
$
0
0

I have two packages, using same DB connection and different file connection mgr

I am requested to use pkg config file, I kept common config for both the pkgs, by defining all the variables specific to each pkg in both the pkgs and using only ones which are needed. 

Now for common db the Connection manger is working fine ( I am passing db & server names in config file)

How to handle File Connection manager, which is not common as both the pkgs have files placed at different sources?

I used expression to provide the source file path, but once I use the exp for one pkg, how make it dynamic to use same File CM for other pkg, which has files at different directory?

Neil



↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>