So Im trying to use a rather complex stored procedure that ends up calling a few other procedures, one of which is part of a commercial system. Ive had lots of trouble with ssis and stored procedures in the past when using them in dataflow tasks. So im curious, what other work arounds have you used? Im pretty sure Ive used a sql task before, then looped through a resultset. Anyone else?
↧
Work around for using complex stored procedures as source in dataflow? sql task?
↧
How to remove carriage return to unwrap flat file
Hey yall,
So I am starting to create a process where I will be loading some info from a file to SQL tables. The problem is the vendor sends us wrapped files. Currently our process consists of manually doing a find and replace where we find '~clm' and replace it a blank. This is our way of unwrapping the file.
Does anyone know of a way to unwrap this within an ssis package?
Here is a preview of some data.
so when we do a find '~clm' that we have to type in to remove the carriage return at the end of each line.
Please let me know if you know of any ways to do this!
Thanks,
Lyndsey
Lyndsey Cupit
↧
↧
Calling windows scheduled Task from SSIS
Hi,
We need to generate csv file and send it to third party via SFTP using SSIS. We are not preferring buying component for SSIS. Free source ones are using plain text to send credentials, which we do not want to do.
Our company uses Axway client to do these SFTP things. It runs under a profile set up with credentials. So, we decided to utilize this so that we don't have to worry about credentials. Axway can run windows schedule task under its credentials. So we are thinking of creating a batch file that does the work and calling this batch file with scheduled windows task. We are planning to execute this scheduled windows task in SSIS package.
I need code or logic to call scheduled windows task in SSIS. Lot of forums say to use 'Execute Process Task', but not syntax or code how to use it for windows scheduled task.
Please let me know how to do this OR is there any good component to use SFTP in SSIS.
Thank You
Spunny
↧
importing ragged right flat file into sql server
I have an issue with this scenario.
I have a big flat file (text file) and I need to load this file data in to corresponding columns in to SQL server 2014 .
The flat file is delimitted by space. The issue is some columns are not seperated by any delimiter like space or tab or comma etc.
The flat file is delimitted by space. The issue is some columns are not seperated by any delimiter like space or tab or comma etc.
The issue is explained below:
I have 2 seperate files.
First file gives all the column names and data types of column and
the 2nd file gives the data .
The 2nd file does not have any column heading and it just gives records.
First file gives all the column names and data types of column and
the 2nd file gives the data .
The 2nd file does not have any column heading and it just gives records.
The issue is that, I need to match the file 1 and file 2 and identifies which values in file 2 goes to each columns if file 1. The file 1 gives the starting position of each column values.
for example,
file 1
........
Column Name Starting Position
........
Column Name Starting Position
..............................
Employee join date 1 10 Date
Employee ID number 11 13 CHAR
Years of Experience 24 5 Integer
Resigned Date 33 10 Date
Department ID 43 3 CHAR
Annual Salary 48 7 FLOAT
File 2
-----------
02/22/20110000000004 00003 01/08/2015002 2000000.00
-----------
02/22/20110000000004 00003 01/08/2015002 2000000.00
My task is to create a table whose column name should be column name described in file 1 and populate this table with records from file 2.
The issue is that, while loading the table, 02/22/20110000000004 should be split in to 02/22/2011 and 0000000004 and goes into Employee join date and Employee ID number columns (I need to split the rows of file 2 based on the starting position described
in file 1 ).
could you please help me to fix this issue? How can I do this with SSIS? Can I schedule a job to import these type of files in to sql server?
↧
Performance Difference - Execute SQL Task with Stored Procedure VS Data Flow Task
Hi Experts,
When it comes to ETL, I have seen 2 approaches usually used by developers -
1) Heavy usage of T-SQL (Execute SQL Task with SPs) and minimum usage of SSIS components
2) Heavy usage of DFT and other tasks and minimum usage of T-SQL.
I understand there are some features which can only be done in either T-SQL or SSIS. However, if there are some business rules/validations which can be achieved with both the approaches which approach would be better in terms of performance.
I have checked many links however did not get satisfactory answer.
Please guide me on the same.
Thank you in advance!!
↧
↧
How to script all Parameters, Connection Managers and References for an already deployed SSIS project in SSISDB, with SQL Server 2012
Hello,
I want to create the scripts for an already deployed SSIS Project, with lots of .dtsx child packages, for its Parameters, Connection Managers and References.
Is there any way I could do that?
Regards,
PS: I have the solution already in VS2012 Professional, so if there is a way from there please let me know.
Regards,
↧
Scripting an agent job, maximum job steps..?
I am creating an agent job with 73 different steps (textfile imports). I have created a script for it, but it gives me an error saying that it is too many steps;
"The specified '@step_id' is invalid (valid values are: 1..31)."
31, is that the maximum steps I can creat wthin a script? I have google it, but I haven't found a clear answear to this.
↧
Webservice TASK
Hi,
I am using webservice task in SSIS and in HTTP connection I am giving a REST API .
Now REST API's doesnt have a WSDL unlike SOAP while webservice task requires WSDL file
Plz suggest form where to get that
↧
Proxy settings
Experts,
Is it possible to add other domain user for proxy settings?
using windows credentials , Domain1\user1 is connected fine to sql server which is in Domain2.
We want to add Domain2\user2 as proxy.
Is it possible?
Thanks
Praveen
Praveen
↧
↧
SSIS log provider for xml files. Release XML, the file stays locked after execution
Hi guys!!Has the title says, I am using the SSIS log provider for xml files in one of my package. The problem is that the file is kept locked after execution. I don't get it!! I've been using the same log provider in all of my packages... but this one? Nope!! Not working properly!! Any idea why ?
Mylene Chalut
↧
Unpivot Transformation Inside For each loop
Hello All,
I am a new SSIS User, I have an requirement, I need to load three excel files to a database table and then I need to move those file to another folder by renaming it.
The Problem is the package should read the files (All three) from second row later Unpivot it and load to a table.
The Source structure are like below(All the excel Files)
The Destination Structure is
Would Be Great Full, If Someone Could help with this...
Thanks In Advance.
↧
ssis object variable
Hi...I am relatively new at this so forgive my ignorance...
I have written a script task that populates a multi-dimensional array. That part seems to be working fine. The array basically has a list of questions (coded by numbers) and the number of responses received. For example: 23568374, 53; 4235234, 53; question, items answered; and so on. It is my hope to put this into a table in my database. column1 would be the question, and column2 would be the return rate. As stated earlier, within the script task the array works like a champ. In SSIS I declared varReturnRate as an object. I was hoping to use a for each loop container to loop through it and place items in the database. The last line in script task is this:Dts.Variables("varReturnRate").Value = returnratearr. (returnratearr is a multidimensional array) This does in fact get the data to where I need it to be. I can put a watch on it. The problem is, when I look at the variable type....It is no longer an object, but is showing as string[]. I need a way to populate the variable, so I can use the ADO enumerator on it to populate my database. Any questions.
Thanks
↧
SQL SSIS log location
Hi All,
SSIS package execution can be viewed from SQL Server management console by right clicking on package and viewing reports.
Is this data pulled from some table? Is this data also gets stored in any log file on the disk?
or is there a log file location for SQL Server integration service that captures packages execution history?
Thanks
Manish
Manish
↧
↧
Recursively validate master package in SSISDB
Hi,
is there any way to recursively validate a package in the SSISDB?
I have a project deployed to the SSISDB with about 250 SSIS packages for a Data Warehouse. Now I would like to validate all packages called from the Master package (using the parent child design paradigma).
If I validate the Master package, then its subpackages are not validated.
If I validate the project then packages that can just be executed on some occasions are validated as well.
How can I validate a master package with all its subpackges that might be called over chaining of Master packages.
regards
Felix
↧
Time gap between the end time of SSISDB.[catalog].[executions] and actual finish time of package
I got 10 seconds gap of package end time between end of view SSISDB.[catalog].[executions] and actual time I log.
below is what I did.
query view for my package or from execution report, I got this
2016-06-02 18:30:59.2813754 +10:00
but when I modify the stored procedure and log the the time of start and end. I got finish time was 08:31:09.
However, in our live environment, It has 100s more
below is what I done
Added the code to log entry in system proc [catalog].[start_execution] before calling
EXEC @return_value =[internal].[start_execution_internal]
@project_id,
@execution_id,
@version_id,
@use32bitruntime
.
Unfortunately, SQL Server doesn't allow me to go a little bit depth as start_execution_internal has been encrypted.
Shawn
↧
using for each loop container
Hi
I tried few times this simple loop , please suggest where i am receiving error ?
Warning: The For Each File enumerator is empty. The For Each File enumerator did not find any files that matched the file pattern, or the specified directory was empty.
i clearly specified the path and selected the connection string but still i get this warning, i am trying to import two text files emp1 and emp2.
m123
↧
Copy 100s of SQL tables
We are in SQL 2014 standard edition.
Using VS2013 - SSIS.
Would like to see if there is a simpler way to create a SSIS package to "only copy data" from list of tables (100s of tables) from one SQL source to destination SQL DB which has empty tables. identity insert to be enabled.
Developer is a part of developer group and gets
"Property DefaultSchema is not available for Database. This property may not exist for this object, or may not be retrievable due to insufficient access rights" when transfer object taks is used.
Any suggestions / work arounds are appreciated.
Thanks,
↧
↧
File system task with Excel sheets
Hi All,
I'm using foreachloop container to loop through multiple excel files and then load the data into the tables. Once this task is done i want to copy the excel files to an archive folder with time stamp and delete the files from the current location. I tried using file system task but not successful so far. can any one please chare has a link to a step by step tutorial on this topic.
Issues am facing is if i have multiple excel files, its loading only the first one. How can i use the file system task to move excel file to a different location with a time stamp and delete the files from the source location?
SV
↧
SSIS Source change from Ado.net to oracle Source
Hi All,
Currently we are using Ado.net source in our ETL but to improve performance of our ETLs we want to change Ado.net to Oracle Source in ETLs. we would like to confirm below points :--
1) what will be the impact if we will change ado.net to oracle source.
2)did we will get good result in terms of performance.
Please let me know if any other impact or suggestion.
Refer below link which I am using for my reference.
http://www.danielbowlin.com/ssis-optimization-techniques-part-10-ssis-oracle-source/
Thanks & Regards,
Akhil
↧
Unable to delete breakpoint in SSIS C# script
I created SSIS script task using C#. I placed a breakpoint within a script while debugging. Now I cannot remove it - execution always stops at this breakpoint.
I tried everything - pressing "F9" while at the breakpoint, going to the list of breakpoints and disabling it (there was no "delete" option), saving project and restarting Visual Studio - nothing helps. Each time I start package via "Start Debugging" execution stops in the script.
Anyone has any suggestions?
I tried everything - pressing "F9" while at the breakpoint, going to the list of breakpoints and disabling it (there was no "delete" option), saving project and restarting Visual Studio - nothing helps. Each time I start package via "Start Debugging" execution stops in the script.
Anyone has any suggestions?
↧