Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Best practice for setting up SSIS with AG and Scale Out for redundancy on Master

July 10, 2020, 5:58 am
$
0
0

I'm looking at building an SSIS Scale Out environment with two AG replicas and two worker servers.  My question though is can I have two Masters?  So I'm thinking of this:

SQL Server 1 and SQL Server 2 setup as AG with SSISDB Catalog DB

SQL Server 3 and SQL Server 4 setup with Worker Service

With this can I install the SSIS Master service on both SQL Server 1 and 2 so if one goes down the other can take over?  My concern is if we need a fifth server for the Master and it goes down what type of redundancy do we have?  

Also we have some add-ins for SSIS like CozyRoc and OLE DB providers. Do these only get added to the Worker servers?  

Thanks.


↧
Search

Exporting data in mulple excel files

July 10, 2020, 9:26 am
$
0
0

Hi everyone,

I am working on creating an SSIS package that will export data from 3 different stored procedures from SQL Server and create 3 different excels for each of them.

I also want a situation when if no data is returned from any of the files, the file should not be created.

Thanks!

me

↧
↧

delete contents of a file based on string using ssis (visual studio 2012)

July 10, 2020, 11:17 am
$
0
0
I am new to SSIS.

There is an existing SSIS package that looks for files in a folder. One of the files in the folder has the string "Output of query _______does not exist."

Because of this sentence , the SSIS process fails. How do I add a condition to remove this string (delete content of file) ?


↧

Issues with data load from SQL tables to Azure Blob container.

July 10, 2020, 5:15 pm
$
0
0

Hi, 

I am working on to load data from SQL tables into Azure Blob using SSIS. I am able to establish the connections and also create  files in Blob storage pulling data from SQL tables.

But, I am facing issue with multiple/duplicate spaces. I have a column which has "duplicate spaces" in it , for eg; "Created according to information     Source : Government data" is one of the column's value.

When creating files into blob, I see a new line/row wherever it encountered these multiple spaces.

Example:

SQL table row : AD.AL|AD|Albany|Albany|Created according to information     Source : Government data|NULL|NULL|NULL

Row in the Azure blob file : AD.AL|AD|Albany|Albany|Created according to information

                                        Source : Government data|NULL|NULL|NULL

So, the record is getting broken into 2 records

Any help appreciated,

Thanks in advance!

harshac

↧

how to create SSIS package with SQL Server 2019/Visual Studio 2019

July 12, 2020, 1:27 pm
$
0
0

How do you create an SSIS package with SQL Server 2019/Visual Studio 2019? I don't see any BI options for SSIS, SSRS, or SSAS when creating a new project in Visual Studio. I have the free SQL Server 2019 Developer edition installed. I also have the Community Visual Studio 2019 free version installed. I installed SSDT with Visual Studio 2019. I selected SQL Server Data Tools under Data storage and processing in the list of workloads. On the Windows Start menu, I searched for Visual Studio (SSDT) but I it doesn't look like Visual Studio (SSDT) is installed. I tried uninstalling and re-installing SSDT with Visual Studio Installer (SQL Server Data Tools under Data storage and processing in the list of workloads). I also installed SSIS in Visual Studio 2019 under Extensions/Manage Extensions. 

↧
↧

SSIS Script Task Error through SQL Server Agent Job - The package execution returned DTSER_FAILURE (1).

August 17, 2016, 4:39 pm
$
0
0

Hello Everyone,

I badly need help here. I tried various suggestions from different posts in msdn and others , but nothing work out . I have created a package using SSDT for VS 2013 which has  a script task with VB 2012 code which refreshes Excel workbooks which in turn pull the data from tabular cubes. This works fine when executed in Visual studio. But fails everytime with below error , when run through SQL server agent Job 

Error - 

Executed as user: *****. Microsoft (R) SQL Server Execute Package Utility  Version 12.0.4100.1 for 32-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  4:12:44 PM  Error: 2016-08-17 16:13:35.23    Code: 0x00000001     Source: FileName      Description: Exception has been thrown by the target of an invocation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  4:12:44 PM Finished: 4:13:35 PM  Elapsed:  50.562 seconds.  The package execution failed.  The step failed.

Couple of suggestions i have tried 

1) Made sure that the Proxy account has read/write access on the folder where the excel file is present

2) Followed instructions in the below thread and created this folder - C:\Windows\SysWOW64\config\systemprofile\Desktop

https://social.msdn.microsoft.com/Forums/en-US/b81a3c4e-62db-488b-af06-44421818ef91/excel-2007-automation-on-top-of-a-windows-server-2008-x64?forum=innovateonoffice

3) Checked the option in job to run  in 32-bit mode

None of these suggestions work for me . Thanks in Advance. This would be a great help 

SG

↧

SQL Server Agent job fails , but works fine when user stays logged in

August 13, 2019, 2:58 am
$
0
0

I have a SQL Server job that runs a SSISDB package. The package does some SQL database connection and creates an excelt file as output. Finally it runs a VB 2015 script that opens the created excel file and rcopies sheets from that to another excel file and then a macro is run. The jobs runs fine as a SQL server agent job while I'm still logged onto the server . However when I log out the VB Script task fails with the message ' Script Task Error: Exception has been thrown by the target of an invocation. '.

help !


↧

ssis help

July 13, 2020, 10:59 am
$
0
0
Hi i have one doubt in ssis

how to append specific data in flatfile destination using ssis package loop iterate

source :  emp (sqlserver) sample data .in this table have 10 lakhs records
here i need to load  specific empid information into flat file destination using loop iterate.
sampe data
empid |Ename
1     |ab
2     |dg
3     |hg

Load the specific empid information maintain in  empmaster table.empids values frequenctly changing in empmaster table

Table : Empmaster
Empid
1
2
5
6

destination : text files emp.txt

empid  |Ename
1      |ab
2      |dg


 I tried like below :

step1 : create 2 variable  1) empid and datatype is varchar 2) empidresult  and datatype is object

Step2 : drag and drop the executesql task and using oledb conneciton and  sql command is : select distinct empdid from empmaster 
and resultset is : fullresultset   and resultset variable select the empidresult

step 3: drage and drop the foreachloopcontainer and continer type is foreachadoenumerator  adoobjectsource variable is : empidresult
 and variable mapping is : empid

step 4 : inside container drag and drop the dataflow task and using oledb source conneciton  sql statment with parametrs

select empid,ename from emp  where empid =? and  paramerter mapping to empid


step 5: drag and drop flatfiledestination 
and configrue emp.txt  file 
here I select the  columns names in the first data row unchecked and overwire data in the file uncheckd 
and executed the package i can append the data for required empids data but header is missing


I tried another methods empmaster records i have converts rows to columns and result store into object with commasepareate buts its supporting only upto 9000 records 

if master table have more than 10,000 records then its not working
I need first row as header then remain each loop iterate should be data append in destination files.

can you please tell me how to achive this task in sql server 




↧
Search

Dropdown option for SSIS package parameter

July 7, 2020, 12:32 pm
$
0
0

Hi guys,

Is it possible to have a dropdown option for package parameter in SSIS with a string data type. I need this because I will like the users to always enter the correct value for the parameter. The result behind this is because I have a conditional variable in the SSIS that will need the input value from this parameter, thus if the wrong value is entered the variable will not work as expected.

me

↧
↧

SSIS - Transfer SQL Server Object Task- Row Count for each Table.

July 7, 2020, 10:54 pm
$
0
0

HI All,

I am new to this forum.

I have a requirement where i need to copy tables from the source SQL database and migrate it to Sql Server 2014.
I have achieved this task using Transfer SQL Server Object Task.
But now i got additional requirement to get the row count from source and the destination table.

how can i achieve this task?

Please let me know if you need more details.

↧

Training Institutes for SSIS, SSRS in Hyderabad

December 9, 2009, 7:16 pm
$
0
0

Hi,

I am seeking for good training institutes for taking training in SSIS, SSRS in Hyderabad. Anyone with knowledge reagrding it please suggest.

Thank you.

↧

SSIS packages failing intermittently with unexpected termination

July 14, 2020, 9:46 am
$
0
0

Hi, A few SSIS packages are failing intermittently with "unexpected termination" on SQL Server 2016. These are scheduled to run daily via SQL Agent job. It does not give very much information in the SQL Server error logs or SSISDB all executions report. There are many other SSIS packages in different projects which run fine daily. This particular project has 5 SSIS packages and all run into this unexpected termination error every once in a while of late.

What could be some of the pointers to look at to mitigate this issue.

Thank you in advance.

Regards.

SQLEnthusiast

↧

Using a variable as a connection string

July 14, 2020, 1:12 pm
$
0
0

I have a for each loop that looks at all files in a folder and then does some work on them.
Obviously I cannot hard code the excel connection string as the file names aren't know until runtime. So I have set it like this

@[User::FilePath] is made up of @[User::RootFolder] + "\\" +  @[User::FileName] which evaluates to z:\blank.xlsx

However the source component has the red X and the error says: 

The result of the expression "@[User::FilePath]" on property "\Package.Connections[Excel Connection Manager].Properties[ConnectionString]" cannot be written to the property. The expression was evaluated, but cannot be set on the property.

Am I doing something wrong?

↧
↧

SQL 2016 - Integration Services Catalog (SSISDB) -Install Fails - Access is denied.

July 14, 2020, 12:34 pm
$
0
0

Hello any guidance is greatly appreciated.

SQL Server 2016EE (Fresh Environment), Trying to install Integrations Services Catalog (SSISDB) 

Installation Attempts

  • Running SSMS 2016 as Admin. - Error below
  • Run Powershell - Error below

 

Error encountered with SSMS\Powershell

TITLE: Microsoft SQL Server Management Studio


The operating system returned the error '5(Access is denied.)' while attempting 'RestoreContainer::ValidateTargetForCreation' on 'f:\SQL\MSSQLSERVER\Data\MSSQL13.MSSQLSERVER\MSSQL\DATA\SSISDB.ldf'.
File 'log' cannot be restored to 'f:\SQL\MSSQLSERVER\Data\MSSQL13.MSSQLSERVER\MSSQL\DATA\SSISDB.ldf'. Use WITH MOVE to identify a valid location for the file.
Problems were identified while planning for the RESTORE statement. Previous messages provide details.
RESTORE DATABASE is terminating abnormally. (Microsoft SQL Server, Error: 3634)

Is there a manual method (TSQL) to restore the SSISDB and configure? 

Thanks Mark 

Covi


↧

Having Trouble Installing SSDT tools for Visual Studio 2017 - Issues started with Integration Services

July 14, 2020, 8:18 pm
$
0
0

Issue: Unable to see BI project options in Visual Studio 2017.

Steps followed to install VS 2017 and SSDT on my laptop which was recently re-imaged:

1. Installed Sql Server 2017 - only Sql Server (not ssas, ssrs, ssis)

1. Installed Visual Studio 2017 (15.9.8)

2. Using SSDT Installer (2017 version), installed ssas,ssis,ssrs

3. I opened Visual Studio and attempted to open an existing project of SSIS and it said this project was not supported. It also would not allow me to reload it. Then I went to create a New BI project to see what BI project options were there and I could see SSRS and SSAS but not SSIS.  I must mention that this existing SSIS project that I tried to open uses an earlier Targeted Version (Sql 2014) but that should not matter as VS gives us that flexibility.

4. I then remembered the Extensions and Updates menu item and thought maybe I needed to download from there as well even though I had already installed it with SSDT installer.   I finally found Integration Services under "Roaming" and it would not allow me to Download. So I selected stop roaming thinking it would enable the Download button but it did not.  It ended up disappearing completed from Extensions and Updates.

5. I then tried multiple SSDT repairs, modifies and ultimately unistalled ssas, ssrs and ssis via the SSDT Installer. Then resinstalled.   In the end, I could no longer find any BI project options(ssas, ssis, or ssrs) in Visual Studio not even in Extensions and Updates. Since I had made things worse, I completed uninstalled visual studio 2017 following the instructions below to make sure it was a clean uninstall. https://docs.microsoft.com/en-us/visualstudio/productinfo/installing-an-earlier-release-of-vs2017

6. I installed a previous version of Visual Studio 15.6.7. Using SSDT 15.8 I installed ssas, ssrs and ssis. I opened Visual Studio to verify if I could now see those projects and they are not there not even in Extensions and Updates.

I am at a complete loss. Can someone please help. This shouldn't be this difficult.  Not long ago I installed the whole thing with no issues.  The only difference was that in addition to installing Sql Server, I also installed ssas and ssis.


↧
Search

SSIS package failed in server when there is no space in hard drive

July 14, 2020, 10:43 pm
$
0
0

Hi experts  

Here is the suitation , when the hard drive c have enough space , the package works fine in  both server in local vs project .

when the hard drive c have no enough space , the package only works fine in local vs project , failed in server ( run as sql agent job) .

while the hard drive c increase rapidly, the server in local area network ，could not connect to the internet .  and the server is particularly to run the ssis jobs (sqlserver agent job and local vs). So there is any ssis cache remain in drive C after each ssis jobs executed ?

Thanks in advance .

Best Wishes

MQ

Best Regards, Eric Liu MSDN Community Support Please remember to click Mark as Answer if the responses that resolved your issue, and to click Unmark as Answer if not. This can be beneficial to other community members reading this thread.




↧

SSIS Job Fails with unexpected termination on Execute of Data Flow Task

July 7, 2020, 5:58 am
$
0
0

Hi there,

I've looked all over the community and none of the possible solutions fixed my problems. Can someone help me look into it?

In the SSIS job history the error message is not helpful. The error message is just unexpected termination. In the event viewer I see that it is categorized as a windows error reporting. There are mdmp files attached to the error, but I'm not able to open it and I'm afraid that even if I do, I won't be able to debug it.

I know that mdmp file is more for windows support, is there anywhere I can send my mdmp files related to my error so that I can find out the root cause of my error and fix it. 

The ssis job succeeds 20% of the time, but 80% of the time it would fail. Not sure why.

Thanks,

Ron

↧
↧

Need SSIS EXPRESSION HELP FOR DATEADD

July 15, 2020, 4:59 am
$
0
0

hi all,

my current variable expression is as per below

 RIGHT("0" + (DT_STR, 2, 1252) DATEPART("dd" , @[System::StartTime]), 2) + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("mm" ,@[System::StartTime]), 2) +  (DT_STR, 4, 1252) DATEPART("yy" , @[System::StartTime]) + "_" +RIGHT("0" + (DT_STR, 2, 1252) DATEPART("hh" , @[System::StartTime]), 2) + RIGHT("0" + (DT_STR, 2, 1252) DATEPART("n" , @[System::StartTime]), 2) + RIGHT("0"+ (DT_STR, 2, 1252) DATEPART("ss" , @[System::StartTime]), 2) 

so basically it is returning me today's date and time.

now i want to add 8 hours in my hour part for specific country local time.so can you please help to write expression.




↧

Using Trim in SSIS derived column

July 15, 2020, 8:21 am
$
0
0

Hi,

I want to get Trim or any function from ID1, and it shows me ID2. I'm looking the correct data for ID2 from ID1.

Could you please help me how can I do it in SSIS ?

ID1               ID2
--------------         ----------------
27NULL101411741   27NULL101411741
9NULL101413231        9NULL101413231
27NULL101411741        27NULL101411741
9NULL101413231        9NULL101413231
2.0901E+12        2090101416321
2036722530        2036722530
203697230        203697230
5.45101E+13        54510140424305
5641014305        5641014305
10167253005        10167253005
2.33922E+12        2339223415420
↧

Empty file found in SSIS Foreach loop

July 15, 2020, 8:01 am
$
0
0

Hi everyone, 

I have a folder that I pick files from using foreach loop in SSIS. I noticed that most times the file it sees does not have any data in it and this makes the task to fail.

I was wondering if there is a proper way to handle this.

Thanks

me

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>