Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

SSIS transactions using MSDTC

December 19, 2018, 3:36 am
$
0
0

Hi,

I'm thinking of introducing transaction logic into a suite of SSIS packages. The sensible way seems to be by using the TransactionOption property and that does look like it gives a lot of flexibility over the scope of transactions. My question is; is this the recommended way to handle transactions in SSIS? Should I consider anything else? One thing I need to bear in mind is that our SQL Server will soon be moving into Azure so I do wonder if there is any potential issue there. I'm aware of the need for MSDTC on the SQL Server; that should not be a problem.

Any advice gratefully received!

Thanks

↧
Search

VS2017 Restart when running packages

December 13, 2018, 7:31 am
$
0
0

I have a number of packages in my project which used to run quite successfully.

When I run them in VS2017 they get part way through and then VS2017 restarts.

If I deploy the package to the database server and execute it from there it runs quite successfully.

Can anyone suggest a way I can diagnose what is happeneing.  I have been unable to fins any logs with pertenant info.

Thanks in advance

Andy

↧
↧

SSIS Flat File destination issue in azure

December 19, 2018, 5:10 am
$
0
0

Hi All,

I have developed ssis package which is having flat file destination in some network folder. It is working fine in local environment and it is not working in Azure and shows compatibility issue in flat file destination. Do we have any other alternates to fix and solution for this. 

Thanks,

Srisai 

↧

Connection Manager and SQL Server Job Agent and Setting up a Job with Credentials

December 19, 2018, 8:36 am
$
0
0

I need a crash course in setting up Connection Managers and Credentials and SQL Server Credentials and setting up those credentials in SQL Server Job Agent. My SSIS Package runs just fine in Microsoft Visual Studio with my provided credentials. Using our Production box from our Test Server as a Linked Manager in order to use Email and [sp_send_dbmail]. But when I schedule to test the Job, it's throwing up Saying "Server is not found or not accessible"

Help!!!

Thanks for your review and am hopeful for a reply.

↧

loading null from CSV

December 19, 2018, 11:14 am
$
0
0

Hi, 

I need help to understand why my import of a CSV file containing null as text does not get loaded as real NULL values. At least I guess that is why I recieve an error. 

[LoadExcel [53]] Error: Data conversion failed. The data conversion for column "Volume" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".

[LoadExcel [53]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "LoadExcel.Outputs[Flat File Source Output].Columns[Volume]" failed because error code 0xC0209084 occurred, and the error row disposition on "LoadExcel.Outputs[Flat File Source Output].Columns[Volume]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.

[LoadExcel [53]] Error: An error occurred while processing file "C:\temp\import_-_19-12-2018.csv" on data row 2.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on LoadExcel returned error code 0xC0202092.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.

↧
↧

SSIS Package runs fine in Micorsoft Visual Studio but fails when attempting to schedule a SQL Server Job in SQL Server Agent

December 19, 2018, 2:15 pm
$
0
0

So my SSIS Package is using a Linked Server to access our Test Server from our Development Server. Our Test SQL Server  is configured to Send and run [msdb].[dbo].[sp_send_dbmail]; our Development Server is not. So my SSIS Package runs an Execute SQL Task to actually run [msdb].[dbo].[sp_send_dbmail] and Send an Email out of the SSIS Package using the Test Server Connection. The Connection Manager to our Test Server is using my credentials and Windows Authentication. The Package runs like a charm producing the Email in Microsoft Visual Studio...Microsoft Visual Studio Professional 2017 Version 15.8.7. Microsoft SQL Server 2014.

So my first question is what is the config file where you can invoke Email capability from SQL Server and where is it normally located? I think I've gone down that rabbit hole before...I just cannot remember the config file contents and syntax to enable Email capability...or am I wwwaaayyy out of whack?

My SSIS Package is currently set-up to use Security/ProtectionLevel/DontSaveSensitive. Now there are Service Accounts to our Test Server and I actually changed the credentials to use the Service Account and provided the ServerName and ServerPassword as Variables and changed the Connection Manager Properties and Expression for Password and ServerName.

(Trying to give as much background as possible....Sorry...)

Now I schedule a SQL Server Job. Simply copied the .dtsx from my Visual Studio Project to where the Development Server normally executes our SSIS Packages. I have no Configuration file currently. I did see that as a possible solution but cannot even seem to find Package Configurations under SSIS in Visual Studio. The Linked Server from our Development Server is defined to useBe made using the login's current security context. I am at the liberty to change that on our Development and Test Servers if need be.

So when it goes and tries to execute the Job in SQL Server Agent it comes back saying...

Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80040E4D  Description: "Login failed for user '<User Name>

I have Googled this and cannot seem to find a solution. Some say to create a Configuration File and reference that in your SQL Server Job Step. Others have said to change the Security on the Package in Project back in Microsoft Visual Studio.

I'm at a point now that I have tried soooooo many things I can't remember what I have tried and what I have not tried.

Can someone please help me grind through this and get this to work. Sooooo frustrating.

Thanks for your review and am hopeful for a reply.

↧

SSIS Error: VS_NEEDSNEWMETADATA

December 19, 2018, 8:34 pm
$
0
0

Hi

A Ssis package read a table from a mysql server.

The problem was when a user added 2 columns in the view which I don't have access.

I had a view in my local mysql server. After that I created 2 columns with the same datatype and length that the view of the another server but when I run the ssis package, this failed.


I replicated the view in my local server and the ssis runs perfect.

I don't understand since the 2 columns have the same datatype in the views of my server and the another server.

I think that SSIS verify not only the structure of views so it showsthe error:SSIS Error: VS_NEEDSNEWMETADATA

↧

Generate Dynamic column based on SQL output

December 18, 2018, 12:16 am
$
0
0

I have source data i.e. is present in SQL like below

Product CodeItem IDItem DescriptionCustomer NameMonth of OrderQty Ordered
10125Food ItemsHanaJan-01-201820
10125Food ItemsHanaFeb-01-201824
10125Food ItemsHanaMar-01-201844


Expected output is when I run at Feb month,this output required in excel

 

    Product CodeItem ID    Item DescriptionCustomer NameJan-18
10125  Food ItemsHana20

Expected output is when I run at Mar month  

Product CodeItem ID Item DescriptionCustomer NameJan-18             Feb
10125    Food ItemsHana2024

Expected output is when I run at Apr month  

Product CodeItem IDItem DescriptionCustomer NameJan-18                 Feb         Mar
10125    Food ItemsHana202444


I need to generate columns dynamically Jan-18,Feb-18,etc.

Like that need to get output in Excel from SQL like above.

Could you please any one help me for this scenario.I hope any one person will help?

Regards,

Niranjan B

↧
Search

how to convert data from XML file to Excel file using EDI source in SSIS?

December 19, 2018, 11:16 pm
$
0
0

Hi All.............

i have xml data file like this,

<?xml version="1.0"?>

-<CHPOI03Payload>
          <DocumentHeader>
                -<DocumentDetails>
                              -<verticalinfo> data is started with like this ....

i have to convert this XML data file into Excel file,

i have tried using XML task, online convertor but data is breaking into multiple sheets i want data in single sheet,i have attached tried file image.




↧
↧

Is there a way to check if this is a permission or configuration error upon Deployment

December 20, 2018, 5:44 am
$
0
0

If permissions... I can't tell what is different between the successful deployment vs the failed one. Same domain same domain service accounts.

SQL 2014 Std
Deploy SSIS project with one package to the Integration services catalog.
Using VS 2017 (SSDT) from SQL server local.

This works on a VM clone of the production database but will not work on the production server.

In SSISDB database under security on the successful deployment
the certificates node has 3 certs
 MS_cert_Proj_1
  MS_cert_Proj_Param_1
   MS_SQLISSigningCertificate
the Symmetric Keys has 2 keys
 MS_Enkey_Proj_1
  MS_Enkey_Proj_Param_1
 
In SSISDB database under security on the failing deployment
the certificates node has 1 cert
   MS_SQLISSigningCertificate
the Symmetric Keys has 0 keys

Deploy wizard causes

Cannot find the certificate 'MS_Cert_Proj_2', because it does not exist or you do not have permission.

this value increments on successive deployment tries

MS_Cert_Proj_2

  
error report

<WizardReport xmlns="http://schemas.datacontract.org/2004/07/Microsoft.SqlServer.IntegrationServices.Wizard.Common.Model" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<Results>
<Children/>
<Header>Results</Header>
<ReportItems>
<SummaryReportItem><Message i:nil="true"/>
<Property>Loading project</Property>
<Status>Success</Status>
<Value i:nil="true"/>
</SummaryReportItem>
<SummaryReportItem><Message i:nil="true"/>
<Property>Connecting to destination server</Property>
<Status>Success</Status>
<Value i:nil="true"/>
</SummaryReportItem>
<SummaryReportItem>
<Message i:nil="true"/>
<Property>Changing protection level</Property>
<Status>Success</Status>
<Value i:nil="true"/>
</SummaryReportItem>
<SummaryReportItem>
<Message>Cannot find the certificate 'MS_Cert_Proj_2', because it does not exist or you do not have permission.</Message>
<Property>Deploying project</Property><Status>Failure</Status>
<Value i:type="a:string" xmlns:a="System.Data.SqlClient.SqlException">http://www.w3.org/2001/XMLSchema">System.Data.SqlClient.SqlException (0x80131904): Cannot find the certificate 'MS_Cert_Proj_2', because it does not exist or you do not have permission.&#xD;
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)&#xD;
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)&#xD;
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)&#xD;
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean&amp; dataReady)&#xD;
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)&#xD;
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task&amp; task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)&#xD;
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task&amp; task, Boolean&amp; usedCache, Boolean asyncWrite, Boolean inRetry)&#xD;
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean&amp; usedCache, Boolean asyncWrite, Boolean inRetry)&#xD;
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()&#xD;
   at Microsoft.SqlServer.IntegrationServices.Common.ObjectModel.SqlHelper.ExecuteSQLCommand(SqlStoreConnection storeConnection, CommandType cmdType, String cmdText, SqlParameter[] parameters, ExecuteType execType, Int32 commandTimeout)&#xD;
   at Microsoft.SqlServer.Management.IntegrationServices.CatalogFolder.DeployProject(String projectName, Byte[] projectStream)&#xD;
   at Microsoft.SqlServer.IntegrationServices.Wizard.Common.Model.DeployExportProjectHelper.DeployProject(CatalogFolder folder, Project project)&#xD;
   at Microsoft.SqlServer.IntegrationServices.Wizard.Common.Model.DeploymentModel.Deploy(CatalogFolder folder, Project project)&#xD;
   at Microsoft.SqlServer.IntegrationServices.Wizard.Common.Model.DeploymentModel.DeployProject()&#xD;
ClientConnectionId:3fc5cd24-04bd-4a66-9841-f286ce8964d4&#xD;
Error Number:15151,State:1,Class:16</Value>
</SummaryReportItem>
</ReportItems>
</Results>
<Summary>
<ReportSection>
<Children/>
<Header>Use the following arguments to perform this deployment from the command line:</Header>
<ReportItems>
<SummaryReportItem>
<Message/>
<Property>Command line</Property>
<Status>None</Status>
<Value i:type="a:string" xmlns:a="/Silent">http://www.w3.org/2001/XMLSchema">/Silent /ModelType:Project /SourcePath:"C:\SSIS\TimeClockPlus\AIST_TimeClock_first_bulk_import\bin\Development\AIST_TimeClock_first_bulk_import.ispac" /DestinationServer:"THESERVER" /DestinationPath:"/SSISDB/AIST_TimeClock_0/AIST_TimeClock_first_bulk_import"</Value></SummaryReportItem></ReportItems></ReportSection><ReportSection><Children/>
<Header>Source</Header>
<ReportItems>
<SummaryReportItem>
<Message/>
<Property>Path</Property>
<Status>None</Status>
<Value i:type="a:string" xmlns:a="C:\SSIS\TimeClockPlus\AIST_TimeClock_first_bulk_import\bin\Development\AIST_TimeClock_first_bulk_import.ispac</Value">http://www.w3.org/2001/XMLSchema">C:\SSIS\TimeClockPlus\AIST_TimeClock_first_bulk_import\bin\Development\AIST_TimeClock_first_bulk_import.ispac</Value>
</SummaryReportItem>
</ReportItems>
</ReportSection>
<ReportSection>
<Children/>
<Header>Destination</Header>
<ReportItems>
<SummaryReportItem>
<Message/>
<Property>Server name</Property>
<Status>None</Status>
<Value i:type="a:string" xmlns:a="THESERVER</Value">http://www.w3.org/2001/XMLSchema">THESERVER</Value>
</SummaryReportItem>
<SummaryReportItem>
<Message/>
<Property>Path</Property>
<Status>None</Status>
<Value i:type="a:string" xmlns:a="/SSISDB/AIST_TimeClock_0/AIST_TimeClock_first_bulk_import</Value">http://www.w3.org/2001/XMLSchema">/SSISDB/AIST_TimeClock_0/AIST_TimeClock_first_bulk_import</Value>
</SummaryReportItem>
</ReportItems>
</ReportSection>
</Summary>
</WizardReport>

↧

Disable option inside Data Flow task

September 23, 2009, 10:37 pm
$
0
0
We can disable a task at control flow.
But there is no option for disabling a component inside a data flow task. Why?
Suppose I have a component and I dont want it inside my package but want to keep it so that I can use it again if required.
Any idea, how to do this....
Nitesh Rai- Please mark the post as answered if it answers your question
↧

CDC Splitter UpdateOutput path - why is this performing inserts rather than updates

December 20, 2018, 9:55 am
$
0
0

Performing a simple CDC package load insert and update records.  Inserts are fine. 

I have an ID field as Primary Key that is coming from source.  If I perform an update on a non-key field in source it flows correctly to the Update path from the CDC Splitter, but I get primary key violation due to an attempted insert on the destination.  Why is it attempting an insert?

I don't see too many ways to customize this or force the update path to only perform updates.  There must be something designating it needs to insert or update that I don't see.

The target PK was created after the initial package was built and I truncated and reloaded the unchanged data ok.  I tried dropping and re-adding destinations after that thinking perhaps the package needed the schema def updated. 

I don't have to use a CDC splitter, I would prefer to though if I can get the behaviors figured out.

(VSTS 2017, Sql server 2017, Sql server data tools 15.1.61808...)

thanks,

Aaron

↧

write ssis variables to an ascii file?

December 20, 2018, 11:41 am
$
0
0

Hi,

      We have a need to write some ssis variables to a file so we can use the values later.   We can think of some ways to do that like maybe use an sql expression and code the ssis variable references in the query.    Is there a task that lets you write ssis variables to an ascii file?

↧
↧

SSIS Server Maintenance - cleanup_server_retention_window Job error

December 20, 2018, 4:24 pm
$
0
0
we are receving the following error message in sql server 2017(CU11) when the SSISDB Maintenance job runs. this job scheduled daily once and SSISDB database recovery model is simple. we have observed this issue after upgraded to SQL 2017 not for all the SQL servers. many SQL server with 2017 are completing without any error. 
when run this cleanup proc step, its blocking other SSIS DB jobs also.

Date12/10/2018 12:00:00 AM
LogJob History (SSIS Server Maintenance Job)

Step ID1
ServerAZ4OBIUCMLSQL03
Job NameSSIS Server Maintenance Job
Step NameSSIS Server Operation Records Maintenance
Duration00:34:21
Sql Severity16
Sql Message ID16916
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted0

Message
Executed as user: ##MS_SSISServerCleanupJobLogin##. A cursor with the name 'execution_cursor' does not exist. [SQLSTATE 34000] (Error 16916)


SQL version : 

Microsoft SQL Server 2017 (RTM-CU11) (KB4462262) - 14.0.3038.14 (X64)   Sep 14 2018 13:53:44   Copyright (C) 2017 Microsoft Corporation  Enterprise Edition:


I have referred below links but no able to fix the issue.

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/9ce94c74-d552-48ec-8621-8be3124c53bc/ssisdb-2017-cu8-running-in-always-on-group-maintenance-procedure-bug?forum=sqlintegrationservices

https://nakulvachhrajani.com/2016/09/19/0386-sql-server-cursor-scope-a-cursor-with-the-name-cursor-name-does-not-exist-msg-16916/

https://stackoverflow.com/questions/50517509/ssis-server-maintenance-job-error
↧

Connecting to Excel downloaded from Reporting Services throws "External table not in expected format" error

December 20, 2018, 7:13 pm
$
0
0

Hi all,

I'm hoping to gain a deeper understanding of the issue as stated in the title. Here's the process flow:

1. Download report from ReportServer as Excel (xlsx)

2. Load the downloaded file on SSIS via ACE OleDB for further processing.

The error is thrown by the SSIS package. A rather common encounter, judging by the discussion on this thread https://stackoverflow.com/questions/1139390/excel-external-table-is-not-in-the-expected-format/51057571#51057571

I confirmed in the thread that the problem has a workaround by renaming the worksheet before feeding it into SSIS. However, upon further testing, I found out that renaming the worksheet isn't even necessary. All that is required is to:

1. Open up the workbook

2. Save (even without any changes)

And that allows the Excel file to be loaded without complaints! Can someone with the knowledge please share your insight on this matter? Thanks in advance!


↧
Search

Mapping column dynamically from SQL to Excel through SSIS package

December 20, 2018, 8:50 pm
$
0
0

I am expecting like below,

SQL Output
ProductID   ItemID     Date        Qty
101 2147 1/1/2018 10
102 2148 2/2/2018 12
103 2149 3/3/2018 15
103 2149 4/4/2018 15

I will convert this output to like below

ProductID ItemID Jan    Feb   Mar    Apr   
101   2147 10 NULL NULL NULL
102   2148 NULL 12 NULL NULL
103 2149 NULL NULL 15 15

I will use above results in OLE DB Source,initially I will prepare required columns in Excel then i can map those columns then i will execute package,

Problem is each and every month automatically one column will generate in SQL output,how can i map that column dynamically to excel.Now I hope you will understand my problem and I hope you will help to me to resolve this issue.

Regards,

Niranjan B

↧

Mapping dynamic column from SQL to Excel through SSIS

December 20, 2018, 9:44 pm
$
0
0

I am expecting like below,

SQL Output
ProductID   ItemID     Date        Qty
101 2147 1/1/2018 10
102 2148 2/2/2018 12
103 2149 3/3/2018 15
103 2149 4/4/2018 15

I will convert this output to like below

ProductID ItemID Jan    Feb   Mar    Apr   
101   2147 10 NULL NULL NULL
102   2148 NULL 12 NULL NULL
103 2149 NULL NULL 15 15

I will use above results in OLE DB Source,initially I will prepare required columns in Excel then i can map those columns then i will execute package,

Problem is each and every month automatically one column will generate in SQL output,how can i map that column dynamically to excel.Now I hope you will understand my problem and I hope you will help to me to resolve this issue.

Regards,

Niranjan B

↧
↧

Error Handling for More than 255 Chars records in Flat file while Export to Excel

December 20, 2018, 10:37 pm
$
0
0

Hi,

We know about Known issue of Export /Import Excel(OLEDB 12.0 Provider) 's column records More than 255 Chars.

So while export to Excel from OLEDB, Shall we capture that Column records in SSIS-Data Flow at excel destination error output,

I tried but still error at excel destination.

↧

Help me choose the correct Excel Driver

December 21, 2018, 12:43 am
$
0
0

Hi,

I have a package with an excel source. The Provider is ACE.OLEDB.12.0. I run it in Visual Studio and it works fine. But running it from the process flow application (Unify) does not work. I google this and understand that it has to do with that Unify is using 64 bit and this driver is not installed on the server.

I guess that I can install it on the server where the package is running, shouldn't be a problem. But the name on the installation package from Microsoft is "Access Database Engine". Access...? Really...? Kind of makes me wonder if I'm using the correct Provider in the first place. Should I do the excel import in some other way in the ssis package?

Regards

↧

Can we get the SQL Queries behind a SSIS package

December 21, 2018, 4:02 am
$
0
0

Hi,

Is there any way we can get the SQL Queries behind a SSIS package. I believe in the backend for each component and its function / process some SQL query gets generated.

Regards,

Manish Virmani 

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>