Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Sql Server 'System.OutOfMemoryException'

$
0
0

I'm using sql server 2016 and I have deployed ETL package in SSISDB and trying to execute the package via sql job getting error 'System.OutOfMemoryException'.

But the same query successfully executed via ETL package without any errors.

Note : I have used script task for data load in ETL package,Because the data load from diff types of source system(MySql,Oracle..) into sql destination and dynamically create table & data load in destination DB depends on the source query.

Server Ram Size : 48 GB

Source Data Volume : 4M

Please help me how to resolve this issue.


Getting error as "An exception occurred while launching the package from sql server"

$
0
0

Hello,

We have an SSIS package which was in SQL Server 2012 imported in MSDB. We are calling it from c# code. Now we upgraded SQL Server 2012 to SQL Server 2016 and  imported SSIS Package in MSDB. When we are calling it from our .net application, we getting error as "An exception occurred while launching the package from sql server". 

We are using same account which have enough permission in SQL Server. We tried various solution but still getting same issue. 

Please help.

Thanks,

SSIS Foreach Loop ignore errors

$
0
0

Afternoon,

I have a foreach loop container that calls a number of execute package tasks to rebuild a data warehouse. The container has a list of database names and within the container these are mapped to a parameter this then gets passed to the execute package task that performs an ETL on a certain table, it then goes to the next table etc. When all tables are complete the loop selects the next database.

The issue I have is if one of the packages fails the task stops. I have since changed MaximumErrorCount to 0 this executes the rest of tables per database even if a table has failed. What I would like to achieve is just to end the foreach loop for the current database, flag an error and go onto the next database in the list if any package fails.

Unable to deploy SSIS package due to A required Prvilage not held by the client

$
0
0

I have an instance of SQL Server 2016 and I'm using SSDT VS 2017. I've created a simple package that loads data from a flat file source and loads it to a database. I have full administrative rights to the machine and SQL Server and the package runs fine in the designer. I have already created a catalogue however when I come to deploy the package i get the following error:

TITLE: SQL Server Integration Services
------------------------------

A .NET Framework error occurred during execution of user-defined routine or aggregate "deploy_project_internal": 
System.ComponentModel.Win32Exception: A required privilege is not held by the client
System.ComponentModel.Win32Exception: 
   at Microsoft.SqlServer.IntegrationServices.Server.ISServerProcess.StartProcess(Boolean bSuspendThread)
   at Microsoft.SqlServer.IntegrationServices.Server.ServerApi.DeployProjectInternal(SqlInt64 deployId, SqlInt64 versionId, SqlInt64 projectId, SqlString projectName)
. (Microsoft SQL Server, Error: 6522)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft%20SQL%20Server&ProdVer=13.00.5081&EvtSrc=MSSQLServer&EvtID=6522&LinkId=20476

------------------------------
BUTTONS:

OK
------------------------------

I even tried to export the package and create an agent job to run it that way but that errors out too, although I can't find the reason why and it's not something I am familiar with. I've searched the internet far and wide but no matter what I try I cannot seem to get any SSIS job working. Any ideas where to begin?

thanks in advance

Flat File Connection Manager changes datatype from source

$
0
0

I am building an SSIS package with an OLE DB data source where the data access mode is SQL Command. In the query, all the date column values are converted to char(10) with style 101 to produce dates like 5/27/2011. When I press the Preview button, everything shows correctly.

I am having trouble with the flat file connection manager. The format has to be Ragged Right to produce a data file in the format the end user needs. In the Advanced tab, for all the “date” columns, I set the DataType to Unicode string (DT_WSTR) and OutputColumnWidths to 10 to match the output from the query. Visual Studio detected the InputColumnWidth as 30 and, initially, I left it that way. However, when I pressed the Preview button, the “date” columns showed values like this: 2011-05-27 00:00:00.

I tried changing the InputColumnWidths to 10, but that did not work. The date column showed 2011-05-27 and all the zeros were picked up in the next column, throwing everything off.

Why is this Connection Manager ignoring the format from the source? Did I miss a step?

I have not been able to find the answer on the web, so any help will be greatly appreciated.

If I move my data from my server to a cluster, can I then leave my SSIS on my server, or do I need a separate licence for that?

$
0
0

Hey!

At several of our customers we have a separate server with SQL Server and SSIS for the ETL process. Now some customers want us to transfer to their cluster. This saves them in the license costs for SQL Server per core.

Can SSIS then remain on our separate server? Or must a separate SQL Server license still be concluded for this?

Because if that is the case, then it makes no difference in licensing costs if we transfer to their cluster. Does someone in that case have a suggestion how we can solve this properly?

Thanks!

Sort transformation drops rows?

$
0
0

While debugging an SSIS package (SQL 2017 target, VS 2015 with SSDT), I noticed that one of the Sort transformations apparently "lost" about 85% of all the records - and the "Remove rows with duplicate sort values" is Unchecked.  Out of 471,226 rows in to the Sort transformation, only 66,899 are reported to be passed to the next transformation (a Merge Join). 

I think I saw that this (Sort -> Merge) may have been an issue in 2008, but I don't recall seeing anything more recent.

What gives?

(My VS shows the following:)

BI Developer Extensions v2.1.1 for Visual Studio 2015 - An add-in to extend SQL Server Data Tools

Microsoft SQL Server Data Tools 14.0.61712.050

Microsoft SQL Server Integration Services Designer
Version 14.0.1000.169

Why i cant read non English characters?

$
0
0

 create table temp2 (
  a varchar(10), 
  b nvarchar(10)
  );
  insert into temp2 (a, b)
  values ('גג','דד');

  select * from temp2; will show '??" where the non English characters were supposed to be. 

why?


Cannot convert DT_NTEXT to DT_STR

$
0
0

Hello, everyone,

I have a simple packages that loads Excel data into a SQL Server table.

One of the Excel Comlums is assigned a data type of DT_NTEXT and it's corresponding database column is varchar(1000). I tried changing the data type to DT_STR in the Advanced editior but it wouldn't let me (reverts back afterwards).

Regards.


My XSD and XML files

$
0
0

I'm using SSIS to copy data from a XML file.I have a xsd but only read a part of my xml, for example doesn't read the three first elements Date, Name and Phone number, only read the elements contained in info elements. Someone can help me?

This is my XSD

<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Liquid Studio 2019 (https://www.liquid-technologies.com) -->
<xs:schema xmlns:xd="http://schemas.microsoft.com/office/infopath/2003" xmlns:s0="http://microsoft.com/webservices/SharePointPortalServer/UserProfileService" xmlns:d="http://schemas.microsoft.com/office/infopath/2009/WSSList/dataFields" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1" xmlns:s1="http://microsoft.com/wsdl/types/" xmlns:ns1="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:pc="http://schemas.microsoft.com/office/infopath/2007/PartnerControls" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:_xdns0="http://schemas.microsoft.com/office/infopath/2003/changeTracking" xmlns:dfs="http://schemas.microsoft.com/office/infopath/2003/dataFormSolution" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:q="http://schemas.microsoft.com/office/infopath/2009/WSSList/queryFields" xmlns:ma="http://schemas.microsoft.com/office/2009/metadata/properties/metaAttributes" xmlns:tns="http://IBKC.int/" xmlns:my="http://schemas.microsoft.com/office/infopath/2003/myXSD/2012-06-21T16:04:39" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:dms="http://schemas.microsoft.com/office/2009/documentManagement/types" attributeFormDefault="unqualified" elementFormDefault="qualified" targetNamespace="http://schemas.microsoft.com/office/infopath/2003/myXSD/2012-06-21T16:04:39" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:import schemaLocation="C:\Users\alfre\OneDrive\Desktop\MaintanceFileDefinition1.xsd" namespace="http://www.w3.org/XML/1998/namespace" />
  <xs:element name="gpDDAMaintenance">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="0" name="Date" type="xs:date" />
        <xs:element minOccurs="0" name="Name" type="xs:string" />
        <xs:element minOccurs="0" name="PhoneNumber" type="xs:string" />
<xs:element minOccurs="0" name="Info">
         <xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="FormStatus" type="xs:string" />
<xs:element minOccurs="0" name="FormName" type="xs:string" />
<xs:element minOccurs="0" name="FormEmailAddress" type="xs:string" />
<xs:element minOccurs="0" name="EmailAddressOverride" />
<xs:element minOccurs="0" name="DepositWorkflow" type="xs:boolean" />
<xs:element minOccurs="0" name="RateWorkflow" type="xs:boolean" />
<xs:element minOccurs="0" name="HiddenPassword" type="xs:string" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
      <xs:attribute ref="xml:lang" use="optional" />
    </xs:complexType>
  </xs:element>
</xs:schema>

This is my XML

<xml version="1.0" encoding="utf-8"?>
<?mso-infoPathSolution name="urn:schemas-microsoft-com:office:infopath:XXXXXXmyXSD-2012-06-21T16-04-39" solutionVersion="1.0.20.207" productVersion="16.0.0.0" PIVersion="1.0.0.0" href="aaaa.com?>
<?mso-application progid="InfoPath.Document" versionProgid="InfoPath.Document.4"?>
<?mso-infoPath-file-attachment-present?>
<my:gpDDAMaintenance xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dfs="http://schemas.microsoft.com/office/infopath/2003/dataFormSolution" xmlns:tns="http://IBKC.int/" xmlns:_xdns0="http://schemas.microsoft.com/office/infopath/2003/changeTracking" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:http="http://schemas.xmlsoap.org/wsdl/http/" xmlns:tm="http://microsoft.com/wsdl/mime/textMatching/" xmlns:ns1="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/" xmlns:soap12="http://schemas.xmlsoap.org/wsdl/soap12/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:s0="http://microsoft.com/webservices/SharePointPortalServer/UserProfileService" xmlns:s1="http://microsoft.com/wsdl/types/" xmlns:pc="http://schemas.microsoft.com/office/infopath/2007/PartnerControls" xmlns:ma="http://schemas.microsoft.com/office/2009/metadata/properties/metaAttributes" xmlns:d="http://schemas.microsoft.com/office/infopath/2009/WSSList/dataFields" xmlns:q="http://schemas.microsoft.com/office/infopath/2009/WSSList/queryFields" xmlns:dms="http://schemas.microsoft.com/office/2009/documentManagement/types" xmlns:my="http://schemas.microsoft.com/office/infopath/2003/myXSD/2012-06-21T16:04:39" xmlns:xd="http://schemas.microsoft.com/office/infopath/2003" xml:lang="en-US">
<my:Date>2019-05-16</my:Date>
<my:Name>Jhon</my:Name>
<my:PhoneNumber>111-111-1111</my:PhoneNumber>

<my:gpInfo>
<my:FormStatus>Saved</my:FormStatus>
<my:FormName>AAA-157758106D-20190516131812</my:FormName>
<my:FormEmailAddress></my:FormEmailAddress>
<my:EmailAddressOverride></my:EmailAddressOverride>
<my:DepositWorkflow>true</my:DepositWorkflow>
<my:RateWorkflow>false</my:RateWorkflow>
<my:HiddenPassword>asd</my:HiddenPassword>
</my:gpInfo>
</my:gpDDAMaintenance>

Importing data from Excel to SQL Server destination table using SSIS

$
0
0

Hi All,

I have a question and in need of some assistance.

I am very new to SSIS and I am trying to import data from an excel file into a SQL Table

I have my Excel Source --> Data Conversion --> Destination, however I am missing a step.  I need to only upload the data if the date in the excel source is greater than the max date in the destination table.

What can I add between the Data Conversion and Destination to accomplish this?

Thanks in advance

Long duration Pre-Execute phase

$
0
0
Does anyone know why SSIS sometimes just sits in the Pre-Execute phase of a data flow and does nothing?  It doesn't matter how elaborate the data flow is or the volume of data.  It can sometimes take 80% of the task's run time.

AzureStorage is not a valid connection manager type

$
0
0

Hello everyone, 

I'm trying to use the Azure Tasks on my SSIS project. Locally everything works fine, but once I deploy to the production server, I get this error:

<name of my ssis package>:Error: The connection type "AzureStorage" specified for connection manager "<Azure Connection Manager Name>" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name

I have installed the SSIS Azure Feature Pack on both my dev machine and the production server, but seems like something is missing. 

Any help is much appreciated.

Connection manager usage

$
0
0

Hello

I have a package that uses multiple connection managers.  The package has 9 containers, each of which have 1 or more data flow tasks.

I need to know which connection manager is being use in each task, two are due to be replaced

Is there an easy way to do this besides manually go through each task or trying to read the xml file?

Thanks

Roy

SCD Type 2 Implementation with merge and merge join.. How to handle Primary key with multiple records from same source file

$
0
0

Hi All,

The Data from the source file looks like below. These are the only columns and there are no additional date columns.

Customer IDCustomer NameSegmentPostal Code
AA-10375Allen ArmoldConsumer85204
AA-10375Allen ArmoldConsumer68104
AA-10375Allen ArmoldConsumer90008
AA-10375Allen ArmoldConsumer97301
AA-10375Allen ArmoldConsumer30318
AA-10375Allen ArmoldConsumer91730
AA-10375Allen ArmoldConsumer37087
AA-10375Allen ArmoldConsumer10035
AA-10375Allen ArmoldConsumer2908

The requirement is to implement SCD type 2 implementation with merge join and merge task in SSIS.  please suggest how i can Implement SCD type 2 and update effective from date and Effective To Date for the above data. 

Thanks

Balaji



balaji


How to move undeployed SSIS package to new dev server

$
0
0

What is the procedure to move an undeployed SSIS package from my current dev server to.  All version software, including OS is the same between servers.  OS Win2016, Visual Studio 2015, SSDT 2015

Any help would be appreciated.

kd

ارقام خدمة عملاء كاريير (01225025360)(01014723434)القاهرة والمحافظات

No data output when executing package through SQLServer Agent job

$
0
0

All...

Any assistance is greatly appreciated in advance.

I have a package (within a project) that executes a SQL transform and then uses a C# script to create an XML output file using a XSL style sheet.  The strangest behavior occurs when the package executes within a SQLAgent scheduled batch job.  The SQLAgent execution does not produce one of the XML elements; but when the package is executed as a stand alone from within the project everything works well.  I have deleted the SQLAgent job (which runs multiple steps) and recreated from scratch, I have created a SQLAgent job with only the one package and still just the one elements does not generate.

Has anyone ever seen similar behavior?  How do I debug and troubleshoot?  How do I correct?

When executing the SQLAgent job the <LINES> element does not generate.  When I right click the package from within SSMS/IntegrationServices Catalog/SSISDB/Project and select execute; then the <LINES> element does generate

<ORDERLIST><ORDER><LINES<><PAYMENTS><COUPONS><ADJUSTMENTS>

Has anyone ever seen similar behavior?  How do I debug and troubleshoot?  How do I correct?


joemac

Using Temp folder in SSIS

$
0
0

Hi everyone,

I was wondering if it is possible to use a temp directory in SSIS. 

I have an SSIS package that loads data from SQL table to a flat file using Data flow task, and then I have a subsequent task that picks this file from a folder to an FTP location using the FTP task.

However, I want the Data Flow Task to drop the flat file in a temp folder and  the FTP task to pick the file from the 'temp folder' which will be deleted after all the control flow tasks has been completed.

The reason why i want to use a temp folder is because there are other processes that the control flow task runs which might task some time and because of the sensitive of the data that is being process, I dont want to give any user the opportunity to have access to the file while the other processes are still running.

Thanks


me

Biml and package/project parameters

$
0
0

I'm trying to create an Execute SQL task using Biml which references a project parameter and a package parameter as input values.  However, all I get out of it is a scope error.  I figured out that User:: is translated to User. and System:: is translated to System.  I can't figure out how to reference package or project parameters.  The project parameter that I'm using is already defined in the project.  Below is a section of the Biml I'm using.  Does anyone have any ideas of what I replace $Project:: and $Package:: with to get this ExecuteSQL task to generate?

<Biml xmlns="http://schemas.varigence.com/biml.xsd"><Packages><Package Name="Account" ConstraintMode="Linear"><Parameters><Parameter Name="SourceBaseFolder" DataType="String" IsRequired="true">C:\Work\InputFiles</Parameter><Parameter Name="SourceFolder" DataType="String" IsRequired="true">Source1</Parameter><Parameter Name="StageLoadID" DataType="Int32" IsRequired="true">1029</Parameter></Parameters><Variables><Variable Name="FileExists" DataType="Boolean">false</Variable><Variable Name="RowCount" DataType="Int32">0</Variable><Variable Name="InputFileName" DataType="String" EvaluateAsExpression="true">@[$Package::SourceBaseFolder]  + "\\" + @[$Package::SourceFolder]   + "\\"  + @[System::PackageName] + "_" +  @[$Package::SourceFolder] + ".csv"</Variable></Variables><Tasks><ExecuteSQL Name="Log Load Results" ConnectionName="Staging Database" ResultSet="None"><DirectInput>EXEC Audit.asp_AddStageLoadResult ?,?,?,?,?</DirectInput><Parameters><Parameter Name="0" VariableName="$Package::StageLoadID" Direction="Input" DataType="Int32" /><Parameter Name="1" VariableName="User.RowCount" Direction="Input" DataType="Int32" /><Parameter Name="2" VariableName="$Package::SourceFolder" Direction="Input" DataType="String"/><Parameter Name="3" VariableName="$Project::SourceBaseFolder" Direction="Input" DataType="String"/><Parameter Name="4" VariableName="System.PackageName" Direction="Input" DataType="String"/></Parameters></ExecuteSQL></Tasks></Package></Packages></Biml>



Mike Hotek mhotek@mssqlserver.com
SQL Server 2012: Data Design and Retrieval
ChampionValleyPress.com


Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>