Database Table column to XML destination - Please help

May 24, 2016, 10:06 pm

≫ Next: Executing SSIS Package on hosting server

≪ Previous: OLEDB Source + check for variable value and run flat file destination to create csv file

Hi,

I have a source table - the DDL is as below

CREATE TABLE [dbo].[PolicyDetail]
(
[PolicyDetailId] [bigint] IDENTITY(1,1) NOT NULL,
[PolicyNumber] [bigint] NULL,
[OriginalXml] xml,
[CreatedBy] [nvarchar](50) NOT NULL,
[CreatedDate] [datetime] NOT NULL,
)

This table has 10k records. In the '[OriginalXml]' column, actual xml file is stored as it is.
So, that means, 10k xml files are stored in the table. Now, I want to pull the originalxml column and create 10k individual xml files in a folder.

Can you please help me for this?

In the SSIS package, I have selected, Data Flow task. In the data flow task, for source, I have selected the table.
What should I select for Destination?

My target is to get the 10k individual files from the originalXML column from the table. Please help!

Thanks in advance for your help!

↧

Executing SSIS Package on hosting server

July 31, 2015, 3:20 pm

≫ Next: XML to database, very difficult requirement - Please help!

≪ Previous: Database Table column to XML destination - Please help

Hi guys,

My Environment:

SQL Server 2014 Standard Edition
SQL Server Installed on a hosted environment (different domain than our company domain)
I have to use DNV\MyUserName when RDP in order to access the hosted environment.
Our Company Domain is IHM\MyUserName
Ultimate Objective is to Execute SSIS package from a .NET application (I'm currently using MS LightSwitch)

I have the SSIS package deployed to SSISDB Catalog, I can run it locally without any problem.

Here is the script:

declare @output_execution_id bigint
exec [dbo].[CopyNewDataToLegacy] @output_execution_id output
print @output_execution_id

If I take the same script and run it from my local machine domain(IHM\MyUserName)

I get the following error

Msg 27123, Level 16, State 1, Procedure create_execution, Line 38
The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an account that uses Windows Authentication.
Msg 27123, Level 16, State 1, Procedure start_execution, Line 32
The operation cannot be started by an account that uses SQL Server Authentication. Start the operation with an account that uses Windows Authentication.

It's very clear to me that I can't execute the package without Windows Authentication.

Is there a workaround to execute SSIS package with SQL Server Authentication?
We have a secondary domain control in the hosted environment company but I can't see our IHM\Users from DNV Environment (I browsed Entire Directory from (SSMS >>> Logins >>> New Login >>> Click Search Button >>> Locations) I only see users under DNV\Users (which is not our domain)
I'm sure we are not the only company having this kind of issues, so I'm wondering how other companies who use hosting environment are working around this limitation.

I appreciate all the help.

Thank you

Codernater

↧

XML to database, very difficult requirement - Please help!

May 24, 2016, 11:04 pm

≫ Next: execute a command for each row

≪ Previous: Executing SSIS Package on hosting server

Hi,

I have 10k xml files in a folder.

The sampe file may be somehting like this

<policy>
<SubPolicy>
<Customer1>
<id>1000</id>
<name>Peter</name>
<term>12months</term>
<vehicletype>car</vehicletype>
<city>Charlotte</city>
<state>NC</state>
</customer1>
<Customer2>
<id>2000</id>
<name>John</name>
<term>12months</term>
<vehicletype>Bike</vehicletype>
<city>Charlotte</city>
<state>NC</state>
</customer2>
</SubPOlicy>
</POlicy>

I want to store the values in the xml files in to differnet tables in a database. Can you please help me on this..

Example, in the customer_maintable, I manually insert the records like this

create table dbo.customer_Maintable
(ID int,
[Type] varchar(20)
)

insert into dbo.customer_Maintable values ('1','name')
insert into dbo.customer_Maintable values ('2','Term')
insert into dbo.customer_Maintable values ('3','VehicleType')
insert into dbo.customer_Maintable values ('4','City')
insert into dbo.customer_Maintable values ('5','State')

then, in the dbo.customer_Transactiontable, the corresponding values should populate like the below output.

create table dbo.customer_Transactiontable
(
customer_TransactionID int identity (1,1),
CustomerID int,
CustomerMainTable_ID int,
Value varchar(20)
)

insert into dbo.customer_Transactiontable values (1000,1,'Peter')
insert into dbo.customer_Transactiontable values (1000,2,'12months')
insert into dbo.customer_Transactiontable values (1000,3,'car')
insert into dbo.customer_Transactiontable values (1000,4,'Charlotte')
insert into dbo.customer_Transactiontable values (1000,5,'NC')

insert into dbo.customer_Transactiontable values (2000,1,'John')
insert into dbo.customer_Transactiontable values (2000,2,'12months')
insert into dbo.customer_Transactiontable values (3000,3,'Bike')
insert into dbo.customer_Transactiontable values (4000,4,'Charlotte')
insert into dbo.customer_Transactiontable values (5000,5,'NC')

Can you please help me on this? In the future, if another tag/xpath comes, example <salary>5000</salary>, then , I should
do a insert an entry in the main table, like the below
insert into dbo.customer_Maintable values ('6','Salary') , then the salary should be correctly loaded in dbo.customer_Transactiontable table, using a
SSIS package or from a stored procedure.

Can yo please help me on this?

↧

execute a command for each row

May 25, 2016, 1:02 am

≫ Next: From SQL to SSIS - no data found

≪ Previous: XML to database, very difficult requirement - Please help!

Hello,

I am trying to load a database. I have a big SQL file of 1.9 GB and it contains 10 million inserts like 'insert into sometable values (1,2,3)'

I tried it using the sqlcmd -i but that is slow. So I switched to SSIS. I created a new package. Added a dataflow to it.

In that dataflow I added a flat file source and pointed to the right file. Specified only 1 column. Did a test run, Flat file source read all the lines in 30 seconds.

Then I added a oledb command. the sql text is easy 'select getdate()' '. At this point it took 12 minutes to execute ten million times select getdate().

Now I need to map the row to execute the insert statement. First try to changing the sqlcommand to ? (question mark). On the advanced editor, I added a column to the external columns on the ole db command input. Designer was happy but execution failed with"

An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80004005 Description: "Syntax error, permission violation, or other nonspecific error". "

Then I switched back to EXECUTE sp_executesql ? no luck

Any ideas ?

Constantijn

↧

From SQL to SSIS - no data found

May 25, 2016, 6:56 am

≫ Next: New columns supplied in CSV to an existing package breaks existing mapping

≪ Previous: execute a command for each row

Hello *.*,

I'm trying to replace some SQL-Statement in my ETL-process with SSIS-elements. There are already some similar transformations but this one gives me a headache.

SELECT SN, MANDKENN FROM V$P$A
WHERE V$P$A.KOPF IN (
	SELECT SN FROM V$VWHERE (
		V$P$A.MANDKENN = V$V.MANDKENN
	)
)

Looks easy, I know. I've made a lookup transformation for this one and connected the rows as following:

The connection works, the results are coming in by this SQL statement:

SELECT SN, MANDKENN FROM V$V

All other options are default. The OLE-connections is also used in several other transformations and works fine there.

When running this dataflow in debug mode, I'm getting the error codes 0xC020901E (no accordance), 0xC0209029 (DTS_E_INDUCEDTRANSFORMFAILUREONERROR) and 0xC0047022 (DTS_E_PROCESSINPUTFAILED).

Any ideas, whats I'm doing wrong?

Greetings, Markus

MCTS (70-642), MCP Please click the "Mark as Answer" or "Vote As Helpful button" if a post solves your problem or is helpful! Bitte klicke auf "Als Antwort vorschlagen" oder "Als hilfreich bewerten", wenn mein Beitrag Dein Problem löst oder hilfreich ist.

↧

New columns supplied in CSV to an existing package breaks existing mapping

May 25, 2016, 7:20 am

≫ Next: Execute SAS VBScript

≪ Previous: From SQL to SSIS - no data found

SSIS package is mapped to 10 columns with a CSV file and is reading data everyday .

Now the supplier starts sending 3 additional columns and puts them in between the existing columns.

When the new CSV file is detected by the package though it had 1-1 mapping with the CSV file column header names it breaks the mapping .Essentially the 3 new columns shift the existing columns from their position causing new columns data to appear in existing columns.

Hence I want to know if this is a expected behavior? And the data is sorted by column position and not column header

If so can this be avoided by asking the supplier to send the new columns at the end so that they dont break the existing mapping with the package and these new columns can be easily mapped later .?

↧

Execute SAS VBScript

April 12, 2016, 8:36 am

≫ Next: Cannot see ODBC oracle connection in SQL bids 2012

≪ Previous: New columns supplied in CSV to an existing package breaks existing mapping

Hi All,

i had to export out data from SAS tables to a txt files using SAS. but i have no clue how to schedule this on server, only option i know of is after you schedule this SAS export it creates a VB script and need to have your machine on!. is it possible to execute this vb script in SSIS? so the plan is to execute the VB script that creates txt files in a shared drive and then i import those files into SQL server tables using SSIS. please help!

↧

Cannot see ODBC oracle connection in SQL bids 2012

May 25, 2016, 8:54 am

≫ Next: Know the status of the report through SSIS

≪ Previous: Execute SAS VBScript

Hello all,

i have WIN 7 64 bit machine, SQL server BIDS 2012 Version.

i installed Oracle 11.2.0.4 both 64bit and 32 Bit . and then i tried to setup DSN in ODBC admin.. i was able to see Oracle 64..so i setup dsn for oracle 64 bit, but i am not able to see odbc 32 bit oracle driver and cannot setp dsn

Now.. in my bids.. i am not able to see any see any Oracle DSN/odbc , so i wen into bids sol properties-> degub mode and set "64bitrunmode" to false.. i still cannot see any thing.. help plsss

↧

Know the status of the report through SSIS

November 10, 2008, 9:28 pm

≫ Next: SSIS Convertion Function from Integer to String

≪ Previous: Cannot see ODBC oracle connection in SQL bids 2012

Hi,

As we all know that cube can be refreshed through SSIS.Similarly is there any option available to schedule or refresh or to know the status of the report?Can anyone help me ?

↧

SSIS Convertion Function from Integer to String

December 1, 2005, 7:31 am

≫ Next: SSIS 2008 script tasks won't compile after SQL 2014 install

≪ Previous: Know the status of the report through SSIS

Hello all

Ho can i convert Integer to String in SSIS Derived column.

Thanks in Advance
Raj Lakamana

↧

SSIS 2008 script tasks won't compile after SQL 2014 install

May 25, 2016, 10:55 am

≫ Next: Confirm: SSIS 2008 Package with Script Task won't Run on a 2012 Server

≪ Previous: SSIS Convertion Function from Integer to String

I have SQL 2008 installed on my local machine, and have quite a few SSIS packages that I have created. I recently installed SQL 2014 Management Studio. Since then, my script tasks no longer compile (in BIDS 2008). After doing some research and testing, I *think* that the issue is that BIDS 2008 no longer recognizes the Microsoft.SqlServer.manageddts dll. I'm not certain that this is the case, though. I have tried several iterations of repair/uninstall/reinstall of both SQL versions, and nothing seems to work. Has anyone else encountered, and more importantly FIXED, this issue? Thanks in advance!

↧

Confirm: SSIS 2008 Package with Script Task won't Run on a 2012 Server

January 16, 2013, 12:22 pm

≫ Next: Data Load of a HUGE National Database File and any hints...pointers

≪ Previous: SSIS 2008 script tasks won't compile after SQL 2014 install

I've been battling this for a long time. I have a package built with 2008 with a script task. The script task is set to run under .Net framework 3.5 (project properties inside the script task code). This package will not run on a server with 2012 installed (under the legacy integration services service). Actually, it does run if I execute the package from my local machine (pointing to the server), but it won't run directly on the server; it gives:

The component metadata for “Script Component, clsid {874F7595-FB5F-4OFF-9BAF-FBFF825OE3EF}” could not be upgraded to the newer version of the component. The PerformUpgrade method failed.

Creating a new 2012 style package and using the same code in a script component works fine.

It seems like maybe the old SSIS service is not able to run my script using the proper framework or something, but there's no way of knowing. Is this scenario simply not going to work?

↧

Data Load of a HUGE National Database File and any hints...pointers

May 25, 2016, 1:06 pm

≫ Next: SSIS package run as a job from sql agent does nothing

≪ Previous: Confirm: SSIS 2008 Package with Script Task won't Run on a 2012 Server

We have tasked to load the Centers for Medicare & Medicaid Services(CMS) NPI file...or otherwise know as the "National Plan and Provider Enumeration System(NPPES)" file. The file is created and posted as a monthly full file cut and then weekly incrementals. We are attempting to load the full monthly file which is 5.7GB. Soooo leads me to my questions...plural...

How much space do I need to allocate to my SQL Server Database and Table? 5.7GB or is there overhead?
For efficiency purposes, what do I need to do to my SSIS Package that will run this efficiently? And the most timely
Should I load this with SSIS or some other means because of the huge volume?
Do I need to explicitly define my data fields on my Flat File Connection Manager? I think I saw somewhere that you should explicitly define your data columns and lengths to avoid any overhead

If there are any additional suggestions, please let me know.

Thanks for your review and am hopeful for a reply.

↧

SSIS package run as a job from sql agent does nothing

May 19, 2016, 11:32 pm

≫ Next: SQL Agent job returns Command Line error for package from Integration Services Catalog

≪ Previous: Data Load of a HUGE National Database File and any hints...pointers

Hi, I am using SSIS for Visual Studio SSDT 2013 and SQL Server 2014 database. In my SSIS package I have a script task which pulls data from an api and writes data to a table. When I run this package in Visual Studio environment everything works as expected but when I run the same from SQL Server Agent as a job it does nothing. Not sure what might be causing this. Please point me in the right direction.

Thanks in advance....

SQLEnthusiast

↧

SQL Agent job returns Command Line error for package from Integration Services Catalog

May 25, 2016, 8:43 pm

≫ Next: The event "OnPipelinePreEndOfRowset" was received, but didn't have correct parameters.

≪ Previous: SSIS package run as a job from sql agent does nothing

I have a package that I am trying to schedule from the SQL Agent, the package is deployed into the Integration Services Catalog, it returns this error

"Option "not" is not valid. The command line parameters are invalid. The step failed."

It has 12 parameters, which could be the issue, but I can't see the command line now that the package is in the Integration Services catalog

Any help much appreciated

Regards

Andy

CRM 4, SQL Server and .Net developer using C#

↧

The event "OnPipelinePreEndOfRowset" was received, but didn't have correct parameters.

May 25, 2016, 10:26 am

≫ Next: Using MSBuild on TFS 2013 to deploy an SSIS package

≪ Previous: SQL Agent job returns Command Line error for package from Integration Services Catalog

Running packages with dtexec - SQL Server 2012.

These packages were upgraded to 2012.

Package executes successfully, but have several messages in logs:

The event "OnPipelinePreEndOfRowset" was received, but didn't have correct parameters.
The event "OnPipelineRowsSent" was received, but didn't have correct parameters.

Can not figure out what these messages mean and what needs to be done so they do not appear.

↧

Using MSBuild on TFS 2013 to deploy an SSIS package

May 25, 2016, 10:32 am

≫ Next: Insert Comma Separated value to CSV in SSIS.

≪ Previous: The event "OnPipelinePreEndOfRowset" was received, but didn't have correct parameters.

Hi,

I am new to SSIS packages and need to find a way to build and deploy an SSIS package. I have looked at a lot of pages on the net and basically all I have seen are snippets of code, mostly in PowerShell that can deploy an SSIS package into the Integration Service on SQL Server. We are using dtexec to run the package from a webservice.

I have the package and I have the code to execute it, but what I cannot do at the moment within TFS is to create a build step which would create a nuget package so that our Octopus Deploy system can then install it

Our current environment is:

Dev

Windows 8.1, Visual Studio 2013 Prof, SQL Server 2014

Server

Windows Server 2012 R2, SQL Server 2014, TFS Server 2015 Update 2, Octopus Deploy

Any help in this would be gratefully received.

↧

Insert Comma Separated value to CSV in SSIS.

May 25, 2016, 9:50 am

≫ Next: Managing the data import into a SQL table with an identity pk inside a data flow task - SSIS 2012

≪ Previous: Using MSBuild on TFS 2013 to deploy an SSIS package

Hi,

I have a Dataset, derived from SQL server table. One of the column has Comma separated values. We need to generate a CSV file out of the given data set.

Problem Statement:

While we try to generate the CSV output, the column having "comma separated value" gets split into different columns.

For Example

Sample Dataset

=========
Id | Address |
=========
1 | A |
=========
2 | B,C |
=========
3 | D,E,F |
=========

I need the Address column to flow in to the csv as it is (Address Column in CSV should have the exact values as in the sql table).For eg. for record 3 the Address Column in Address should "D,E,F ".

Thanks,

Sidd

↧

Managing the data import into a SQL table with an identity pk inside a data flow task - SSIS 2012

May 25, 2016, 7:06 am

≫ Next: Data mining viewer not showing all input attributes from mining model (SSAS mining Structure)

≪ Previous: Insert Comma Separated value to CSV in SSIS.

Hi,

I'm creating a SSIS pkg to feed a SQL Server table having an identity primary key.

I'm trying to use a data flow task, but the run goes in error, or because I cannot to pass a NULL valuer or because I cannot to pass a specified value.

Any suggests to solve this issue? Thanks

↧

Data mining viewer not showing all input attributes from mining model (SSAS mining Structure)

May 26, 2016, 12:05 am

≫ Next: To bulid SSIS package taking more time after providing the value through expression to arguments property in execute process task

≪ Previous: Managing the data import into a SQL table with an identity pk inside a data flow task - SSIS 2012

Hello Everyone,

I have created one mining structure using ssas.

I am not able to see all attributes in mining model viewer which I selected in mining model, I tried by changing content for eg:- Discrete, ordered, cyclic, but still not appearing in mining model viewer.

Can anyone tell me which property/step I am missing?

↧

Latest Images