Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Best way to move data from Dev DB to Test DB in different server?

April 12, 2016, 8:54 pm
$
0
0

Hi,

I have a development database with more than 100 tables with plenty of data in it. Data comes to this DEV server from a different source on weekly basis.

Now, it is my responsibility to move data from DEV to Test database for all the tables.

Can you please let me know which is the  best way to move data from DEV database in server A into  Test Database in Server B.

The data is going to be same - No changes or transformations. The data is going to be the same.

SSIS packages are good? Or any other simple ways so that on weekly basis, data will be moved?

Thanks in advance for your input.


↧
Search

Find Error Column Name in DFD ( Data conversion/Truncation)

April 13, 2016, 2:13 am
$
0
0

Hi,

I have a ssis package in which i am loading data from flat file to Staging table. I am redirecting error rows (datatype mismatch/Truncation) to another table but with SSIS i can just have Error description or error rows , lineage ID. How can i find the actual Column name whose got truncated or datatype mismatch.

 i have read no. of articles to read lineage id and then column name in scripts but didnt find a single easy and useful code in VB.net.. VB is standard in our project so can anyone help me with VB script????

Will highly appreciate...i am using 2012



↧
↧

SSIS - There were errors while the package was being loaded/ Failed to decrypt XML node "DTS:property"

April 13, 2016, 4:55 am
$
0
0

Hello everyone

I have a SSIS package that worked until last night (2008 R2). Now when i try to open the package i get an error telling me:

"There were errors while the package was being loaded.

The package might be corrupted.

See the Error list for details."

The error is:

"Error loading XXX.dtsx: Failed to decrypt protected XML node "DTS:Property" with error 0x800900B "Key not valid for use in specified state". You may not be authorized to access this information. This error occurs when there is a cryptographic error. Veryfy that the correct key is available"

I have seen several post/ etc telling that when this error occurs changing the projects property ProtectionLevel from EncryptSensitiveWithUserKey (by default) to DontSaveSensitive. However when i change it and try to save or debug it wants me to save it as a new file ones witch is odd but if i do accept or try to create a new file i get this error:

"The operation could not be completed"

At this point i dont know what to do. Does anyone have any suggestions? Yes i have restarted the program/ logged off the server and so on, nothing helps.

Thanks in advanced

Daniel

↧

Buffer Issue with ODBC Driver SSIS 2012.

February 10, 2016, 8:51 am
$
0
0

Hello ,

I have a package with the below tasks.

Source : MYSQL5.6 - Table - 50000000

Source Connection : ODBC Driver

Destination : SQLServer -Table

Destination Provider : OLEDB Driver.

I am trying to read 5000000 rows at a time. But It's throwing the below buffer error. 

Error: A buffer failed while allocating 10485024 bytes.
Error: The system reports 29 percent memory load. There are 32211845120 bytes of physical memory with 22597099520 bytes free. There are 4294836224 bytes of virtual memory with 116981760 bytes free. The paging file has 40801779712 bytes with 30649970688 bytes free.
[ODBC Source [2]] Error: Failed to add row to output buffer.

[ODBC Source [2]] Error: Open Database Connectivity (ODBC) error occurred.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on ODBC Source returned error code 0x80004005.  The component returned a failure code when the pipeline engine called PrimeOutput(). 

I am new to MYSQL and do not have the visibility on buffer settings of it.

Is it fair to assume that if we change buffer size on ODBC will solve the problem.

How can I solve the problem ? any inputs appreciated.

Thanks in advance.

↧

Execute rowcount query and pass value as variable in a control flow

April 13, 2016, 4:59 am
$
0
0

I know there is an abundance of information out there regarding what i am trying to do, and i have tried many of them with no luck.  I am sure it is something simple i am missing.  As usual, suggestions are gratefully welcomed !

I am using INTERSECT to find the duplicated between 2 tables.  I am obtaining the @@ROWCOUNT and assigning it to a column.  The query is such:

USE edgp
GO

SELECT @@RowCount as RCD1
FROM
(
 SELECT
   [txpid]
  ,[txpname]
  ,[txpadd1]
  ,[txpadd2]
  ,[txpcity]
  ,[txpst]
  ,[txpzip]
  ,[txphone]
  ,[txp_contac]
  ,[txpdba]
  ,[txp_begin]
  ,[phyadd1]
  ,[phyadd2]
  ,[phycity]
  ,[physt]
  ,[phyzip]
  ,[naic]
  ,[naic_desc]
  ,[Month]
 FROM edgp.DBO.EconDev_NewBusinessLicenses_Geocoded
 INTERSECT
 SELECT
   [txpid]
  ,[txpname]
  ,[txpadd1]
  ,[txpadd2]
  ,[txpcity]
  ,[txpst]
  ,[txpzip]
  ,[txphone]
  ,[txp_contac]
  ,[txpdba]
  ,[txp_begin]
  ,[phyadd1]
  ,[phyadd2]
  ,[phycity]
  ,[physt]
  ,[phyzip]
  ,[naic]
  ,[naic_desc]
  ,[Month]
 FROM edgp.DBO.EconDev_NewBusinessLicenses_2016
) D;

SELECT @@RowCount as [RCD]

The control flow is very simple, if duplicates exist, do not merge and exit.  If all values are unique, merge to the master data set.

I have set the execute SQL task general section as such:

Result set, i have created a variable User::RowCountDuplicate as int32 with value 0 (zero).  I have tried the value 0 and RCD as the Result Name, neither worked.

Based on the @@ROWCOUNT, i have set up constraint and expression:

When executing, this is the output I get:

SSIS package "C:\Users\oliver\documents\visual studio 2010\projects\NBL\NBL\Merge.dtsx" starting.
Error: 0x0 at Execute INTERSECT to detect duplicates before merging, want to avoid multi-merges of same dataset: No result rowset is associated with the execution of this query.
Error: 0xC002F309 at Execute INTERSECT to detect duplicates before merging, want to avoid multi-merges of same dataset, Execute SQL Task: An error occurred while assigning a value to variable "RowCountDuplicate": "Exception from HRESULT: 0xC0015005".

I think that covers it.  Again, I appreciate the help !


↧
↧

deploy packages

April 13, 2016, 6:00 am
$
0
0
We are running sql server 2008 r2.    When currently deploying an SSIS package to the development instance, I sign into integration services thru sql server management studio, expand the msdb folder and import the package into the appropriate subfolder.   Due to some company reorganization, we may be losing the permissions to deploy in this manner in the future.     What are other ways to deploy the packages?      
↧

Job failure occured due to communication and partial data loded

April 13, 2016, 7:17 am
$
0
0

Hi can any one suggest me how to resolve the below issue

If Job failure occurred in the middle of the process due to network issue and data loaded partially from source to dwh.

When we restart the job process, I don't want to load data that has already been loaded.

for example :

I've got about 100 million rows that I'm moving in SSIS 2012 via a Data Flow Task.  It's pretty much a straight table data copy from source to destination. 

Due to communication problem job has been terminated and loaded 50million records into destination table.

to resolve this issue what is the best approach

please share your ideas from your experience.

thanks in advance

HarSan



↧

SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.

April 12, 2016, 2:44 pm
$
0
0

[Excel Source [1]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009.  There may be error messages posted before this with more information on why the AcquireConnection method call failed.

My package was working fine then all of a sudden i started seeing this error. I already have the "Run64BitRuntime" property set  to False.

I have no idea why it stopped working, its very weird though.

I observed this second error in Execution task

[Connection manager "Excel Connection Manager"] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available.  Source: "Microsoft Access Database Engine"  Hresult: 0x80004005  Description: "The Microsoft Access database engine cannot open or write to the file ''. It is already opened exclusively by another user, or you need permission to view and write its data.".

All files are closed, no user is using it.
↧
Search

Test for the existence of a SharePoint list

April 8, 2016, 7:20 am
$
0
0

I have a scenario where data is to be read from a SharePoint list but the name of the list is unknown. The name will always follow the format:

        List_XX_YY

with XX being a two-digit year and YY being the two-digit representation of the following year. However I won't know whether the current year is represented by the XX or the YY at design time.

For example, this year's list could be:

        List_15_16

or

        List_16_17

I know I can parameterise the name of the Collection to be used in an OData source but I either need a way to try one list and use the other if it isn't there or to get a list of all the lists from which I can select the appropriate one to use.

Has anyone had a similar situation and, if so, did they find a solution?

Thanks.

↧
↧

Variable values doesn't change

April 13, 2016, 8:19 am
$
0
0

Hi 

I created a template and resusing when needed , The SSIS Variable values still pointing to old package name whenever i change the package name. 

Ex: I have built a package with name as Package1 and assigned the value to a variable as @[System::PackageName] when i changed the package name as TestPackage i was still getting the value in the variable as Package1. 

Please help

Thanks,

Rajesh

↧

SSDT 2013 script task exceldatareader issue

April 13, 2016, 8:56 am
$
0
0

Hi, I am using SSDT 2013. In the script task I am reading data from excel spreadsheet with ExcelDataReader using the code below.

string filePath;

filePath = @"\\Share\Data\Excel.xls";

FileStream stream = File.Open(filePath, FileMode.Open, FileAccess.Read);

//Reading from a binary Excel file
IExcelDataReader excelReader = ExcelReaderFactory.CreateBinaryReader(stream);
excelReader.IsFirstRowAsColumnNames = true;
DataSet result = excelReader.AsDataSet();
DataTable myDataTable = result.Tables["Data"];

For some reason the script task is failing without even getting into the main method. When I tried to debug line by line this is where it was failing

"IExcelDataReader excelReader = ExcelReaderFactory.CreateBinaryReader(stream);"

Could you please point me in the right direction.

Error Message:

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()

Thanks in advance......

SQLEnthusiast



↧

Job teriminated due to communication in the midle of the process, data loded partially into destidation table, what is best approach?

April 13, 2016, 8:22 am
$
0
0

Hi can any one suggest me how to resolve the below issue

If Job failure occurred in the middle of the process due to network issue and data loaded partially from source to dwh.

When we restart the job process, I don't want to load data that has already been loaded.

for example in detail :

I've got about 100 million rows that I'm moving in SSIS 2012 via a Data Flow Task.  It's pretty much a straight table data copy from source to destination. 

Due to communication problem job has been terminated and loaded 50million records into destination table.

to resolve this issue what is the best approach

please share your ideas from your experience.

thanks in advance

HarSan


HarSan

↧

Stop Debugging

March 7, 2013, 11:15 am
$
0
0

Hi I'm new to SSIS and am trying to figure out how it works.  I created a data source and exported the data to a flat file.  So, on my control flow tab the only thing there is one data flow task.  If I expand the data flow task, I have a ADO.Net data source and a flat file destination.  Once I execute the package it runs and creates the file, but just sits there until I click stop debugger.  Is there some type of end execution task that I should be including?  It just doesn't make sense to me that after the last step it doesn't stop. 

Thank you

↧
↧

SSIS Package Failed

April 13, 2016, 1:53 pm
$
0
0

I am getting following error when I was trying to run the package. could some explain me why this error is throwing.

*************************************************************************************************

Executed as user: NSCORP\pxtCRM. ...sion 10.50.4000.0 for 32-bit  Copyright (C) Microsoft Corporation 2010. All rights reserved.    Started:  12:27:40 PM  Error: 2016-04-13 12:27:52.38     Code: 0xC002F304     Source: Populate Minor Group TF Dynamics CRM Destination [31]     Description: An error occurred with the following error message: "SOAP security negotiation with 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' for target 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' failed. See inner exception for more details.".  End Error  Error: 2016-04-13 12:27:53.42     Code: 0xC002F304     Source: Populate Railroads TF Dynamics CRM Destination [31]     Description: An error occurred with the following error message: "SOAP security negotiation with 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' for target 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' failed. See inner exception for more details.".  End Error  Error: 2016-04-13 12:27:54.31     Code: 0xC002F304     Source: Populate Customer (CRM) TF Dynamics CRM Destination [16]     Description: An error occurred with the following error message: "SOAP security negotiation with 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' for target 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' failed. See inner exception for more details.".  End Error  Error: 2016-04-13 12:27:55.18     Code: 0xC002F304     Source: Populate Major Group TF Dynamics CRM Destination [31]     Description: An error occurred with the following error message: "SOAP security negotiation with 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' for target 'http://gatucitcrm03d.nscorp.ad.nscorp.com/CRMDEV/XRMServices/2011/Organization.svc' failed. See inner exception for more details.".  End Error  Error: 2016-04-13 12:27:56.05     Code: 0xC0047017     Source: Populate Minor Group SSIS.Pipeline     Description: component "TF Dynamics CRM Destination" (31) failed validation and returned error code 0xC020801C.  End Error  Error: 2016-04-13 12:27:56.93     Code: 0xC0047017     Source: Populate Railroads SSIS.Pipeline     Description: component "TF Dynamics CRM Destination" (31) failed validation and returned error code 0xC020801C.  End Error  Error: 2016-04-13 12:27:57.89     Code: 0xC0047017     Source: Populate Customer (CRM) SSIS.Pipeline     Description: component "TF Dynamics CRM Destination" (16) failed validation and returned error code 0xC020801C.  End Error  Error: 2016-04-13 12:27:58.78     Code: 0xC0047017     Source: Populate Major Group SSIS.Pipeline     Description: component"TF Dynamics CRM Destination" (31) failed validation and returned error code 0xC020801C.  End Error  Error: 2016-04-13 12:27:59.73     Code: 0xC004700C     Source: Populate Minor Group SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2016-04-13 12:28:00.73     Code: 0xC004700C     Source: Populate Railroads SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2016-04-13 12:28:01.80     Code: 0xC004700C     Source: Populate Customer (CRM) SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2016-04-13 12:28:02.71    Code: 0xC004700C     Source: Populate Major Group SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2016-04-13 12:28:03.59     Code: 0xC0024107     Source: Populate Minor Group      Description: There were errors during task validation.  End Error  Error: 2016-04-13 12:28:04.49     Code: 0xC0024107     Source: Populate Railroads      Description: There were errors during task validation.  End Error  Error: 2016-04-13 12:28:05.33     Code: 0xC0024107     Source: Popul...  The package execution fa...  The step failed.

↧

catalog.start_execution SYNCHRONIZED in Transaction

March 17, 2016, 8:08 am
$
0
0

Hi, I am trying to execute a SSIS Package from a stored procedure within a transaction. I also want the execution to be in Synchronized mode. This is an example of my stored procedure code:

begin try
begin transaction

	/*
	inserts, updates
	*/

	declare @execution_id bigint
	EXEC [SSISDB].[catalog].[create_execution]	@package_name=N'My Package.dtsx',
												@execution_id=@execution_id OUTPUT,
												@folder_name=N'MyFolder',
												@project_name=N'MyProject',
												@use32bitruntime=False,
												@reference_id=Null
	EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,
															@object_type=50,
															@parameter_name=N'LOGGING_LEVEL',
															@parameter_value= 1
	EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id,
															@object_type=50,
															@parameter_name=N'SYNCHRONIZED',
															@parameter_value= 1
	EXEC [SSISDB].[catalog].[start_execution] @execution_id;

	/*
	throw error if execution status is not = 7 ("Success")
	*/

commit transaction
end try
begin catch
	if @@Trancount > 0
		rollback transaction;
end catch

The problem is that the package reaches a timeout and gets a status = 5 ("Pending").

If I remove the Synchronized setting, it works. If I remove the transaction, it works. But a combination of both will not work. Can anyone think of a simple workaround for this?

Thank you,

Filip

↧
Search

Real Time Interview Questions

April 13, 2016, 10:41 pm
$
0
0

Hi Guys,

I want to know where can I get real time questions for SSRS and SSIS for experienced candidates?

Please share links and some interview questions which you came across?

I am not asking theoretical questions but some questions like after transferring data from various source to SQL Db  what you will do ?

SQL BI Developer (Starting to learn more about SQL&BI)

↧

Cache Transformation fails with Buffer Memory error during load on SQL 2014 SSDT 2013, potential upgrade issue.

September 23, 2014, 7:58 am
$
0
0

My issue is loading 140 million records into a Cache Transformation fails constantly since upgrading to SSDT 2013 from 2010.

The error is reproducible 100% of the time while running the package in debug mode (dtsDebugHost.exe).

The server is rather large having the following configuration.

Windows Server 2012 Datacenter

Processor Xenon E5-4650 @ 2.70 GHz (4 proc)

768 GB RAM

SQL Server Max Memory 600 / Min 520GB

This issue has also been tested on a different server with similar results.

The source of the Cache is an OLE DB Connection to the local SQL server pulling 13 columns all having ID values.

The destination is a standard Cache Destination.

The errors are as follows and occur while both writing to file and full memory cache near 20.5 million records for both Cache types:

[SSIS.Pipeline] Information: The buffer manager detected that the system was low on virtual memory, but was unable to swap out any buffers. 14 buffers were considered and 14 were locked. Either not enough memory is available to the pipeline because not enough is installed, other processes are using it, or too many buffers are locked.

Information: Buffer manager allocated 4 megabyte(s) in 2 physical buffer(s).

Error: A buffer failed while allocating 1703856 bytes.

Error: The system reports 92 percent memory load. There are 412146331648 bytes of physical memory with 29133062144 bytes free. There are 4294836224 bytes of virtual memory with 8069120 bytes free. The paging file has 429326200832 bytes with 43001991168 bytes free.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Cache LicenseMaster" (2) failed with error code 0xC0208252 while processing input "Cache Input" (5). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

Our initial thoughts are that SQL server is not configured correctly to handle the usage of so many buffers.

I have tried both extremes of Cache property settings for MAXROWS and MAXBUFFERSIZE.

I have read many articles, but none seems to discuss buffer management in detail.

We appreciate your insights and recommendations.

Scott M

↧
↧

Master of Import/Export (SSMS) do not work

April 14, 2016, 2:20 am
$
0
0

Hello all,

Windows 7 Ultimate  x64

SQL Server 2014 ver. 12.0.4100.1 with SSIS

When I do right click to Data Base - then click Tasks - then click Import Data... - after this immediatelly show Error message:

TITLE: the import Wizard and export of SQL Server
------------------------------

Failed to create a data flow task SSIS. Make sure that the library DTSPipeline.dll available and registered. The wizard cannot continue and will be closed.

------------------------------
FOR MORE INFORMATION:

Cannot create a task from XML for task "", type,", {1CF081CA-677D-4561-967A-D06798F62DD6}" because of error 0xC0010014 "there was one or more errors. Should be more specific errors preceding this one that explains the details of the errors. This message is used as return value from functions that encounter errors.".
({89A919A6-2EF2-418C-A376-207A939AF8F3})

------------------------------
BUTTONS:

OK
------------------------------

I deinstalled and install the SQL Server again....but it didn't help me

Please help me to solve this problem?

↧

Dataflow Task modifications propagate invalid column names

April 14, 2016, 6:42 am
$
0
0

Hi!

I have a Data  Flow Task  which starts from a XML source from which many transformations (multicast, sort, join) descend.

The issue I’m facing is if I edit XML location of XML source, many Sort components show a red cross and if I double clicked them, “Restore Invalid column Reference Editor” shows with "Invalid columns", but I don’t know what to do with it. Nothing different in XML structure, only the location.

I can’t rebuild my solution. The only workout I found was to delete components  (Sort, join, etc) and re-create them, but that’s too much work . Do you know how I could make modifications without these cascade drawbacks?

Mary

↧

SSIS Error: "Microsoft SQL Native Client" Hresult: 0x80040E21 Description: Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.

April 14, 2016, 3:47 am
$
0
0

Hi Experts,

We have the mentioned error whicle processing some XML files via SSIS packages and having the Data read from an XML Source componenta and populated to a OLEDB destination.

This error as of now is random and occurs only for few files (say 40 out of 200), also if the failed files are processed again they go in well except some rejected again (say 40 in error processed again and 10 move to error again, 10 processed 2 move to error and finally all processed fine after this iterations).

This issue is happening after we migrated the package to SQL 2008 R2, the same packages on SQL 2005 works fine with no issues.

We have tried using different version sof SQL client (9,10,11) and all have same issue. Also tried checking handles and connection pooling and found no issues.

Have anyone else faced this in past and provide any advice to resolve this??

Thanks & Regards

VV

Vinish Viswanathan

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>