Hi all,

i have a ssis 2012 package with two exec sql task.

first sql task get the the date and stores in the variable @datevalue (this varibale is created as datetime in the package)

second sql task's sql command DELETE from table where datecol > = ? (@value)

table's datecol is of dattime type

here is the problem it won't delete the record in the table with the value 2014-06-16 08:24:02.850 when the @datevalue is 6/16/2014 8:24:02 AM and this only happens in ssis 2012. if I run that package in 2008 it deletes that record but not in 2012?

any ideas plss?  

I am new to SSIS and am trying to create SSIS package.

I have value per line in a text file. There is a table consisting of records with each of the values in the text file. The SSIS package should generate text files for each of the value in the text file with the data from the table.

Details:

Text File – abcd.txt

10

11

12

13

Table

The SSIS package should create the following text files with the records that are associated with 10, 11,12 and 13:

10.txt

11.txt

12.txt

13.txt

Thanks in advance

I have this source table
 

Employee(ID, Name)
 
I have this Destination Table
 EmployeeImported(ID,Name,DesgnationID)
 
During transformation I want to Insert DesignationID from C#. I add a parameter in dataflowtask. I Don't Know What to do before the destination source. Mean I have to write script task, sqlcommand or something else? I'm newer and stuck in it. Kindly guide me?

I am trying to install BIDS feature from sql server 2008 R2 on window server 2008 R2 , but installation is failing with below error. Have some one encountered similar error.

Description:
  SQL Server 2008 R2 Setup has encountered an error. 

Problem signature:
  Problem Event Name:SQL100MSI
  Problem Signature 01:10.50.2550.0
  Problem Signature 02:9.0.21022
  Problem Signature 03:vs_shell.msi
  Problem Signature 04:0x162A16FE
  Problem Signature 05:0x1603
  Problem Signature 06:Install_VS_Shell
  OS Version:6.1.7601.2.1.0.274.10
  Locale ID:1033

Additional information about the problem:
  LCID:1033

Read our privacy statement online:
  http://go.microsoft.com/fwlink/?linkid=104288&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline:
  C:\Windows\system32\en-US\erofflps.txt

Hi experts,

 Can you pls explain the use of Transaction option in SSIS package ? I am little confuse on the part.

Thanks

Thanks for reading this.

I have just implemented a new database on our live environment, this database is fed information by a service brokers from our two transactional databases.  When the messages come into the new database we have a series of stored procedures which manipulate the feed for each specific user.

I also have another set of stored procedures which re-manipulate the data by user instead of by vehicle, these are used by our maintenance screens to ensure we can change users visibility of the data. 

The below stored procedure keeps locking up  I have now amended this sp to update a temp table first and then just undertake one update against the main database, however this has not aided the situation, I have also amended so I can update in smaller batches and if this fails keep retrying until it is successful.  This works 100% on our dev environment but on live it keeps locking.   The throughput on the servicebrokers is not great enough to register the number of failures, I therefore believe I am locking myself within this sp?

Any ideas would be greatfully received, apprecieate this is a heavy sp, I have included ---'THIS UPDATE KEEPS LOCKING'  at the point of failure.

regards

Tom

USE [Isight]
GO
/****** Object:  StoredProcedure [dbo].[UserVisibilityForVehicles]    Script Date: 07/18/2014 08:47:07 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO



ALTER PROCEDURE [dbo].[UserVisibilityForVehicles]
	-- Add the parameters for the stored procedure here
@Username VARCHAR(50) 
 WITH RECOMPILE
AS
BEGIN
	-- SET NOCOUNT ON added to prevent extra result sets from
	-- interfering with SELECT statements.
	SET NOCOUNT ON;

	PRINT 'UserVisibilityForVehicles Started'
   
	-- Now Start to check security for user.
	
	IF EXISTS ( SELECT ID FROM dbo.SecurityTable WHERE userid = @Username AND Deleted = 0)
	BEGIN
	
	
			CREATE TABLE #VehicleToUsers(ID BIGINT, NewRecord BIT DEFAULT(0))
			CREATE CLUSTERED INDEX IDX_VehicleToUsers ON #VehicleToUsers(ID)
			CREATE NonCLUSTERED INDEX IDX_NewRecord ON #VehicleToUsers(ID)
			
			INSERT INTO #VehicleToUsers
					( ID )
			(
				SELECT Distinct Veh.[ID]
				FROM [iSight].[dbo].[Vehicle] Veh WITH (NOLOCK) 
				INNER JOIN SecurityTable WITH (NOLOCK) ON  Veh.[System] = SecurityTable.[System]
				WHERE SecurityType = 1  AND UserID = @Username AND SecurityTable.deleted = 0 
			)
			
			INSERT INTO #VehicleToUsers
					( ID )
			(
				SELECT DISTINCT Veh.[ID]
				FROM [iSight].[dbo].[Vehicle] Veh WITH (NOLOCK) 
				INNER JOIN SecurityTable WITH (NOLOCK) ON Veh.[System] = SecurityTable.[System] AND Veh.CurrentSite = SecurityTable.[Site]
				WHERE SecurityType = 2 AND UserID = @Username AND SecurityTable.deleted = 0 
			)
			
			
			
						BEGIN
							PRINT 'UserVisibilityForVehicles: ' + @Username
							INSERT INTO #VehicleToUsers
										( ID )
								(
							SELECT  DISTINCT   Vehicle.ID
							FROM         Manufacturer WITH (NOLOCK) INNER JOIN
												  ManufacturerAgreementSubcustomer WITH (NOLOCK) ON Manufacturer.ID = ManufacturerAgreementSubcustomer.ManufacturerID INNER JOIN
												  ManufacturerMake WITH (NOLOCK) ON Manufacturer.ID = ManufacturerMake.ManufacturerID INNER JOIN
												  Vehicle WITH (NOLOCK) ON ManufacturerMake.Make = Vehicle.Make AND ManufacturerAgreementSubcustomer.Agreement = Vehicle.CurrentAgreement INNER JOIN
												  SecurityTable WITH (NOLOCK) ON Manufacturer.ManufacturerGroupID = SecurityTable.ManufacturerGroupID AND Vehicle.System = SecurityTable.System
							WHERE     (SecurityTable.SecurityType = 3)  AND (SecurityTable.UserID = @Username) AND ManufacturerMake.Deleted = 0 AND ManufacturerAgreementSubcustomer.Deleted = 0 AND SecurityTable.deleted = 0 
								)
								INSERT INTO #VehicleToUsers
										( ID )
								(
							SELECT DISTINCT Vehicle.ID
							FROM         Manufacturer WITH (NOLOCK) INNER JOIN
												  ManufacturerAgreementSubcustomer WITH (NOLOCK) ON Manufacturer.ID = ManufacturerAgreementSubcustomer.ManufacturerID INNER JOIN
												  ManufacturerMake WITH (NOLOCK) ON Manufacturer.ID = ManufacturerMake.ManufacturerID INNER JOIN
												  Vehicle WITH (NOLOCK) ON ManufacturerMake.Make = Vehicle.Make AND ManufacturerAgreementSubcustomer.Agreement = Vehicle.CurrentAgreement INNER JOIN
												  SecurityTable WITH (NOLOCK) ON Vehicle.System = SecurityTable.System AND Manufacturer.ID = SecurityTable.ManufacturerID
							WHERE     (SecurityTable.SecurityType = 4) AND (SecurityTable.UserID = @Username) AND ManufacturerMake.Deleted = 0 AND ManufacturerAgreementSubcustomer.Deleted = 0 AND SecurityTable.deleted = 0 
								)
									INSERT INTO #VehicleToUsers
										( ID )
								(
									SELECT DISTINCT Vehicle.ID
									FROM         Manufacturer WITH (NOLOCK) INNER JOIN
														  ManufacturerAgreementSubcustomer WITH (NOLOCK) ON Manufacturer.ID = ManufacturerAgreementSubcustomer.ManufacturerID INNER JOIN
														  ManufacturerMake WITH (NOLOCK) ON Manufacturer.ID = ManufacturerMake.ManufacturerID INNER JOIN
														  Vehicle WITH (NOLOCK) ON ManufacturerMake.Make = Vehicle.Make AND ManufacturerAgreementSubcustomer.Agreement = Vehicle.CurrentAgreement INNER JOIN
														  SecurityTable WITH (NOLOCK) ON Vehicle.System = SecurityTable.System AND Manufacturer.ID = SecurityTable.ManufacturerID AND 
														  ManufacturerAgreementSubcustomer.ID = SecurityTable.ManufacturerAgreementSub INNER JOIN
														  ManufacturerUserAgreementSubcustomer WITH (NOLOCK) ON 
														  ManufacturerAgreementSubcustomer.ID = ManufacturerUserAgreementSubcustomer.ManufacturerAgreementSubcustomerID AND 
														  SecurityTable.ManufacturerID = ManufacturerUserAgreementSubcustomer.ManufacturerID AND 
														  SecurityTable.UserID = ManufacturerUserAgreementSubcustomer.UserName
									WHERE     (SecurityTable.SecurityType = 5) AND (SecurityTable.UserID = @Username) AND (ManufacturerMake.Deleted = 0) AND 
														  (ManufacturerAgreementSubcustomer.Deleted = 0) AND (ManufacturerUserAgreementSubcustomer.Deleted = 0) AND SecurityTable.deleted = 0 
								)
										INSERT INTO #VehicleToUsers
										( ID )
								(
									SELECT DISTINCT Vehicle.ID
									FROM         Manufacturer WITH (NOLOCK) INNER JOIN
														  ManufacturerAgreementSubcustomer WITH (NOLOCK) ON Manufacturer.ID = ManufacturerAgreementSubcustomer.ManufacturerID INNER JOIN
														  ManufacturerMake WITH (NOLOCK) ON Manufacturer.ID = ManufacturerMake.ManufacturerID INNER JOIN
														  Vehicle WITH (NOLOCK) ON ManufacturerMake.Make = Vehicle.Make AND ManufacturerAgreementSubcustomer.Agreement = Vehicle.CurrentAgreement AND 
														  ManufacturerAgreementSubcustomer.Subcustomer = Vehicle.CurrentSubCustomer INNER JOIN
														  SecurityTable WITH (NOLOCK) ON Vehicle.System = SecurityTable.System AND Manufacturer.ID = SecurityTable.ManufacturerID AND 
														  ManufacturerAgreementSubcustomer.ID = SecurityTable.ManufacturerAgreementSub INNER JOIN
														  ManufacturerUserAgreementSubcustomer WITH (NOLOCK) ON 
														  ManufacturerAgreementSubcustomer.ID = ManufacturerUserAgreementSubcustomer.ManufacturerAgreementSubcustomerID AND 
														  SecurityTable.UserID = ManufacturerUserAgreementSubcustomer.UserName AND 
														  SecurityTable.ManufacturerID = ManufacturerUserAgreementSubcustomer.ManufacturerID
									WHERE     (SecurityTable.SecurityType = 6) AND (SecurityTable.UserID = @Username) AND (ManufacturerMake.Deleted = 0) AND 
														  (ManufacturerAgreementSubcustomer.Deleted = 0) AND (ManufacturerUserAgreementSubcustomer.Deleted = 0) AND SecurityTable.deleted = 0 
								)
						END
			CREATE TABLE #VehicleToUserCopy(ID BIGINT, vehicleTableID BIGINT, Deleted BIT DEFAULT(1), UpdatedAt DATETIME DEFAULT (GETDATE()), UpdatedBy VARCHAR(50) DEFAULT('UserVisibilityForVehicles-Update'), NextToUpdate BIT DEFAULT(0))
			CREATE CLUSTERED INDEX idx_ID ON #VehicleToUserCopy(ID)
			CREATE NONCLUSTERED INDEX idx_VehicleTableID ON #VehicleToUserCopy(vehicleTableID)
			CREATE NONCLUSTERED INDEX idx_NextToUpdate ON #VehicleToUserCopy(NextToUpdate)
			INSERT INTO #VehicleToUserCopy
			        ( ID ,
			          vehicleTableID ,
			          Deleted
			        )
			(
			SELECT ID, vehicleTableID, Deleted
			FROM dbo.VehicleToUser
			WHERE Username = @Username
			)


			PRINT 'Starting to do updates'
			--Not required as default set to 1
			----UPDATE VehicleToUser
			----SET DELETED = 1
			----,UpdatedAt = GETDATE()
			----,UpdatedBy = 'UserVisibilityForVehicles'
			----FROM dbo.VehicleToUser WITH (NOLOCK)
			----LEFT JOIN #VehicleToUsers AS UsersVehicles ON VehicleToUser.VehicleTableID = UsersVehicles.ID
			----WHERE UserName = @Username AND UsersVehicles.ID IS null
			
			PRINT 'Starting to do updates - Set Deleted = 0'
	
	
	
	
			SET LOCK_TIMEOUT 1000 -- set to  second
			DECLARE @Tries tinyint
		
	
			UPDATE #VehicleToUserCopy
			SET Deleted = 0
			FROM #VehicleToUserCopy AS VehicleToUserCopy
			inner JOIN #VehicleToUsers AS UsersVehicles ON VehicleToUserCopy.VehicleTableID = UsersVehicles.ID
			
			
			
			
			INSERT INTO VehicleToUser(UserName, VehicleTableID, DELETED, UpdatedAt, UpdatedBy)
			(
				SELECT DISTINCT @Username, TempVehicle.ID, 0 , GETDATE(), 'UserVisibilityForVehicles-Insert'
				FROM #VehicleToUsers AS TempVehicle
				LEFT JOIN (
							SELECT VehicleTableID
							FROM #VehicleToUserCopy WITH (NOLOCK)
						  ) AS [VehicleToUser] ON TempVehicle.ID = [VehicleToUser].VehicleTableID
				WHERE [VehicleToUser].VehicleTableID IS null
							
			)
			
			
			SELECT TOP 1 * FROM #VehicleToUserCopy
			WHILE @@rowcount > 0 
				BEGIN
						SET ROWCOUNT 5000
						SELECT @Tries = 1
						WHILE @Tries <= 3

							  BEGIN

							 BEGIN TRANSACTION

							 BEGIN TRY
									
									UPDATE #VehicleToUserCopy SET NextToUpdate = 1
				 				 
				 				 
				 					 ---'THIS UPDATE KEEPS LOCKING'
									UPDATE dbo.VehicleToUser
									SET Deleted = VehicleToUserCopy.Deleted
									, UpdatedAt = GETDATE()
									, UpdatedBy = VehicleToUserCopy.UpdatedBy
									FROM VehicleToUser
									inner JOIN #VehicleToUserCopy  AS VehicleToUserCopy ON VehicleToUser.ID = VehicleToUserCopy.ID
									WHERE VehicleToUserCopy.NextToUpdate = 1			
								
									DELETE FROM #VehicleToUserCopy WHERE NextToUpdate = 1	
									COMMIT    

							  -- therefore we can leave our loop
							  BREAK

							 END TRY
							 
							 BEGIN CATCH

									ROLLBACK --always rollback 


								PRINT 'Rolled Back '
							  -- Now check for Blocking errors 1222 or Deadlocks 1205 and if its a deadlock wait for a while to see if that helps
								 SELECT ERROR_MESSAGE()
								  IF ERROR_NUMBER() = 1205 OR ERROR_NUMBER() = 1222

									BEGIN

										 -- if its a deadlock wait 2 seconds then try again
									   IF ERROR_NUMBER() = 1205
										 BEGIN	-- wait 2 seconds to see if that helps the deadlock
												
												WAITFOR DELAY '00:00:02'
										 END   

									   -- no need to wait for anything for BLOCKING ERRORS as our LOCK_TIMEOUT is going to wait for half a second anyway
									   -- and if it hasn't finished by then (500ms x 3 attempts = 1.5 seconds) there is no point waiting any longer

									END      

							  
									 SELECT @Tries = @Tries + 1  -- increment and try again for 3 goes

							  -- we carry on until we reach our limit i.e 3 attempts
							  CONTINUE    

							   END CATCH

							  END
				
				
				
					SELECT TOP 1 * FROM #VehicleToUserCopy	
				End
			
			 
			SET ROWCOUNT 0
		
			
			
		
			
			--PRINT 'UserVisibilityForVehicles Stopped'
			--PRINT 'UserVisibilityForVehicleImages starting'
			--	EXEC UserVisibilityForVehicleImages @Username
			--PRINT 'UserVisibilityForVehicleImages Stopped'
				
			--PRINT 'UserVisibilityForVehicles_Tags starting'
			--	EXEC UserVisibilityForVehicles_Tags @UserName
			--PRINT 'UserVisibilityForVehicles_Tags Stopped'
			
			--PRINT 'UserVisibilityForVehicle_TS3 starting'
			--	EXEC UserVisibilityForVehicle_TS3 @Username
			--PRINT 'UserVisibilityForVehicle_TS3 Stopped'
	
	END
	ELSE
	
	BEGIN
			DELETE FROM dbo.VehicleToUser WHERE username = @Username
			DELETE FROM dbo.VehicleToUser_UserCurrentImageCount WHERE username = @Username
			DELETE FROM dbo.VehicleToUser_UsersCurrentVehicles WHERE username = @Username
	
	End
END

We have two data sources:

1 - mdb files (one file downloaded every hour)

2 - xls files (multiple files downloaded every week)

These files are downloaded to different folders.

I am currently using a Windows Service/File Watcher to monitor the directory where the mdb files are download. This then executes an SSIS package to upload the data to SQL Server. All is working well.

For data source two, I was considering doing something similar.

Firstly, does this approach seem ok?

And could I experience issues if both SSIS tasks are importing data at the same time? (i.e. to the same table). Is there a way around this?

Thanks,

Darren

In my SSIS package, I'm using a WMI file watcher combined with a ForEach loop to process excel files and load them into a SQL Server database as they are deposited in a drop box folder. Recently I've been getting truncation errors on one of my fields (Comments). I know that this is because Excel is scanning the first 8 rows to determine column length and there are several records that have a comments field that is greater than what excel determined.

I've tried going to the excel editor in my advanced editor and setting the data type length of my external and output columns to a larger length (500 W_STR vs 255 W_STR) but I get a warning saying that the data type is not valid. I've tried adding IMEX=1 to my extended properties in my excel connection string but no dice. Still getting truncation errors.

I was wondering if anyone had any thoughts or suggestions? I'm always getting new files so the comments field will potentially always encounter this type of error. This SSIS package runs on a production server so making a Registry change to the excel drivers regarding the scanning of the first 8 rows is not an option for me.

Migince

Hi all,

I'm currently exporting my XML files from Magento, the webshop client. However I cannot use the XML task in ETL because I get the error about namedspaces. I can open the XML file itself no problem, it contains a few columns with data, some empty values. What I do to solve the error is create the script in ETL that converts the XML file so that I can make the SDX schema. The ETL script itself works 

<?xml version="1.0" encoding="utf-8" ?> 
<xsl:stylesheet version="1.0"         xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> 
  <xsl:output method="xml" indent="no" /> 
  <xsl:template match="/|comment()|processing-instruction()"> 
    <xsl:copy> 
      <xsl:apply-templates /> 
    </xsl:copy> 
  </xsl:template> 
  <xsl:template match="*"> 
    <xsl:element name="{local-name()}"> 
      <xsl:apply-templates select="@*|node()" /> 
    </xsl:element> 
  </xsl:template> 
  <xsl:template match="@*"> 
    <xsl:attribute name="{local-name()}"> 
      <xsl:value-of select="." /> 
    </xsl:attribute> 
  </xsl:template> 
</xsl:stylesheet> 

But when I open the newly created file he looks really odd: See below for the initial file and the new file (based on how many times I open the file, i get another weird look...). Please help me out it's killing me :)

Hi all,

   I am having a file name as T001_2014_07,how to get the file name and create table with same as File name in runtime,pls help me,

Thanks

Hi,

I have heterogeneous sources (txt,foxpro,ora tables etc) to be loaded to SQL Server, which I've SSIS pkg working fine to load these data sources . I need to do a load balance to check the records in the file to compare it with loaded rec in sqlserver tables. Please guide me with the logic to start with, also if there are any generic code available which can be edited to use for load balancing. 

Thanks

Neil

I have a plain text data file to be imported into SQL format.

The file contains 11 columns, the first 10 columns are length fixed, the last column is not.

Is there a way to handle this in SSIS? or Bulk Insert format?

Thank you very much.

I'm looking to write some SQL code to transfer customer confidential data into anonymous data.

To be honest i'm struggling, has anybody done something similar? 

Umar Javed

The text files I'm importing always contain a fixed set of columns for example total number of full set of columns is 60, or a subset of those columns (some csv contain 40 columns, some contain 30 or 20 or any other number.) .  I would like to import these csv based on the column header inside the each csv, if it is a subset of full column set then the missing columns can be ignored with null value.

At the moment in SQL 2012, if I import a subset of columns in the csv file, the data doesn't import...I assume because the actual file doesn't include every column defined in the flat file source object? 

Is it possible to accomplish this without dynamically selecting the columns,  or using script component?

Thanks for help.

Sea Cloud

How can I use script component to check the existence of the file?  I have a excel file which I am loading to table, one of the column of excel file has complete file name with path for each line item, I want to use this to check if the file exists or not.

I want to do this in Data flow task using script component, and pass Yes or No value as a output column to the OLEDB destination to update a log table, weather the file exists or not

excel file

ID     FilePath

1c:\file1.txt

2      c:\file2.accdb

3.     c:\file3.csv

Target table I want to update it as

IDFilePath              FileExist

1c:\file1.txt           Y

2      c:\file2.accdb      Y

3.     c:\file3.csv          N -- in case of file do not exist 

Thanks

Neil

I'm trying to use the destination assistant in an SSIS project in Visual Studio v2010 to configure a database to hold fields from a flat file source.  I select the Native OLE DB/SQL Server Native Client 11.0 as a provider.  No connections show up in the connection manager.

I select 'MSSQL2008_R2', but there are no databases listed there even though I have several databases showing up in SSMS.  I've double checked the database name and connections; I've got windows authentication set as the authentication method.  I am listed in these databases as dbo and I'm an SA user.

Is there another step I need to enable the OLE DB connection from SSMS?

 I have designed a package which will  search for record and return the result to an excel destination.

first data flow task-1: this will take input from excel and record them into recordset destination.

second data flow task-2:  search for the records which are present in the record set and results into excel destination.

Requirement : create new excel file on every package execution .

below is the package :

I have a column which has 05MAY2006:04:34:00.000000 it is stored as varchar(25). I need to save it as datetime in the same column. I have tried using 

update tablename
set columnname = (SUBSTRING(columnname,1,2) + '-' + SUBSTRING(columnname,3,3) + '-' + 
SUBSTRING(columnname,6,4) + ' ' + SUBSTRING(columnname,11,8));

and then 

alter table tablename

alter columnname datetime;

but later it shows up the error

Msg 242, Level 16, State 3, Line 1
The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.

How do I change it any other opinion or any modification for the above query. Please help. Thank you.

The World Cup is here again!

Not balls... brains!

And YOU have been selected to play on our team!

Yes forum reader, step up and take a shot!

Slam some techie tips in the back of our nets!

No dribbling please, just lots of problem tackling.

So come on Gurus and use your head!

Show us your skills, wow us with your technique, and win the hearts of your nation!

All you have to do is add an article to TechNet Wiki from your own specialist field. Something that fits into one of the categories listed on the submissions page. Copy in your own blog posts, a forum solution, a white paper, or just something you had to solve for your own day's work today.

Drop us some nifty knowledge, or superb snippets, and become MICROSOFT TECHNOLOGY GURU OF THE MONTH!

This is an official Microsoft TechNet recognition, where people such as yourselves can truly get noticed!

HOW TO WIN

1) Please copy over your Microsoft technical solutions and revelations toTechNet Wiki.

2) Add a link to it on THIS WIKI COMPETITION PAGE (so we know you've contributed)

3) Every month, we will highlight your contributions, and select a "Guru of the Month" in each technology.

If you win, we will sing your praises in blogs and forums, similar to the weekly contributor awards. Once "on our radar" and making your mark, you will probably be interviewed for your greatness, and maybe eventually even invited into other inner TechNet/MSDN circles!

Winning this award in your favoured technology will help us learn the active members in each community.

Feel free to ask any questions below.

More about TechNet Guru Awards

Thanks in advance!
Pete Laker

#PEJL

Got any nice code? If you invest time in coding an elegant, novel or impressive answer on MSDN forums, why not copy it over to the one and onlyTechNet Wiki, for future generations to benefit from! You'll never get archived again!

If you are a member of any user groups, please make sure you list them in the Microsoft User Groups Portal. Microsoft are trying to help promote your groups, and collating them here is the first step.