Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Avoiding Error on Flat file with extra column

May 27, 2015, 4:07 pm
$
0
0

Hello everyone. I am extremely new to SSIS, but have programmed in Base sas for years (4+).  I am trying to replicate some of the code I have in my base sas program in SSIS and am finding it extremely difficult (or impossible as of right now).  I am hoping someone can clarify on some easy questions I had.

#1) how can SSIS handle extra columns being found in a flat file connection manager and simply ignore those columns on load?

I work with a company that gets files from clients. Say the file should have 3 columns... Account #, Name, and Address.  however the next month they add the field "Age" in the second column position... Now the file has Account #, Age, Name, Address.. I can't figure out how to have SSIS do ANY of the following (I need it to be able to do all 3).

Solution 1) Simply ignore the column, and read in Account #, Name, Address

Solution 2) Read in the column and dynamically update the source destination (sql database) with the new field information

Solution 3) Error and send an email to a set of business users letting them know the file has problems.

Should I not be using a connection manager task to accomplish this in SSIS?

Thanks!

↧
Search

Looping through destination folder and checking if the file exist

May 29, 2015, 10:01 am
$
0
0

Hey guys, I just have a question? How to design ta SSIS package which loops through DESTINATION folder files and checks whether that file is there in the SOURCE or not.

If the file exists then I have to check the modified date on DESTINATION file if it is greater than 1 day delete that file. If the modified date is less than that SOURCE file then I have to copy that file to DESTINATION<o:p></o:p>

 <o:p></o:p>

If there are files which exists in SOURCE and not in DESTINATION, then how shall we copy all the files to the DESTINATION that are created on the day of execution of package.<o:p></o:p>

↧
↧

error while sending mail using script task in ssis 2008

September 15, 2014, 11:23 pm
$
0
0

Hi,

    i am trying to send mail using ssis 2008 script task.for my requirement i am not able to use send mail task.

code i have used is

declared read only variables system::packagename

 Dim PACKAGE As String
        PACKAGE = Dts.Variables("System::PackageName").Value.ToString()

        Dim myHtmlMessage As MailMessage
        Dim mySmtpClient As SmtpClient
        myHtmlMessage = New MailMessage("vijayabhaskar.perala@gmail.com", "vijayabhaskar.perala@gmail.com", "PACKAGE STATUS", PACKAGE + "WAS FAILED")
     mySmtpClient = New SmtpClient("smtp.gmail.com")
        mySmtpClient.Credentials = New NetworkCredential("vijayabhaskar.perala@gmail.com", "mypassword")
        mySmtpClient.EnableSsl = True
        mySmtpClient.Port = 587
        mySmtpClient.Send(myHtmlMessage)

------------------------------------

error i am getting is

Error: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Net.Mail.SmtpException: The SMTP server requires a secure connection or the client was not authenticated. The server response was: 5.5.1 Authentication Required. Learn more at
   at System.Net.Mail.MailCommand.CheckResponse(SmtpStatusCode statusCode, String response)
   at System.Net.Mail.MailCommand.Send(SmtpConnection conn, Byte[] command, String from)
   at System.Net.Mail.SmtpTransport.SendMail(MailAddress sender, MailAddressCollection recipients, String deliveryNotify, SmtpFailedRecipientException& exception)
   at System.Net.Mail.SmtpClient.Send(MailMessage message)
   at ST_c121e07caaa94c21bb1355d4f753112f.vbproj.ScriptMain.Main()
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, CultureInfo culture)
   at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()

-------------------------------------------------------------------------------------------------

can any one tell me where i am going wrong



↧

SMTP server issue

April 11, 2013, 12:19 pm
$
0
0

 Hi Everyone

I have an issue with SMTP server " send mail task ".  we see the following error

[Send Mail Task] Error: An error occurred with the following error message: "The SMTP server requires a secure connection or the client was not authenticated.

In SMTP connection manager what we should give in SMTP SERVER, if we use gmail id's in send mail task editor.

Looking for your help

Regards

Kumar

↧

Email doesn;t send

May 2, 2012, 11:19 am
$
0
0

I have a bi package that restores databases.  It works okay, but doesn;t always send a notification email as it is designed to do.  It seems to work if I have logged on the the machine.   It seems to fail if I haven't logged on in a while.  Here is the error log.

Send Mail Task Send Mail Task     Description: An error occurred with the following error message: "The SMTP server requires a secure connection or the client was not authenticated. The server response was: authentication required for relay (#5.7.1)".  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  3:00:00 AM  Finished: 3:01:07 AM  Elapsed:  67.159 seconds.  The package execution failed.  The step failed.

Same package works when I log on and run it.  Is this a credentials thing on the sql server agent?

Help

dave

↧
↧

Loading datetime field gives wrong dates in ssis

May 29, 2015, 6:13 pm
$
0
0

I am using Sql Server 2012. I have a table which has a field as Datetime (it is a table in Dynamics CRM 2011 so I have no control of the data type).

Say this field is called BisStartDate.

If I run this query in management studio,

selectBisStartDate, BisStartDateutcfrommyTable where _bisnumber=10375

I will get:

BisStartDate                                             BisStartDateutc               

2014-07-29 00:00:00.000                      2014-07-29 05:00:00.000

*in CRM, datetime is saved in 2 fields, one is the current time, the other one is the utc time.

You can see the offset  between the datetime and utc is 5 hours.

However when the same statement was running inside a SSIS package on the server, theresult returned is:

BisStartDate                                             BisStartDateutc               

2014-07-28 23:00:00.0000000            2014-07-29 05:00:00.000

And if I do

datediff(MINUTE,CRMAF_BisSection.ttc_section_startdatetimeutc,CRMAF_BisSection.ttc_section_startdatetime)

I will get -5 if I run it in ManagementStudio and -6 is running on server package(running inside VisualStudio will be -5, same as running a query in ManagementStudio).

I think when the record was saved, “date” is 5 hours offset to UTC time but now the system use the current utc offset which is 6 hours. I just want to use the BisStartDate as it is. How do I let the SSIS turn off the conversion.

The same datatime is saved in another system then we compare them to check the data entry. Now because of this one hour difference, sometime the Day will be different.

 

 

Thanks.

Holly

↧

Converting Stored Proc to Inline Query

May 29, 2015, 5:17 pm
$
0
0
Hello,

I have a stored proc that is being executed from an OLE DB Source component in my Data Flow. Takes one input parm, has several variables declared within, a derived table which is used in a join and all within a try catch block with transaction handling. No updates, just returning data, works great, except now I have been asked to replace these stored procs with inline queries.
ALTER PROCEDURE [dbo].[usp_Get_Test]
(
	@numberOfMonths				int
)
AS
BEGIN
	SET NOCOUNT ON
	set transaction isolation level read uncommitted

declare @TESTY table (TEST_ID char(12), TEST_SEQ_NO smallint, TEST_ID_ADJ_FROM char(12), TEST_IND char(1))

declare @BeginDate datetime
        ,@EndDate datetime
        ,@TestKey bigint
        ,@TestNumber varchar(255)

select   @BeginDate = dateadd(MONTH, @numberOfMonths, GETDATE())
select   @EndDate = GETDATE()

BEGIN TRANSACTION

BEGIN TRY

insert into @TESTY
        ( TEST_ID ,
          TEST_SEQ_NO ,
          TEST_ID_ADJ_FROM ,
          TEST_IND
        )
select  tl.TEST_ID
,       cl.TEST_SEQ_NO
,       tl.TEST_ID_ADJ_FROM
,       case
            when cl.TEST_ID is null then 'N'
            else 'Y'
        end as TEST_IND
from    TESTL tl
join    TESTCL cl on tl.TEST_ID = cl.TEST_ID
where   tl.TEST_DT > @BeginDate
        and tl.TEST_DT < @EndDate

select	td.TestNumber as [TEST_ID]

from    TestDetail td

left
join    @TESTY TY on td.TESTDataSource = 'TESTY' and td.TestNumber = TY.TEST_ID
        and TY.TEST_SEQ_NO = td.TestLineNumber

where   td.TestMonth > @BeginDate
        and td.TestMonth < @EndDate
        and td.TESTStatus in('tested','retested')

END TRY
BEGIN CATCH
    SELECT
        ERROR_NUMBER() AS ErrorNumber
        ,ERROR_SEVERITY() AS ErrorSeverity
        ,ERROR_STATE() AS ErrorState
        ,ERROR_PROCEDURE() AS ErrorProcedure
        ,ERROR_LINE() AS ErrorLine
        ,ERROR_MESSAGE() AS ErrorMessage

    IF @@TRANCOUNT > 0
        ROLLBACK TRANSACTION
END CATCH

IF @@TRANCOUNT > 0
    COMMIT TRANSACTION
END
The problems I have run into so far are...

SQL command text in OLE DB Source Editor does not like:
- TRY/CATCH block
- Will not let me use my input parm (@numberOfMonths int)
- When I hard code in my input parm (select   @BeginDate = dateadd(MONTH, -1, GETDATE())) I can parse query and run the step but no results are returned. So I am let to assume that it does not like the @TESTY derived table.

The query here as a sample has had pivots removed as well, but research suggests this should be an issue in the SQL command text.

Also, I know not even to try the Build Query... cause it will complain about any variable 
declarations (i.e., declare @BeginDate datetime).

So, if someone could confirm some things for me I would really appreciate it. For instance, can I pull this off with an Execute SQL Task? The problem is I don't see this available in the toolbox for the Data Flow. 

Also, would my error handling be done in the Event Handlers tab now and if so, is there a good example of this?

Thanks,
Buster

BusterCoder

↧

SSIS Package failed but SQL Agent Job Report Success

May 29, 2015, 1:09 pm
$
0
0

I created a simple SSIS package to check if a file exists. If the file does not exist then the SSIS package fails. I use SQL Agent job to execute this SSIS package. If the SSIS package fails then the Agent job should fail and send email to alert me the file does not exist.

I manually executed the SSIS package and made sure it failed (no file exists) so I can test the SQL Agent job to see it generated an email alert. Strangely, the SQL Agent job ran successfully for this failed SSIS package. Why?

↧
Search

Maintenance Plan : Subplan failed

April 10, 2015, 1:57 am
$
0
0

Hello~ I have a plan which have 3 subplans. one of it has problem ~and have the following error:

 EXECUTE msdb..sp..." failed with the following error: "

"The INSERT statement conflicted with the FOREIGN KEY constraint "FK_sysmaintplan_log_subplan_id". The conflict occurred in database "msdb", table "dbo.sysmaintplan_subplans", column 'subplan_id'. 

I have checked some forum and ask me to check dbo.sysmaintplan_subplans,sysjobs,sysmaintplan_log

and nothing special

and I know this problem is come from sp_maintplan_open_logentry, how can I know what parameter is passing to this SP ?

when the job is run ?

↧
↧

How avoid comma in csv file in SSIS

May 30, 2015, 4:52 am
$
0
0

I folks 
       source                                            Oledb destination
 column:id,name,address                    id,name,address

  data:1,ramu,ameerpet,hyd   

       in this case my source contain address column contian comma

Intervier questionos:
1. what will the out put? i said error
2.How you resolve this issue? 
3.if data is like this how you came to know?

thanks in advance waiting for your ans 



↧

SSIS Sort and Hyphens

March 14, 2014, 11:26 am
$
0
0
I have read about issues of collation when sorting in SQL vs. SSIS.

In my case, I am performing two SSIS sorts between 2 datasets:

Dataset1
email id flag
zzz-yyyy.lastname@hotmail.com 1 1
ZZZYYYY.LASTNAME@HOTMAIL.COM 1 2


Dataset2
ZZZYYYY.LASTNAME@HOTMAIL.COM 1

I sorted both datasets in SSIS with ignore case check for email address. I did a merge join (left join between dataset2 and dataset1) between the two datasets based on id,email. The expected result is:

ZZZYYYY.LASTNAME@HOTMAIL.COM 1 2

I am getting:
ZZZYYYY.LASTNAME@HOTMAIL.COM 1 null

The left join is returning null for the flag column as if they do not match.
If I remove row 1 of dataset1, it works fine.
If I have ignore case in the sort unchecked it works but I need it checked for other cases.

I am not mixing SQL and SSIS sorts. Everything is within SSIS sorts.

Any help is appreciated.

Thanks.
↧

SSIS DQS cleansing component mapping window missing check box

May 30, 2015, 9:53 pm
$
0
0

 am using DQS cleansing component inside SSIS 2013. In mapping window there is no check box to select input columns to map them to domain fields!!! Anyone any idea?

MOSS LOVER

↧

Modify SSIS package saved on server

May 10, 2006, 7:34 am
$
0
0

   I have saved an SSIS pacakage on my sql server.  I am able to see and run the package through Integrations services.  My question is, is there a way to edit the package through Management studio?

 

Thanks

Jim

↧
↧

Creating Dynamic Sql Server Tables through SSIS as per XML data files metadata

February 14, 2011, 10:58 pm
$
0
0

Hi All,

I have a scenario, need to create SQL server Tables dynamically.

I Have multiple xml data file on a particular location, and want to load those XML data into sql server tables, but he metadata of each xml data files are not same.

Hance the approach is that,

1. Pick first file from that location

2. Create a table according to that xml data file metada

3.  load data on newly created table.  

4. Pickup the next xml data files.

5. loop through, till the XML data files are exists on that location.

 

any suggestions, Ideas.

Thanks

Sandeep Vaishya

↧

Is there a Row limit for Excel ACE Driver?

May 31, 2015, 10:56 pm
$
0
0

I have a SSIS package running well in production however sometimes the package will fail when the excel file contains more than 25000+ rows.

The SSIS package is run by SQL Server Agent and is set to run in 32bit mode.

I checked the data by loading in batches and all data loaded successfully. But the funny thing is when I run the same package on my local development PC using BIDS and the same data file. The package loads all 25000+ rows successfully.

Is there some setting that is preventing all rows loading in the server environment.

↧
Search

Upgrade SSIS packages

May 28, 2015, 12:21 am
$
0
0

Hi ,

Can anyone tell me the detailed steps to upgrade the SSIS packages. We are migrating from SQL 2005 Enterprise Edition to SQL 2014 Standard Edition. Database migrations are happening smoothly , but i am facing difficulty in SSIS package migration.

What are the best practices for migrating/ upgrading the SSIS packages ? Does the Editions of SQL (2005 Enterprise to SQL 2014 Standard) impact the package upgrade.

Your help would be much Appreciated. Thanks in Advance :)

↧

Loading multiple XML file with different Metada In SQL Server Tables with SSIS 2008

February 16, 2011, 9:37 pm
$
0
0

Hi, 

I have multiple xml data file in a directory say C:\XMLData\ abc1.xml, abc2.xml, abc3.xml etc.

Need to loop through each file in ssis with Foreach loop container, and get the file name say abc1, and load the data of abc1.xml to abc1 table in sql server DB.

next iteration will pick up the abc2.xml and find the abc2 table in sql server DB then insert the data in abc2 table.

while each iteration, xml source should also point each xsd file correspondingly.

 Tables are already created in DB

 I solved my problem up to getting the file name from ech iteration and assigned file name to variable,

in oledb destination data access mode I select Table or view name variable, then corresponding table will get selected for data insertation.

 Just wanted to know how can I read each xsd file for each xml data files while iteration. 

Any early help will be appreciated. 

Sandeep Vaishya

Thanks Sandeep Vaishya
↧
↧

Set SSIS Package parameters in SQL Agent

May 28, 2015, 5:05 pm
$
0
0

Hello,

I have defined two package parameters in my 2014 SSIS package

1. ssas_backup_dir

2. ssas_instance

I would like to set them through the "Set Values" tab in a sql agent job. I know variables can be set as below

Property Path: \Package.Variables[User::VariableName] 
Value: SomeValue

These are not variables, but package parameters. I have tried many different variations trying to set the parameters but keep getting errors unable to set the parameter. 

DTExec: Could not set .......

Any ideas on how to set the project parameters in the set values tab in sql agent?

Thx



↧

Can not able to see 64 bit ODBC driver from SSIS odbc connection

May 31, 2015, 9:46 pm
$
0
0
Hi,

I am using SSIS 2014 with the below .net framework version and installed in Windows server 2012 R2 . I have installed my client's odbc drivers (both 32 bit and 64 bit) in my production server and created ODBC system DSNs for 32 bit and 64 bit.

When i open SSIS 2014 and tried to create the odbc connection but i can able to see only the 32 bit system DSN connection ,i can't able to see my 64 bit odbc system dsn connection.

Microsoft Visual Studio 2012 Shell (Integrated)
Version 11.0.50727.1 RTMREL
Microsoft .NET Framework
Version 4.5.51650

SQL Server Integration Services   
Microsoft SQL Server Integration Services Designer
Version 12.0.1524.0
And i installed my client odbc drivers(32,64 bit) and created ODBC system DSNs in my local system and when i open ssis 2014 and i can able to see both the ODBC system DSNS(32,64) connections from SSIS ODBC connection.

I am using below version of .net framework in my local system which was installed in windows 7 and i have SSIS 2012 also installed in my system and i can able to see both ODBC connections using 2012 as well in my local system.

Microsoft Visual Studio 2012 Shell (Integrated)
Version 11.0.50727.1 RTMREL
Microsoft .NET Framework
Version 4.5.50938

SQL Server Integration Services   
Microsoft SQL Server Integration Services Designer
Version 12.0.1524.0

Can any one please suggest me why i can not see the ODBC 64 bit system DSN connection from SSIS in my production server ??

Thanks,

Nag.
↧

Professional Microsoft SQL Server 2008 Integration Services: Chapter 8 SSIS exercise issues with Corporation Extract

May 28, 2015, 9:32 pm
$
0
0

Hello I am new to SSIS, I am doing this exercise in the book where I need to import the data file at chapter 8 

Link to the data file.

http://www.wrox.com/WileyCDA/WroxTitle/Professional-Microsoft-SQL-Server-2008-Integration-Services.productCd-0470247959,descCd-DOWNLOAD.html

The name of the data file is "010305c.dat", I have successufully created a file source, and when I preview the data in the file source it is displayed to be ok, but when I viewed it in a data viewer from a Union all dump, the double quotes are in the data and the data column to values are missaligned for some columns. 

Why does this happen what am I doing wrong?

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>