Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Importing special characters into SQL Server from XML source

August 22, 2012, 1:45 am
$
0
0

Hi,

I'm having great difficulty dealing with special characters in XML source file when importing them in SQL server via a data flow task. I have the following character 'ú' in one of my xml files (which is correctly defined in the XSD file as a string)  and it results in the following error: -

[XML Source [24575]] Error: The component "XML Source" (24575) was unable to process the XML data. Invalid character in the given encoding. Line 6279, position 23.

Is there any way I can import such a character or alernatively redirect the record to another table via an Error Output. Any help would be greatly appreciated. Thanks.

Mike

↧
Search

Multiple look up

August 22, 2012, 1:05 am
$
0
0

Hello,

I have seven lookups in sequence, one after other if  match found update some flag column as Y  else update it as N and both the match and no match outputs has to flow to next lookup and  continue on with same logic  then finally write all those values to a table .

First thing came to my mind when i thought about the solution for this is to union the lookup match n no match and send it to next look up but my manager told not to do it in that way as it may affect performance.

How can I do this? I am not sure, are there any other component to do this?please advice

↧
↧

Is it possible to change sample size on suggest datatypes in import wizard from 10000 to all?

August 21, 2012, 9:25 pm
$
0
0

I am getting the truncating error when importing large files as I can't sample more than 10000 rows.

This needs to be a automated import so moving large files to the top of the file is not an option.

I know that there is a registry value that can be changed for excel is there something similar for flat files (.csv)

↧

Error in Expression in Derived

August 22, 2012, 1:22 am
$
0
0

HI,

Getting issue in following expression in Derived col.

(DOB_DT == (DT_DATE)"12/31/9999" || YEAR(DOB_DT) < 1753) ? NULL(DT_DATE) : DOB_DT

[Derived Column [104]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "Derived Column" failed because error code 0xC0049067 occurred, and the error row disposition on "Derived Column.Inputs[Derived Column Input].Columns[DOB_DT]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.

Any thoughts how to handle <1753 year? Please suggest

Porus

↧

Export to CSV file - strange data going in to a DateTime field

August 22, 2012, 1:29 am
$
0
0

Hi all,

I am doing a simple export to a csv file from a table.

One of the fields, DateAllocated, is a DateTime field.

Some of the NULLS in that field are being converted to TRUE/FALSE values while 2 of them are being converted into what looks like a GUID eg {3554-YTT4-etc etc}

I have checked the offending 10 records and they all have NULL or Valid dates in the table.

This is causing massive problems with subsequent Imports.

Anyone seen this before ?

Thanks in advance.

↧
↧

Broken Data Flow Task keeps validating components downstream

August 21, 2012, 11:53 pm
$
0
0

I have a Data Flow task containing several steps. If I break the logic of the flow ie to insert another lookup, the flow starts validating and generates more than a thousand errors in the error list. The most ignoring thing is that it starts counting at 1 error to let's say 1000 and than you see it start counting at 50 and increasing to 1000. Each loop starting at a higher number. All in all this can last for several minutes. All errors are: 'Validation error. <Data Flow Task Name>: The metadata for "input column "" (<ID>)" does not match the metadata for the associated output column.' (The Data Flow Task contains a lot of Union All transformations)

I already set the DelayValidation property of the Data Flow and the package to true, but this does not change the behaviour.

Is there a way to stop this behaviour?

↧

select all columns which are not null

August 21, 2012, 6:59 pm
$
0
0

I have table which have 60 columns.

first row have null values in 32,44,55, columns

second row have null values in 11,22,24 columns.....etc for all the other 600 rows.

I need to select the data which do not have null values.

result set should not display columns which are not null.

Thanks in advance.

↧

Storing a Money data item (in SQL Server) in a double SSIS variable

November 4, 2007, 10:05 pm
$
0
0

Hi

 

I have a table in SQL Server with following spec

Table1(Grossamount(money))

 

I have a SSIS variable called grosstot of type double and use following sql in Execute SQL task in SSIS

 

Select Sum(Grossamount) from Table1

 

I then assign the result of above sql stmt to the SSIS variable grosstot within the same Execute SQL task.

 

it gives me the error :

[Execute SQL Task] Error: An error occurred while assigning a value to variable "grosstot ": "The type of the value being assigned to variable "User::grosstot " differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object. ".

 

 

I tried the following sql to no avail

 

Select CONVERT(numeric (12,2), Sum(Grossamount) from Table1

 

Your help very much appreciated


 

↧
Search

Running an execute package task with the project deployment model

August 22, 2012, 6:59 am
$
0
0

Please point out anything I forgot to add or got wrong.

I'm working with SSIS in SQL Server 2012 and I am attempting to write to a parent package variable from a child package. The hitch here is that the child package is not in the same project as the parent package.

From what I've read, the new project model requires an execute package task to target packages stored in the same project. If I'm using an external reference, the task is unable to target packages stored in projects in SSISDB and can only reference MSDB or the file system.

Inside the child package I have a single script task that contacts a web service and invokes several methods and contains logic around the invocations. It also accesses several parent variables, some to read through parameters, some to write by directly referencing them. Ideally this script task would be centralized so that if the logic needs to change, I only need change the centralized package, and the cost to link a new package with the child package should be minimal, which is why I liked the idea of using an execute package task and a child package in the first place. These are the options that I see right now:

1) I realize I could just have the child package in every single project that gets created, but this will most likely lead to maintenence problems down the road.

2) I could call a stored procedure in SSISDB to kick-off the child package but I do not believe the child package will have access to the parent package's variables, as, from what I can tell, it would be initiating the package with no ties to the parent's execution ID.

3) I could also store the package in the file system, but I have doubts about the acceptance of using the file system by my administrators.

4) I could compile a .dll with the code from the script task in the child package. The problem is, I don't know how the new model treats referenced .dll's.

Can anyone provide advice on how I should proceed?

↧
↧

SSIS BOL and API docs for SQL Server 2012...are they out yet?

August 22, 2012, 6:57 am
$
0
0

SIAP, but I"ve asked twice over at the SSIS Team blog and nobody ever took the time to get back to me...any idea when the BOL documentation and API 'Technical Reference' comes out for SQL 2012?

Lee Everest http://www.texastoo.com

↧

How to start a SQL process with my smart phone?

August 17, 2012, 11:59 am
$
0
0

Hi All,

Question: How can I use a smart phone to fire off any sort of SQL Server process?

Situation: Our SQL Agent Jobs for ETL sometimes fail.  Sometimes the ETL Jobs succeed but the cube processing Job will fail.  Failures generate email notifications to  the BI group, since we have no weekend or evening support staff.  Somebody from the BI group is typically at home with their laptop and is able to start up the VPN, log into the SQL Server, and manually kick off the Job.  Increasingly, though, Jobs cannot be manually restarted in a reasonable amount of time because nobody from the BI group is near their laptops when they get the failure notification email.  (Think sitting at the lake on Sunday afternoon with your family when a BI manager calls and says you need to restart the Job.  "Uh, yeah, I'll get right on it, boss."  And then, immediately afterward, "Say, why are you calling me on my personal phone, anyway?  Ready to give me back that company phone yet?")

Since every BI developer and DBA in the group owns a smart phone, I'm wondering how they can be used to help remedy this situation.  Can I somehow use my smartphone to run a stored procedure or kick off a Job?  I'm certainly willing to be creative--can a server be configured to run a batch file after receiving a certain text method?  I've never used a "web service" before, but it sounds like this could work, too.  One option  that just came to mind is a web page that allows users to start off certain Jobs, but I'm looking for the easiest possible method, like "1. receive the email, 2. reply to the email, 3. relax while your email gets the Job running."

Thanks,
Eric

↧

ForEach Loop File Enumerator SSIS

August 17, 2012, 2:52 pm
$
0
0

Hello Guys,

 Need help with a strange situation. I have a SSIS package that executes Sql Scripts store in a folder ( 132 scripts in all). I use a for each loop and a Execute Sql task with File connection and the connection string is from the variable mapped to the foreach loop fully qualified name.

This has been running for till about a week ago. Lately it has been skipping files, and only executes anywhere from 33 to 101 files and some time the complete 132 scripts. I dont know what is causing this, and i have tried restarting the services, server but still same problem.

It is not consistent, and i did notice that on some files the ForEach task just hangs for a while.

Please help, any advice how ever strange would be very much appreciated.

Thank You

↧

SET IDENTITY_INSERT ON required

August 22, 2012, 11:01 am
$
0
0 
I want to insert records using  the following code in ssis variable
if ((SELECT OBJECTPROPERTY( OBJECT_ID(N'[dbo].[TableName]'), 'TableHasIdentity')) = 1)
begin
SET IDENTITY_INSERT  [dbo].[TableName] Off
 insert into [dbo].[TableName] select * from [efi_dot_com].[dbo].[TableName]  
SET IDENTITY_INSERT  [dbo].[TableName] On
end
else
 insert into [dbo].[TableName] select * from [efi_dot_com].[dbo].[TableName]

Then I got error

An explicit value for the identity column in table 'dbo.TableName' can only be specified when a column list is used and IDENTITY_INSERT is ON.

The reason behind the erroir is try to use insert into with select * from ... with a table has identity (primary key) column.It is required to list out all the column name.

I can't list out all column names since the above code use dynamically for various tablename. How can I include the lsit of column name dynamically into the insert sql statement?

Is there other solution?

Thanks
↧
↧

Flat file source - split delimited address field to additional fields.

August 22, 2012, 9:33 am
$
0
0
A client of ours can only export data in the following .csv delimited format:

ID, Name, Company, Address, Town, County, Postcode, Country

The Address field contains anything from 1 to 5 lines of local address data separated by Carriage Return.

We need to get this data into our SQL database using SSIS on a daily basis.

I have managed to import the file as Flat File, and have successfully changed the Carriage Returns in the Address field to commas.

I now have data in the Address field that represents:

Address
"20 Test Street, Testville"
"22 Test Avenue"
"Flat 2, 41 Test Close, Testville"

I would like the data to look like:

Add1,Add2,Add3,Add4,Add5
20 Test Street,Testville,,,
22 Test Avenue,,,,
Flat 2, 41 Test Close, Testville,,
Flat 5, Test House, Test close, Test Road, Testville

I have tried using Derived Columns and the FindString and SubString commands, but I can't seem to get it to work because of the differing number of commas in each record. Sometimes there are none (second row in above examples) and sometimes more (other rows in above example).

To get the first value in the field I tried:
SUBSTRING(AddressCommas,1,(FINDSTRING(AddressCommas,",",1) - 1))

but as stated it doesn't work with the differing number of commas.

Are there any other ways I can go about this?

Note: the client has no control over the quality of their data.
↧

SQL Agent Job, Set Values, DTS variables and job rescheduling

August 20, 2012, 3:15 pm
$
0
0

Techies--

I have an SSIS job that begins with a script task. The script task accepts the starting datetime and the ending datetime. For qa purposes, my plan is to begin the start datetime @ 7/1/2012 00:00:00 AM and set the end datetime @ 7/1/2012 01:00:00 AM, easy enough--for this I set the values for the sql agent job mapped to the dts variables.

Now, here is the real question: How would I automatically reschedule the sql agent job so that when I set it up to run once every hour,  the range changes to add an hour from the end datetime of the last run (e.g. 7/1/2012 01:00:00 AM to 7/1/2012 02:00:00 AM) ?

Outside of QA testing, the job will normally need to do essentially the same thing--with current datetime variables.

I should mention that I am storing the previous start/end range in a control table, so I know I can update the info that way, however, I'm looking for input from the group for alternatives


↧
Search

ETL Design Concern:

August 22, 2012, 9:28 am
$
0
0

Hi, 

I have the following staging tables:

1.ContactDemographics

2.Contact ( The contact table has a foreign key reference to ContactDemographics)

3.Address 

4.AddressInstance  ( The addressInstance  table has foreign key references to Contact and Address Table)

5. HomeImprovements ( The HomeImprovements table has foreign key references to Contact table)

Since HomeImprovments table and the AddressInstance are not referenced by any other tables I can truncate these 2 tables for the staging load. 

For the others I cannot truncate tables as they are being referenced in other tables : The option that I have is to truncate HomeImprovments table and the AddressInstance and then delete data from Contact  and ContactDemographics. 

 I totally understand why the table it is not possible to truncate a table which has references to other table.  ( Foreign Key Constraints)

My question/concern is

1. What is the best practices to handle staging load when it comes to loading data into staging tables with referential integrity ? Is it a good practice to truncate tables which are not referenced and then delete data ( reset identity) from the tables which are being referenced in a particular order OR it is a good practice to alter table and disable/drop constraints and then enable/create constraints when the package finishes. I have a single SSIS package for staging load for each table

Please suggest best practice in such kind of scenario?

Thanks

EVA05

↧

Help with data flow errors

August 22, 2012, 4:07 am
$
0
0

Hi All,

I'm trying to move some data from an excel spreadsheet into a table in a MySQL database using SSIS (2008 R2). I've been getting errors relating to data types, so I'm now just extracting and loading one field as a test, whereby I'm comfortable that there are no data type issues. The errors I am getting is below. Would anyone have any idea as to what the problem might be and how to resolve it? 

[ADO NET Destination [2931]] Error: An exception has occurred during data insertion, the message returned from the provider is: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"URN") VALUES (p1), (p1), (p1), (p1), (p1), (p1), (p1), (p1), (p1), (p1), (p1), ' at line 1

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "ADO NET Destination" (2931) failed with error code 0xC020844B while processing input "ADO NET Destination Input" (2934). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

Any thoughts appreciated. Thanks in advance.

Tom


↧
↧

Error 0x80004005 when trying to specify connect string for connection manager with config environment variable

August 22, 2012, 3:36 am
$
0
0

I'm using a configuration which specifies a connect string to a config database via an environment variable:

Data Source=QASQLONEPRI;Initial Catalog=SSISConfig;Provider=SQLNCLI10.1;FailoverPartner=QASQLONEFAL;Integrated Security=SSPI;

When I run my package from a remote machine, with a config environment variable set up as above I get, the following, this is despite the fact that I can connect to the database server: QASQLONEPRI from the remote machine using management studio without any issues, I have also tried using the OLDEDB client, but this leads to the same problem also. Can someone please advise as to how I can resolve this issue.

Error: 2012-08-22 11:34:10.54
   Code: 0xC0202009
   Source: ImportMain Connection manager "SSISConfig"
   Description: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred.
 Error code: 0x80004005.
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 10.0
"  Hresult: 0x80004005  Description: "Login timeout expired".
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 10.0
"  Hresult: 0x80004005  Description: "A network-related or instance-specific err
or has occurred while establishing a connection to SQL Server. Server is not fou
nd or not accessible. Check if instance name is correct and if SQL Server is con
figured to allow remote connections. For more information see SQL Server Books O
nline.".
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 10.0
"  Hresult: 0x80004005  Description: "Named Pipes Provider: Could not open a con
nection to SQL Server [53]. ".
End Error
Warning: 2012-08-22 11:34:10.54
   Code: 0x80012059
   Source: ImportMain
   Description: Failed to load at least one of the configuration entries for the
 package. Check configuration entries for "Inbound Interface Config" and previou
s warnings to see descriptions of which configuration failed.
End Warning
Started:  11:33:09 AM
Finished: 11:34:10 AM
Elapsed:  60.875 seconds

↧

SSIS package execution Insufficient memory to continue the execution of program Problems with the query, "ResultSet" property not set correctly, parameters not set correctly

August 20, 2012, 12:41 pm
$
0
0

Hi,

     I am created a Stored Procedure and it will return XML result set.  I am calling this storedprocedure in SSIS package ( using the execute SQL task) and writing the result of the StoredProcedure in a file ( by using the script task ) .

Now i configure the SP to return only 4 rows, the SSIS package is getting executed and a xml file will be written in a folder.

If the SP is configured to return all the rows, then there is  Insufficient memory to continue the execution of the program error.

Detail error:

"Insufficient memory to continue the execution of the program.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly."  while executing the Execute SQL task.

Execute SQL task : 

 Result Set = XML

Can anyone please let me know how to resolve this error and why there is an error when all the rows are returned ( the xml file with all rows is returned from the Stored Procedure ).

-Prasad

↧

Compact ad Repair List of Databses

August 22, 2012, 9:22 am
$
0
0

I have a list of databases in Access that I need to compact and repair. I have looked up information about "shredding" the dataset but have not been able to find what I need.

I am unsure as to the setup of the SSIS Package as well as the vairables needed and the I also am looking for the VB script to compact and repair all databases in a list.

I am new to SSIS, so please simplify answers, I also have already referenced most online material including:

http://www.sqlis.com/post/Shredding-a-Recordset.aspx - which discusses shredding records for email NOT script task

http://sqlblog.com/blogs/andy_leonard/archive/2007/10/14/ssis-design-pattern-read-a-dataset-from-variable-in-a-script-task.aspx- which I understand but am unsure about the xml reference in the script task as well as the actual VB code to script task

I am not very good in VB either but any help would be greatly appriciated!

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>