Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Suspended process during simultaneous update and insert on same table

March 31, 2014, 6:11 am
$
0
0

LS,

I have to upload a CSV into a table, the CSV will mostly contain new data although it will also contain updated data (based on a 4 column key).

At the moment I've a very straightforward package, which performs a lookup on the table based on the 4 column key. The match output routes to an OLE DB Command with an update statement and the error output routes to an OLE DB Destination which performs a fast table load.

The CSV holds approx. 15k records, and I've got two SQL environments: a Q&A with the table containing 1.2M rows and a PRD environment with the table holding 5.4M rows.

At the moment I am experiencing suspended processes in the activity monitor on the Q&A environment when the package executes; i.e. the Update is blocked by the Bulk insert, both processes are suspended.

I already tried to uncheck the Table lock on the OLE DB Destination, but without any success.

All the settings on the control and dataflow elements are default.

My questions are:

  • Should I play with the settings FastLoadMaxInsertCommitSize on the OLE DB Destination, currently 0;
  • Should I play with the settings DefaultBufferMaxRows and DefulatBufferSize on the Dataflow, both on their default now, 10000 and 10485760 respectively;
  • How to get to the root cause of the suspended process or blocking?

Regards,

René

↧
Search

ssis job fails on scheduled time. But works fine when executed manually in BIDS.

March 24, 2014, 11:54 pm
$
0
0

Message
Executed as user: Answers\Administrator. Microsoft (R) SQL Server Execute Package Utility  Version 9.00.5000.00 for 64-bit  Copyright (C) Microsoft Corp 1984-2005. All rights reserved.    Started:  5:00:01 AM  Error: 2014-03-04 05:00:05.77     Code: 0x00000002     Source: SCT create SSRS Skus Not Agreement Report      Description: The script threw an exception: Object reference not set to an instance of an object.  End Error  Error: 2014-03-04 05:01:39.40     Code: 0x00000003     Source: SCR Collect Info      Description: Precompiled script failed to load. The computer that is running the Script Task does not have a compiler capable of recompiling the script. To recompile the script, see the Microsoft Knowledge Base article, KB931846  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  5:00:01 AM  Finished: 5:01:39 AM  Elapsed:  97.782 seconds.  The package execution failed.  The step failed.

Any input on this would be appreciated.

Thanks !!!

↧
↧

Maintenance plan for backup executes endlessly.

March 31, 2014, 8:17 am
$
0
0

Hi All,

I have a maintenance plan for a full back which is scheduled to run once every night. It has been running quite peacefully with out any error and usually takes 50min average. But Now(the last three days), it starts executing but never completes. I tried to run sp_who2 and saw a block when the execution reaches at a specific database. I stopped the execution, killed the spid on the 'Blk By' and then restarted the execution but same problem show up again. Whats going on? Any ideas?

Thanks.

↧

unable to understand SCD codes logic

March 31, 2014, 9:33 am
$
0
0

Hi Experts

I'm stuck on Slower Changing Dimension scenario. I did the following steps:

Step 1 --Pull OLE DB SOURCE under Data Flow Designer and insert the following query.

          Select cast ('100' as varchar) as emp_id, cast ('PPP' as varchar)  as dname,
         cast('zzz'as varchar) as title

Step 2 -- Create Dimension table like below:

 CREATE TABLE [dbo].[emp_dest](
 [emp_id] [varchar](10) NULL,
 [dname] [varchar](40) NULL,
 [start_date] [date] NULL,
 [end_date] [date] NULL,
 [title] [varchar](50) NULL
) ON [PRIMARY]

Step 3 -- Pull SCD and configure emp_id as my business key, dname and title as my historical attributes, and use start and end date to identify current and expired records and assigned System::start time  as my set dates value.

Now I can see the below picture in my data flow designer

 Now, Its working fine but I don't understand why wizard created derived column under  Historical Attribute Inserts Output and then OLD DB Command and how come system know that some changes exist in data. Please advice....

Historical Attribute Inserts Output:
↧

usage of Temp tables in SSIS 2012

March 28, 2014, 5:34 am
$
0
0

Hello,

We have many SSIS packages (2008 R2) which imports data to a temp table and process it from there. 

We are upgrading to SQL server 2012 and facing the issue with temp table as working table and our ssis packages fail in 2012. While investigating found that SQL Server 2012 deprecates FMTONLY option and instead uses sp_describe_first_result_set , which does not support using of temp tables as import table.  SSIS works fine in our workstations but not in the DEV box.  With SQL 2012, I can execute from my workstation, which has (11.0.2100.60) where as DEV server has SQL Server version 11.0.3000.0 

Also when I ran profile with that of the DEV box, it gives two different statements 

from workstation (11.0.2100.60)

 CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )

go
declare @p1 int
set @p1=NULL
declare @p3 int
set @p3=229378
declare @p4 int
set @p4=294916
declare @p5 int
set @p5=NULL
exec sp_cursoropen @p1 output,N'select * from #temp',@p3 output,@p4 output,@p5 output
select @p1, @p3, @p4, @p5
go


it works fine

But with the DEV server (version 11.0.3000.0), it executes the below sql and it fails to get the meta data

 CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )

exec [sys].sp_describe_first_result_set N'select * from [dbo].[#temp]'

On checking the assembly difference between the versions, I could only see Microsoft.SqlServer.ManagedDTS.dll being 11.0.3000.0, which I replace by 11.0.2100.60 version. but still getting the same result. 

The other different I found is with ,Net framework libraries.

Could you advise whats the assembly causing this issue between our workstation and DEV server  i.e 11.0.2100.60 and 11.0.3000.0 

Many thanks

↧
↧

Dont want to use Slower Changing Dimension

March 31, 2014, 9:52 am
$
0
0

Hi All

I don't want to use Slower Changing Dimension wizard and instead of this I want to do this scenario manually. I only want to use  Historical Attribute Inserts Output with start and end date scenario. Any suggestion about it... ?. Thanks in advance

Regards

Muz

↧

Output select query in OLEDB Command

March 31, 2014, 10:09 am
$
0
0

Hi I am trying to write a simple output query to get the Count of rows  in OLE Db Commnad Adavnced editor and getting error. Please help!

Declare @Count INT;

Select @Count(*) AS ? FROM <tablename>WHERE Column 1  =?  AND Column2 = ?

Please help as this is my first project!

Thanks in advance.

↧

Integrating VS 2012 Shell with TFS

October 21, 2013, 1:54 pm
$
0
0
We have a new instance of SQL Server 2012 and I've installed the Visual Studio 2012 shell in order to create and manage SSIS packages. We have an instance of TFS 2010 on our network. For the life of me I can't figure out how to get TFS to show up as a Source Control option in the VS 2012 Shell program (or get the "Team" menu option to appear). There seems to be nothing online about this. Help!

jdf35

↧
Search

.csv flat file loading data issue

March 31, 2014, 9:18 am
$
0
0

Environment: SQL Server 2008 R2
Purpose: Load source data which are stored in a csv file to the destination table
Problem: After setting up flat file transformation, including the columns in the output alias, see the following picture:

,because SSIS set datatype to 50 char while the length is 65. I also set up response string filed to 100 in the destination table, An error message showed as the following picture

What I could do differently to overcome this issue. please advise



↧
↧

SQL server 2012 job intermittently unable to kick-off or start SSIS 2012 package - The operation failed because the execution timed out

January 27, 2014, 7:37 am
$
0
0

Our SQL server 2012 jobs intermittently fail bacause they are unable to kick-off or start the associated SSIS 2012 package.

The error we get is: 

Failed to execute IS server package because of error 0x80131904. Server: Package path: \SSISDB\ETL\xxx

Environment reference Id: NULL.

Description: The operation failed because the execution timed out

Source: .Net SqlClient Data Provider
Started:  7:00:03 AM
Finished: 7:00:10 AM
Elapsed:  6.974 seconds
Microsoft (R) SQL Server Execute Package Utility
Version 11.0.2100.60 for 64-bit

Is there any limitation on how many SSIS 2012 packages can run con-currently?

How to find out at a given time (both current as well as in past) how many SSIS 2012 packages were running con-currently?

Any solution other than changing the start times of SSIS package jobs?

↧

Table name varies, depending on data from query

March 31, 2014, 11:43 am
$
0
0

It is kind of difficult to explain what I need, but here goes...   I need to do a query where the data from the first select gives me the table nameto query in the second part of the query.

For example....

select CatalogID, Title, Tester

     from CatalogListing

     where CatalogTitle like '%xxx%'

select InventoryID, MachineID, User

     from Inventory_CatalogListing.CatalogID

The first part of the table named Inventory_ is constant.  The second part of that table name varies, depending on the CatalogID returned from the first select line.  How to I get the table name to combine the 2 pieces? Nothing I have tried works. 

↧

OLE DB destination error on insert to iSeries DB2 table error column -1 error code -1071607685 description No Status Available

March 31, 2014, 7:58 am
$
0
0

I am trying to use SSIS to move records from a DB2 table on an IBM iSeries  to an identical table on another iSeries.  Getting errors on the insert to the destination table.  When I use a data viewer to look at the error being generated the error column shows as -1, the error code is -1071607685 and a description of No Status Available.

How can I tell what column(s) is causing the issue?

I've tried removing records from the record layout of source and destination tables and have been able to successfully copy over a subset of the original record layout and finding along the way that I must convert unicode string [DT_WSTR] to string [DT_STR] and database timestamp [DT_DBTIMESTAMP] to database date [DT_DBDATE] but after doing the conversion to all the strings and dates in the original record layout am still getting error on the insert.  I could continue to increase the number of columns in the record layout one at I time until I found which column(s) were generating the issue but the record layout contains dozens of columns.

Can someone show me how to determine which column(s) are causing the issue?  Or perhaps suggest what other conversons need to be done to data types?

Thanks in advance for any help, Roscoe

↧

ssis will not accept DB2 ZOS non-unicode data to sql server non-unicode

March 31, 2014, 7:57 am
$
0
0
We need to load 600+ tables from DB2 zos to SQL Server 2012.  Both source and destination data is NON-UNICODE.  SSIS will not accept the source data as it seems to change it to unicode automatically, and  rejects it on the destination side. How do we shut this off so that it will process the input data as it originally is (non-unicode) to sql server non-unicode?   We need this on a global basis as it is a massive effort to use a conversion task for each of the 600 + tables that need to be migrated.   We do not want to use unicode for this database.  How can this be done?  
↧
↧

SQLISPackage100

March 29, 2011, 6:15 am
$
0
0

I am using SSIS log provider for Windows Event Log.
In the windows event log, my packages appear with the name "SQLISPackage100".
I have not found out how I can rename my packages for the log. Is this even possible?



↧

file system task and ibm iseries file share

March 31, 2014, 10:59 am
$
0
0
Hi.  I'd like to use the file system task (FST) to copy files en masse (folder to folder regardless of file name) from an as400 file share.  We run ssis 2012 enterprise.  I'm not an expert on as400 shares but hopefully that doesnt matter.  Generally, I have to provide a userid and pswd once a day when reviewing this as40 share manually.  It is mapped to one of my drive letters via ip address and a specific folder name.  I dont see anywhere in ssis' FST component where credentials can be provided, hard coded or otherwise.  Has anybody done this kind of thing before or know of any options?
↧
Search

SSIS package flow based on a scheduling configuration table.

April 1, 2014, 1:14 am
$
0
0

I have a table called Tbl_Scheduler.
This scheduler table contains
scheduler id ,scheduler description,as shown below.
1  ,Runs between 10 th of current month and 20th of current month
2  ,Runs at last day of the month
3  ,Runs at first day of the month

Scheduler Id will come from first task(it is given in a configuration table),
second task should check this scheduler id on scheduler table and
verify whether the current run date  is falling under the criteria,which is given as scheduler description.
If it is not,need to stop the package flow.
SSIS package is scheduled from SQL agent job in every 10 mins.Could you please let us know,how I can design the same.
Also please feel free to advise to change the table design.

 

↧

Consuming oData service from SSIS 2008 R2

March 29, 2014, 4:34 am
$
0
0
Is there possibilities to consume oData service feed from SSIS 2008-R2, any component that can be used ?
↧
↧

Trying to add ODBC to SSIS11 XML Mapping file in SSMS

March 28, 2014, 7:10 am
$
0
0

Hello I am trying to make a mapping file for SSMS using .Net Framework Data Provider for ODBC. I am trying to import ODBC data to SQL Server 2012 database using SSMS.

I created a file ODBCToSSIS11.XML and it says it can't find a file.

I need to know what file name SSMS is looking for when you are using .Net Framework Data Provider for Odbc as your data source

It is a pain in the butt SSMS does not tell you what file name it is looking for, and I can't find a list on google of what I am supposed to name the file. Any help would be awesome!


↧

Custom Datasorce Reader cast before write to buffer

January 28, 2014, 2:29 am
$
0
0

I have created a custom datasource reader, that gets information from a webservice. The Data returned is in Json format.  Below is an example of the Json returned.  I edited it a bit theres a lot more stuff that is returned from the api that is not relivent to this question.  Trust me the Json returned works.

    {
     "columnHeaders": [
      {
       "name": "ga:date",
       "columnType": "DIMENSION",
       "dataType": "STRING"
      },
      {
       "name": "ga:entrances",
       "columnType": "METRIC",
       "dataType": "INTEGER"
      },
     {
     "name": "ga:timeOnSite",
     "columnType": "METRIC",
     "dataType": "TIME"
      }
    "rows": [
      [
       "20130604",
       "180",
       "1.1"
       ],
       [
       "20130603",
       "200",
       "1.1"
       ]

     ]
    }

As you can see the second column is an Int and the thrid column is Time which is really float.  Now the way the API returns everthing in json is string "becouse of the".   In my task i have created the output columns using the corect datatype supplyed by the api. 

My question is do i need to cast results before writing them to the output.  They are always corectly formated there is no chance that a string will be returned in an intiger column.

Currently my code looks like this: For each of the diffrent datatypes I parse them before I wright them to the buffer. 

    case "INTEGER":
         ParseFail = Int32.TryParse(ColumnValue, NumberStyles.Any, CultureInfo.InvariantCulture, out parsedInt32);
          if (!ParseFail)
            {
            msg = "Error: Data Feed Converting INTEGER string to int32: failed " + ColumnValue;
            ComponentMetaData.FireError(0, ComponentMetaData.Name, msg, string.Empty, 0, out pbCancel);
            throw new Exception(msg);
            }
            else
            {
            buffer[mycol.OutputColumnIndex] = parsedInt32;
            }
            break;


My question is do i need to do this?  The only reason i could think to be foreced to do this is that the decimal point in all of the floats is a `.`  if the server the task is installed on has EU settings which would make a decimal point a `,` Will the insert fail if i dont do the cast first?

↧

Deploying standard changes to SSIS packages

April 1, 2014, 11:27 am
$
0
0
A couple glitches were discovered in our SSIS template.  Approximately 100 packages were built from the template.  Is there a mechanism to update all packages with the changes?  Or do I have to open each one and manually change?
↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>