Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Shared Memory Provider: Timeout error [258]

April 1, 2014, 1:38 am
$
0
0

Hi All,

Hopefully there is somebody that can help me...

When running the etl I'm getting the error: <SSIS Task>: Shared Memory Provider: Timeout error [258] ; followed by the message "Communication link failure".

What is special about this message that it happens on a SQL Execute task (random task) and the Timeout is after 2 minutes.

When executing the packages separatly it is working fine. The SQL Tasks that are failing are also quit heavy, but reasonable and takes between >2min and 10 - 15 min. Statements are stored procedures that puts an index on 3 mil. records or update statements,...

I had a look to all my (SSIS-etl) timeouts and they have the default value 0, the "remote query timeout" of the server is set to 10 minutes. According to me, these are the only one that exists?

There are 2instances on the server each instance has 24GB allocated, the server has 64 in total. Also when the etl runs (that results in an error) no other etl is running on the 2 instances. I'm working with the oledb \sql server native client 11.0 provider: SQLNCLI11.1.

It is frustrating because I don't have a clear error message. Maybe there or other places to look? I had a look on the application log & sql server log but it did not made me any wiser...

Any help is appriciated,

Bram

↧
Search

Access to path denied in SSIS 2012 script task

July 31, 2012, 10:47 am
$
0
0

Note: this applies to SQL 2012 only.

I'm using the project deployment model in SQL SSIS 2012

I have a script task inside the SSIS package that renames some files on a domain share. (\\mydomain\myfileshare)

The database owner is a domain user (mydomain\svclogin)

I start the package from a stored procedure like so:

EXEC [SSISDB].[catalog].[start_execution] @execution_id

When the package is deployed to my local machine, it executes without error.

When it is deployed to a remote server on the same domain (2012), I get an error:

Access to the path \\mydomain\myfileshare is denied.

The report log show that the caller is mydomain\svclogin. I added some informational logging of my own and I see that the variable System::UserName is also mydomain\svclogin

I logged on to windows as mydomain\svclogin and verified that I can modify/delete/change files on  \\mydomain\myfileshare

On older packages, I used credentials and proxy accounts to set up access to domain resources.

1. How do you specify the account (a domain account) that will access domain resources in an SSIS 2012 package ?
2. How do I fix the error?

John Lucas - Code Monkey

↧
↧

No Column information was returned by the SQL command.

November 13, 2015, 9:49 am
$
0
0

I can preview the SQL command in the OLE DB Source Editor and bring back all columns and results just fine but when I click on the Columns I get

TITLE: Microsoft Visual Studio
------------------------------

The component reported the following warnings:

Error at Data Flow Task [OLE DB Source [1]]: No column information was returned by the SQL command.

The columns are there in the preview - why can't SSIS get the column information?

TIA

Harry


↧

Column Names using Lineage ID

November 16, 2015, 10:56 pm
$
0
0

Hi,

   I am working on a custom component to implement some rules based on the column name. I am looking for ways to identify the column name using lineage id. Is there anyway we can derive column name using the lineage id?

Thanks,

Raj

Raj

↧

Data Conversion in ssis

November 16, 2015, 11:09 pm
$
0
0

Hi ,

I am using flat file as source.I have quantity column in flat file which is a  Numeric datatype and target table quantity datatype is Numeric.

I am able to load data from source to target but when I am comparing data from source to target I am not getting exact record from source flat files

Source having data like

Source        >>  Target

31.61          >>   31.0000000000
00029.430 >>    29.0000000000

as we can see that data are not matching with source

I can not change the target table quantity data type, is there any thing which I can do with source column data type.

Thanks in advance :)

Thankx &amp; regards, Vipin jha MCP

↧
↧

Implementation of bothe Authentication Mode( SQL and Window ) in SSIS package at same time ...

November 16, 2015, 11:00 pm
$
0
0

Hello Everyone ,

Can we implement both authentication mode i.e SQL Authentication & Window authentication , while development of SSIS package same time .

 As due to Business requirement , we need both authentication mode .

Any guidence will be very helpful.

Thanks
Alok
↧

OData to extract Sharepoint List data .. how to handle dates??

November 16, 2015, 2:16 pm
$
0
0

I'm trying to move SharePoint List data to a SQL Server (2014) destination using the OData tool in my SSIS package.

I have a column in my SharePoint List that is date values, and this is where I run into trouble ..

From the OData > Advanced Editor > Input and Output Properties dialog, and then clicking on theOutput > External Columns and then looking at the Data Type Properties for my date column, the DataType defaults to Unicode text stream [DT_NTEXT].  When I change the DataType to database timestamp [DT_DBTIMESTAMP], I get the error message listed below.

How should I handle SharePoint List date data when ETLing it to SQL Server?

Thanks

Error Message snip:

"Error at Extract [OData Source [2]]: The data type of output columns on the OData Source cannot be changed, except for DT_WSTR and DT_NTEXT column"

Full error message, below:

Error at Extract [OData Source [2]]: The data type of output columns on the OData Source cannot be changed, except for DT_WSTR and DT_NTEXT columns.

Error at Extract [OData Source [2]]: System.Runtime.InteropServices.COMException (0xC020837D)
   at Microsoft.SqlServer.IntegrationServices.OData.ODataSource.SetOutputColumnDataTypeProperties(Int32 iOutputID, Int32 iOutputColumnID, DataType eDataType, Int32 iLength, Int32 iPrecision, Int32 iScale, Int32 iCodePage)
   at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostSetOutputColumnDataTypeProperties(IDTSManagedComponentWrapper100 wrapper, Int32 iOutputID, Int32 iOutputColumnID, DataType eDataType, Int32 iLength, Int32 iPrecision, Int32 iScale, Int32 iCodePage)

↧

RPC Server not available

November 5, 2015, 8:43 am
$
0
0
I'm having issues trying to connect to a remote SSIS server. Connection between servers within the DC works.  Port TCP 135 and 139 are open.  Also tried to connect using IP address but the same error occurs.  Do i need to open more ports? The SQL version installed in 2014.
↧
Search

SSIS 2012 and required package parameters

November 4, 2015, 8:22 am
$
0
0

Hello Guys,

I'm after running into something in SSIS 2012 that I fail to grasp.

I have a package that provides a service to other packages. In order to provide that service it needs 4 parameters provided by the caller. So naturally I'm thinking I make those 4 parameters 'required'.

The caller uses Execute package task and provides the 4 parameters on the parameter mapping tab.

Yet the packages fails with the error message that one or more required parameters weren't provided.

What am I doing wrong?

regards

Carsten

↧
↧

Need SSIS Complex interview questions and Real time scenarios

May 18, 2013, 9:42 am
$
0
0

Hi..

I am newly searching a job on MSBI with 3 years of experience.

Could any body help me regarding the interview questions and mainly on real time scenarios. Because I attended three interviews, all are asking real time scenarios. 

Or

Where can I get the  real time scenario examples 

↧

Pivot in SSIS, how does it calculate

November 17, 2015, 4:27 am
$
0
0

Hi All,

I was looking this example on MSDN, In this example, I could not make out how did it calculate HL Mountain Hire for 2002 and 2003 as it is not there in the source.

If I just copy the source into excel and generate pivot table it shows me according to source, Any Idea about it please?

This is excel output in pivot

Sum of Total Column Labels   
Row Labels 2002 2003 2004 Grand Total
HL Mountain Tire  1504884.151504884.15
Road Tire Tube 35920.5 35920.5
Touring Tire62364.225  62364.225
Water Bottle – 30 oz.  28052805
Grand Total 62364.225 35920.5 1507689.15 1605973.875

This is pivot transformation output is down below.

The link of the below source is

https://msdn.microsoft.com/en-us/library/ms140308.aspx 

Pivot Transformation

SQL Server 2016
 

The Pivot transformation makes a normalized data set into a less normalized but more compact version by pivoting the input data on a column value. For example, a normalized Orders data set that lists customer name, product, and quantity purchased typically has multiple rows for any customer who purchased multiple products, with each row for that customer showing order details for a different product. By pivoting the data set on the product column, the Pivot transformation can output a data set with a single row per customer. That single row lists all the purchases by the customer, with the product names shown as column names, and the quantity shown as a value in the product column. Because not every customer purchases every product, many columns may contain null values.

When a dataset is pivoted, input columns perform different roles in the pivoting process. A column can participate in the following ways:

  • The column is passed through unchanged to the output. Because many input rows can result only in one output row, the transformation copies only the first input value for the column.

  • The column acts as the key or part of the key that identifies a set of records.

  • The column defines the pivot. The values in this column are associated with columns in the pivoted dataset.

  • The column contains values that are placed in the columns that the pivot creates.

This transformation has one input, one regular output, and one error output.

Sort and Duplicate Rows

To pivot data efficiently, which means creating as few records in the output dataset as possible, the input data must be sorted on the pivot column. If the data is not sorted, the Pivot transformation might generate multiple records for each value in the set key, which is the column that defines set membership. For example, if a dataset is pivoted on a Namecolumn but the names are not sorted, the output dataset could have more than one row for each customer, because a pivot occurs every time that the value in Name changes.

The input data might contain duplicate rows, which will cause the Pivot transformation to fail. "Duplicate rows" means rows that have the same values in the set key columns and the pivot columns. To avoid failure, you can either configure the transformation to redirect error rows to an error output or you can pre-aggregate values to ensure there are no duplicate rows.

Options in the Pivot Dialog Box

You configure the pivot operation by setting the options in the Pivot dialog box. To open the Pivot dialog box, add the Pivot transformation to the package in SQL Server Data Tools (SSDT), and then right-click the component and click Edit.

The following list describes the options in the Pivot dialog box.

Pivot Key

Specifies the column to use for values across the top row (header row) of the table.

Set Key

Specifies the column to use for values in the left column of the table. The input date must be sorted on this column.

Pivot Value

Specifies the column to use for the table values, other than the values in the header row and the left column.

Ignore un-matched Pivot Key values and report them after DataFlow execution

Select this option to configure the Pivot transformation to ignore rows containing unrecognized values in thePivot Key column and to output all of the pivot key values to a log message, when the package is run.

You can also configure the transformation to output the values by setting the PassThroughUnmatchedPivotKeyscustom property to True.

Generate pivot output columns from values

Enter the pivot key values in this box to enable the Pivot transformation to create output columns for each value. You can either enter the values prior to running the package, or do the following.

      • Select the Ignore un-matched Pivot Key values and report them after DataFlow execution option, and then click OK in the Pivot dialog box to save the changes to the Pivot transformation.

      • Run the package.

      • When the package succeeds, click the Progress tab and look for the information log message from the Pivot transformation that contains the pivot key values.

      • Right-click the message and click Copy Message Text.

      • Click Stop Debugging on the Debug menu to switch to the design mode.

      • Right-click the Pivot transformation, and then click Edit.

      • Uncheck the Ignore un-matched Pivot Key values and report them after DataFlow execution option, and then paste the pivot key values in the Generate pivot output columns from values box using the following format.

        [value1],[value2],[value3]

    Generate Columns Now

    Click to create an output column for each pivot key value that is listed in the Generate pivot output columns from values box.

    The output columns appear in the Existing pivoted output columns box.

    Existing pivoted output columns

    Lists the output columns for the pivot key values

    The following table shows a data set before the data is pivoted on the Year column.

    Year

    Product Name

    Total

    2004

    HL Mountain Tire

    1504884.15

    2003

    Road Tire Tube

    35920.50

    2004

    Water Bottle – 30 oz.

    2805.00

    2002

    Touring Tire

    62364.225

    The following table shows a data set after the data has been pivoted on the Year column.

    2002

    2003

    2004

    HL Mountain Tire

    141164.10

    446297.775

    1504884.15

    Road Tire Tube

    3592.05

    35920.50

    89801.25

    Water Bottle – 30 oz.

    NULL

    NULL

    2805.00

    Touring Tire

    62364.225

    375051.60

    1041810.00

    To pivot the data on the Year column, as shown above, the following options are set in the Pivot dialog box.

    • Year is selected in the Pivot Key list box.

    • Product Name is selected in the Set Key list box.

    • Total is selected in the Pivot Value list box.

  1. The following values are entered in the Generate pivot output columns from values box.

    [2002],[2003],[2004]

        • Configuration of the Pivot Transformation

        You can set properties through SSIS Designer or programmatically.

        For more information about the properties that you can set in the Advanced Editor dialog box, click one of the following topics:

        MH

        ↧

        Same package huge different time performance. Why?

        November 17, 2015, 3:26 am
        $
        0
        0

        Hi guys, I am running in a strange, really strange behavior. DB Sales. I've got a package A, this package has two ole db connection: an Oracle connection and a SQL connection. The issue is in the data flow from [ORACLE] --> [SQL]. I use the original Oracle language. In the package A it tooks two minutes to load 3.000.000 rows in SQL. Cool.

        In another project I added an existing package (A) and I renamed it packageB. I build a new DB, DB Marketing. I amended the package B and it is totally equals to the package A. So, Data Flow from [ORACLE] --> [SQL]. I didn't change any comma in the original Oracle SQL. The issue is that it takes twenty minutes to load the 3.000.000 rows instead of two minutes. It's very slow. But the data flow is the same.

        I had some doubt about the DB, is the only thing that is changed (unless copying the package I lost some important detail). The two DB:

        Sales (package A) DATA Initial Size 8005 AutoGr. 1000 MB to 70000 LOG I.S. 3751 A.Gr. 250 to 2097152 

        Marketing (Package B) DATA Initial Size 5000 AutoGr. 1000 MB to 10000 LOG I.S. 2000 A.Gr. 250 to 100000

        Do you think can depend on the DB setting?

         
        ↧

        Oracle provider for OEL DB is not showing in Connection manager in SSIS

        November 17, 2015, 4:43 am
        $
        0
        0

        I have installed Client 64 bit for oracle11g and when i open the BIDS 2010. I try to create the new OLE Db Connection and click on Provider dropdown list. Is it not showing "Oracle provider for OEL DB"?

        When i installed Client 32 bit for oracle11g it display the Oracle provider.

        Please help me to make compatible with client 64 bit.




        ↧
        ↧

        SSISDB Project deployment , where is the physical .dtsx file stored ?

        May 22, 2014, 12:34 am
        $
        0
        0

        Greetings everyone !

        I'm in process of migrating few packages from SSIS 2008 to SSIS 2012 . The packages are migrated and are successfully deployed over to SSIS2012  SSISDB Catalog ( Integration Services Catalog /SSISDB/Projects/Project Name/Package Name ) .

        When deployed projects to SSIDB catalog using Project model deployment , I was anticipating the .dtsx meta data  are saved some where in SSISDB database ( in previous versions , Packages when deployed to MSDB saves package metadata inside sysssispackages table object )  , But The Project file .ispac seems to be creating and saved  on a file system . I think i viewed the SSIDB database views , they only have references to the project/Executions but no actual .dtsx meta data .

        1) While executing packages ( Ryt click and execute from SSISDB Catalog  OR Scheduling it via agent ) SSISDB Catalog seems to be  referencing the package via  Physical .ispac file saved on File System and then execute it with configured parameters ??

        2) What benefit do i get as  a part of SSISDB catalog deployment other than the parameter/Environment features  ? I'm specially concerned about backup of my package/Project  objects .

        Thanks in advance for your pointers and help .

        Rajkumar Yelugu

        ↧

        What is the best approach to assess the impact of changes across hundreds of SSIS packages?

        November 16, 2015, 8:08 am
        $
        0
        0

        I know we can scan the SSIS package XML files, but is there another approach for assessing the impact to SSIS packages when the database changes (without purchasing a third party tool, like Pragmatic Works Doc-xpress)?

        We are currently using SSIS 2008R2 but upgrading to SSIS 2014 early next year. Does the SSISDB provide any functionality to help with this?

        Thank you so much for helping! Kathy

        ↧
        Search

        Script Transformation Task

        September 13, 2011, 10:38 pm
        $
        0
        0

        Hi All,

        I wrote a C# code to split delimited rows into columns. I created script transformation editor with two output columns as primary_emailaddr and secondary_emailaddr. I added the following code but it is throwing me an error as

        Error    1    The name 'Output0Buffer' does not exist in the current context    

         

        using System;using System.Data;using Microsoft.SqlServer.Dts.Pipeline.Wrapper;using Microsoft.SqlServer.Dts.Runtime.Wrapper;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
publicclass ScriptMain : UserComponent
{publicoverridevoid CreateNewOutputRows(Output0Buffer Row)
    {
        EmailAddresses seperatedAddresses = SplitEmail(Row.EmailAddr);

        Output0Buffer.AddRow();
        Output0Buffer.Primary_EmailAddr = seperatedAddresses.PrimaryEmail;
        Output0Buffer.Secondary_EmailAddr = seperatedAddresses.SecondaryEmail;

    }

    public EmailAddresses SplitEmail(string emails)
    {var delimeters = newstring[] { ",", ":", ";", "'", "/", "|" };var splitResults = emails.Split(delimeters, StringSplitOptions.None);returnnew EmailAddresses
        {
            PrimaryEmail = splitResults[0].Trim(),
            SecondaryEmail = splitResults[1].Trim()
        };

    }

    publicclass EmailAddresses
    {publicstring PrimaryEmail { get; set; }publicstring SecondaryEmail { get; set; }
    }


}

        Any help would be really helpful.

        Thanks
        Tarak

        ↧

        sql server agen job using script

        November 16, 2015, 10:39 am
        $
        0
        0

        hi

        i am trying to create sql server agent job ,but when i add parameter its giving me error missing quotes in below line

        @command=N'/ISSERVER "\"\SSISDB\HIF\package\package.dtsx\"" /SERVER "\"DAYDEVSSIS01\DEV_108\"" /Par PGender;"\"\\day-appdev05\Applications\Hif\\"" /Par LookUp;"\"\\day-appdev05\Applications\Hif8\\"" /Par "\"$ServerOption::LOGGING_LEVEL(Int16)\"";1 /Par "\"$ServerOption::SYNCHRONIZED(Boolean)\"";True /CALLERINFO SQLAGENT /REPORTING E',

        where i am missing quote.job created successfully but when i run its giving me error

        ↧
        ↧

        SQL Server 2012: What would be the best way to estimate the length of time a transform workload will take?

        November 17, 2015, 7:38 am
        $
        0
        0

        Fellow Techies--

        We're migrating an ETL process from a hardware environment with about 240 GB of installed memory to one with 1.5 TB for physical memory, 1.5 TB of virtual memory.  This will be the "maiden voyage" on this new box.  On the old box, the process took roughly 5 days to process from start to finish, with the longest part of the process surrounding multiple sets of MERGE statements against very large data sets.  In calculating what we can expect for performance on the new hardware, what should I be looking at?

        ↧

        SSIS package testing

        November 17, 2015, 1:34 pm
        $
        0
        0

        I recently started a new position at a large corporation that utilizes SSIS heavily. I am a developer supporting what I call the guts of SSIS. Basically the majority of our packages are programmed in c# and we have developed, in house, our own set of tasks beyond the stock stuff.  My problem is that while I can test the custom tasks easily enough I can't find a way to effectively test the packages.  I recently added a property to one of our custom connection managers and missed setting the value in 1 connection manager which ended up effecting 7 packages out of a couple hundred.  It was a simple mistake that could have been avoided via testing.

        I don't see how you can test packages though that are defined to connect to various databases or make stored procedure calls.  And what about the ones with complex data flows?  I could imagine an entire staff dedicated to just testing packages - which I, of course, can't do.

        Minimally I'd like to be able to at least validate the package - like what BIDS does when you first open a package.  That alone would have caught my last blunder.  I thought the validate method from package would do that but as far as I can tell it performs a custom validation based on what the programmer wants to check which means you need to know what you're checking for.

        So I'm not really sure how to effectively test a package without having to run it in production, a scary thought, or have a staff run some sort of extravagant test environment.

        ↧

        SSIS Bulk Insert Error truncation

        November 17, 2015, 1:09 pm
        $
        0
        0

        Hello everyone

        I have am having some issues bulk inserting from a flat file (CSV) to the database. I have also tried this by using the import and export wizard and get the following error:

        I dont understand what the issue. 

        the table that i have created looks like this:

        CREATE TABLE IderaPatchAnalyzer
(
IP_Adresse varchar(64) NOT NULL,
Release_ varchar(50) NOT NULL,
Level_ varchar(50)NOT NULL,
Edition_ varchar(50) NOT NULL,
Build varchar(50)NOT NULL,
Updates_Available varchar(50)NOT NULL,
Supported_  varchar(50)NOT NULL,
Support_Status varchar(50)NOT NULL,
KB_Available varchar(50)NOT NULL,
PRIMARY KEY (IP_Adresse)
);

        I have in the changed the outputcolumnwidth in Ip_Adresse to 64. The length of the cells are not near 50 however i want it to be sure that its not the case.

        What is wrong? when i try to do the same in my SSIS project, i also get an error. I do get a warning: Truncation may occur due to inserting data from data flow column """"KB Available""" with a length o..... 

        in that column there are max 5 varchar:  "yes" and "no". 

        The  """"KB Available""" is the column name in the flat file (CSV), I have made checkmark in Column names in the first data row. 

        I have used the following guide for my SSIS project:

        Journey to SQL Authority with Pinal Dave

        Thanks in advanced



        ↧
        Viewing all 24688 articles
        Browse latest View live
        Search
        <script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>