Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

SSIS package exclusive lock on source table

$
0
0

I have created a package that copies data from one table to another table on a different server. When I run the package, most of the time it runs for a while and then hangs.  I am using the method found in this website: http://blogs.msdn.com/b/jorgepc/archive/2010/12/07/synchronize-two-tables-using-sql-server-integration-services-ssis-part-i-of-ii.aspx for synchronizing the tables.  In BIDS as I am watching the execution of the package all objects turn Yellow in the Data Flow window and the counts do increase till it hangs.  Then the counts stop.

I have run the SQL Server "Resource Locking Statistics by Objects" report on the source server/database and it reports there are two locks on that table by my userid at the time I am running the package (a page lock and a object lock).  All I am doing with the table is reading it.  In the package the OLE BD Source(SQL Server Native Client 10.0) I am using a query "select * from table where id between ? and ?".  "id" is an identity column and a clustered index.

This package does work once in a while.  There was never any problems in the Development environment.  This is occurring in Production where another application is writing data to the table all day long.  The Development environment did not have another app running at the same time.

The package also copies 6 other tables with no problem.  The problem table is larger (more columns) than the other tables.

Any ideas of what the problem might be?

Fred


Fred Schmid




Best way to import csv file into sql server

$
0
0

Hello DBAs

always thankful for your help and one question, what is the best way to import a csv file into sql server ?

How do you update a field in database with script task?

$
0
0
Pretty new to SSIS - So not sure if I have the order correct. 

Goal:  I would like to cycle thru each row in a table.  Pull field X, write some code that transforms it from Hexidecimal to Decimal and then write the Decimal value to field Y.  

Problem:  I can pull field X and convert to Decimal with a Script Task but I can't successfully update Field Y in database with the new value.   

Current set up is a a ForEach Loop with a script task that pulls a variable and updates a variable.  What do you need to do to update the database Field with new value. 

How do you update this column and save it.   I currently can pull the value of Field Y but can't get it to save with new value. 

Cheers. 

deployement of ssis packages

$
0
0

Hi all,

in my dev enviorement i have 10 packages but i want to deploy only 5 packages in testing enviorement .

Can an SSIS package be run/invoked multiple times while the previous invocation is still running?

$
0
0

Hi All,

I tried a Google search on running SSIS packages "asynchronously" but that mainly comes up with results describing synchronous or asynchronous processing of data *within* a package. That's not what I'm trying to figure out.

What I want to know is simply: can an SSIS package be invoked multiple times and have different invocations of that package running concurrently? 

For example, user uploads a data file to a web application, and then the application calls the SSIS package. It takes a minute or so to complete. While it's running another user uploads a file, then another user uploads, etc. 

Will the instances of the package execution run concurrently, or will they be queued up by SSIS? Or (even worse) will we have to come up with some sort of queuing system ourselves?

Thanks in advance for any info you can provide

Dylan

Dynamic columns with Excel Source?

$
0
0

I have Excel file number 1 with columns A ja B.
I have database table with columns A,B,C,D,E,F,G,H etc. (there are 100 columns)

I know how to import normally data with Excel Source and OLE DB Destination with Excel file 1.

Now I have new need.
I should be able to import dynamically Excel files with any combination of columns.
It should be automatic import with For Each Container.
When new files like below is imported, I should not be make any changes to SSIS.

Excel 2) Columns A, G, X (so column A data must be added to column A in database etc)
Excel 3) Columns B, C, G, Y (so column B data must be added to column B in database etc)
Excel 4) Columns D, X

IS THIS POSSIBLE WITH SSIS? HOW?

CUSTOM CODE IS NEEDED? ANY POINTERS TO SOLUTION?


Kenny_I



Related to execute process task

$
0
0

Folder will have 5 zip files.Iam unable to unzip all files into a specific folder

but unzipping of  single file by passing  argumets:"-e -o"+" E:\\File20131201_1.zip"+" E:\\zipfiles"

please give suggestions to unzip multiple file

Forcing outcome of “ failed” SSIS package to log as “Succcess” in SQL Server 2008 Job Log File Viewer

$
0
0

I have a SQL Server Agent  job that calls an SSIS package.The first task of the SSIS package is a C# script that checks for the existence of a file in a folder. The file may or may not exist in the folder.

If the file exists the task completes successfully and the next task in the package is invoked. The job retries a number of times before terminating.

The issue I have is that if the file does not exist , when the job finishes it writes an entry in the History  Log that the “Job Failed”.The job is scheduled to run every day, but the file will appear only once a month, and which day that happens is unknown.So I end up with a History Log with job fail entries every day except once a month.As far as I am concerned , the job is a success if it runs, if no file exists, that is not a failure in this context. So I do not want a job failure written to the History Log of the job.

I still want the job the  package to terminate at the first task if a file is not found, but I do not want fail to be written to the History Log of the Job.

Is there some amendment to the C# script I can make to achieve this?

Or is there some other way?

I have been trying to implement the recommendations at these locations below but without success so far, they seem to point in the direction  of creating an event handler to achieve this..

http://sqlblog.com/blogs/rushabh_mehta/archive/2008/04/24/gracefully-handing-task-error-in-ssis-package.aspx

http://dougbert.com/blog/post/Faking-success-in-SSIS.aspx

Any guidance  greatly appreciated.

Thank you

The C#  script is below

using System;
using System.IO;
using System.Data;
using System.Collections.Generic;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Text.RegularExpressions;

namespace ST_e72b194f12ee4fbab7090f4187b09a3c.csproj
{
    [System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };

        private Int32 fileIndex;

        public void Main()
        {
            String[] DEPOSFiles = Directory.GetFiles(Dts.Variables["SourceDataFileFolder"].Value.ToString(), "CMGDEPOS_*");
            if (DEPOSFiles.Length != 0)
            {
                for (int i = 0; i < DEPOSFiles.Length; i++)
                {
                    File.Move(DEPOSFiles[i],DEPOSFiles[i].Replace("DEPOS", "Corpos"));
                }
            }

            String[] opicsFiles = Directory.GetFiles(Dts.Variables["SourceDataFileFolder"].Value.ToString(), Dts.Variables["LogicalFileName"].Value.ToString() + "_*");
            FileInfo file;
            if (opicsFiles.Length != 1)
            {
                try
                {
                    Int32 fileLen = 100;
                    for (int i = 0; i < opicsFiles.Length; i++)
                    {
                        if (opicsFiles[i].Length < fileLen)
                        {
                            fileLen = opicsFiles[i].Length;
                            fileIndex = i;
                        }
                    }
                    if (Regex.Matches(opicsFiles[fileIndex], "_").Count == Regex.Matches(Dts.Variables["LogicalFileName"].Value.ToString(), "_").Count + 1)
                    {
                        file = new FileInfo(opicsFiles[fileIndex]);
                    }
                    else
                    {
                        throw new ArgumentException("No source data files exist that match the LogicalFileName(" + Dts.Variables["LogicalFileName"].Value.ToString() + ")", "LogicalFileName");
                    }
                }
                catch
                {
                    throw new ArgumentException("No source data files exist that match the LogicalFileName(" + Dts.Variables["LogicalFileName"].Value.ToString() + ")", "LogicalFileName");
                }
            }
            else
            {
                if (Regex.Matches(opicsFiles[0], "_").Count == Regex.Matches(Dts.Variables["LogicalFileName"].Value.ToString(), "_").Count + 1)
                {
                    file = new FileInfo(opicsFiles[0]);
                }
                else
                {
                    throw new ArgumentException("No source data files exist that match the LogicalFileName(" + Dts.Variables["LogicalFileName"].Value.ToString() + ")", "LogicalFileName");
                }
            }
            while (IsFileLocked(file))
            {
            }
            Dts.Variables["SourceDataFilePath"].Value = file.FullName;
            Dts.TaskResult = (int)ScriptResults.Success;
        }

        protected virtual bool IsFileLocked(FileInfo file)
        {
            FileStream stream = null;
            try
            {
                stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
            }
            catch (IOException)
            {
                return true;
            }
            finally
            {
                if (stream != null)
                    stream.Close();
            }
            return false;
        }
    }
}


SSIS called via sql job does not use defined configuration files!

$
0
0
Anybody knows how to handle this.

I deployed a SSIS.
Then I create a job in which I add the config files.

When running the SSIS, the config files used are the ones the developper was using instead of the ones defined in the job.

I tried, redeploying, redefining the config files, deleting and recreating both the job and the ssis.  Still, the config files I define are not used.  When looking at the sql generated, the files are the adequate ones.  However, at run time, I receive the message for a different file then the one defined:

If I script the job and look at the command I get:

@command

=N'/SQL "\Import" /SERVER SQLSVR
/CONFIGFILE "E:\Mssql\SSIS\Import\Configuration\LIVE\cnfg_Import_Properties.dtsConfig" 
/CHECKPOINTING OFF /REPORTING E'
,

When looking at the generated log file, I get:
OnWarning,SQLSVR08,GREGORY\IntServices,Import,{A08A9F3D-1DDB-4AAC-8791-371AD44D3B65},{3C719D08-E954-4396-A5B6-A76910E27ED9},1/6/2010 11:00:17 AM,1/6/2010 11:00:17 AM,-2147409902,0x,The configuration file name "C:\WD\Development\Main\SSIS\Import\Import\cnfg_Import_Properties.dtsConfig" is not valid. Check the configuration file name.

Is there something I can do to force the usage of the config file defined at the job level and to forget the one the developper was using on his local computer?

Project Deployment Model + Source Control, Multiple Developers

$
0
0

How does the project deployment model work with multiple developers + source control + iterative deployment?

SSIS 2008R2/package deployment model allows multiple developers to work separate packages independently in a single project, and deploy those packages independently onto dev/test or even production if it's a bugfix

What happens in the project deployment model where one dev has a new package ready for deployment in a data warehouse project with say 30-40 different packages, but there's other developers that have been making changes to existing packages or working on their own and those changes are ongoing/have broken those packages

My understanding of the project deployment model means developer A would need to deploy the entire project, including the broken/unfinished packages to be able to test their work. Is this correct or am I missing something?


Jakub @ Adelaide, Australia


I want to know how many records are updated and inserted i want to due in ssis

$
0
0

HI all,

i want to develop one package while iam running the package i came to know how many records are updated and insert

can any one help me regarding this with step by step process.

Become our FIRST Microsoft TechNet SSIS Guru of 2014!!

$
0
0

Happy New Year!

Time for a fresh start!

We're looking for the first Gurus of 2014!!

This is your chance to make your mark on the Microsoft developer community.

All you have to do is add an article to TechNet Wiki from your own specialist field. Something that fits into one of the categories listed on the submissions page. Copy in your own blog posts, a forum solution, a white paper, or just something you had to solve for your own day's work today.

Drop us some nifty knowledge, or superb snippets, and become MICROSOFT TECHNOLOGY GURU OF THE MONTH!

This is an official Microsoft TechNet recognition, where people such as yourselves can truly get noticed!

HOW TO WIN

1) Please copy over your Microsoft technical solutions and revelations toTechNet Wiki.

2) Add a link to it on THIS WIKI COMPETITION PAGE (so we know you've contributed)

3) Every month, we will highlight your contributions, and select a "Guru of the Month" in each technology.

If you win, we will sing your praises in blogs and forums, similar to the weekly contributor awards. Once "on our radar" and making your mark, you will probably be interviewed for your greatness, and maybe eventually even invited into other inner TechNet/MSDN circles!

Winning this award in your favoured technology will help us learn the active members in each community.

Feel free to ask any questions below.

More about TechNet Guru Awards

Thanks in advance!
Pete Laker


#PEJL

Got any nice code? If you invest time in coding an elegant, novel or impressive answer on MSDN forums, why not copy it over to the one and onlyTechNet Wiki, for future generations to benefit from! You'll never get archived again!

If you are a member of any user groups, please make sure you list them in the Microsoft User Groups Portal. Microsoft are trying to help promote your groups, and collating them here is the first step.


how to find a certain usage of a specific value inside an SSIS package

$
0
0

Hi,

In T-SQL, there are many tools to search for instances of a specific value in all the objects such as tables and SPs.

The only way I can think of in SSIS, is to search for a value in the XML of the DTSX, but it is not that user friendly.

Are there any existing tools or other ways for a value search that can assist ?

Thanks,

Dani

EXCEL DESTINATION ERROR IN SSIS

$
0
0

I am transferring data from production.product table of adventureworks to excel and it giving me below error

[Excel Destination [414]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E09.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Excel Destination" (414) failed with error code 0xC0202009 while processing input "Excel Destination Input" (425). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

How to divide data per client from a single database?

$
0
0

I am looking for best practices to separate data from a single database into multiple client specific databases. Currently my application dumps data for all the customers into single database and I need to separate data for each client(for security reasons I can't use single database) into respective client databases.

Can we use SSIS effectively to perform above task? Is there any way this SSIS package will understand that new customer is added (New ClientID) and create a new database for it?


Visual Studio does not recognize Integration services being installed Sql Server 2008 R2

$
0
0
I am trying to modify .dtsx packages that we run daily in Visual Studio. However when I select edit all I get in Visual studio is an error that says integration services needs to be installed. I already have integration services installed though im not sure why Visual Studio does not recognize that.

In ssisdb catalog is it possible to look at package version build number?

$
0
0
With SSIS 2008 and packages stored in the MSDB catalog, by rightclicking on the folder with packages you can obtain a report that shows the package name and version build number. In SSIS 2012 with project deployment in the SSISDB catalog, you can only see the deployment date of the project and nowhere can I find the build version numbers of the installed packages. Is it possible to see the build version numbers of packages deployed as a project in the SSISDB catalog?

Jan D'Hondt - Database and .NET development

SQL server 2012 job intermittently unable to kick-off or start SSIS 2012 package - The operation failed because the execution timed out

$
0
0

Our SQL server 2012 jobs intermittently fail bacause they are unable to kick-off or start the associated SSIS 2012 package.

The error we get is: 

Failed to execute IS server package because of error 0x80131904. Server: Package path: \SSISDB\ETL\xxx

Environment reference Id: NULL.

Description: The operation failed because the execution timed out

Source: .Net SqlClient Data Provider
Started:  7:00:03 AM
Finished: 7:00:10 AM
Elapsed:  6.974 seconds
Microsoft (R) SQL Server Execute Package Utility
Version 11.0.2100.60 for 64-bit

Is there any limitation on how many SSIS 2012 packages can run con-currently?

How to find out at a given time (both current as well as in past) how many SSIS 2012 packages were running con-currently?

Any solution other than changing the start times of SSIS package jobs?

transcation

$
0
0

I have

4 Execute Sql statement inside package.

i want 1st Execute statement not part of transcation (transcation option =NotSupported at task level)

2nd Execute Sql staetment : transcation option =Required

3rd and 4th  Execute Sql staetment : transcation option =Supported

package level set = supported

But it is not working . giving error at 2nd task.

Note :i have try for pacakge level = reuired.. but same error

Cannot insert the value NULL into column

$
0
0

This this apparently is a pretty common problem. Here is my twist. This process has worked fine in the past and just today it decides to blow up on me. I'm inserting data from a flat file into a table. The column that is blowing up is a primary key that is marked as an identity with auto increment. There are 50K records and it blows up around 9k record loaded. I looked at the file around the area that is blowing up and there is nothing unusual there.

In SSIS keep nulls is unchecked and I ignore the field in the mapping. What am I missing here?

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>