Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Calculations in query taking long time and to load target table

$
0
0

Hi,

I am pulling approx 45 Million records using the below query in a ssis package which pulls from one DB on one server and loading the results to another target table on the another server. In the select query I have a calculation for 6 columns. The target table is trunctaed and loaded every day. Also most of the columns in the source which I used for the calculations is having 0 and it took approximately 1 hour 45 min to load the target table. Is there any way to reduce the load time? Also can I do the calcultions after once all the 47 M records loaded during query running and then calculate for the non zero records alone?

SELECT T1.Col1,
T1.Col2,
T1.Col3,
T2.Col1,
T2.Col2,
T3.Col1,
convert( numeric(8,5), (convert( numeric,T3.COl2) / 1000000)) AS Colu2,
convert( numeric(8,5), (convert( numeric,T3.COl3) / 1000000)) AS Colu3,
convert( numeric(8,5), (convert( numeric,T3.COl4) / 1000000)) AS Colu4,
convert( numeric(8,5),(convert( numeric, T3.COl5) / 1000000)) AS Colu5,
convert( numeric(8,5), (convert( numeric,T3.COl6) / 1000000)) AS Colu6,
convert( numeric(8,5), (convert( numeric,T3.COl7) / 1000000)) AS Colu7,
FROM Tab1 T1 
JOIN Tab2 T2
ON (T1.Col1 = T2.Col1)
JOIN Tab3 T3
ON (Tab3.Col9 =Tab3.Col9)


Anand


Design Query - Read ChangeVersion

$
0
0

I have a ChangeTracking table with a single ChangeVersion INT column with an initial value of 0.

I have an SSIS package which contains 2 Data Flow Tasks, each with 2 Stored Procedures. Each of theStored Procedures requires the current ChangeVersion value each time the package is executed.

Once package execution is complete, an Execute SQL task is used to increment the ChangeVersion value by 1.

Should I:

1 - Read the ChangeVersion value into the SSIS package and store in a variable to be used by eachStored Procedure?

OR

2 - Create a UDF to return the ChangeVersion value on each run and call from within each SP? 

Should I be aware of a performance consideration with these options? Option 1 uses an additional DB read where option 2 involves invoking a UDF 4 separate times on each run.


scheduling packages in ssis

$
0
0

Hi

  I have deployed 2 packages in that first i scheduled one package after execution immediately second package should execute

how can i schedule a job for this scenario?

Server not running SSIS job in 32bit mode

$
0
0

I have a package that I developed in VS2012 (using the SQL Data Tools component) which collects data from a DBF file using the VFPOLEDB provider, and puts it into a database on a SQL Server 2012 X64 server. The project containing the package has the DebugOption of Run64BitRuntime set to false. I have imported this package to the SSIS Package Store of a test and live server (identical setups). The VFPOLEDB provider is installed on both, and I can see in the registry of both machines that it's there for 32bit runtimes.

The package runs fine on the test machine, but fails on the live machine. The live instance of SQL does not appear to recognize the 32bit VFPOLEDB provider that is installed.

The only difference in the instances of SQL is that the live environment has an Integration Service Catalog set up, where as the test does not. Looking at the logs of the servers, when the live starts, it runs sp_ssis_startup, and then logs messages about unsafe assemblies being loaded in. This SP is not run on the test environment because there is no catalog.

The jobs I create have the flag set to use the 32bit runtime, but I can't help but feel that the SSIS Catalog is having an issue with the VFPOLEDB I am using, and not loading it.

I don't really know anything about the SSIS catalog, so is anyone able to suggest any direction I could move in?

CHANGE_TRACKING_CURRENT_VERSION

$
0
0

I am tracking insert changes only on 4 tables. I am using an SSIS package and stored procedures to return incremental changes to all tables to populate a data warehouse.

When the initial data is loaded into the database, the CHANGE_TRACKING_CURRENT_VERSION changes to 1. 

If I then run the SSIS package for loading the DW, it will not return any records as we are using the CHANGE_TRACKING_CURRENT_VERSION as the last sync version. 

Must we do an initial load before we enable change tracking and use CHANGE_TRACKING_CURRENT_VERSION?

Do we need to track changes to each table separately?

Do we need to use our own custom change tracking id?



The File name property is not valid. The file name is a device or contains invalid characters

$
0
0
I have an SSIS task that has run successfully for years. They just moved the dtsx to a new server and now it is failing. 

Issue:

The task has a data flow that writes a raw file destination. It then has a subsequent data flow that reads the raw file destination. 
The path and name of the raw file is passed to each of the data flows via the same variable. 
The raw file resides in a folder on the server in the same path that the dtsx resides in. 
The task fails with the "The File name property is not valid. The file name is a device or contains invalid characters" error when trying to access the raw file.

What we have done to troubleshoot:

1. I ran the task successfully on my local machine so it is definitely a server issue.
2. The first thing we were seeing was that it failed trying to write to the raw file. The task is set to "Create Always". 
3. We saw that the dba's copied over the entire directory structure from the old server which included a previously created raw file, so we deleted the raw file.
4. We reran the task and it successfully wrote a new raw file but then failed on reading the raw file with the same error as above.
5. We reran the task AGAIN and this time it failed trying to write the raw file throwing the same error as above.
6. The dba looked at the directory and the account that performs the task has full control.
7. The dba looked at the raw file that was created and verified that the account that performs the task has full control of the file and that it is the owner of the file.


Summary:

The task fails when trying to access the file EXCEPT when the file does not exist. In that situation it can write the file but subsequently fails in accessing it again.


cannot convert between unicode and non-unicode string data types.

$
0
0
I'm trying to copy the data from 21 tables in a SQL 2005 database to a MS Access database using SSIS. Before converting the SQL database from 2000 to 2005 we had this process set up as a DTS package that ran every month for years with no problem.  The only way I can get it to work now is to delete all of the tables from the Access DB and have SSIS create new tables each time. But when I try to create an SSIS package using the SSIS Import and Export Wizard to copy the SQL 2005 data to the same tables that SSIS itself created in Access I get the "cannot convert between unicode and non-unicode string data types" error message. The first few columns I hit this problem on were created by SSIS as the Memo datatype in Access and when I changed them to Text in Access they started to work. The column I'm stuck on now is defined as Text in the SQL 2005 DB and in Access, but it still gives me the "cannot convert" error.

Fail to start project

$
0
0

Hi, everyone

When I click start to run the project, there is a error message: 

Failed to start project

------------------------------
ADDITIONAL INFORMATION:

Method not found: 'Boolean Microsoft.SqlServer.Dts.Design.VisualStudio2012Utils.IsVisualStudio2012ProInstalled()'. (Microsoft.DataTransformationServices.VsIntegration)

I installed vs2012 iso and integrated, my version is 11.0.5058.0.

Any help appreciate.


Rounding Numbers in SSIS

$
0
0
I have imported data from excel and i m exporting it to text file. In excel i see one value as 0.85 but when i am exporting it to text it has changed to 0.84999999999999999999. How to overcome this situation is SSIS.

Import wizard doesn't come up! (DTSWizard.exe faults in dbgtrace.dll)

$
0
0

This has been happening intermittently for a few weeks, but now it seems pretty constant. When I click on the "Tasks -> Import Data..." option for my databases, no wizard window pops up. When I look in my Application event log, I see entries like this:

Application Error 1000

Faulting application name: DTSWizard.exe, version: 2011.110.5058.0, time stamp: 0x5373fe54
Faulting module name: dbgtrace.dll_unloaded, version: 9.0.1404.0, time stamp: 0x52949283
Exception code: 0xc0000005
Fault offset: 0x00002b09
Faulting process id: 0x998
Faulting application start time: 0x01d0462a28ed3ffb
Faulting application path: C:\Program Files\Microsoft SQL Server\110\DTS\Binn\DTSWizard.exe
Faulting module path: dbgtrace.dll
Report Id: 6846f3d4-b21d-11e4-9797-00219b02078a
Faulting package full name:
Faulting package-relative application ID:

Perflib 1008

The Open Procedure for service "BITS" in DLL "C:\Windows\System32\bitsperf.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

The Open Procedure for service "Lsa" in DLL "C:\Windows\System32\Secur32.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

The Open Procedure for service "msdrda" in DLL "C:\Program Files\Microsoft OLE DB Provider for DB2\system\MSEIDPM.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

The Open Procedure for service "MSDTC" in DLL "C:\Windows\system32\msdtcuiu.DLL" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

.Net Runtime 1026

Application: DTSWizard.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: exception code c0000005, exception address 60025357

Have tried my best Google-fu, but can't find any leads on what is going wrong or how to fix it. I'm using SQL server 2012 service pack 2 on Windows 8.1 Pro, a 32-bit system.

Much appreciations in advance for any assistance.

Package Help needed For each Loop container

$
0
0

Hello all,

I need a bit of advice, I`m a newbee to SSIS so I`m learning. Here`s what I need. I have a EDI 837 file which needs to be converted and inserted into a sql table. These are multiple files and I need to maintain them with a monthly load. So I have a source folder where these files will be dumped. I created a Package which will grab the files process the EDI aspect inserts into the data into table. This is the story on a high level.

Here`s what I did.

Created a For each loop container.

I have a stored proc which takes in a 837 files and gets the fields what I want and it inserts it into a staging table from there I have a data flow task which pipes it into my production table.

Inside the loop first task is to exec the Stored Proc but my question is once a file is picked up from for each loop I need the file to be inserted into a table which I am referencing in my store proc. All the EDI conversion takes place in my SP and it references the table..

My question is the For each loop container picks up the files but where does the data goes?? Can someone help me out


FM

Adding a New Date Column

$
0
0

This is probably simply answered, but I can't find it on the web...

I'm importing data from an existing Excel file, and I need to add a few columns using the Derived Column Transformation tool. As I add strings, I simply put double quotes and it types tehm fine; numerics, I just put 0 and, again, they type fine.

But, I don't know how to put in a date to get it to type as DT_DATE. I tried "0000-00-00," '0000-00-00,' and 0000-00-00, but they all error out, unable to parse.

What do I put in the Expression field on the tool to get it to initialize the column as DT_DATE?

Thanx!

Year Month Format

$
0
0

I have an input column value of Sep-15, i want to see output as 201509. what is the most efficient way to achieve it in SSIS 2008R2?

The source is Excel not SQL Sever.

Report Model with mulitple data sources

$
0
0

Hi,
We have requirement to develop report model in 2008 having multiple data sources which includes SQL SERVER 2012 and Oracle database so end user can all the relevant data in one place to run in report builder. So please advise how this can be achieved.

One option I can think of pulling all the data into one database through SSIS and then build a smdl on it.

I heard it might be possible to build Tabular cube and then use Report builder on it not sure which is the best option unless any other solution.

Thanks,

Cannot load script for execution.

$
0
0

hi 

 iam creating  the  sisi package  with in script task . script task   was  build successful .  but when i was executive the ssis packages vs 2012.

 it get  error :Cannot load script for execution..

 the code is 

/*
   Microsoft SQL Server Integration Services Script Task
   Write scripts using Microsoft Visual C# 2008.
   The ScriptMain is the entry point class of the script.
*/

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;



namespace ST_8cdbf584a74b48abb384d8f03cf42f4d.csproj
{
    
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        #region VSTA generated code
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

        /*
The execution engine calls this method when the task executes.
To access the object model, use the Dts property. Connections, variables, events,
and logging features are available as members of the Dts property as shown in the following examples.

To reference a variable, call Dts.Variables["MyCaseSensitiveVariableName"].Value;
To post a log entry, call Dts.Log("This is my log text", 999, null);
To fire an event, call Dts.Events.FireInformation(99, "test", "hit the help message", "", 0, true);

To use the connections collection use something like the following:
ConnectionManager cm = Dts.Connections.Add("OLEDB");
cm.ConnectionString = "Data Source=localhost;Initial Catalog=AdventureWorks;Provider=SQLNCLI10;Integrated Security=SSPI;Auto Translate=False;";

Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.

To open Help, press F1.
*/

        public void Main()
        {
            byte[] dataByte = null; String errInfo = "";
            try
            {
                String sDirectory = Dts.Variables["User::Directory"].Value.ToString();
                String CreatePartitionFileXMLA = sDirectory + "\\partitionssales20010701to20010731.XMLA";
                String ProcessPartitionFileXMLA = sDirectory + "\\partitionssales20010701to20010731.XMLA";
                // Create XMLA Script
                Dts.Variables["User::CreatePartitions"].Value = ReadFile(CreatePartitionFileXMLA, errInfo);
                if (errInfo.Length > 0)
                {
                    Dts.Log("Error while reading XMLA file: " + errInfo, 0, dataByte);
                    Dts.TaskResult = (int)ScriptResults.Failure;
                }
                //Process XMLA Script
                Dts.Variables["User::ProcessPartitions"].Value = ReadFile(ProcessPartitionFileXMLA, errInfo);
                if (errInfo.Length > 0)
                {
                    Dts.Log("Error while reading XMLA file: " + errInfo, 0, dataByte);
                    Dts.TaskResult = (int)ScriptResults.Failure;
                }
                Dts.TaskResult = (int)ScriptResults.Success;
            }
            catch (Exception ex)
            {
                Dts.Log("Error Message: " + ex.Message, 0, dataByte);
                Dts.TaskResult = (int)ScriptResults.Failure;
            }
        }

        public String ReadFile(String FilePath, String ErrInfo)
        {
            String strContents;
            StreamReader sReader;
            try
            {
                sReader = File.OpenText(FilePath);
                strContents = sReader.ReadToEnd();
                sReader.Close();
                return strContents;
            }
            catch (Exception e)
            {
                MessageBox.Show(ErrInfo);
                ErrInfo = e.Message;
                return "";
            }
        }
    }
}

 how to solve this issue

 pleasse help me..

 


check for text files in folder

$
0
0
How do we check if ther exist any text file in a particualr folder.

Problem with sampling data using Percentage Sampling

$
0
0

I am using Percentage Sampling transformation to sample data of 10 records. As you can see below, out of the 10 incoming rows, only 4 rows are going to the output, rest of the 6 rows are not getting sampled at all. Does anyone know why this is happening?

http://i.imgur.com/WHpbzue.png

How to check edition of SSIS (no db engine)?

$
0
0

Is there a way to check the edition of SSIS? i've tried connecting to SSIS through SSMS and looking at the reports, but that just gives me the version number.

there is no database so SELECT @@version doesn't apply. It's also possible to have different editions of sql db engine and SSIS installed on the same server so that's more of an inferred edition rather than the actual ssis edition. i want to find out from SSIS directly what edition it is


Jakub @ Adelaide, Australia

Convert SQL Query to Derived Column Expression

$
0
0

I'm in the process of reworking some SSIS packages I inherited. A lot of these use Execute SQL Statement components to update certain fields in the tables once they've been loaded. I'd like to replace these individual UPDATE statements with Derived Column components.

I've got 95% of them converted, except for a couple, one of which has me a bit perplexed and was hoping someone could provide some insight.

Essentially, there is a column called POSTPERIOD which is a date in string format YYYYMM (i.e., 201503). The SQL update parses out the month piece and converts it to the month name and populates a column called POSTPERIOD_MONTH.

Here is the SQL:

select    DateName(month , DateAdd(month, CONVERT(INT, SUBSTRING(POST_PERIOD,5,2)), 0 ) - 1 )

I'd like to accomplish this using a Derived Column, but not sure how to go about doing this...

If someone could point me in the right direction, it would be greatly appreciated!

Thanks!


A. M. Robinson

XML as source in SSIS: needed one single tab as destination

$
0
0

Hi

As source in SSIS I have an XML file and as destination an Excel file. SSIS produces in Excel as many tabs as blocks in XML. The last column in every tab contains some IDs like for instance employee_id. Such IDs are not found in the XML file. In my understanding SSIS produces itself these IDs to keep track of connected (nested) blocks.

Currently I use vlookup on those IDs to "manually" put together all tabs in one single tab. Isn't it possible to automatically get all data from XML in one single Excel tab?

By simpy importing XML in Excel (without SSIS) you also get not only one single tab. Well, depending on some import options you can actually get one single tab but it's not easily readable because some shifts - as you go to the right the line number increases like this:

Name   Town   Age

Paul

            London

                          44

Thanks!

Gabi


gec

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>