Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

script task in ssis

$
0
0

hi

i have 28 lac's record in my file ,below is script task in VB 2010 which uses to attach header.

when i select top 10 its  running fine.

but will all the records its giving me exception:

here is my code:

Public Sub Main()
        Dim fileContents As New StringBuilder()
        Dim vars As Variables

        Dim finalFile As String

      
        Dts.VariableDispenser.LockForRead("FinalFile")
        Dts.VariableDispenser.GetVariables(vars)

        finalFile = CType(vars("FinalFile").Value, String)
        vars.Unlock()

     
        fileContents.Append("HBPFACT" & Dts.Variables("Increment").Value.ToString() & Dts.Variables("CurrentDate").Value.ToString())

        fileContents.AppendLine()


        fileContents.Append(File.ReadAllText(Dts.Connections("BP").ConnectionString))
        File.WriteAllText(finalFile, fileContents.ToString())

        Dts.TaskResult = ScriptResults.Success


what to do?



Selecting records and inserting records in two different SQL servers

$
0
0
We are migrating data from 2008 SQL server to 2012 SQL Server. We need to move data using Import wizard available in the SSMS. Since 2012 database tables has 2008 table data in it (Till dec2014) we need to insert only records for FY2015 from 2008 to 2012. So planned to write a SQL query for Insert into Select, but we are not sure how to specify the 2008 server details and database in that query so that it fetches data from 2008 to 2012.

Sarvan

Managing rightly the transaction log of a db for a massive import to avoid storage saturation

$
0
0

Hi,

I'm working on a virtual production SQL Server environment that hosts 10-15 databases. The data storage has a capacity about 400-500 GB. All databases are in simple recovery model. A my procedure on a my db (in simple recovery mode) reads 4-5 millions of row data from 4-5 Oracle tables having 20-30 columns. The SQL Server stored procedure is called by a SSIS 2012 task. During the procedure execution, the transaction log of my db is increased exceeding 120-130 GB. This growth has caused a storage saturation not pleasant for my customer.

So, my customer has asked me to investigate about the right management of the transaction log growth. In particular, is it possible to avoid to write in the transaction log for a certain procedure or T-SQL operation? My customer has experienced with the Oracle dbms and has said me that Oracle manages better the storage space also with blob fields.

Also if it is possible to perform a shrink operation for the transaction log, it is need to avoid the exponential growth of the transaction log before that the storage saturates.

Any helps to me, please? Many thanks

SSIS packages are failing after upgrade to SQL server 2014

$
0
0

Hi,

I have some SSIS packages running on SQL server 2012 .

After I upgraded to SQL server to 2014 from 2012 , the SSIS jobs are failing on the SQL agent.

And i can see its related to Data source connectivity to the SQL agent. I hope it not able to identify the connection manager in SQL agent. And connection adapter is not upgraded.

I read some articles about this and they say it not able to connect to SQL server agent job.

And, I can see that the package is running if i run manually using SQL 2012 run time.

Why its not running on SQL 2014 ?

did i  miss anything while upgrading SQL server 2014 ?

Please give me some suggestions to solve this issue. 

And is there any way i can change the SQL server agent 2014 to adapt this and run ?

Below is the error : 

The Package filed to load due to error 0XC00100014 " One or more error occurres. There should be more specific errors preceding this one that expalins the details of the erroes. This message is used as a return value from functions that encounter errors.: This occures when CPackage::LoadFormXML fails.

Regards,

Vinodh Selvaraj.



Schema advice for huge csv file

$
0
0

Guys, I need an advice: Huge csv file (500 millions rows) to load in a table and I did. But now I need to alter the columns (they came all as varchar(50)). I'm just change one column and it's taking age...what kind of schema should I adopt? So far I applied a simple data flow but I am wondering if I should do something like:

drop table

create table (all varchar)

data flow

alter table

No sure about

Location of SSIS Packages

$
0
0

Hello all,

I have started to work for a company and they have some SSIS packages in place. I need to find them and I don't know where to look at. I don't want to ask others all the time.

I tried to find  Microsoft Visual Studio, but they don't have this application on their server.

Where should I find the SSIS packages without asking from others?

Thanks,

GGGGGNNNNN


GGGGGNNNNN


Issue with Excel table as data source

$
0
0

Hi , I am working on an SSIS package which has excel source. I am able to get the package working when my excel file is opened but when I close the file and run the package in BIDS it fails. I realized the  Excel source sheet has Excel table which I need to Query and I dont know how to do that. I looked in web and found your blog "Excel in Integration Services, Part 2 of 3: Tables and Data Types". 

Please help.
Rakhi Agrawal

A rowset based on the SQL command was not returned by the OLE DB provider.

$
0
0

Hi

I am calling a stored procedure using ssis.

i am creating a work table--temp in the procedure and using that to join and select columns from other tables.

i am gettig the error

[Positions [22612]] Error: A rowset based on the SQL command was not returned by the OLE DB provider.

please help


SSIS Execute Process Task to Unzip Remote Files

$
0
0

I have created an SSIS Execute Process Task to unzip files on a remote server.

Executable: \\servername\c$\Program Files (x86)\7-Zip\7z.exe

Arguments: -o\\servername\d$\DBFiles\ -y x \\servername\d$\DBFiles\somefile.zip

The zip file size is about 40MB.

The SSIS task runs very slowly, close to 5 minutes. I know it should not take this long. When I remote logon to the server and run the 7z program to unzip the same file, it unzips in less than 10 seconds.

It seems that the way it is set up now, it constantly sending instructions back and forth from my PC to the server when I run the SSIS package on my PC.  

I have 50 files of varying sizes that need to be processed daily. The SSIS unzip task takes 2 hours just to unzip these files.

How should I set up the SSIS Execute Process Task so that it can unzip files on remote server efficiently?

Many thanks in advance

2012 SP1 no longer recognizes common data types

$
0
0

Since installing SP1 for SQL 2012 on my workstation, the import/export tasks from SSMS no longer recognize the varchar and nvarchar fields:

CREATE TABLE [dbo].[TestTable](
	[ID] [int] IDENTITY(1,1) NOT NULL,
	[VariableCharacterField] [varchar](50) NULL,
	[UnicodeVariableCharacterField] [nvarchar](50) NULL,
	[DateTimeField] [datetime] NULL,
	[DateField] [date] NULL,
	[TimeField] [time](7) NULL,
	[BitField] [bit] NULL,
	[BigIntField] [bigint] NULL,
	[CharacterField] [char](50) NULL,
	[UnicodeCharacterField] [nchar](50) NULL,
	[BinaryField] [binary](50) NULL,
	[DecimalField] [decimal](18, 0) NULL,
	[FloatField] [float] NULL,
 CONSTRAINT [PK_TestTable] PRIMARY KEY CLUSTERED
(
	[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]

The destination is a table that does not exist on another SQL 2012 database. I've also noticed that it changes nchar to nvarchar, binary to varbinary, datetime to datetime2, and time to sql_variant.

The same thing from an installation of SSMS that does NOT have SP1:

It seems whatever detects the data types in SP1 when SQL Server Import/Export Wizard is launched from within SSMS is not correctly identifying the data types, and in the case of nvarchar/varchar, results in garbage.

How to get value from a parent tag in XML source?

$
0
0
Hi All,

Can anyone please help me achieving the below requirement?

I have an XML source in the format.

<RecordBody>
<Case>
<CaseNumber>IC001</CaseNumber>
<Department>MSD</Department>
<IsProcessed>No</IsProcessed>
<Customer>
<Name>Name1</Name>
<Designation>Officer</Designation>
<EmailAddress>email01</EmailAddress>
<EmailAddressAt>@</EmailAddressAt>
<EmailAddressDomain>gmail</EmailAddressDomain>
</Customer>
<Services>
<ServiceList>
<Service>
<ServiceID>345345432423</ServiceID>
<ServiceType>I</ServiceType>
<Status>Active</Status>
<ServiceProvider>
<Type>ISO</Type>
<Name>Provider1</Name>
<Location>Location1</Location>
</ServiceProvider>
<ServiceJobs>
<ServiceJobsList>
<ServiceJob>
<JobID>98732</JobID>
<Date>20150201</Date>
<Hours>0630</Hours>
</ServiceJob>
<ServiceJob>
<JobID>98732</JobID>
<Date>20150201</Date>
<Hours>0630</Hours>
</ServiceJob>
 </ServiceJobsList>
 </ServiceJobs>
</Service>
<Service>
<ServiceID>35576655</ServiceID>
<ServiceType>E</ServiceType>
<Status>Active</Status>
<ServiceProvider>
<Type>ISO</Type>
<Name>Provider2</Name>
<Location>Location2</Location>
</ServiceProvider>
<ServiceJobs>
<ServiceJobsList>
<ServiceJob>
<JobID>9873</JobID>
<Date>20150301</Date>
<Hours>0830</Hours>
</ServiceJob>
<ServiceJob>
<JobID>9832</JobID>
<Date>20150205</Date>
<Hours>0630</Hours>
</ServiceJob>
</ServiceJobsList>
</ServiceJobs>
</Service>
</ServiceList>
</Services>
</Case>
</RecordBody>

Requirement:

I created tables for each entity in staging (Case, Customer, Services, ServiceProvider, Service Jobs) and want to pull the data to respective table with the Parent ID's (foreign key).

For eg. Need to pull the Case Number in to Customer table or vice versa when pulling the data to staging. Same with ServiceID for Service Providers and Service Jobs.

What is the best possible way to achieve it?

Any help is greatly appreciated.

Thanks & Regards,

Sri


Sql Server agent Jobs running

$
0
0

Hi All,

We have a scenario where we have to run 30 pkgs at same second. Each pkg is designed to fetch 60 lakh per day records and load it into Oracle/OLEDB destination. For which we have created 30 SQL agent jobs for each pkg running every 5 mins 24/7. We want to test running with option(1) these 30 jobs on single sql server instance and option(2) 10 jobs each on 3 sql server instances. Can someone please help me out to understand which would give me the better results.

a) The main advantage of multiple sql server instance scenario when compared with single sql server instance?
b) How much does the hardware configurations impact on the performance?

Thanks in Advance....

Unable to Run SSIS Package Through SQL Agent Job

$
0
0

Hi,

I recently upgraded SQL server 2008 R2 to SQL Server 2012. I also upgraded all the packages on the server. The package runs fine from BIDS. However when I try to run the package through the SQL Agent Job it fails with the error below:

Executed as user: A. Microsoft (R) SQL Server Execute Package Utility  Version 11.0.2100.60 for 64-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  11:43:04 PM  Could not load package "\FolderA\Package.dtsx" because of error 0xC00160AE.  Description: Connecting to the Integration Services service on the computer "S2345WE" failed with the following error: "Access is denied."    By default, only administrators have access to the Integration Services service. On Windows Vista and later, the process must be running with administrative privileges in order to connect to the Integration Services service. See the help topic for information on how to configure access to the service.  Source:  Started:  11:43:04 PM  Finished: 11:43:04 PM  Elapsed:  0.016 seconds.  The package could not be loaded.

Using Windows Authentication I am able to login to Integration services through SSMS. In the SQL Agent job I am using package store to execute the package. I have admin permission on the server. The integration services currently uses my credentials while running.I am not sure why I am getting this error.

Please advice..

Thanks,


EVA05

How to Run SSIS Package through SQL AGENT Job

$
0
0

I am good with SSIS, I created a package which will load the excel files to SQL SERVER  tables. To Implement this I wrote the C# code to extract the first sheet name only, and then package will load the data present the first sheet only. The package working fine when i run this through BIDS. But the same package throwing error at C# code step when i am running this package through SQL AGENT JOB. When i searched for this, i have seen many posts saying that, this is happening becoz of using Microsoft.interop.excel references inside the c# code. Please help me with this. Is there any alternate way in c# to extract the first sheet name(not based on ascending order) of an excel file with out using interop. Or

How to configure SQL AGENT job to run suceessfully when using interop inside the code.

My UAT server is of 64 bit. And i have to run this in the same server.

I tried creating  "Desktop" folder in "C:\Windows\System32\config\systemprofile\"

Error:Description: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object.     at ST_ecfa668f250a45e18c95639c9ffd64d4.csproj.ScriptMain.Main()    --- End of inner exception stack trace ---     at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)     at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)     at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)     at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)     at System.Type.InvokeMember(String name, BindingFlags invokeAttr, Binder binder, Object target, Object[] args, CultureInfo culture)     at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()  End Error  Error: 2015-02-27 11:24:00.23     Code: 0x00000001    Source: User Mail      Description: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)     at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)     at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)    at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity, SqlConnection owningObject)    at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, Int64 timerExpire, SqlConnection owningObject)     at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(String host, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, Int64 timerStart)     at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)     at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)     at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)    at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)     at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owning... The package execution fa...  The step failed

using System;
using System.Data;
using System.Diagnostics;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Data.OleDb;
using Microsoft.Office.Interop;
using System.Runtime.InteropServices;

namespace ST_ecfa668f250a45e18c95639c9ffd64d4.csproj
{
    [System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {

        #region VSTA generated code
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion


        public void Main()
        {   /*Passing the file path via User::File_Name Variable*/
            string FileName = Dts.Variables["User::File_Name"].Value.ToString();
            Microsoft.Office.Interop.Excel.Application xlApp = null;
            Microsoft.Office.Interop.Excel.Workbook excelBook = null;
            try
            {
                xlApp = new Microsoft.Office.Interop.Excel.Application();
                excelBook = xlApp.Workbooks.Open(FileName, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing, Type.Missing, Type.Missing,
                                                Type.Missing, Type.Missing);
                string[] excelSheets = new string[excelBook.Worksheets.Count];
                int i = 0;
                foreach (Microsoft.Office.Interop.Excel.Worksheet wSheet in excelBook.Worksheets)
                {
                    excelSheets[i] = wSheet.Name;
                    i++;
                }
                Dts.Variables["User::WorkSheetName"].Value = excelSheets[0] + "$";
            }
            catch (Exception ex)
            {
                excelBook.Close(false, FileName, null);
                Marshal.ReleaseComObject(excelBook);
                string error = ex.Message;
            }
            finally
            {
                excelBook.Close(false, FileName, null);
                Marshal.ReleaseComObject(excelBook);
            }

            Dts.TaskResult = (int)ScriptResults.Success;
        }
    }
}



DB2 SSIS Connection may not have configured properly or you may not have permissions

$
0
0

Hi Everyone,

  Can you please help me on this. I am trying to access DB2 using SSIS. I created a connection manager and when i click on test connection it succeded and also when i run command on DB2 i can run the command. But when i run the execute SQL task on SSIS, i see the below error. 

 "  Failed to acquire connection. Connection may not be configured correctly or you may not have right permissions on this conection. "

What might be the solution?



Export Fixed Width File

$
0
0

Hello I am exporting a fixed width text file and it works fine until the end users try importing it into there system. I have the format set to Ragged Right so it separates each record to its own row. When i try importing the file into an external system it would throw an error then i messed around with the file and removed one space at the end. That seemed to do the trick. Is there something that i am configuring incorrectly or need to do in SSIS to fix this? I am using SSIS 2012.

Do you think the using Ragged Right format is adding another column?


Thanks !!

SSIS 2012 package scheduling with SQL Server 2008 R2

$
0
0

Hi, We have a ssis 2012 package with parameters & would like to schedule this package in SQL Server 2008 R2 agent. Is there a way to pass parameters to the ssis package from SQL Server 2008 R2 agent?

Thanks in advance..........


Ione

Importing Excel data into SQL database on WIN 7 seems to hit a limit with regard to record count. What is limit

$
0
0

I have a user with a workstation running windows 7, importing data to a SQL 2008 R2 database.  She is able to import smaller files up to 1/2 million records, but gets a fatal error when attempting files with 750 K of records.  

Is there a hard limit for doing this that she is reaching?  Are there any bugs related to this that would affect larger imports from an excel spreadsheet?

Thanks

Jeff Bennett, St. Louis, MO

Balanced Data Distributor in SSIS 2012

$
0
0

I have been trying to get Balanced Data Distributor to install and show up in my toolbox in SSIS 2012 (version 11.0.31030.0 Q11REL) but so far I have not no success.  The versions dated 7/8/2012 fail to install but the 32 bit version dated 8/16/2012 installs without error.  BDD does not, however, ever show up in my toolbox even after refreshing.  The install instructions are incorrect because they say to go to "Choose Items..." and ultimately the "SSIS Data Flow Items" tab but that is applicable to versions prior to 2012.  

Any ideas on how to get BDD installed and working would be much appreciated.

Regards,

Mike

 

SSIS FASTLOAD

$
0
0

Hi Guys

I was using fast load option @ destination..! Some of Rows are going to error rows table without any error and its saying

error description as "No Status Available"

Then I Removed Fast load and loaded using table / View Now it is fine. but Very Much Slow @ destination

How to tune it....?to get fast load performance.

Thanks in Advance


ADKR

Viewing all 24688 articles
Browse latest View live




Latest Images