When configuring an excel destination in SSIS it only allows for a 32 bit 2007 version. If the destination is a higher version of excel and is 64 bit what is the work around.
↧
How to load a 64 bit Excel(2010) if SSIS only provides tools for 32 bit and only up to 2007
↧
Rebuild Index/Refresh Index and Stats improves ssis package performance
Hi,
My SSIS package is running very slow taking so much time to execute, One task is taking 2hr for inserting 100k records, i have disabled unused index still it is taking time.
I am rebuilding/Refreshing indexes and stats once in month if i try to execute on
daily basis will it improve my SSIS Package performance?
↧
↧
SSIS VB.NET - Object reference not set to an instance of an object
Hi,
I am working on SSIS package with vb.net script in it, I am facing an error saying object reference not set to an instance of an object. Here is the below code, Please check the code and help me with what exactly I am doing wrong here. Appreciate for your responses.
Imports SystemImports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports System.Data.SqlClient
Imports System.Windows.Forms
<Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute> _
<CLSCompliant(False)> _
Public Class ScriptMain
Inherits UserComponent
Dim connMgr As IDTSConnectionManager100
Dim sqlConn As SqlConnection
Dim sqlCmd As SqlCommand
Dim sqlParam As SqlParameter
Public Overrides Sub StagingCustomerInput_ProcessInputRow(ByVal Row As StagingCustomerInputBuffer)
'This IF structure is creating the CrossRef in the customer master.
'MessageBox.Show("Compare to see if parent: Inv = " + Row.cscidinv.ToString().Trim() + ", CscId = " + Row.cscid.Trim())
If (Row.cscidinv.Trim() <> Row.cscid.Trim()) Then
'MessageBox.Show("Not Equal")
'Find parent qw_csc_id with parent csc_id_inv which is the parent csc_id and update csc_id_inv with parent qw_csc_id
Row.newCscIdInv = GetParentQwCscId(Row.cscidinv.Trim())
Else
'MessageBox.Show("Equal")
'update the newCscIdInv with current qw_csc_id because I am the parent
Row.newCscIdInv = Row.qwcscid.Trim()
End If
If (Row.cscidpays.Trim() <> Row.cscid.Trim()) Then
'get parent qw csc_id
'update csc_inv with parent id because it is different
Row.newCscIdPays = GetParentQwCscId(Row.cscidpays.Trim())
Else
'update with current qw_csc_id because I am the parent
Row.newCscIdPays = Row.qwcscid.Trim()
End If
If (Row.csciddel.Trim() <> Row.cscid.Trim()) Then
'get parent qw csc_id
'update csc_inv with parent id because it is different
Row.newCscIdDel = GetParentQwCscId(Row.csciddel.Trim())
Else
'update with current qw_csc_id because I am the parent
Row.newCscIdDel = Row.qwcscid.Trim()
End If
If (Row.cscourbillto.Trim() <> Row.cscid.Trim()) Then
'get parent qw csc_id
'update csc_inv with parent id because it is different
Row.newCscOurBillTo = GetParentQwCscId(Row.cscourbillto.Trim())
Else
'update with current qw_csc_id because I am the parent
Row.newCscOurBillTo = Row.qwcscid.Trim()
End If
End Sub
Public Overrides Sub AcquireConnections(ByVal Transaction As Object)
connMgr = Me.Connections.StagingConnection
sqlConn = CType(connMgr.AcquireConnection(Nothing), SqlClient.SqlConnection)
End Sub
Public Overrides Sub PreExecute()
End Sub
'Get the qw_csc_id if not the parent and update
Public Function GetParentQwCscId(ByVal LegacyCscId As String) As String
Dim strParentQwCscId As String
Dim reader As SqlDataReader
sqlParam = New SqlParameter("@legacyCscId", SqlDbType.VarChar)
sqlCmd = New SqlCommand("select top 1 qw_csc_id from qwcustomermaster WHERE(csc_id = @legacyCscId)", sqlConn)
sqlCmd.Parameters.Add(sqlParam)
sqlCmd.Parameters("@legacyCscId").Value = LegacyCscId
' '--MessageBox.Show("Execute Reader")
reader = sqlCmd.ExecuteReader()
If (reader.Read() = True) Then
strParentQwCscId = reader.Item(0).ToString().Trim()
Else
strParentQwCscId = LegacyCscId.Trim()
End If
reader.Close()
Return strParentQwCscId
End Function
Public Overrides Sub ReleaseConnections()
'MessageBox.Show("Release Conn")
connMgr.ReleaseConnection(sqlConn)
End Sub
End Class
↧
SSIS Balanced Data Distributor not visible in Visual Studio 2013 (BIDS)
I've used the Balanced Data Distributor with Visual Studio 2010 and SQL Server 2012 successfully. I've now installed BIDS 2013 and I'm trying to get the Balanced Data Distributor to appear again in the SSIS Toolbox.
I've installed the x86 and x64 version and it won't appear. I've tried the new version which is supposed to have a fix for some SQL Server 2008 issues but I get the message:
The installation is not successful. Check the following prerequisites: 1. Either Integration Services or BIDS has to be installed. 2. The version of these components has to be either SQL Server 2008 SP2 (or future SPs) or SQL Server 2008 R2 (or future SPs).
That seems this version only works for SQL Server 2008.
Any help would be appreciated
↧
Load multiple files from Using For each loop container
Hi All,
I have a requirement to load multiple flat files in target table .
I have created the package which used to load files into target table using For each loop container.
But now requirement has been changed now I have to take only those files from table where status="Success" and max JobId. By the query I am to get those records which need to load into table.
Below query I am using to get the files which need to load.
select [JobLogKey],[SrcNm],[DestNm]
FROM [ConfigRep].[dbo].[JobLog]
Where [JobId]=
(Select Max(cast([JobId] as Int)) Jobid
FROM [ConfigRep].[dbo].[JobLog]
Where [JobStat]='Success')
Output:-
268H:\Data Platform\Source File\Client2\Loc\HGSSpec\Location.txtLocation.txt
269H:\Data Platform\Source File\Client1\Loc\HGSSpec\Location.txtLocation.txt
I have to load using above 2 files which are under SrcNm.
I have created one variable called FileToLoad as Object and mapping to result set of above query.
I have create JobId,SrcNm and DestNm variable to catch the record at every loop.
I have created 2 For each Loop container
Below screen shot of outer Foreach loop.
Till here Its working fine.Inner for each loop container not executing any task under that.
Help me to get it done
Thanks in Advance .
Thankx & regards, Vipin jha MCP
↧
↧
SSIS 2012 Script Tasks failed to run when it run through Task Schedular
Hi,
We are getting the error message "The task has failed to load. The contact information for this task is ""." when we run the SSIS package through Task Schedulerthough the package is executing successfully when we debug it in SQL Server Data Tools 2012.
When we try run it through batch File its executed successfully.
Thanks in Advance
↧
Changing the FilePath of a FlatFile Connection in a Script Task
I need to grab the name of a file in a folder and use it as an input source for data. This file is generated with a time stamp at the end so it is close to never the same. I am not sure how to cannect to my Flat File Connection so I can manipulate the file path to always point at this file. To grab the file I am using: string[] name = System.IO.Directory.GetFiles(@"file path"); I want to create the path from this and set it in the File connection. There will only ever be one file in the folder and the path will always be the same.
Thanks in advance
↧
Choosing a File Format for destination
Hi,
Is there a recommended file format (csv, xml, txt) when choosing a file destination for SSIS? Does a file format impact the performance in terms of loading? Let's say i have chosen to use a .csv as my file destination (this has 15million rows and 50 columns with 2 bigint and the rest binary(32)) and later on, i would need to reload them back to table using SSIS. Is using csv faster than e.g. xml when reloading? Does it have performance impact at all?
What's your experience?
↧
Call SAP RFC program from SSIS Package
How do we connect and get data from SAP database using SSIS 2012/2014.
Currently we are extracting data using WebService, and BAPI function call for RFC, but it seems not so much reliable, due to timeout and others.
Can somebody help us ion providing the link for download of .NET Framework Data Provider for mySAP Business Suite
and some sample to invoke and get the data using SSIS.
↧
↧
SSIS: Data Provider For SAP: ADOSource Connection
Can somebody help us, why we are not able to see the table list in the drop-down,
1. On change of [Data Access Mode] from [SQLCommand] to [Table].
2. When we try to find record cound from table by using "Select Count(*) Cnt" or when we try to use "group By", "Distinct" clause. it does not work.
↧
Querying the SSIS catalog info - SQL Server 2012
Hi,
I've deployed a SSIS project (having some packages) on SSIS catalog and I've created a job to invoke the related packages. I'd like to query the SSIS catalog to get the info about the execution of the job to understand if it is completed with success.
I've seen the tables internal.executions and internal.operations. Which of these two tables contains the more useful info about the execution job (f.e. the job name, the execution status)?
Many thanks
↧
Help needed translating SSIS script from VB.Net to C#
Can anyone help to translate this code to C#? Thanks.
Imports System
Imports System.IO
Imports System.Text
Imports System.Windows.Forms
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
' Get the unmanaged connection object, from the connection manager called "HTTP Connection Manager"
Dim nativeObject As Object = Dts.Connections("HTTP Connection Manager").AcquireConnection(Nothing)
' Create a new HTTP client connection
Dim connection As New HttpClientConnection(nativeObject)
' Download the file #1
' Save the file from the connection manager to the local path specified
Dim filename As String = "C:\Temp\Sample.txt"
connection.DownloadFile(filename, True)
' Confirm file is there
If File.Exists(filename) Then
MessageBox.Show(String.Format("File {0} has been downloaded.", filename))
End If
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
Imports System
Imports System.IO
Imports System.Text
Imports System.Windows.Forms
Imports Microsoft.SqlServer.Dts.Runtime
Public Class ScriptMain
Public Sub Main()
' Get the unmanaged connection object, from the connection manager called "HTTP Connection Manager"
Dim nativeObject As Object = Dts.Connections("HTTP Connection Manager").AcquireConnection(Nothing)
' Create a new HTTP client connection
Dim connection As New HttpClientConnection(nativeObject)
' Download the file #1
' Save the file from the connection manager to the local path specified
Dim filename As String = "C:\Temp\Sample.txt"
connection.DownloadFile(filename, True)
' Confirm file is there
If File.Exists(filename) Then
MessageBox.Show(String.Format("File {0} has been downloaded.", filename))
End If
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
↧
Scientific Notation while taking data from EXCEL into SSIS
Hi ,
I have one problem regarding format of mixed data columns in excel. I am using this query to retieve data from my excel spreadsheet
Select
* fromOPENROWSET('Microsoft.Jet.OLEDB.4.0','Excel 8.0;IMEX=1;IHDR=yes;Database=E:\test.xls;HDR=YES'
,'SELECT * FROM [original$]'
)
to retrieve data from excel sheet.
I am able to retreive all data (numaric and text) , but for few cells I am getting wrong values.
like this
15106749310-> 1.51067e+010
70400672657-> 7.04007e+010
48110753143-> 4.81108e+010
registry setting are “text and 0″. I am also using IMEX=1 in connection string.
I have numeric and text data both in same column,
eg:
70400503549
70400672657
KF1080420908
KF1260370908
KF1260880908
KF1260960908
Could you please help me to get the correct value from excel .
thanks in advance ,
↧
↧
Conditional Split Column < Today
Hey guys, I'm trying to write a conditional split where I want to bring in only records where the date is less than today, but my problem is that I can't simply do this Column < GetDate() because if something comes in today, it takes the time into account and it will bring that record for today. You can do this in SQL, but I'm not sure how to do that in SSIS
SELECTCAST(CONVERT(char(8),GetDate(),112)ASdatetime)
↧
Import multiple text file into individual excel file dynamically
Hi,
I would like to load multiple flat file into separate(individual) excel file dynamically. I have used for each loop container and assigned a variable to it (filepath and file name). Then in dataflow I have a flatfile source and excel destination. I created other two variables (destination and destination path) When I run my package , I get all the records in one single file. But I want to have individual excel file for each text file.
Please save me.
↧
Drop ISC Project, Environment in SQL Job is invalid
So, I have been successful in CREATING ssis projects within VS and deploying them to the Integration Services Catalog (ISC). I can then create a job using a SQL script and hook up the step to the environment variable I want. It runs! YEAH!
Then it is time to correct an error in one package in a project of 4 packages. Hmmm, looks like I have to drop the project first and then create it again. (Don't like that)
utilizing-powershell-with-2012-ssis-deploys
Then I have to add the existing environment variable to the newly created project. This results in a new environment id.
SELECT ER.reference_id AS ReferenceId , E.name AS EnvironmentName , F.name AS FolderName , P.name AS ProjectName FROM SSISDB.catalog.environments AS E INNER JOIN SSISDB.catalog.folders AS F ON F.folder_id = E.folder_id INNER JOIN SSISDB.catalog.projects AS P ON P.folder_id = F.folder_id INNER JOIN SSISDB.catalog.environment_references AS ER ON ER.project_id = P.project_id where E.name = N'aEnvVar' and F.name = N'aProjFldr' and P.name = N'aProject'
Well, that causes an error when I run the job again.
Is there a way to replace just one package in an existing project?
If not, I am going to have to gen T-SQL scripts to repair the job steps.
Thanks for the help, as always!
↧
How to download files from a web page before loading into SQL Server 2014?
How to download files from a webpage before loading into SQL Server tables? I have the following URL and under the Downloads & Resources section, I have different file formats.
By doing hover on the download tab for each file type, I see that there is a link that is associated with it just like the following:
For CSV - https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD
For XML - https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.xml?accessType=DOWNLOAD
The above is just an example for your reference/understanding.
In the sample data from the internal website I have, I need to do a similar operation. The only difference would be that I would be having multiple XLS files with a description for each.
Example:
Sales Q1 - <xls download tab>
Sales Q2 - <xls download tab>
Sales Q3 - <xls download tab>
Sales Q4 - <xls download tab>
<li><sub>Sales for Calendar Year 2015--All Countries </sub><a href="/Data/Downloads/Documents/Sales/Sales_Quarter1.xlsx"><sub>[XLS]</sub></a><sub> , <a href="/Data/Downloads/Documents/Sales/Sales_Quarter1.pdf"><sub>[PDF]</sub></a><sub></sub></sub></li>
I need to download the file based on the month/quarter every time.
Any help with this is much appreciated.
Thanks,
Bangaaram
Known is a DROP, Unknown is an OCEAN.
↧
↧
Dynamically create sql table using SSIS
Hi,
Please I have a requirement to dynamically create a sql table within my SSIS package.
I have some flat files to load into sql server with different file names and structure. I would like the package to create the table before loading the file. The table name should be taken from the flat file name and after each load, it should be archived to a folder.
Could you help! please.. I am new to SSIS.
Thanks
Chuk
↧
Editing container in SSIS get me crazy
I am literally getting crazy with SSIS visualization. I am working with the containers but every time they go at the bottom of the page, when I try to enlarge them they become smaller, if I try to put the container at the top of the editor it takes minutes....it's
only me? Am I doing something wrong?
↧
Recording E-mail sent by SSIS Package to table
We have multiple SSIS job that send e-mail but we want to log each e-mail that goes out to end users in table with data such as From, To , Subject and sent time possibly some of body. Is there a way i can achieve this after executing send mail task ? i need actual data sent not whats set in variable already.
↧