Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

SSIS package running from SQL Server Agent

$
0
0

I have an SSIS package running from the SQL Server Agent which creates several CSV files in for each loop. It has been running with no problems for over a year but the problem seems to be a limit on how large the CSV file can be. As time goes by the file because larger and does not up date the CSV file. It does update the smaller CSV file but not the large ones.  This package runs correctly from my computer in BIDS.  I don't know really what the problems is or how I can fix it.

Any help will be greatly appreciated.


SQL server Source Datetime Column conversation to Teradata.Destiation using SSIS.......?

$
0
0

I have Datetime Column Say 'Sales Date' which is in format '1900-01-01 00.00.000' I want to insert this in Teradata but the task is getting failed...i had kept Target table column datatype as Timestamp3 in Teradata..!

Please help me out


Shiv


Excel Loading Scenario - Which Component ?

$
0
0

Hi, i have a xlsx workbook with 20 worksheets, each worksheet has a number of ranges in it (a1 - d4, a6 - e20 and so on).

I would like to redev it with the most suitable excel component (the previous developer used Excel + / CozyRoc but i am not going to put this on a prod server so am curious as to the alternative).

A nice to have would be a enable a loop to utilise spreadsheet metadata which is loaded to a set of tables, and just have 1 DFT to process all 20 worksheets and constituent ranges in each w/sheet. 

Can anyone suggest a suitable component for this scenario ? I would like to avoid parsing the file using c# if at all possible.

Thanks in advance


SSIS Server Failover

$
0
0

Hi All,

Whether Microsoft CDC for SQL Server Supports High Availability on Database Mirroring

Whether Attunity CDC for Oracle supports High Availability on Oracle RAc.

Whether SSIS supports High Availability 

Can we switch to SQL Jobs in Secondary physical server from SQL Jobs in primary physical server and continue with SQL Jobsprocessing when the primary physical server crash.

Thanks in Advance...

SSIS package "The Microsoft Jet database engine cannot open the file"

$
0
0

Dear all,

I am trying to import data from an excel spreadsheet which is save on application sharing server. When I try to save same file locally (On my desktop) my package has no problem to read it.  Problem comes when I change the file path to the share location I am getting below error message,

"Version 11.0.2100.60 for 32-bit  Copyright (C) Microsoft Corporation. All rights reserved.    Started:  13:26:46  Error: 2014-12-05 13:26:50.30     Code: 0xC0202009     Source: Incremental MasterStyle DL Connection manager "Excel Connection Manager"     Description: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.  An OLE DB record is available.  Source: "Microsoft JET Database Engine"  Hresult: 0x80004005  Description: "The Microsoft Jet database engine cannot open the file ''.  It is already opened exclusively by another user, or you need permission to view its data.".  End Error  Error: 2014-12-05 13:26:50.30     Code: 0xC020801C     Source: Data Flow Task 1 Excel Source [36]     Description: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER.  The AcquireConnection method call to the connection manager "Excel Connection Manager" failed with error code 0xC0202009.  There may be error messages posted before this with more information on why the AcquireConnection method call failed.  End Error  Error: 2014-12-05 13:26:50.30    Code: 0xC0047017     Source: Data Flow Task 1 SSIS.Pipeline     Description: Excel Source failed validation and returned error code 0xC020801C.  End Error  Error: 2014-12-05 13:26:50.30     Code: 0xC004700C    Source: Data Flow Task 1 SSIS.Pipeline     Description: One or more component failed validation.  End Error  Error: 2014-12-05 13:26:50.30     Code: 0xC0024107     Source: Data Flow Task 1     Description: There were errors during task validation.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  13:26:46  Finished: 13:26:50  Elapsed:  3.401 seconds.  The package execution failed.  The step failed."

I am trying to run this package as an sql server agent job and  server is installed on my Machine. I have created proxy account with my own credential to execute it. I have full excess to the spread sheet and can open it in an excel program. Please suggest what could be the reason why sql job is not able to access this spreadsheet. I can think of below reasons

  1. File is password protected and I am not using its password to unlock it?
  2. My domain account is a part of It Admin group with high previllages
  3. when I installed my SQL server I installed it as a standalone instance of SQL server?

please advise.

regards

JKH


SSIS / Catalog / PowerShell

$
0
0

Hi,

I have the following PowerShell script which :

$PsFileDir="C:\MyFolder\01_ETL\PS\PkgExec"
$SlnDir="C:\MyFolder\TestAutomatPowerShell\MyFolder\MyProj.dtproj"
$PkgDir="C:\MyFolder\TestAutomatPowerShell\MyFolder"
$TargetSrv="MySQLServer"
$DevEnvPath="\Microsoft Visual Studio 11.0\Common7\IDE\devenv.exe"

#Build the project
Write-Host "Building the project ..."&"${env:ProgramFiles(x86)}${DevEnvPath}" ${SlnDir} /build

Start-Sleep -s 180

# Load the IntegrationServices Assembly
$loadStatus = [Reflection.Assembly]::Load("Microsoft"+".SqlServer.Management.IntegrationServices" +", Version=11.0.0.0, Culture=neutral" +", PublicKeyToken=89845dcd8080cc91")

# Store the IntegrationServices Assembly namespace to avoid typing it every time
$ISNamespace = "Microsoft.SqlServer.Management.IntegrationServices"

Write-Host "Connecting to server ..."

#Connect to SQL server which has SSIS Package
$sqlConnectionString = "Data Source=$TargetSrv;Initial Catalog=master;Integrated Security=SSPI;Trusted_Connection=True;"
$sqlConnection = New-Object System.Data.SqlClient.SqlConnection $sqlConnectionString

# Create the Integration Services object
$ssis = New-Object $ISNamespace".IntegrationServices" $con

#Connect to Integration Service
$ssisServer = New-Object Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices $sqlConnection

$ssisCatalog = $ssisServer.Catalogs["SSISDB"]

#Check if Folder is already present, if not create one
Write-Host "Creating MyProj project folder ..."
$ssisFolderName = "MyProj"
$ssisFolder = $ssisCatalog.Folders.Item($ssisFolderName)

# Read the project file, and deploy it to the folder
Write-Host "Deploying MyProj project ..."
[byte[]] $projectFile = [System.IO.File]::ReadAllBytes("C:\MyFolder\TestAutomatPowerShell\MyProj\bin\Development\MyProj.ispac")
$ssisFolder.DeployProject("MyProj", $projectFile)

# Creating an environment
Write-Host "Creating MyProj environment ..."
$environment = New-Object $ISNamespace".EnvironmentInfo" ($ssisFolder, "MyProj", "Projet New MyProj")
$environment.Create()

Write-Host "Adding server variables ..."
	# Adding variable to our environment
	# Constructor args: variable name, type, default value, sensitivity, description
	$environment.Variables.Add("L_7ZIP_PATH", [System.TypeCode]::String, "C:\MyFolder\01_ETL\Tools", $false, "")
	$environment.Variables.Add("L_DWHLOADER_PATH", [System.TypeCode]::String, "C:\Program Files\Microsoft SQL Server Parallel Data Warehouse\100", $false, "")
	$environment.Variables.Add("L_PDW_PWD", [System.TypeCode]::String, "MyPWD", $false, "")
	$environment.Variables.Add("L_PDW_USER", [System.TypeCode]::String, "MyUSR", $false, "")
	$environment.Variables.Add("L_SRC_FILES_PATH", [System.TypeCode]::String, "C:\MyFolder\01_ETL", $false, "")
	$environment.Alter()



#xxxx
Write-Host "Adding environment reference to project ..."

	# making project refer to this environment
	$project = $ssisFolder.Projects["MyProj"]
	$project.References.Add("MyProj", "MyProj")
	$project.Alter()


Write-Host "Adding reference to variables ..."

	$project.Parameters["L_7ZIP_PATH"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, "L_7ZIP_PATH")
	$project.Parameters["L_DWHLOADER_PATH"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, "L_DWHLOADER_PATH")
	$project.Parameters["L_PDW_PWD"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, "L_PDW_PWD")
	$project.Parameters["L_PDW_USER"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, "L_PDW_USER")
	$project.Parameters["L_SRC_FILES_PATH"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, "L_SRC_FILES_PATH")


$TABLENAME=(get-childitem $PkgDir | ?{!$_.PSIsContainer -and $_.name -match "\\*.dtsx"})

$TotStartTime=Get-Date
Write-output "Total Start Time : $TotStartTime"

foreach ($pkg in $TABLENAME)
{
                $StartTime=Get-Date
				Write-output "Executing $pkg ..."

				# Run the package
				Write-Host "Running package ..."

				# When executing, we need to specify two parameters
				# 1 arg is a bool representing whether we want to run
				# 32bit runtime on 64 bit server
				# 2 arg is a reference to an environment if this package depends on it

				$pkgToExec = $project.Packages[$pkg.Name]
				Write-output "Package Name : " $pkgToExec

				# retrieving environment reference
				$environmentReference = $project.References.Item("MyProj", "MyProj")
				$environmentReference.Refresh()

				$executionId = $pkgToExec.Execute("false", $environmentReference)

				$EndTime=Get-Date
                $ElapsedTime = ($EndTime - $StartTime)

                Write-output "${pkg},${ElapsedTime},${StartTime}, ${EndTime}" >>$PsFileDir\Logs\$pkg.log

}
$TotEndTime=Get-Date
$TotElapsedTime = ($TotEndTime - $TotStartTime)
Write-output "Total End Time : TotEndTime"
Write-output "TotalElapsedTime : $TotElapsedTime"

  1. Build a SSIS project
  2. Deploy the project to SSIS Catalog
  3. Run each of the packages in the solution

I don't have errors when executing the PowerShell.

But none of my packages succeed.

Any expressions using Project variables are not evaluated right because the value from the Project variable is not found. I have blanks anywhere I use Project variables.

Any idea ?

Thanks for your help.

Regards,


Bertrandr

Distribute file to multiple emails using ssis

$
0
0
I want to send an email to multiple email recipients.  The list of all the email recipients are present in a Flat File.  My SSIS package should get the list of email recipients from the Flat File, and send an email to all of them.  All the recipients would receive the same message.  How do I do this.

how to make my SSIS package execute successfully, even when there is no input feed file which is actually expected.

$
0
0

Hello,

Currently I am running a SSIS package scheduled daily at 7 A.M. It expects two feed files from two different folders. 

The first step in my package will rename the input files in those folders to  names which the package can understand. I have created two variables in my package to read the files with those file names. I used these variables in connection managers.

If any of these folders doesn't have input files when the package runs, the package will fail. Can someone please let me know how to make the package the run successfully even there is no input feed?

Any help is highly appreciated.

Thank you.


Package to backup all the available databases in the instance

$
0
0

Hi 

In an interview i was asked to create package which backup's all the DB's in an instance.

In first case using BIDS 2008R2.

In second case, How would you simplify the package using BIDS 2012 which has DB tasks in built?

Can someone help me to get the out structure of the package?

SSIS : Read Rows from an Object variable in SSIS Script Task which is looped many times.

$
0
0

Hello All,

Here is what I am trying to do...

1. I am having two rows, one column in an Object Variable. (vLoopCountObj).

2. I am having 30 Rows, 2 Columns in my second Object  Variable (vTableRowsObj)

3. I have a FOR EACH LOOP which will run for number of rows in vLoopCountObj i.e 2 times here.

4. I have a Script Task inside the FOR EACH LOOP to display all the rows of vTableRowsObj.

5. When I execute, 30 Rows gets displayed only once. It will not display when the loop goes for second iteration. Please help me to display for second time.

6. I have used below code to display the rows whithin the script task which is in SSIS dataflow task.

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports System.Xml
Imports System.Collections
Imports System.Data.OleDb

Public Class ScriptMain

  Public Sub Main()

    Dim oleDA As New OleDbDataAdapter
    Dim dt As New DataTable
    Dim col As DataColumn
    Dim row As DataRow
    Dim sMsg As String


    oleDA.Fill(dt, Dts.Variables("vTableRowsObj").Value)
    MsgBox("Number of Rows " + dt.Rows.Count.ToString())
    For Each row In dt.Rows
      For Each col In dt.Columns
        sMsg = sMsg & col.ColumnName & ": " & row(col.Ordinal).ToString & vbCrLf
      Next
      MsgBox(sMsg)
      sMsg = ""
    Next

    Dts.TaskResult = Dts.Results.Success

  End Sub

End Class

Error Message: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020

$
0
0

I have a data flow in SSIS (2008 R2) that has an OLE DB Source --> Script Component (Transformation via Visual Basic) --> OLE DB Destination. The query, when ran in SSMS, brings in a little over 2.6 million records from the DB source. (just to give you an idea).

At about 17,600 records the package fails with the follow error message:

Error Message: The attempt to add a row to the Data Flow task buffer failed with error code 0xC0047020

Data Flow Task - Concatenate Service Request Notes  Error Code:  -1073450952  Error Message:  SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on component "OLE DB Source - INT_FCT_SERVICE_NOTE Internal" (1) returned error code 0xC02020C4.  

Data Flow Task - Concatenate Service Request Notes  Error Code:  -1073450910  Error Message:  System.Runtime.InteropServices.COMException (0x80004005): Error HRESULT E_FAIL has been returned from a call to a COM component.

   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.HandleUserException(Exception e)

   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.ProcessInput(Int32 inputID, PipelineBuffer buffer)

   at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostProcessInput(IDTSManagedComponentWrapper100 wrapper, Int32 inputID, IDTSBuffer100 pDTSBuffer, IntPtr bufferWirePacket)

Now, the current Default Buffer Max Rows = 1000 and Default Buffer Size = 10485760


Could anyone be of assistance?



impact of a change to a custom tool box item

$
0
0

Hi we run std 2012.  Just to be safe I'm posting this question.

We r contemplating use of a custom toolbox item for a very common update to sql.  A gac'd dll would likely run a proc in this component and i'm assuming that is not unusual for custom toolbox items.  I'm also assuming that passing one or more ssis var params to a custom toolbox item is very doable.

Our alternatives r either a specialized sub pkg or same c# script component where/whenever this needs to run.

If the toolbox item changes does every pkg that uses it need to be visited/validated/rebuilt so that the underlying dtsx can somehow be updated?

I guess there r 3 scenarios involved:

1) passed param list changes but new params are already "known" to all of the pkgs involved anyway.

2) passed param list changes (eg new vars, different data types on existing vars) and there are new vars involved.  Clearly this would require a change to the pkg vars but does any additional step need to be taken in order for ssis to understand there is new plumbing between the pkg and the custom component?  Or is that more of a dynamic thing at run time?  

3) change to dll is otherwise transparent to the plumbing (eg dll changes but still requires same param list) between ssis and the component. 

Import excel file error

$
0
0

Hello - I am having issues running package to import a excel file into a sql table. It works fine in SSDT (BIDS) but when i deploy it to the server I get errors the below errors:

I have installed the driver from here http://www.microsoft.com/en-us/download/details.aspx?id=13255 and also set my package to run in 32bit mode with delay validation. This doesn't still seem to work.

I have a Microsoft server 2008 R2 64 bit and I am have Sql Server 2012 SP2 on this server.

Thanks!

Create a connection manager for a set of already created SSIS packages

$
0
0

Hi folks, 

I have a solution with 300 SSIS packages. Half of which contains a send mail task with a defined SMTP server connection manager. The connection manager was hard coded for all of these packages. Now the SMTP is about to change. What i did is that I added a hard coded property with the new server name. This property overwrote the old value and I was about to celebrate. Now, The packages that had a connection manager defined, got the new value and they are working fine. The packages that never had a SMTP connection manager, are now getting an error. Simply because the config file is referring to the connection manager that doesn't exist in this case. 

What I am thinking now is to add a SMTP connection manager for all packages and refer to this connection manager from my property file (or config file). This the best that I can think of so far. Not sure if this is doable though. 

I thought about the idea of adding a variable with the connection string for all packages. But don't know how to set the connection string property to take value from this variable automatically. It is impossible to do this manually given the number of packages and environments. 

Thanks, 

Import json file to sql server

$
0
0

I have 1000 records in text file. My requirement is to insert data into sql server. For that, if i can convert to xml or push to script component also fine.

Text file consists in this format...

[{"ID":1,:"Name":"test1","State":"AP"},{"ID":2,:"Name":"test2","State":"KN"}{"ID":3,:"Name":"test3","State":null}]

I'm new to coding. This is the code I wrote in scriptcomponent.

#region Class
[Micorosoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryAttribute]
public class ScriptMain: UserComponent
{
public override void CreateNewOutputRows()

{

string jsonstring = System.IO.File.ReadAllText(@"c:\t.txt");

WorkGroup[] outPutMetrics = new GetWebService(wUrl);

foreach (var metric in ouPutMetrics)
{
	TDBuffer.AddRow();
	TDBuffer.ID = metric.id;
	TDBuffer.Name = metric.nm;
}
}
}

private WorkGroup[] GetWebService(string wUrl)
{
WorkGroup [] jsonResp = null;
string jsonstring = System.IO.File.ReadAllText(@"c:\t.txt");
JavaScriptSerializer sr = new JavaScriptSerializer();
jsonResp = sr.Deserialize<WorkGroup[]>(jsongstring.trim('"');
return jsonResp;
}
#endregion
#region JSON Class

class 	WorkGroup
{
public string id {get; set;}
public string nm {get; set;}
}

#endregion

I'm getting these errors when I build solution -

Expected class, delegate, enum, interface, or struct
Identifier expected
Expected class, delegate, enum, interface, or struct
Identifier expected
Type or namespace definition, or end-of-file expected
The type or namespace name WorkGroup could not be found (are you missing a using directive or an assembly reference")
GetWebServiceResult does not exist in the current context
The name wUrl does not exist in the current context

Need guidance on switching career from dot-net to MSBI

$
0
0

I want to switch my career from dotnet to MSBI.

I am having more than one year of experience in dotnet technologies.I have good analytic skills and good programming skills.I am very interested in SQL Server.I want to learn MSBI,but i don't have any idea how to start?, which one need to start first among SSIS, SSRS, SSAS. Which is best in these three in MSBI?

Is my decision is good in career selection?

will i get the better opportunities MSBI with dotnet?

Please suggest me on this...

Thanking you......

Parameter Error

$
0
0

I have requirement to put following query in a SSIS OLE DB Source. But I'm getting following error:

"Parameters cannot be extracted from the SQL command. The provider might not help to parse parameter information from the command. "

I'm wondering if I can rewrite and pass the parameter outside of the subquery. 

Here is the query.

SELECT a.Store
,a.Sales
FROM (

SELECT 
       L.Store As Store, 
       SUM(Sales) AS Sales
FROM Sales S
       INNER JOIN Weeks W ON S.WeekDate = W.MaxFullDate
       INNER JOIN Locations L ON S.StoreID = L.Store          
WHERE S.WeekDate BETWEEN ? AND ?
GROUP BY L.Store

Union All

SELECT 
      L.Region As Store,
      SUM(Sales) As Sales
FROM Sales S
       INNER JOIN Weeks W ON S.WeekDate = W.MaxFullDate
       INNER JOIN Locations L ON S.StoreID = L.Store         
WHERE S.WeekDate BETWEEN ? AND ?
GROUP BY  L.Region
 
Union All

SELECT 
        L.Division + '0' As Store, 
       Sum(Sales)AS Sales
FROM SalesActuals_Stage S
       INNER JOIN  Weeks W ON S.WeekDate = W.MaxFullDate
       INNER JOIN  Locations L ON S.StoreID = L.Store         
WHERE S.WeekDate BETWEEN ? AND ?
GROUP BY L.Division

) a
ORDER BY 1
,2

Thanks.


ZK

SSIS Job execution hangs without errors when executed from IS catalogs or Sql Server Agent

$
0
0

Hi,

I have a small package with 2 loops to read the data from excel and load that into sql server table. One loop for reading multiple files and another inner loop for reading the multiple sheets in each file. When I execute the package from my local machine, it works without any issues but when I execute the same package from Intergration services catalogs or sql agent it hangs after processing 3 or 4 excel files without any errors or warnings. I am able to load 10-15 files without any issues from my local machine using BIDS in VS 2012. I am not sure why it gives issues when running from SQL Server 2012 catalogs or sql server agent. I installed SQL Server SP2 patch also ins erver but this didn't fixed the issue. I checked for locks also and I don't see any. If I choose the logging option as none then it processes more files around 8-9 but it still hangs at the 10th file processing. I don't see a definite solution from the forums. Any help is greatly appreciated.

Regards,

Vijay



How to upgrade ssis 2012 packages into ssis 2014

$
0
0
we upgraded ssis 2012 packages into ssis 2014 but when i execute the packages its showing as validation mode .

Expression -puzzling(help needed)

$
0
0

Hi All,

Is there a possibility that my expression fails on the first run trying to load into a column in the table having truncation error,but ultimately runs at the second time(having that column alone empty while populating other columns?).

Help needed...ANy help on this topic wil be much helpful!!


Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>