Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Need to send data to excel spreadsheet

March 16, 2014, 2:01 am
$
0
0

Hi All

I'm new in SSIS and need your help in one scenario.

I created 1 flat file connection manager  that link to 1 CSV file. I also created 1 Flat file Source and use that flat file connection.

After that I created 1 unpivot transform and link it with that flat file source and then  configure unpivot transformation and its working fine. Now I want the unpivot result in excel spread sheet and I don't know how to assign result in excel spread sheet.

I tried to make excel connection  manager and give excel file name and then connect it with unpivot but no luck. I think I'm missing something. Please help me out. Thanks

↧
Search

SSIS - merge table and split transformation function

March 11, 2014, 9:08 am
$
0
0

i would like to ask if i have to update the existing record or insert new record from SSIS.

here is my step in data flow control:

1. create a source table A and table B

2. merge two tables by left join A to B with ID,NAME,TEL column

3.add split transformation control and then set condition as belows:

   3a) output name: insert row   ; condition: isnull(ID)

   3b) output name: update row ; condition !isnull(ID)

4. route to different OLD DB destination  (one is update, another is insert row)

------------------------------------------------------------------------------

but i don't understand the step 2 and step 3. when i left join table A and B by ID,NAME,TEL and then check isnull(ID)/!isnull(ID) in step 4 from the result set of merge join.

for step 3, is it only check if the result set of merge table's "ID" column is null then route to the step to Insert row. not check the join key from step 2?

copy from http://itssmee.files.wordpress.com/2010/10/solution.jpg

 


↧
↧

Upgrade to SQL Server 2012 SP1 Developer (windows 8.1 professional)SSDT NOW hangs opening a Tabular Model.bim

March 16, 2014, 2:36 pm
$
0
0

Hi all,

Any leads would be gratefully appreciated. :)

1) I am able to CREATE a new model ( 1 in 4 times) 3 of the 4 freeze / hang once the model.bim file is open. The other 25 % of the time I am able to link to the relational tables, create the model and deploy. 

HOWEVER

After saving the '25%' example, leaving SSDT and then coming back into SSDT, THIS model will also freeze/hang upon trying to open it(to do further work). Each time, I have to use the task manager to kill the process.

Symptoms:

Memory is OK

CPU usage jumps to 100% (almost as if the process is 'thrashing'), SSDT is actually clocking ( I have tried to let it finish however it never does).

For every project started, Connectivity to the database is fine and I am using the SP1 setting as prescribed in the wizard.

Thanks a million in advance,

regards Steve Simon SQL Server MVP BI

↧

SSIS to get the particular file from multiple files and copy to different folder--Need Help

March 16, 2014, 1:45 pm
$
0
0

Friends,

I need your help in writing an ssis package. I’ve folder and have so many different files in there. I need a a particular files which I want to copy to different folder and that file has a specific file name with the previous date in the name. Package should take that file and copy into different folder. We get the files everyday and I need to copy it to a different folder. Could you please help ..I am not really an ssis guy and need your help . I would really appreciate if you someone can provide step by step since I am not really an ssis expert

File folder :-   D:\SSIS_Test

Need to copy the file to :- D:\SSIS_Test2

File format : act.trect_bcdw_01.20140314.output.dat

↧

.csv flat file source problems when moving to ole db destination question

March 13, 2014, 10:37 am
$
0
0

Does anyone know of a utility or process in SSIS Data Tools using Visual Studio 2012 when working with .csv flat file data source where I can filter out rows based on columns that have invalid characters? So, basically I want to skip rows that  would otherwise make the load stop or bomb and then send the physical data that is incorrect to a file location  that the source developer can troubleshoot and correct?

Another way to say this is;

I’m loading data into a SQL Server table from multiple .csv flat file sources and the problem is that there are handfuls of rows where the delimiter is getting thrown off due to strings that have invalid characters. Can I skip those, throw them into another .txt file and continue only loading the rows that are correct?

I know with a fuzzy look-up I could work with tables but not sure how to do this using a flat file .csv source? Any ideas? 

Thank you

↧
↧

Instruct DataFlow task to Redirect bad input rows to Log file?

March 26, 2009, 9:48 am
$
0
0
Hi all,

I am new to SSIS 2005 packages but have written many in DTS 2000.

I am importing a CSV flat file into a SQL server table with SSIS 2005. How do I tell the Dataflow task to redirect rows that don't have the proper number of columns into a Text log file so I can inspect the bad rows later?

I have pointed the Red Error arrow to a text file and defined all the Columns to redirect the error to the text file. I have set the Max error count to 200. However, the entire package fails at the first bad row in the PrimeOutput() method, indicating that Column 13(the last one in the Row) is missing in a particular row. I have tried setting Ignore the error, but that doesn't work either.

I have created a DTS 2000 package that works on the same CSV input and the same SQL table destination. It seems to have no problem with my instructions.

I must be missing something. Can someone tell me what it might be? I really would like to use SQL 2005 or even SQL 2008 because of the new features available, but I am stuck on what I thought would be a very easy first step (because it is very easy in DTS 2000).

Any help would be appreciated. Thanks...
↧

Execute SQL Task Editor: The query failed to parse. Syntax error, permission violation, or other nonspecific error.

March 17, 2014, 1:49 pm
$
0
0

Environment: SQL Server 2008 R2
Code:

CREATE TABLE dbo.PkgAudit
  (
     PkgAuditID      INT IDENTITY(1, 1),
     PackageName     VARCHAR(100),
     LoadTime        DATETIME ,
     NumberofRecords VARCHAR(50),  
     Status1          VARCHAR(50), 
     Status2          VARCHAR(50),
  )

The following code is inserted in the SQL Task Execute Editor

INSERT INTO dbo.PkgAudit(PackageName
,LoadTime
,NumberofRecords
,Status1
,Status2
) VALUES(?,?,?,?,?)

Screen Shot (Parameter Mapping):


Problem: an error Message occurred when I hit Parse Query Button in the Execute SQL Task Editor, "Execute SQL: Task Editor: The query failed to parse. Syntax error, permission violation, or other nonspecific error". How I could Solve this problem  


↧

Issue with SSIS calling script in C#

March 17, 2014, 7:36 am
$
0
0

I have a SSIS package that runs locally on my machine just fine but when I move it to my integration server and try to run it through VS 2010 Shell on the Integration server I get the following error when loading the DTS Script:

DTS Script Task has encountered an exception in user code:
Project Name: ST_d3f7f00f7f3447299e9f5179f3d80872
Exception has been thrown by the target of an invocation.

at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo method, Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeType typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()

Mind you, this same code runs without an issue on my local machine.  I've had two other people look at the code and verify that the variables are all spelled correctly (which is what I understand to normally throw this error).

Would anyone have an idea on why I'm getting this error?  Is it because locally I use VS 2010 Professional version 10.0.40219.1 SP1Rel with .Net Framework 4.0 and the Integration Server uses VS 2010 Shell?

Thanks,

Bob

↧
Search

Error in importing multiple excel sheets to SQL Server

March 17, 2014, 3:06 pm
$
0
0

I have a package which imports multiple excel sheets to SQL server using a For each Container. However I am getting the following error message "Excel Source failed validation and returns validation status "VS-NEEDSNEWMETADATA".

Can you please advise me of the steps to potentially resolve this issue ?

Many thanks

Scott

↧
↧

Multiple namespaces in an XML document

June 18, 2008, 7:37 pm
$
0
0

I have an XML document that I am trying to use within a data flow (XML Source)--the problem is that I keep getting an error message stating that the XML document has multiple namespaces and therefore, SSIS will not create the XSD for me.  If I take out the second namespace, I am able to successfully get the task to execute, but this XML is created with 2 namespaces and there is no way to get around this (I cannot get the report server to change these parameters)--my question is: does anyone know of a way to get SSIS to handle multiple namespaces so that I can process this XML document and extract the necessary data elements from it.

 

Any assistance would be greatly appreciated.

 

Thank you!!!!

↧

Max limit on size of query in SSIS OLEBD Source component

March 17, 2014, 10:52 am
$
0
0

Hello Everyone,

I have a question as below.

What is the Max limit on size of query in SSIS OLEBD Source component ?

Thanks for your co-operation.

Thanks

Fredyyy

↧

Retrieve ErrorColumn Name in Error Flow SSIS

March 13, 2014, 3:15 am
$
0
0

Hi All,

I am trying to retrieve the ErrorColumn name from the error raised in a dataflow. this is how i set up.

in the list column names i have written the code from the link.

http://angelia-thompkins.blogspot.com/2011/05/retrieving-ssis-error-column-names.html

but its raising the below exception. 

how can i solve this to retrieve the ErrorColumn name .

Thanks in advance

↧

Can SSIS web services task write to a HTTPS site, not an http site, or is it best to script this?

March 12, 2014, 2:52 pm
$
0
0

I see a lot of information on using web services task to http, but it doesn't look like everyone is in agreement on using this task to write to a https site.

I'm being tasked with a uploading a file daily to a https site and wanted to see what others are using? I am not much of a programmer, but if the consensus is to script this, does anyone have some examples they could share? We are using ssis 2012

thanks

kam

↧
↧

improve performance to insert data from SQL SERVER to AS400

March 12, 2014, 9:01 am
$
0
0

how can i improve the performance  to insert data from sql server to AS400.

now i have added a OLE DB source as a source from sql server

and then added OLD Destination as destination to AS400.

but i have around 1240000 data need to insert to AS400. it run for a long time to finish the job.

how to improve the performance?

↧

How to capture full error message passed back from db engine?

March 11, 2014, 7:25 pm
$
0
0

If i have the following table in sql server

create dbo.null_test (id int not null, Code varchar(10) not null)

and I try to insert a null

INSERT INTO dbo.null_test values (1, null)

I get the following error

Msg 515, Level 16, State 2, Line 1

Cannot insert the value NULL into column 'Code', table 'dbo.null_test'; column does not allow nulls. INSERT fails.

The statement has been terminated.

But if i try the same in a data flow I only get the the following from the oledb destination

"The value violated the integrity constraints for the column."

Not quite to the same level of usefulness. Is this a bulk insert issue that we're stuck with? ie does the bulk insert command/sqlBulkCopy return these generic error messages and SSIS is just showing what it's given?

Jakub @ Adelaide, Australia

↧
Search

Attunity Connector double byte Character

March 11, 2014, 3:15 am
$
0
0

Hi All,

I am importing data to Sql server through Oracle attunity connector. It is pretty fast however I have some Chinese and Japanese characters in my source and all looks ? in my destination table. I used data conversion to convert dt_str to dt_wstr but no luck. Is there any properties that I can set? Does someone have any idea?

thanks

↧

Create a status bar inside SSIS C# Script?

March 10, 2014, 1:07 pm
$
0
0

I am moving rows from one table to another using a C# script. Please don't tell me to use SSIS components for that. I am looping the incoming dataset using a for loop and I want to see the current iteration, ie for loop index in a GUI box. I can useMessageBox.Show("Text"), but I need to press ok/cancel to allow the code to continue. So, I thought of using a status bar instead. I tried an example I got online - http://www.c-sharpcorner.com/uploadfile/mahesh/statusbar-in-C-Sharp/

The line this.Controls.Add(mainStatusBar); in the example (below) causes the error -

csproj.ScriptMain' does not contain a definition for 'Controls'
and no extension method 'Controls' accepting a first argument of type '.csproj.ScriptMain' could be found (are you missing a using directive 
or an assembly reference?)

This happens despite adding the reference - System.Windows.Forms.dll and doing a save all (ieCtrl+Shift+S). The script has a import using System.Windows.Forms; already.

Why am I getting this error and how do I fix it ?

Code -

protectedStatusBar mainStatusBar =newStatusBar();protectedStatusBarPanel statusPanel =newStatusBarPanel();protectedStatusBarPanel datetimePanel =newStatusBarPanel();privatevoidCreateStatusBar(){// Set first panel properties and add to StatusBar
    statusPanel.BorderStyle=StatusBarPanelBorderStyle.Sunken;
    statusPanel.Text="Application started. No action yet.";
    statusPanel.ToolTipText="Last Activity";
    statusPanel.AutoSize=StatusBarPanelAutoSize.Spring;
    mainStatusBar.Panels.Add(statusPanel);// Set second panel properties and add to StatusBar
    datetimePanel.BorderStyle=StatusBarPanelBorderStyle.Raised;
    datetimePanel.ToolTipText="DateTime: "+System.DateTime.Today.ToString();
    datetimePanel.Text=System.DateTime.Today.ToLongDateString();
    datetimePanel.AutoSize=StatusBarPanelAutoSize.Contents;
    mainStatusBar.Panels.Add(datetimePanel);

    mainStatusBar.ShowPanels=true;// Add StatusBar to Form controlsthis.Controls.Add(mainStatusBar);}privatevoid button1_Click(object sender,EventArgs e){
    statusPanel.Text="Button is clicked.";}privatevoid checkBox1_CheckedChanged(object sender,EventArgs e){
    statusPanel.Text="CheckBox is checked.";}privatevoid textBox1_TextChanged(object sender,EventArgs e){
    statusPanel.Text="TextBox edited.";}
↧
↧

How to Investigate and Fix Dependency Chains

March 7, 2014, 1:12 pm
$
0
0

I have a package that is currently running but is displaying the following warning message:

Warning: 0x80049305 at Data Flow Task, SSIS.Pipeline: The package contains data flow components with multiple inputs. Run-time dependencies that exist among these components may increase memory usage and reduce performance. Consider removing the dependency chain for the following component IDs and output IDs: 1790(1851) 667(671) 1890(1915) 680(752) , to improve performance.

How do I go about investigating and removing the dependency chain for the listed component and output IDs? I can't seem to locate them.

Thanks!

↧

C# Application.LoadPackage with Project Level DataSource Connections

March 6, 2014, 3:01 pm
$
0
0

I'm trying to use the .NET C# framework to make changes to a number of SSIS 2012 packages.

I am trying to use the LoadPackage method of the Application class to open the package, update a variable, and save the package.

The issue that I'm encountering is that the package contains a reference to an project-level connection.  When I save the change, it saves the change fine, but marks all the external connections as invalid.

connectionManagerID="{B0E1693D-0BCA-4D24-AEFC-97F81E37432C}:external"

becomes

connectionManagerID="{B0E1693D-0BCA-4D24-AEFC-97F81E37432C}:invalid"

I assume this is happening because I only have the one project open and therefore the LoadPackage has no context of the project and it's project-level connections.

Is there a way to give the LoadPackage the context of the available project-level connections so that it doesn't mark all the connections as invalid?  When I save the one valid change that I made?

Here is the code for reference:

// Initialize an Application and Package object
Application app = new Application();
Package package = null;

// Load the package in package object
package = app.LoadPackage(pkgPath, null);

// Update the Variable
String value = package.Variables["TableName"].Value.ToString();
if (value.IndexOf(".") == -1){
	package.Variables["TableName"].Value = "tstg." + package.Variables["TableName"].Value;
}
// Save the package
app.SaveToXml(pkgPath, package, null);

Thanks in advance,
Jason

↧

2K8R2: Flat file manipulation suggestions (copy,unzip,load) ~3500 files

March 17, 2014, 5:55 pm
$
0
0

Hi all,

I need to load some data from a large amount of flat files (identical schema). The files reside on a file share and are zipped. There's a file for every day since 2004 so there's about 3500 files.

There's one csv file per zip. The filenames have a YYYYMMDDHH24Mi timestamp embedded in them (both the zip and csv).

This is not a once off load; once I load all 3500 current files the process needs to run daily, loading the new files when that arrive.

I'm canvassing for suggestions on how to process the unzip/csv copy in the most robust and quickest fashion. And also support a delta unzip, so that I only process new files that arrive in the folder

I have an ETL framework in place that saves a delta date and that works fine for loading from a database source, but I'm not sure what the best way is to pull out a list if files greater than some filemask and then process them.

I guess the first question is how should I get a filtered list of files to process.

Powershell fired by an execute process task?

C# transform task? - Want to avoid this

a foreach loop won't work because it'll loop through all 3500 files in each execution and this will just grow.

Jakub @ Adelaide, Australia

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>