Hi Guys -
Is there any way we can download sharepoint folders using ssis. I have a specific link(View on sharepoint) which I would like to download using ssis, Is it possible?
Thanks.
- please mark correct answers
↧
Sharepoint folders using ssis
↧
How to convert a field of VARCHAR datatype to an INT datatype in SSIS 2014?
After searching a lot on the Internet, I have to post this question here. I know it is simple but I'm breaking out my head since the last couple of hours.
I'm trying to convert a column in my source table of datatype varchar(6) to a column of datatype int in my destination. I tried using the Derived Column/Data Conversion transformations but none of them worked. So, I tried using the following C# (credits to the original poster) and getting an error during compilation.
Note: "MyCol" is the Input Column I've specified in the Script Component and "CleanCol" is the Output column I've specified as datatype [DT_I4].
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
int colOut = 0;
if (!Int32.TryParse(Row.MyCol, out colOut))
{
Row.CleanCol_IsNull = true;
}
else
{
Row.CleanCol = colOut;
}
}The best overloaded method match for 'int.TryParse(string, out int)' has some invalid arguments
The other expression I've tried was:
ISNULL(MyCol) ? (dt_i4)"" : (dt_i4)MyCol
From the above code, you might have understood that the source field has some blank records as well in the MyCol field.
Please let me know the best possible way to do the conversion from a String to Int or help me in fixing the error from the above.
Thanks,
Bangaaram
Known is a DROP, Unknown is an OCEAN.
↧
↧
Refresh All Excel file using C# code
Hi,
Please find the below C# code for Excel Refresh All. I am not familiar in C# code. Here I have Included Thread.Sleep time. Is there any possibilities to Refresh excel workbook without using thread.sleep, because some excel file refreshed within 5 sec but its wait 30 seconds (thread.sleep) to go for next excel. Please provide any suggestions.
using System;
using System.Data;
using System.Collections.Generic;
using System.Text;
using Excel = Microsoft.Office.Interop.Excel;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.IO;
using System.Runtime.InteropServices;
using System.Data.OleDb;
using System.Xml;
namespace ST_5f600caa86a04a4b8080eef2563c791e.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
Excel.Application xlapp = new Excel.Application();
xlapp.Visible = false;
string path = Dts.Variables["User::rfilename"].Value.ToString();
//string path = "C:\\SSISDev\\1510001_Cluster Penetration\\Final_File\\AE Assigned & BP Details - Template.xls";
Excel.Workbook excelwb = xlapp.Workbooks.Open(path, 0, false, 5, "", "", false, Excel.XlPlatform.xlWindows, "", true, false, 0, true, false, false);
OleDbDataAdapter A = new OleDbDataAdapter();
System.Data.DataTable dt = new System.Data.DataTable();
A.Fill(dt, Dts.Variables["User::SName"].Value);
Excel._Worksheet oSheetGame;
foreach (DataRow row in dt.Rows)
{
string Name;
object[] array = row.ItemArray;
Name = array[0].ToString();
Excel._Worksheet oSheet;
oSheet = (Excel._Worksheet)
excelwb.ActiveSheet;
oSheet.Copy(oSheet, Type.Missing);
oSheetGame = (Excel._Worksheet)excelwb.Worksheets["Temp"];
oSheetGame.Name = Name;
Excel._Worksheet rsheet = (Excel._Worksheet)excelwb.Worksheets["Temp (2)"];
rsheet.Name = "Temp";
}
oSheetGame = (Excel._Worksheet)excelwb.Worksheets["Temp"];
xlapp.DisplayAlerts = false;
oSheetGame.Delete();
excelwb.RefreshAll();
//excelwb.SaveCopyAs(Dts.Variables["User::rfilename"].Value.ToString());
System.Threading.Thread.Sleep(Convert.ToInt32(60000));
excelwb.Save();
excelwb.Close(false, Type.Missing, Type.Missing);
xlapp.Quit();
GC.Collect();
GC.WaitForPendingFinalizers();
//Dts.TaskResult = (int)ScriptResults.Success;
}
}
}
↧
Failed to create an instance of empty child package.
Hi Champs,
I am executing a package deployed in SSMS. I am using service code to execute the package. Below is the error I am getting.
"[XXXService] Exit Final Run - ConsistencyCheck Package Errored out.
Error 0x80070005. Failed to create an instance of empty child package.
The Distributed Component Object Model (DCOM) configuration or the
installation of SQL Server Integration Services, may be corrupted on your machine."
Please suggest any solution and please let me know the issues.
Regards,
Mojahid
↧
How to handle bad data
Hi folks,
I am having a requirement where I need to load the correct data into the target table and needs to save the bad data for analysis, how can I do that in SSIS.
Thanks in advance
↧
↧
SSISDB Backup & RESTORE (WHY USE MAster Key)
SSISDB Backup & RESTORE I backed up the SSISDB and restored on another server with replace where catalog existed...restored perfectly fine and all packages were restored...I dont understand why I need to restore the master key and what is it used for??
I understand I need additional steps if I need to restore on a server where I had no SSISdb catalog....but Not able to understand why I need master key restore ...please guide.(SQL Server 2012)
↧
Performance Tips for slow Insertion in OLEDB Destination SSIS
Hi
My one task is taking so much time for inserting 60k records it is taking 30min for insertion.
Task contains 12 lookups and destination table contains billions rows and destination table was having 25 indexes.I disabled some indexes using unused index script now it is taking less time compared to previously but still not satisfied for 60k it is taking 30min.
Can anyone tell me any other techinques except disabling and enabling indexes.
Batch size, commit size, maxbuffer rows size etc will improve my performance or any property in SSIS will improve the performance.
↧
DTExec: The package execution returned DTSER_FAILURE (1).
I have 3 SSIS packages on our DEV machine - a 64 bit machine.
When I run in Visual Studio I receive error:
[Connection manager "History Database (mdb)"] Error: The requested OLE DB provider Microsoft.Jet.OLEDB.4.0 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode. Error code: 0x00000000.
An OLE DB record is available. Source: "Microsoft OLE DB Service Components" Hresult: 0x80040154 Description: "Class not registered".
I can then select 32 bit runtime in Debug options and all runs fine.
I have now (package) deployed these SSIS packages to our PROD machine - a 64 bit machine.
I have created a SQL Agent Job to execute these one of these packages. Within this package the other two packages are called.
I continue to receive this error:
DTExec: The package execution returned DTSER_FAILURE (1).
I have tried setting the 32 but runtime on the step settings but the same problem occurs.
Any ideas??
↧
Schedule ETL jobs using SQL Agent or Windows Task Scheduler?
Hi,
When do I use the SQL Server Agent and when do I use the Windows Task Scheduler? What are the pros and cons of each? Currently I need to run ETL jobs (SSIS) at scheduled times, and I want to know which of these two scheduling tools to use for this.
Venkatesh
↧
↧
reading text file, problems after first record
I'm trying to read in a text file, fixed width, very long records (over 7000 characters) in an ssis package. The first record appears ok in the 'preview' in the connection manager setup, but each record after that is offset by 2 characters (record 2 offset
by 2, record 3 offset by 4, record 4 offset by 6 and so on), like it's inserting special characters. I figure I'm doing something wrong with the settings in the connection manager, but not sure what.
↧
Error connecting to Sharepoint
I am trying to get going with the SSIS SharePoint List Source component. I followed the tutorial, created a SharePoint credential in my project set to "Use credentials of executing process", dragged a SharePoint List Source onto the design surface, chose the new credential as my connection manager, filled in the SiteURL and Site Name from the URL I use to access the site in my browser.
The URL is
http://w8.teamworks.td.com/Teams/5628/Lists/Design%20Exception%20Reviews/AllItems.aspx
so I set:
SiteListName: Design Exception Reviews
and
SiteUrl: http://w8.teamworks.td.com/Teams/5628
I left SiteListViewName blank
Then I clicked refresh and got this error message:
Error at Data Flow Task [SharePoint List Source [2]]: Microsoft.Samples.SqlServer.SSIS.SharePointUtility.SharePointUnhandledException: Unhandled SharePoint Exception ---> System.ServiceModel.FaultException: Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException' was thrown.Server stack trace:
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoap.GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.ListsService_ListsSoap_GetListAndView(GetListAndViewRequest request)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.GetListAndView(String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointFields(String listName, String viewId)
--- End of inner exception stack trace ---
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointFields(String listName, String viewId)
at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.GetFields(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.GetAccessibleSharePointColumns(String sharepointUrl, String listName, String viewName)
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.ValidateSharePointColumns()
at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.Validate()
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostValidate(IDTSManagedComponentWrapper100 wrapper)
How can I resolve this?
↧
How yo run a package without BIDS or Data Tools installed in the machine
Hi masters, as far as you aware is there any way to run a package without having VS Data Tools or BIDS installed in the local machine? Scenario: I build a package and I put the package in a folder. I would like that one of my colleague run the package all by himself but I don't want install BIDS or DTT in his machine. Is there any plug in, trick or something else I can do?
↧
Share Point Source RSSBus Adapter
Hi All,
I want to use the excel sheets located on a share point site as source, for this i downloaded the adapter from the following link
I got the connection manager to work, but when coming to load the excel files to the destination location its not doing what is intended. For example i want the destination location my local drive for the excel files, how can i do that which destination should i use?
Thanks
↧
↧
Implementation of bothe Authentication Mode( SQL and Window ) in SSIS package at same time ...
Hello Everyone ,
Can we implement both authentication mode i.e SQL Authentication & Window authentication , while development of SSIS package same time .
As due to Business requirement , we need both authentication mode .
Any guidence will be very helpful.
Thanks
Alok
Can we implement both authentication mode i.e SQL Authentication & Window authentication , while development of SSIS package same time .
As due to Business requirement , we need both authentication mode .
Any guidence will be very helpful.
Thanks
Alok
↧
can i execute sp_start_job sp from execute sql task by passing jobname as todaydate
Hi,
can i execute sp_start_job sp from execute sql task by passing jobname as todaydate
that job is not created in sqlagent.
need immedite response
Thanks
AVS
↧
SSIS Memory Errors without any Memory Pressure
After upgrading an SSIS package from SQL Server 2008R2 to SSIS 2014 (12.0.442) I am seeing the package fail with the following error:
Error: A buffer failed while allocating 104856448 bytes.
The next set or information indicates however that there does not actual shortage of memory on the instance as per below log
Error: The system reports 11 percent memory load. There are 206157959168 bytes of physical memory with 182876856320 bytes free. There are 4294836224 bytes of virtual memory with 1266438144 bytes free. The paging file has 244410011648 bytes with 225109983232 bytes free.
Information: Buffer manager allocated 2530 megabyte(s) in 13653 physical buffer(s).
Information: Component "LkpPlanMember" (125) owns 2530 megabyte(s) physical buffer.
[SSIS.Pipeline] Error: The Data Flow task failed to create a buffer to call PrimeOutput for output "OleSrcAtunePlanInvestment" (269) on component "OLE DB Source Output" (280). This error usually occurs due to an out-of-memory condition.
The Server is running SQL Server 2014 Enterprise Edition version - 12.0.442 with 192GB of Ram. The Max Memory on the SQL Server instance was reduced to 64GB to ensure that plenty of memory is available
This package has been running for years on a SQL Server 2008 R2 version with only 128GB of Ram
Does anyone have ideas on how to correct this error?
↧
Cloning a Sequence container programmatically (C#)
Hi all,
I am currently implementing a tool for creating SSIS packages programmatically (C# based). The Approach is template-based and I am right now looking for ways to clone a (complex) Sequence Container which is part of a package template. The Container might include DFTs and SQL Tasks as well as precedence constraints. All objects contained in the Sequence Container (and their children) should be copied by the searched function/method. My problem is that the Sequence Class does not implement the Serializable Interface. Thus, I am not able to implement the Deep Copy method suggested in other forums.
Any help is very much appreciated.
Thank you very much!
↧
↧
Text File didn't save properly (because I created multiple tabs??) and now all my work is gone - Please help!
Good Morning (11/19/15),
I was working on a fixed asset additions text file to upload in to SAP. To make the process more efficient, my boss suggested I create separate tabs for the different asset types so I could copy down some of the like data so I did. I separated the work I had completed the previous day by asset type and put them on their respective tab, and then started adding data on the tab for 1st asset type tab. I saved as I went along. At the end of the day, I did my final save and answered yes to keeping the file in the same format, etc. I went to re-open it to verify that everything was aligned in the columns properly because on occasion I'd miss that a cell I was copying and pasting was wrapped and it will mess up the alignment on the text file. In those cases I'd just fix that line. So when the file re-opens, none of the work I did that day was there, and even the additional tabs I created with the different asset types were gone!?! OMG!!! I just started a new job and am so upset. =( We looked at trying to restore a previous version of the text file, but even it didn't show my multiple tabs... My co-worker, a fellow accountant, said it was because text files are flat file and won't save properly if you have more than one tab because it's a flat file (I have no idea), which I have no idea what that means. She said you have to save it as an Excel file and then convert it back to text for it to work; at least you think. There has to be some way to retrieve my 3 days Mworth of work since I saved up until the minute I did my final close, right?!? Please advise. Thanks for any help you may offer, and quickly please.
↧
Can we join 3 Object variables inside the package?
Hi,
I have a critical requirement.
I have 3 object variables inside my SSIS 2012 package. They are like 'table' structure .
Eg: First 'object' variable has fields like EmpID,EMpName,Empsalary
Second 'object' variable has fields like ProductID,Productname,product description
Third 'object' variable has fields like SalesID,salesdesc,salesamount
Now can I join these 3 object variables like we do for normal tables and pull the fields from these 3 'object' variable output into another Object variable like 'var_output' . Becasue I may need it to give 'var_output as a input to ForEach loop ADO Enumerator?
Please help!
↧
importing Access data ... empty fields are being imported as Null
Hi..
I am using SQL server 2014
I am importing data from MS Access
I want my blank fields to be imported as blank and not converted to null..
Just an aside - why do people want a value of null to be entered for blank fields?
thanks
M
↧