Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Discussion: How to best load data from Excel Sheet saved in Office 365 Sharepoint and into SQL DB

November 18, 2016, 4:06 am
$
0
0

Hi all

I have a question that I would like some input on. I thought that it was straight forward, but currently it seems rather tricky to actually be able to get data from an Excel sheet from a document folder in Office 365 Sharepoint and into a SQL Database using SSIS.

The setup is as follows.

I have a Office 365 Sharepoint setup, with a set of Subsites - one for each Customer Area.

So Customer1@company.com have access to https://company.sharepoint.com/customer1

On the site there is a document folder where the user can upload their timesheets. Ex. Timesheet_Jan2016.xls ... Timesheet_Nov2016.xls is stored in the document library called

https://company.sharepoint.com/customer1/DocumentsForUpload/Forms/AllItems.aspx

So question is now, how can I, from the Server running SSIS and SQL create a package that is allowed to access the folder and extract the data. I know how to design the SSIS flow to cycle through the files and import, but it's more about actually getting to the folder in Sharepoint. I have seen examples where, instead of a document folder, the user is directed to a OneDrive Location and then on the SSIS/SQL server the SQL agent account is then granted access to the OneDrive folder and after syncronizing it to the local OneDrive folder, it can then be accessed through SSIS. This however seems very cumbersome and at the same time, the OneDrive Sync seems only to be initiated once the agent account is logged on to the desktop..

Alternative i'm thinking of just booting up a FTP, join it to the Azure AD, grant the users access to respective folders on the FTP and then redirect the users to the FTP instead of using the Document folder

\Christian

 

↧
Search

Can I Reference a SSIS variable from inside a SQL Query? Maybe apart of Execute SQL Task or Lookup Task.

August 19, 2011, 1:37 pm
$
0
0

Scenario:

I created a variable in SSIS called DealerNamePartPath. By using the variable’s property expression I grab only the dealer name in the file path.   The datatype is String and its value returns GNJ, being the dealer name. I want to use this variable in a SQL query.  Is that possible with Lookup Task or Execute SQL Task? I heard to use the mapping parameter tab apart of SQL Task, however for what I’m doing it seems like the Lookup task is a better option. I’d say under Connections tab in Lookup task

 

SELECT * FROM DDR.Part_Txn_HeaderTemp

Where dealer_id in (SELECT DealerID

                            FROM DealerIDFactTable

                            WHERE DealerName = 'GNJ')

 

But here instead of hardcoding ‘GNJ’ I want to use User::DealerNamePartPath.

Evan Johnson
↧
↧

for each loop container doesnt stop

November 21, 2016, 10:41 am
$
0
0
I have for each loop container in ssis package. Im trying to loop through some 100 folders and under those folders say I have 100,000 files. Once it is done looping for each loop container doesn't stop. I would like to stop for each loop container once it has done looping through all folders. ANy help much appreciated.
↧

SSIS 2016 Deployment Failed - Changing Protection Level

November 21, 2016, 3:59 pm
$
0
0

Hi experts,

I created an SSIS package and successfully deployed to the DEV and SYST environments which has SQL Server SSIS 2016 installed. However, my deployment team can't deploy the same package to the UAT environment which also has SQL Server SSIS 2016 as the other environments.

They got this error: Changing Protection Level - Failed and actual error message is:

Value does not fall within the expected range. (Microsoft.SqlServer.ManagedDTS)

Any idea?

Thanks,

Sam Lam

↧

UNION operator plus XML PATH

November 21, 2016, 1:28 am
$
0
0

Hi guys,

This query does not work at all:

SELECT distinct flagenric  + ', ' AS 'data()'
FROM HQ_NAV_PRO.DBO.TEMPCDRCO
FOR XML PATH('')

UNION

SELECT distinct flagenric  + ', ' AS 'data()'
FROM HQ_NAV_PRO.DBO.TEMPCDRPO
FOR XML PATH('')

Msg 156, Level 15, State 1, Line 5

Incorrect syntax near the keyword 'UNION'.

Thanks for your thoughts,


↧
↧

Minimize the lookup and Union All

November 21, 2016, 4:57 am
$
0
0

Hi Experts,

I have an existing ssis packages which contains numerous Lookup and Union all. I read on a resource that such patterns impacts the performance and also currently its looks very shabby if someone need to troubleshoot. Could you please suggest some alternate approach for achieving the requirement. 

Thanks 

Priya

↧

SSIS - create a variable table Create table and can not create table because it already exits.

November 21, 2016, 9:15 am
$
0
0

I'm receiving an error message table already exists when I'm creating an creating a table based off of a variable. A variable that creates a table.  The source variable dynamically creates a table with month and year. Received an error message that can not create table NOV - 16 because it already exists.  I looked around in google and suggestion if table already exists in DB but this can not be the problem due to the variable creates the table dynamically.

Any direction on how to find root cause would be greatly appreciated.

Paul Hinostroza

↧

Unable to connect to SAS Server while using SAS Providers for OLE DB

November 21, 2016, 2:20 am
$
0
0

Hi Technet!

I want to connect to a SAS Server in order to retrieve some data from a Sas Server. I tried the use of theSAS Share Provider 9.43 as oledb source with two different connection settings but I have not managed to set up a connection. In the first case, at the connection manager window I populate the field "Server or file name" with the specified server name and the "location" field with the specified domain name of the server and of course the necessary access credentials (username and password). In the second case, i insert the same values but in the "location" I put the path name of the data location folder in the server. Below, i record the error message i get for each case accordingly:

Case 1 Message error using server domain name at the "location" field :

Test connection failed because of an error in initializing provider. A server error has occurred: TCP method failed rc = -1009 (Unable to locate service).SASUPSQLPassthru::Connect; context =YHRELSCE was returned; 

Case 2 Message errorusing data location path folder at the "location" field :

Test connection failed because of an error in initializing provider. A server error has occurred: TCP method failed rc = -7 (No such host is known).vice).SASUPSQLPassthru::Connect; context =YHRELSCE was returned;

Is there something i am doing wrong or any further connection properties i  am omitting?I would apreciate i you can give me some advice.Thanks in advance.
Regards,

Vangelis

↧
Search

Invalid Character Value For Cast Spefication on SSIS 2012

November 21, 2016, 7:46 pm
$
0
0

Hi,

We're trying to upgrade our SSIS 2005 packages to 2012 and I encountered a casting/invalid character issue in Data Flow Task.

This table will get populated by a pipe-delimited file

CREATE TABLE [dbo].[STKORD]
(
[Catalogue] [varchar] (255) COLLATE Latin1_General_CI_AS NULL,
[Title] [varchar] (255) COLLATE Latin1_General_CI_AS NULL,
[Total Stk Avail] [int] NULL,
[Stk on Order] [int] NULL,
[Rank] [varchar] (255) COLLATE Latin1_General_CI_AS NULL,
[QOH] [int] NULL,
[Allocated] [int] NULL,
[Reserved] [int] NULL,
[BackOrd] [int] NULL,
[WIP] [int] NULL,
[Cleared] [int] NULL,
[Held] [int] NULL
) ON [PRIMARY]
GO

Data Flow

This used to be working in 2005 but not in 2012.

Now my question is, does 2005 handle casting better over 2012? Which forces me to add a Data Conversion task in 2012?

Many thanks.


↧
↧

OnError Event not firing if executing package from job or SSISDB

November 20, 2016, 5:30 pm
$
0
0

Hi I create a simple package with event handlers : OnPreExecute, OnError and OnPostExecute.

It works fine if I execute the package on SSDT (2013) (all 3 fired, if having error)

But but if I deploy the package on SSISDB and execute it there, the OnError event does not get fired (only Pre and Post fired, even having the error)

The same thing, if I execute the package from a job, the event does not get fired either(only Pre and Post fired, even having the error).

I did try

1 - DelayValidation : true/false - both do not work

2 - DisableEventHandler : true/false - both do not work

Combine all possibilities of 1 and 2 - do not work either

Do you have any idea

Thanks

 

↧

OnPostExecute triggers before execution completes

November 17, 2016, 7:48 am
$
0
0

A quick brief of the issue I am facing, there is an audit table that captures the execution status of the package. The package has an OnPreExecute event handler to update this table with status 'Started' and an OnPostExecute event handler to change the status to 'Completed'. I have set a breakpoint in one of the tasks to halt the progress. At this point when I check the execution table the status shows as 'Completed'.

My understanding is that the OnPostExecute event handler defined at the package level should execute only after the execution of the package completes, or am I wrong?! Kindly share your views.

Also, when I disable to OnPostExecute event handler the OnPreExecute handler gets triggered and the status gets updated to 'Started'. So I know for sure that, that part works as expected.

↧

BufferTempStoragePath - Global setting?

July 13, 2006, 10:28 am
$
0
0

By default the BufferTempStoragePath is mapped to the user running the package's Documents and Settings folder.  This is problematic when numerous packages are running simultaneously and using this disk location (i.e. sorts), and you don't have a large disk for your C: drive.

The property of course can be changed.  However the property is specific to a data flow task, so this would require developers to change the property is every data flow task of every package.  Is there a global setting to change the default location that SSIS will use?

An alternative is to use configurations, however a configuration will be required for every data flow, as it is specific to the data flow task (and name of that data flow task)

Any ideas?

Thanks

 

 

 

↧

Loading character greater than 255 into an excel column

November 17, 2016, 9:01 am
$
0
0

I have having problems when trying to load into an excel sheet with over 250 characters. I have recreated my issue with just one row to remove other noise in the package. so the package extract from one excel cel with one row(character greater that 255) into another excel. I used two conversion in between from Dt-ntext to str then from str to unicode str. But it still fails please any guide will be appreciated. Thanks

I get this Error:

Error: 0xC020901C at Data Flow Task, Excel Destination [22]: There was an error with Excel Destination.Inputs[Excel Destination Input].Columns[Copy of Copy of Test] on Excel Destination.Inputs[Excel Destination Input]. The column status returned was: "The value violated the schema's constraint for the column.".
Error: 0xC0209029 at Data Flow Task, Excel Destination [22]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "Excel Destination.Inputs[Excel Destination Input]" failed because error code 0xC0209076 occurred, and the error row disposition on "Excel Destination.Inputs[Excel Destination Input]" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.
Error: 0xC0047022 at Data Flow Task, SSIS.Pipeline: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Excel Destination" (22) failed with error code 0xC0209029 while processing input "Excel Destination Input" (33). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.

↧
↧

Your easier chance to become a medal winner SSIS Guru winner for November!

November 22, 2016, 3:20 pm
$
0
0

Dearest Microsoft Technologists!

This is your last minute call for November Gurus!

You have just over a week left to submit anything you post to TechNet Wiki, into our competition, and you could win BIG!

With the management in turmoil due to MVP Summits and RL interruptions, there has been low publicity this month for the competition.

This simply means any half decent submission to TechNet Wiki can win a medal... and a place in history!

Changes are under way in this competition, and medals will count towards REAL WORLD PRIZES in the new year...

So get in while you can, and start making a name for yourself in your favourite technologies

All you have to do is add an article to TechNet Wiki from your own specialist field. Something that fits into one of the categories listed on the submissions page. Copy in your own blog posts, a forum solution, a white paper, or just something you had to solve for your own day's work today.

Drop us some nifty knowledge, or superb snippets, and become MICROSOFT TECHNOLOGY GURU OF THE MONTH!

This is an official Microsoft TechNet recognition, where people such as yourselves can truly get noticed!

HOW TO WIN

1) Please copy over your Microsoft technical solutions and revelations toTechNet Wiki.

2) Add a link to it on THIS WIKI COMPETITION PAGE (so we know you've contributed)

3) Every month, we will highlight your contributions, and select a "Guru of the Month" in each technology.

If you win, we will sing your praises in blogs and forums, similar to the weekly contributor awards. Once "on our radar" and making your mark, you will probably be interviewed for your greatness, and maybe eventually even invited into other inner TechNet/MSDN circles!

Winning this award in your favoured technology will help us learn the active members in each community.

Feel free to ask any questions below.

More about TechNet Guru Awards

Thanks in advance!
Pete Laker - Azure MVP

#PEJL
Got any nice code? If you invest time in coding an elegant, novel or impressive answer on MSDN forums, why not copy it over toTechNet Wiki, for future generations to benefit from! You'll never get archived again, and you could win weekly awards!

Have you got what it takes o become this month's TechNet Technical Guru? Join a long list of well known community big hitters, show your knowledge and prowess in your favoured technologies!

↧

SAP BW source to OLEDB

November 17, 2016, 6:04 am
$
0
0

Hi,

I'd like to integrate with SAP BW (I have the Bex Query and MultiProvider from SAP BW) and have the data in the SQL Server.

How can I do it?.

Thanks in advance!.

↧
Search

parameterise source flatfile path

November 22, 2016, 3:04 am
$
0
0

i'm new to SSIS, developing a package on a development server.

i'd like to transfer this to a test server, and from there to a live server.

on the "Flatfile Connection Manager Editor" page, "General" tab is a "File Name" box.

this currently has the full absolute path to a text file.

i have about twenty of these.

moving to a different server, this path will be different.

i'd prefer only to have to edit one thing and have it propagate automatically.

is this possible?

perhaps to insert a parameter for the folder part?

how would that work?

as a side note, i've read that it won't be possible to do what i want anyway, since all objects are identified by GUID. creating new objects (for connections, etc) breaks everything and causes more work than starting again from scratch. is this just alarmism? no longer true?

many thanks in advance,
e

↧

OLEDB Deprecation and SSIS

November 22, 2016, 8:04 am
$
0
0
Hi, 

Though this question has been asked numerous times before, I think I am one of the few people still who are still not 100% sure about what the deprecation of OLDDB means for SSIS/ SSAS,  

The deprecation was announced several years ago (more than 5?) and here we are with SSIS 2016 and there is still no support for non OLEDB connection managers for components like Lookups. SSAS Multidimensional still reads via OLE DB drivers. There are other OLEDB only implementations too. Is there any definitive guide for OLEDB deprecation and what it means for SSIS (and SSAS Multidimensional)?

I have trawled through numerous forum posts and links but can’t say for sure if the direction is clear. Switching to ODBC doesn’t seem like an option due to performance reasons. ADO .Net is faster but still isn’t quite as fast as OLEDB. 
Ofcourse we could start switching to something like ADO.Net for new development but this is still not what  Microsoft recommended as OLEDB had been deprecated in favour for ODBC and also the fact that data loading performance using ADO.Net / ODBC connection manager is still not at par with OLEDB 

Does anyone know what exactly is happening around it please? Any link or details published officially would be really helpful. 


↧
↧

Term Lookup Exclusions

November 22, 2016, 1:13 pm
$
0
0
I am importing a damn excel feed that has all financial activity from a particular vendor. I was trying to use a term lookup to find rows that I don't want to look at further. Say terms are "pop tart", "toner cartridge", etc. I no longer want these rows so I thought I could send the matches to an exclusions table and the ones that do not have a match to the error output and on to the next step. Technically these are not errors and I don't have the option to redirect no matches to the error output on that transformation. How can I achieve this in SSIS? I get the excluded rows but nothing out the error output.

JC

↧

How to get a format date like that dd-mm-yyyy

November 22, 2016, 7:44 am
$
0
0

Hi,

I have a flat file with this the format date October 01, 2015 i want to change to01-10-2015 

How can i do that

Thanks 

↧

Excel Source, Failed to retrieve long data for column (Unicode Text stream Data Type)

November 20, 2016, 9:58 am
$
0
0
I am Exporting Excel File data(passing the file names dynamically) from a folder into another Excel File. I have done all the proper conversion including handling the long data columns. But at the excel source connection I am having "Failed to retrieve long data for column" I have tried different soon like increasing the size of defaultDefaultMaxRows to 20000 also I have increased the size of DefaultBuffersize, but I am still getting the error, also specified new temp locations for BLOB and buffer. But still having the same error "Failed to retrieve long data for column". is there any thing I am missing Please

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>