I'm trying to take data from three separate text files and combine into one destination table. Specifically, there are 8 columns in my destination table, which are a combination of data from the three source files. Each of the three source files have different
columns that I would like to map to the specific columns in the destination table. The source files do not have the same columns, and the destination table has a combination of a select few columns from each of the source files. Is there a way to make this
work?
↧
How to Transfer Data From Multiple Sources to One Destination
↧
SSIS: Change Code page 1252 ANSI to UTF-8 65001
Hi All;
I have created SSIS Package to import data from OLE DB to a flat file. The code page in the flat file connection manager was set to default1252 ANSI - Latin I. Now I have been asked to output the flat file inUTF-8 format. I have changed the code page to 65001 UTF-8 but the package will not run. I got the following message:
the code page on input column COLUMN_NAME (184)is 1252 and is required
to be 65001
I have searched around and i have read that i need to change the column data type fromVARCHAR to NVARCHAR!! Is this the solution?
Is there is any other option? Thanks.
↧
↧
Problems with Excel Formatting in SSIS Output
Hi,
I'm in the process of upgrading a lot(100+) of SSIS packages to a newer version of Visual Studio. While upgrading i am also taking the opportunity to update any packages using xls files to xlsx.
Most of the jobs are about taking data from a SQL Query, exporting to excel and emailing the file to end users so formatting is a key aspect. I'm not bothered about fonts/colours/borders etc but what is important is that numbers are exported as numbers and that dates are sometimes required to be date/time, just date or just time.
I had previously controlled this by using a template xls file with a hidden dummy row under the header containing example formatting. When the data was exported the process would "magically" follow the formats set out in the dummy row. Now that i've moved over to xlsx this only partially happens. If i put a date in a dummy row the export brings it out as a date(rather than text - not using the dummy row at all dumps everything out as text values that don't sort or filter correctly) but it's always DD/MM/YYYY format even if the dummy specified DD/MM/YYYY HH:MM. xls files worked perfectly in this regard. Even pre-formatting the columns in the way i would like doesn't have the desired effect as the export process ignores any pre-existing formats.
It's looking like my only option is to roll back to using xls which feels like a step backwards i wasn't really wanting to take. Is anyone able to give me any advice on how best to control formatting when outputting to excel? I've seen things on other threads about adding scripts to format data after export but to do this to 100+ packages isn't realistic when it's actually not adding much value to the process i currently have with xls files.
Regards,
Andy
↧
Perform transformation on No-SQL database using ETL
Hi,
I want to perform transformation on Cosmos DB data and create data warehouse using ETL tool such as SSIS,Azure data factory.
Is there any specific scenarios where we can use ETL with No-SQL database?
Thanks,
↧
Can SSIS source data from a Power BI dataflow?
I would like to use Power BI dataflows as sources for SSIS. Is this possible?
↧
↧
No Column information was returned by the SQL command.
I can preview the SQL command in the OLE DB Source Editor and bring back all columns and results just fine but when I click on the Columns I get
TITLE: Microsoft Visual Studio
------------------------------
The component reported the following warnings:
Error at Data Flow Task [OLE DB Source [1]]: No column information was returned by the SQL command.
The columns are there in the preview - why can't SSIS get the column information?
TIA
Harry
↧
Abort Data flow task
I have data flow, importing from Excel to table.
Current setup is to ignore Row if First Column in Excel is blank continue rest of Sheet.
instead
Is it possible to Abort Data flow task if First Column is blank , don't want to read further from Excel.
Current setup is to ignore Row if First Column in Excel is blank continue rest of Sheet.
instead
Is it possible to Abort Data flow task if First Column is blank , don't want to read further from Excel.
↧
[Execute SQL Task] Error: The value type (__ComObject) can only be converted to variables of type Object.
Hi we run 2017 std. The tail end of my single row resultset query is shown below in the code block, a bit anonymized. I'm getting an error mapping the output (command) to an ssis string variable called extractCommand. Presumably because of the varchar(max) portion. This error follows the first in the progress pane.
[Execute SQL Task] Error: An error occurred while assigning a value to variable "extractCommand": "The type of the value (DBNull) being assigned to variable "User::extractCommand" differs from the current variable type (String).
Variables may not change type during execution. Variable types are strict, except for variables of type Object.
".
The query being generated dynamically ends up being close to 21k bytes in length for 22 locations. The generating query runs fine in ssms and the generated query runs fine in the db where it is intended to run.
Its my understanding so far that sql wont even run this unless the first item in the string_agg function has a data type (thus the casts) big enough to hold the results. The generated query ends up looking something like this and the number of conditions in parens separated by "or" for each location is not necessarily the same as the others...
select ... from location1_replicated.dbo.tbl where (id=... and dt >... and dt <= ...) or (...) or (...) union all select ... from location2_replicated.dbo.tbl where (...) ....
The compound predicates were built separately, one row for each location in #predicates.
Does anybody know of an elegant way around this problem?
select substring(command,12,len(command)-11) command
from
(
select string_agg(cast(' union all select ' as varchar(max))+ '''' + a.loccode + '''' +' loccode,* from ' + case when a.loccode='abc' then 'xyz' else a.loccode end + '_replicated.dbo.tbl where ' + b.[predicate],'') command
from #whichlocs a
join #predicates b
on a.loccode=b.loccode
) x
↧
is ssis appropriate for parsing free form text for mood and favorability indicators?
Hi this vague but perhaps there is a must read recommendation if someone is looking to ingest (perhaps in an etl tool) free form text from both the company's captured comments and social media looking for and recording "(un)favorability" indicators
that might provide insights into declining sales.
↧
↧
Could not load file or assembly 'Microsoft.SqlServer.ManagedDTS
Hello,
I have Sql Server 2016 (13.0.4001.0) installed on my machine and I downloaded SSDT for VS2017 (15.1.61810).
While SSDT was on setup step, I have choosen all features for Sql Server Analysis Services, Reporting Services and Integration Services. So after the installation, I tried to Create a new project under Business Intelligence > Integration Services > Integration Services Project, but as soon as I give a name to the new project and clicked on OK, I got the following error:
Could not load file or assembly 'Microsoft.SqlServer.ManagedDTS, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.
Then I tried to create a Reporting Service Project, it is created successfully.
So please help me understand the main problem and kindly suggest me a solution.
Thanks,
↧
Connecting oracle
Hi All,
We installed oracle 11 G in windows server 2016 operating system and are trying to connect to oracle .
Created Sql agent job, when we start the job , job was keeps running and not getting error or not complete .
I connected oracle successfully through SQLPLUS however sqlagent job is not working.
Please let me know of nay one has idea about this is issue,
Thanks.
↧
expression
I have the query to return records along with row number 1,2,3.....
if RowNo <15, i want the row sets to green, if it's between 15 and 25, set to yellow, else, set to red. Not sure what I have wrong from the expression below but it didn't work for me. Can you please help?
=IIF(Fields!RowNo.Value<15,"green",IIF((Fields!RowNo.Value>=15 and Fields!RowNo.Value<=25,"yellow"),"red"
))
↧
Multicolumn-Single row excel data to single column multiple rows excel destination -SSIS
Hi Guys,
Extraction of Data from Excel source to Excel destination.
Excel source format:
need to extract the data from excel source ( multiple columns - single row ) into Excel destination ( single column - single row spilt into multiple rows in same column
i.e based on input datarow where columns are static , need to load to excel destination with single row split into multiple rows in one column
This i am trying to do in SSIS.
↧
↧
[OLE DB Source Editor] Redirect rows
Hi,
I have OLE DB Source Editor which have SQL doing conversion of almost all columns, and failing on one of column doing conversion to date. I configured Error Output and set Error and Truncation for all columns to Redirect Row but unable to receive any record. I am wondering that is possible or not as the error while executing query.
Muhammad Yasir
Many Thanks, Muhammad Yasir
↧
Dased on Parameter value run data flow task
Based on the Paramater value I wan to start the data flow excecuation A or B.
For example lets say the parameter name is InputFlag
when i set the precedience constraint to InputFlag== "ABC"||InputFlag== "PQR"||InputFlag== "XYZ" THEN it should run the task 1 else it should run the task 2
↧
auto generated identity field as business key
I have 4 Fields let's say tab_id, Col_A,Col_B,Col_C, Start_Date, End_Date in source table .
In building a dimension
Tab_Id is not a unique field that classifies as a business key.
Col_A,Col_B,Col_C are all required SCD.
Can I add an auto generated number as a business key in source table to use as the business key in dimension table?
or can I create a composite business key using the fields tab_id, Col_A,Col_B,Col_C while still keeping Col_A,Col_B,Col_C as SCD?
Thanks,
↧
How to write case statement in SSIS expression Derived Column.
Hello ,
I have a data flow task where I am getting a flat file and importing into our staging table. But before I import I am trying to remove the domain from the field called computer name. Here is the expression written in SQL -
Update table A
set [Computer Name]=case when CHARINDEX('.',[Computer Name])>0
then case when [Computer Name] like '[1-9]%.[1-9]%.[1-9]%'
then [Computer Name]
else SUBSTRING([Computer Name],0,CHARINDEX('.',[Computer Name])) end
else [Computer Name] endThis works if I run this outside of dataflow. But due to requirement change I need to include this inside data flow and have derived column that will use this expression.
How do I rewrite this under derived transformation?
↧
↧
Email Script Works on VS2015, But Not VS2017
Hello, all -
I have an email script that works fine and dandy on a box running VS2015. But, we will be losing those systems at some point and we have been given new laptops with VS2017 to use concurrently, until we lose the old laptops.
My email script is in VB.NET and leverages Outlook. It has two lines;
objDoc = OutMail.GetInspector.WordEditor
objBkm = objDoc.Bookmarks("_MailAutoSig")that are used to grab the default signature and this works as it should on my box running VS2015. But, on the VS2017 version, when I hover the cursor over objDoc, the value shows as "Nothing."
My limited knowledge tells me that WordEditor is not working, for some reason, but I don't know why.
Might anyone be able to give me some assistance on what may be happening?
Thanx in advance for any help!
↧
How to configure TFS to automatically build an ISPAC file on check-in
Hello Everybody,
We are checking our SSIS 2016 packages into TFS. We always make sure we can successfully build the project, too, which creates the *.ispac file for us locally. When it comes time to deployment, we manually pass that ISPAC over to the Operations group. The Operations person then logs onto the production SSIS server and double-clicks the ISPAC file in order to bring up the SSIS deployment wizard.
Sometimes this process doesn't work for us due to human error. Typically, that's a developer forgetting to check in the project they built into an ISPAC. That ISPAC gets deployed, but when the next developer makes changes to the same project, we reintroduce a bug that was fixed in the previous deployment.
Is there a way to force TFS to create the ISPAC for us? In this way, we could ensure that what gets checked in is what gets deployed (and likewise what doesn't get checked in doesn't get deployed).
Thanks,
Eric B.
↧
unable to connect to SSIS(2008) from SSMS (2016) remote server
Hi I am trying to connect SSIS 2008 from remote Server SSIS(2016) having error as follows. . Please note I can connect from locally. Mean from server itself I can connect to Integration service. Issue is while connecting from Remote server. I granted permission DCOM on local server. I am not sure need to do on remote server as well . Do any one knows how to resolve this issue.
↧