Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Help in getting SSIS to load a Data Partition

$
0
0

Hello, I have created a 1200 Version (2016) tabular cube. And in that cube I have create a number of data partitions for my large fact tables. To focus on one of those data partitions I’m trying to load, lets call it Usage_201806. In that one D/P that I’m trying to get to load local SQL Server data into, I have this SQL statement Inside that data partition, I have this SQL statement inside: Select Colname1, Colname2, etc, from tablename where date = 201806.

In my SSIS package, instead of my fact table loads going straight to a Analysis Services Processing Task, that executes a full process on the cube and loads everything into one large partition, I have in its place an Analysis Services Execute DDL Task. In that task, in the editor, on the DDL tab, the connection is set to the correct target cube, Source Type is Direct Input, and in SourceDirect I placed this script:

{

  "refresh": {

    "type": "automatic",

    "objects": [

      {

        "database": "Subs Cube",

        "table": "Usage",

        "partition": "Usage_201806"

      }

    ]

  }

}

(Obtained script by the method of going into the deployed target cube with the new partitions I created, and went to table, process, then scripted out to new, etc…)

There are no errors, and when I run it, no errors, but it’s not processing the local data into the data partition. This is a test, once done, I’ll setup many more using TSML and get all the target partitions loaded, once this is good.

What do I need to look at, alter, or is there a better approach? I’m trying to make this simple, use SSIS pkg tools and minimal script, with each new month, some interaction/alteration to get the next target month to load, at this time no fancy automation.

Thank you


How to get restartability without using Checkpoints?

$
0
0

Hi All,

It takes too long to re-execute our SSIS 2012 packages when they fail several hours into execution.  I had problems with Checkpoints when I tried it years ago, so I'm wondering if there's some other way to support re-executing the package by starting at the Task immediately after the last successfully completed Task.

Any ideas or suggestions?

Thanks,
Eric B.


SSIS reusable transformations

$
0
0

Hi,

I am using SSIS 2015.   

I have a table which has 30 DATE columns. I want to apply transformation logic on these date fields. the logic is same but it has to be applied on all 30. Instead of writing the code again and again in derive transformation expression 30 times, is there a way to write kind of function, which when called by passing my date field as parameters returns me the expected result.

similar to this I have transformations for multiple data types. So is there a way to write re-usable transformation expression in SSIS

Thanks 

Create dynamic File Name using Destination Table Name in SSIS

$
0
0

Hi,

    I have a source OLEDB staging table . I multicast the data from staging to loadreadytable as well as flat file.

I have close to 1000 tables and hence 1000 flat file. so am creating new Flatfile connection for each of the tables.

Instead is there a way to dynamically create the path and name of flat files . And the names of the files should be name of my load ready tables.

Web service reference issue

$
0
0

Hi 

I am updating someone else's code in a SSIS package that downloads data from a webservice using a C# script task. Recently a few more attributes were exposed in that webservice that I now need to add to my SSIS package.

When I click on the "Update Service Reference" in the solution explorer, the service references gets updated and I can access the new attributes (the intellisense and object explorer all show the new attributes) but as soon as I build the project, it somehow still references an old file (probably reference.cs file) and can't find those new attributes and an throws error that the new attributes can't be found

Is there a step I am missing? I am using Visual Studio 2015 




Login failed for user 'NT AUTHORITY\ANONUMOUS LOGON'. Reason: Could not fin a login matching the name provided

$
0
0

I have created a dts package and deployed to SSIS Dev server, with project deployment model. The package executes a stored procedure in a database named TCUCDB, which resides in another server tcsqldev01.
I have the connectionmanager created with Integrated Security=True, and tested connection is ok. I have environment set for QA, and when I try executing the package, selecting the Environment "QA", and I can see the connection string in "Connection Managers" tab as Integrated Security=True. But on execution from SSMS (Service catalog->package->right click -> Execute), packages fails to run with the error message below, on report

Failed to acquire connection "TCUCDB". Connection may not be configured correctly or you may not have the right permissions on this connection.

I checked with DBA, and they confirmed error logged on access to tcsqldev01 is as below

"Login failed for user 'NT AUTHORITY\ANONUMOUS LOGON'. Reason: Could not fin a login matching the name provided"

Why the login using this user account, instead my windows account, even after setting Integrated Security=True? 

How can I resolve this issue, and make sure package is executed on my windows account?

The dts package execution report also confirm the caller is my windows account 'DOMAINNAME\sbalakrishnan'

Pivot/Unpivot in dataflow task SSIS

$
0
0

I have the data in following format:

MONTH	Number of Installs	STATION	          Month Name
Jul	0	                ALPENA	                Jul
Jul YTD	0                       ALPENA	                Jul
Jul	0	                BIG RAPIDS	        Jul
Jul YTD	0	                BIG RAPIDS	        Jul
Jul	0	                CADILLAC	        Jul
Jul YTD	14	                CADILLAC	        Jul


I need to convert it to following format:

YTD   MTD        STATION
0     0         ALPENA
0     0         BIG RAPIDS
14    0         CADILLAC

MTD column should show values of rows with abbreviated month name

YTD column should show values of rows with abbreviated month name and YTD

I can use pivot transformation, but it requires pivotKey values to be specified as Jul and Jul YTD. I need to avoid this because the MONTH column changes its values based on current month. Since this is july month, it has Jul and Jul YTD values. If it is August, it will have Aug and Aug YTD values. So pivot should happen dynamically rather specifying Jul and Jul YTD as pivotKey values.

Any ideas on how to do this? Thanks in advance




EDIT: Sorry for the table format.. I tried many times, it is not showing appropriately

SSIS: Could not find installable ISAM

$
0
0

Dear All,

I have office 2013, SQL server 2016 and visual studio 2017 installed. I need to connect to an OLEDB provider to read data from multiple sheets of .xlsx file. I have installed Microsoft Access database engine 15.0 32 bit olebd provider but I'm getting an error in the test connection. I'm not sure which version should be installed. 

Test connection failed because of an error in initializing provider. Could not find installable ISAM.

Please suggest.

Thank You

Julie


SINGLE CONNECTION WITH MULTIPLE DATABASE IN SSIS PACKAGE

$
0
0

i need your help in ssis. I have one server having 40 database. i have to use all the database in one connection means one oledb connection, i can change the database name on the runtime Is there any way

Please guide me with screen shot 

I have done through object but fail

40 database in one server, but do not look good if I take 40 connections from same server for each database  we are using azure database not sql server

Thanks

Reply soon

Rohitsethi123@gmail.com


Documentation explaining connection to SharePoint Online

$
0
0

Hello,

Can anyone point me at Microsoft documentation detailing the connection process, particularly authentication, from an SSIS package to a SharePoint Online (not on-prem) data source?

I've been experimenting with this for a while using the OData connector (it works but is difficult to protect the password) and believe that I must use the 'Microsoft Online Services Authentication' method in my Connection Manager, but I can't find an explicit reference to confirm this.

In using this authentication type I also believe that I must provide a username and password rather than some form of integrated security, but again I can't find an explicit reference to confirm this.

What I can find is https://docs.microsoft.com/en-us/sql/integration-services/connection-manager/odata-connection-manager?view=sql-server-2017 which implies what I've explained above but doesn't go into any detail or explicitly state that it is necessary to do it this way. There are a few tutorials explaining how to configure the OData connection to SharePoint Online but all seem to be happy storing a password in the package.

My dilemma is that I don't want to save a password in my package as it makes it difficult to deploy to multiple servers when they connect via different credentials, I don't want to save the password in a package configuration file as it's stored in plain text, I don't want to pass in a password from the SQL Job when calling the package as it's saved to the msdb. Therefore I'm stuck as to the best way to make the connection and not expose the password. I'm hoping that there is some documentation out there which I've been unable to find which explains the correct way to resolve this puzzle.

Is there anyway to connect to a SharePoint Online instance with some form of integrated security (MS Flow?) and if not how do I protect the password used in the package?

Thanks,

Mike

SSIS Job failed: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host, A connection cannot be made. Ensure that the server is running.

$
0
0

Hi All,

one of my Production SSIS job is failing with the below error. All of the required permissions are given for the account as far I know.

Some details of the environment: Windows Server 2008 R2, SQL Server 2008 R2 SP3, Disabled Win Firewall, Symantec Endpoint Protection

Microsoft (R) SQL Server Execute Package Utility
Version 10.50.6560.0 for 64-bit
Copyright (C) Microsoft Corporation 2010. All rights reserved.
 
Started:  6:21:40 PM
Error: 2018-07-04 18:22:46.21
   Code: 0x00000000
   Source: Process Full Process Full
   Description: An existing connection was forcibly closed by the remote host
End Error
Error: 2018-07-04 18:22:46.23
   Code: 0x00000000
   Source: Process Full Process Full
   Description: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
End Error
Error: 2018-07-04 18:22:46.23
   Code: 0x00000000
   Source: Process Full Process Full
   Description: A connection cannot be made. Ensure that the server is running.
End Error
Error: 2018-07-04 18:22:46.23
   Code: 0x00000008
   Source: Process Full 
   Description: The script returned a failure result.
End Error
DTExec: The package execution returned DTSER_FAILURE (1).

 Thanks!

Dynamic Excel Connection Manager with changing data types

$
0
0

Hi, this is my first question here...

I've created a dynamic excel connection manager to pull in data (to Visual Studio community 2017) from multiple spreadsheets before sending them to a SQL Server table.  My 'For Each File' loops works fine and my connection manager works fine before I make it dynamic.  However, when I make the connection dynamic that's when the package fails.  I've tried various things which all result in either error messages with the metadata and/or datatypes. 

I'm confident I know what the problem is - the spreadsheet datatypes are coming across differently from one spreadsheet to another.  It may not help that (ideally) I'm not picking up the data from row 1 but from row 19 onwards.

I'm wondering if I can force the datatypes in some way at the point of the connection (and then deal with the type conversion once the data is in SQL)?  Or do I need a wholly different approach?

Any thoughts would be much appreciated. 

VS2013 and SSIS packages running WRONG package

$
0
0

Hi,

I have an integration project with many SSIS packages.   I open one package, and then click "Start" to run that one package, then visual studio decides to pick another package in the project at random and run that one instead.  I have my machine and a server with VS2013 and they both do it.  Even when I right-click the package and choose "execute package", there is a 50% chance it will still run something else.  

Anyone else encountering this or know how to stop it from doing this?

Thanks!

How to compare 2 versions of SSIS package excluding the version details? (do not have any version system like tfs, etc)

$
0
0
How to compare 2 versions of SSIS packages excluding the version details? (do not have any version system like tfs, etc)

SSIS: Error: The Assertions OleDbSchemaGuid is not a supported schema by the 'Microsoft.ACE.OLEDB.12.0' provider.

$
0
0

Hello,

I have created a foreach loop with Microsoft office 12.0 access database engine to load data from multiple sheets of excel file. But i'm getting this error while running the package.

Thank you

Julie


SSRS Expression

$
0
0

Hi,

I need to display the title for 5 sections from my report such as below:

2 records are late
10 records due today, 7/10/2018
25 records due 07/11/2018
5 records due 07/12/2018
15 records due 07/13/2018

with the expression below, always give the error, i am not sure what i have wrong here, please help.

--EXpression ---

=CStr(CountDistinct(Fields!t_orno.Value & Fields!t_oset.Value)) + " records are " + 
  IIF(Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!sortGroupBy.Value, mydts)=1,"LATE" ,
     
  IIF(Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!sortGroupBy.Value, mydts)=2,"due today, "+
  Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!dt.Value, mydts),

IIF(Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!sortGroupBy.Value, mydts)=3,"due "+
  Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!dt.Value, mydts)

 IIF(Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!sortGroupBy.Value, mydts)=4,"due "+
  Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!dt.Value, mydts)

 IIF(Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!sortGroupBy.Value, mydts)=5,"due "+
  Lookup(Fields!pKey.Value, Fields!pKey.Value, Fields!dt.Value, mydts)
))

SSIS reusable transformations

$
0
0

Hi,

I am using SSIS 2015.   

I have a table which has 30 DATE columns. I want to apply transformation logic on these date fields. the logic is same but it has to be applied on all 30. Instead of writing the code again and again in derive transformation expression 30 times, is there a way to write kind of function, which when called by passing my date field as parameters returns me the expected result.

similar to this I have transformations for multiple data types. So is there a way to write re-usable transformation expression in SSIS

Thanks 

Sensitive property of conneciton strings in project param

$
0
0

Is it a security risk if my sqlserver is hosted on premise server and if I am setting the Sensitive value of connection string to 'False' in project managers.My SSIS package is set to Donotsavesensitive

Do i need to keep the Sensitive value set to always 'True'.

Thanks

Unpivot Columns to rows

$
0
0

Hi , I have some csv files where i need to unpivot columns to rows, I have no idea on number of files i will get from source so i need to loop the DFT and process the files. I was able to process the files perfectly and had a problem where the columns are different  for some of the files. SOme files have 5 min interval data and some files have 15 min data. i.e few files have 96 columns (15 min interval data) and few files have 288 columns (5 min interval) data. Can you please help me how do i automate this scenario. 

dynamic query

$
0
0

Hi,

I want my report to show the request date field and value when the company !=50, but the syntax below keep getting an error, can anyone please help?

Thanks

DECLARE @tableHTML  NVARCHAR(MAX) ;

SET @tableHTML =
   N'<head>' +
    N'<style type="text/css">h2, body {font-family: Arial, verdana;} table{font-size:11px; border-collapse:collapse;} td{background-color:#F1F1F1; border:1px solid black; padding:3px;} th{background-color:#99CCFF;}</style>' +
   N'<h2><font color="#0000ff" size="4">Work Order </font></h2>' +   
   N'</head>' +
N'<body>' +
N' <hr> ' +
N' ' +


    N'<table border="1">' +
    N'<tr><th>Work Order ID</th><th>Product ID</th>' +
   case when @companyID !=50  then N'<th>Request Date</th>'  end +
    N'<th>Expected Revenue</th>

    </tr>' +
    CAST ( ( SELECT td = wo.WorkOrderID,  '',
                    td = p.ProductID, '',
case when @companyID !=50  then td = wo.DueDate  end ,'',               
                    td = p.ListPrice 
              FROM WorkOrder as wo
              JOIN Product AS p
              ON wo.ProductID = p.ProductID
              WHERE DueDate > '2006-04-30'
                AND DATEDIFF(dd, '2006-04-30', DueDate) < 2 

              FOR XML PATH('tr'), TYPE 
    ) AS NVARCHAR(MAX) ) +
    N'</table>' ;

EXEC msdb.dbo.sp_send_dbmail @recipients='test@cdd.com',
    @profile_name = 'mypr',
    @subject = 'work order',
    @body = @tableHTML,
    @body_format = 'HTML' ;

                          
Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>