Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Script task that only processes files that have one of two names

June 7, 2016, 9:18 pm
$
0
0

I need to look in a directory that has 2 files but many iterations and find the current file for each filename

20160608020003_FULL_EXAM_EXTRACT
20160607020004_FULL_EXAM_EXTRACT
20160606020003_FULL_EXAM_EXTRACT

and 

20160606040000_ACC_REPORT
20160530040003_ACC_REPORT
20160406040000_ACC_REPORT

I need to find the current file of each filename

I have the below code that finds the current file but how to modify to

do what I need. 

  var directory = new DirectoryInfo(Dts.Variables["User::VarSourcePath"].Value.ToString());

            FileInfo[] files = directory.GetFiles();
            DateTime lastModified = DateTime.MinValue;

            foreach (FileInfo file in files)
            {
                if (file.LastWriteTime > lastModified)
                {
                    lastModified = file.LastWriteTime;
                    Dts.Variables["User::VarFileName"].Value = file.ToString();
                }
            }

↧
Search

pass comma separated value to ssis source query

June 8, 2016, 3:10 am
$
0
0

I am using a variable to pass value to sql server, source query

When I pass a single value it works fine, but when I use multiple values separated by  comma ',' it doesn't work.

There is no error, but no values gets picked from source.

If I type values 'A1','A2' in the query in the IN clause bracket directly, it works fine, but same value if I place in variable and link it to the query using '?' it doesn't work

Here I want to point out, when I use write the value directly in the source query I need to put it as 

SELECT        ID, Name, DateModified, Age
FROM            Account where Name IN ('Acc1')

But I pass through the Variable , I pass value without single quote for it to work.

I cannot use Sql server stored procedure/function, its a project constraint

 

↧
↧

SETS issue

June 8, 2016, 7:12 am
↧

return output parameter value from dynamic sql in execute sql task in ssis

June 8, 2016, 7:53 am
$
0
0

Hello,
I have a dynamic sql as follows:

declare @sql nvarchar(max) = ''

set @sql = 'SELECT fieldname + fieldname2 AS MyOutputValue FROM '
set @sql += @TableName + ' where ID = ' + @ID

Question:
How is it possible to return this MyOutputValue and assign it to my valriable in ssis. The variable in ssis is called vMyOutputValue

Thank you

↧

SQL 2016 RC3, SSIS package randomly stops processing

May 16, 2016, 1:12 pm
$
0
0

I have an SSIS package that runs every minutes. It's handling some sync logic between two systems. The checks are queries against the CDC tables to see what if anything has changed and then syncing those changes records (in denormalized form) to a second database. The job can run up to 5 days without any issues. Then at other times it locks up every few hours.  The exact location in the SSIS package varies and there are never any queries running against the database server etc.  It appears to be at the end of sequence containers.  Wondering if anyone else is having similar issues, or know of a way to figure out what is causing the problem.  Verbose logging is showing nothing of import that I can tell.

↧
↧

How to design ssis etl package for ssas cube process?

June 8, 2016, 10:50 am
$
0
0

Hello All,

My task is to create SSIS ETL package to process SSAS cube.

Can someone please help me to understand how to design ssis etl package for ssas cube process.

Please post some useful link or video.

Thanks

Thanks, Shyam.

↧

SSIS package with OData connection - can't connect

February 19, 2015, 1:51 pm
$
0
0

I need to read some data from a SP list. I have the OData connection manager setup. When I test the connection to my sharepoint site, it says it's successful. I added in a data flow task and then an OData Source. In the OData Source editor, I select the Odata connection manager. I want to use a collection. However, it says "Unable to load collection for the connection Odata Source, reason: Cannot open this connection"


Anybody know how to resolve this?


↧

how to Unzip to specific folder

June 8, 2016, 4:23 pm
$
0
0

Can someone help me with the argument on how to put the unzipped file to specific location.

Here's my Executable and arguments

Executable: C:\Program Files\7-Zip\7z.exe

Arguments: e "abc.zip" -aoa

This argument puts the unzipped files in same directory, how can I put it to specified location using arguments?

Thanks

sree

↧
Search

Query - dealing with XML files - Please help

June 8, 2016, 6:01 pm
$
0
0

Hi,

I have a table with xml file in one of the columns. Please find the DDL.

create table dbo.Src_data
(policyid int,
originalxml xml)

The rows are like below DDL. 

insert into dbo.Src_data values (1001,'
<session id="4" fragment="0">
   <properties dateModified="2015-02-27" id="p74B82ECE85254095"  context="Manager">
    <userName>XXXDomain\Manesar</userName>
   </properties>
 <data id="d6D7B8845441C0EB1">
  <policy id="1000">
    <EffectiveDate>2015-02-27</EffectiveDate>
    <Term>10</Term>
    <ExpirationDate>2016-02-27</ExpirationDate>
    <QuoteNumber>ABCD1234</QuoteNumber>
  </policy>
  <coverage id="1">
          <Type>abc</Type>
          <NoOfEmployees>5</NoOfEmployees>
  </coverage>
  <coverage id="2">
          <Type>mno</Type>
          <NoOfstudents>5</NoOfEmployees>
  </coverage>
  <coverage id="3">
          <Type>qrs</Type>
          <NoOfEmployees>50</NoOfEmployees>
  </coverage>
  <coverage id="4">
          <Type>xyz</Type>
          <NoOfStudents>40</NoOfEmployees>
  </coverage>
 </data>
</session>')


insert into dbo.Src_data values (1002,'
<session id="4" fragment="0">
   <properties dateModified="2015-02-27" id="p74B82ECE85254095"  context="Manager">
    <userName>XXXDomain\Manesar</userName>
   </properties>
 <data id="d6D7B8845441C0EB1">
  <policy id="1001">
    <EffectiveDate>2015-02-27</EffectiveDate>
    <Term>12</Term>
    <ExpirationDate>2016-02-27</ExpirationDate>
    <QuoteNumber>ABCD1234</QuoteNumber>
  </policy>
  <coverage id="1">
          <Type>abc</Type>
          <NoOfEmployees>43</NoOfEmployees>
  </coverage>
  <coverage id="2">
          <Type>mno</Type>
          <NoOfstudents>34</NoOfEmployees>
  </coverage>
  <coverage id="3">
          <Type>qrs</Type>
          <NoOfEmployees>32</NoOfEmployees>
  </coverage>
 </data>
</session>')

I would like to get the output like


PolicyID Coveragetype NoOfEmployees  
1001       abc           5
1001       mno           NULL
1001       pqr           50
1001       xyz           NULL
1002       abc           43
1002       mno           NULL
1002       pqr           32


Can you please help me?

↧
↧

Data Dump in EXCEL using SSIS 2008

June 8, 2016, 6:42 am
$
0
0

In the SSIS Pkg there are some DFT which perform some operation and after that 1 Stored Procedure is invoked from Oledb Source which write the data to excel   using EXCEL Destination .

The data is not getting written into the excel while executing the whole pkg, but while executing only the Data Flow Task to load the data into EXCEL it is working . Not sure what is going wrong.

xlsx is the file extension Please suggest


↧

Data flow task error

June 9, 2016, 3:54 am
$
0
0

Hello-

I have an OLE DB source with a SQL command running a SP inserting data into a staging table. I’m passing parameters and using a WITH RESULT SETS but when I run the package I get an error:

 “[OLE DB Source [52]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E14.

An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80040E14  Description: "EXECUTE statement failed because its WITH RESULT SETS clause specified 16 column(s) for result set number 1, but the statement sent 17 column(s) at run time.".”

The table the SP reads from has 16 rows but the staging table it transfers the data to has 17 the additional column is ActiveAuthorityStagingId and it’s a PK, INT, NOT NULL. Does anyone know away to resolve this error, I’ve been researching all morning and find a lot of discussions on this issue but can’t find any solution that solves my dilemma?

Below is the code and design I’m passing in my SQL command which is where the data is coming from:

Simple dataflow task

Code inside the source:

Destination mapping:

So I'm sure it's the ActiveSalesAuthorityID column problem is I'm not sure how to get around this issue as the column does not exist in the source only in the destination. Any pointers greatly appreciated!

↧

How to Delete Sheet1 from EXCEL without INTEROP and only JET driver is Avaiable

June 9, 2016, 4:03 am
$
0
0

How to DELETE the default sheet using C# , INTEROP driver is not available , and EXCEL is not installed in the box where data dump is happening. Creating Table and column on the runtime and populating in the new sheet .

Need to clean the default sheet 


↧

SSIS Ole db source as stored procedure in openquery with parameters

June 9, 2016, 4:04 am
$
0
0

Hi, all.

I am using stored procedures from linked server. I want to use the procedure in ole db source.
I wrote the query, which works in SSIS.
select ID,
LAST_NAME_ENG,
LAST_NAME_G,
FST_NAME_ENG,
FST_NAME_G,
CONTACT_ID,
BIRTHDATE,
OWNER_ID,
PARENT_ACCOUNT_ID,
GENDER_CODE,
LANGUAGE,
EMAIL,
POSITION,
STATUS,
EMP_TYPE,
MOBILE_PHONE,
PERSON_NUM,
IS_DELETED from openquery (linkedserver, 'exec [linkedserver].get_records @SESSION_ID = 12 , @SYSTEM = ''oCRM'', @ENTITY_NAME = ''CLIENT'' WITH RESULT SETS (([ID] [int] NOT NULL,
[IS_DELETED] [bit] NULL,
[CONTACT_ID] [int] NOT NULL,
[LAST_NAME_ENG] [varchar](50) NOT NULL,
[LAST_NAME_G] [varchar](50) NOT NULL,
[FST_NAME_ENG] [varchar](50) NOT NULL,
[FST_NAME_G] [varchar](50) NOT NULL,
[BIRTHDATE] [date] NOT NULL,
[OWNER_ID] [int] NOT NULL,
[PARENT_ACCOUNT_ID] [int] NULL,
[GENDER_CODE] [varchar](10) NOT NULL,
[LANGUAGE] [varchar](50) NULL,
[EMAIL] [varchar](100) NULL,
[POSITION] [varchar](100) NULL,
[STATUS] [varchar](50) NOT NULL,
[EMP_TYPE] [varchar](50) NULL,
[MOBILE_PHONE] [varchar](50) NULL,
[PERSON_NUM] [varchar](50) NOT NULL))');

I can use it in SSIS ole db source and successfully get required data. But in the next step there is the problem:
I need to pass the parameter to the @SESSION_ID from SSIS instead of '12'. And I cannot find the right way to do it.
There are a lot of advices to use dynamics sql and construct full query string with required parameter values and then exec it, but if I will do it - the SSIS couldnt get columns data from the dynamics query.
Are there ways to solve it?
Any ideas will be helpfull.

Thank you.

With regards, Yuriy.



↧
↧

TCP/IP SSIS rather than ftp

June 9, 2016, 3:39 am
$
0
0

Hi,

I am trying to move a file to an internal server via an SSIS package.

I cant use a move command so currently using ftp.

The ftp server is having problems so I need to find an alternative.

Is it possible to use TCP/IP ? and if so how ?

Thanks in advance

↧

Importing excel files with decimal values fails

September 19, 2013, 12:01 pm
$
0
0

In have made a package that loops over multiple workbooks and multiple sheets. In these files I have decimal values like:
48,95
95 (no decimal)

After import in DB and in the data viewer placed just after the Excel Source values are converted to:
4895.0000000000
95.0000000000

Somehow the decimal disappears, which means that imported value in some case are correct e.g. 95, but some are not e.g. 4895.

OLE DB Destination columns are set to decimal (18,10)

Connection string:
Provider=Microsoft.ACE.OLEDB.12.0;Data Source=[FileName].xlsx;Extended Properties="EXCEL 12.0;HDR=YES;IMEX=1";

I have tried to set output column on source to decimal. But this does not help me out.

Is there anyone that had have same issue?

Thanks.


↧
Search

execute permissions on integration tasks

June 6, 2016, 8:07 am
$
0
0

Hi,

I am executing an SSIS package using a c# application.  this SSIS packages extracts about 700K-2M records from a database and outputs to text or excel files.  the application allows the user to execute the package on demand.

This application uses these dlls:

Microsoft.SqlServer.ConnectionInfo

Microsoft.SqlServer.Management.IntegrationServices

Microsoft.SqlServer.Management.Sdk.Sfc

Microsoft.SqlServer.Smo

The package runs perfectly if the app is run under my login but fails when run with the user's login. (error message pasted below)

My hunch is that i need to grant execute permission on one or more of these dlls to the user account but i don't quite know how or where. Or is it some dcomcnfg permission?

Any help would be most welcome.

Regards

Habib




---------------------- Execution error ------------------

Failed to connect to server .\n   at Microsoft.SqlServer.Management.Common.ConnectionManager.Connect()

   at Microsoft.SqlServer.Management.Common.ConnectionManager.PoolConnect()

   at Microsoft.SqlServer.Management.Common.ConnectionManager.get_ServerVersion()

   at Microsoft.SqlServer.Management.Common.ServerConnection.get_TrueName()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices.Microsoft.SqlServer.Management.Sdk.Sfc.ISfcDomainLite.get_DomainInstanceName()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices.Key.GetUrnFragment()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcKeyChain.ToString()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcKeyChain.Microsoft.SqlServer.Management.Sdk.Sfc.IUrn.get_XPathExpression()

   at Microsoft.SqlServer.Management.Sdk.Sfc.Environment.GetObjectInfo(Object ci, RequestObjectInfo req)

   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.GetObjectInfo(Object connectionInfo, RequestObjectInfo requestObjectInfo)

   at Microsoft.SqlServer.Management.Sdk.Sfc.Enumerator.Process(Object connectionInfo, RequestObjectInfo requestObjectInfo)

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.GetSupportedResultTypes(ISfcConnection connection, Urn urn)

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.Initialize()

   at Microsoft.SqlServer.Management.Sdk.Sfc.SfcInstance.MarkRootAsConnected()

   at Microsoft.SqlServer.Management.IntegrationServices.IntegrationServices..ctor(SqlConnection sqlConnection)

   at HSalimCo.Pivots.DataExport.bkworker_DoWork(Object sender, DoWorkEventArgs e)

----------------------------------------


↧

Some times SqlServer Agent account failed but it has sysadmin access?

June 9, 2016, 7:40 am
$
0
0

Hi Experts,

I have checked everything it has access to db(sys.admin) not always only some time it fails

How to fix and trouble shoot it

Thanks in advance


ADKR

↧
↧

CDC source component: What actually happens behind the scenes during TFSTART?

May 12, 2016, 4:14 pm
$
0
0

Gurus,

I've come across a performance issue which I am having a hard time understanding.

To make a long story short, I have an SSIS package that uses the CDC components.  This has been active for 4 years; working flawlessly 24/7-- populating millions of rows within 10 minutes. Monday, this package slowed to a crawl, taking 20 hours to write out 169,000 rows. This finished successfully with a TFEND state dropped into the dbo.cdc_states table.

Here's what happened last Sunday: there was an upgrade to the source server where the CDC tables live.  The upgrade took mssql from 2008 R2 to 2012.  

On the current run, SSIS has picked up the range and started processing, dropping a TFSTART state record into the dbo.cdc_states table--I'm 2 hours in and SSIS is still running...and running ... and running...

What is the component actually doing at this stage? The documentation reads:

TFSTART : This is the state on a non-initial run of the trickle feed update package, after the GetProcessingRange operation call to the CDC Control task.  This indicates that a regular CDC run is started but has not finished or has not yet finished, cleanly (MarkProcessedRange).

https://msdn.microsoft.com/en-us/library/hh758667(v=sql.110).aspx

What is meant by "trickle feed" in this context? 


↧

SSIS 2012: Update statement in a SQL Execution task did not appear to complete

November 17, 2015, 7:13 am
$
0
0

Fellow Techies--

I have an SSIS package that has reliably executed for over 3 years.  There is a SQL Execution task that issues a single update statement which updates a bit flag from 0 to 1.  Normally, this update statement processes against 1-1.5 million rows, however, in a recent scenario (read: reload/rebuild), the processing was against 900 million rows.  The SSIS package completed successfully, however, not all of the rows received the update.  About 1/4 of them did.  The process has been executed since then under its usual load (1-1.5 million rows) with all the rows receiving the update.

Even though this was a reload, it did not pass through a different workflow for the flag setting.  I am trying to determine what may have caused an incomplete update.  Does anyone have any suggestions on where to look?


↧

No column information was returned by the SQL command while accessing the data from a cube.

June 9, 2016, 8:03 am
$
0
0

Hi All,

I have created an SSIS Package in 2012, it reads the data from and 2008 Analysis Services Cube. I have 4 ForEach Loop Containers nested within one another. I am building dynamic MDX Query with the values I am getting from all the Execute SQL Tasks prior to each of the ForEach Loop Container and within the last ForEach Loop Container, it connects to the SSAS Cube and loads the data into SQL Server 2012 table.

Everything is working as designed, but for some of the conditions there is no data available within the cube and during that time package is failing, giving the below error.

"[OLE DB Source [100]] Error: No column information was returned by the SQL command.
[SSIS.Pipeline] Error: "OLE DB Source" failed validation and returned validation status "VS_NEEDSNEWMETADATA"."

In order to overcome this, I have set up the Event Handlers and set the Propogate value to False. Even then it is failing, I have run short of ideas on resolving this, could you please help me.

Thanks
Vineesh

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>