Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

How to get only new lignes and update lines in the next Extraction

November 18, 2016, 5:19 am
$
0
0

Hi,

Every day i do an extraction in flat file,My source is my my sql server database and i have a query with some join and my destination is a flat file.

What i want to do is : in each extraction i want to get only the new lines and the update lines for example :

Today the query that i do extraction return me the line as bellow :

id           Subproduct            QTE
1             Laptop               1000
2             Computer             1800
3             Phone                2000

The flat file now contains 3 lines

and Tomorrow the query that i do  extraction return me the line as bellow :

id           Subproduct            QTE
1             Laptop               1000
2             Computer             1800
3             Phone                3000
4             Other                5000

I want my flat file contains only the two last line because i have an update in Quantity for a product and i have a new line

How can i do this mechanism with SSIS. i would a solution using only sql query if it's possible

Thanks for help !!!

↧
Search

SQL job never finishes

November 18, 2016, 5:56 am
$
0
0

I have an ssis package that writes to a log file as the sql agent job is finishing up.  It's basically,

insert tablename values ('The job is done',getdate())

The package is inserting the data but every day or so the package / job fails to finish.   If I look in activity monitor is shows me the job is still running and I can't stop the job.  I have to restart the SQL Agent to proceed.   I know the job is successful because the data is in the table. 

SQL 2014 SP2 enterprise edition running on the same server as the integration services.  The connection manager uses the SQL 11 Native Client.  

--Burt King

↧
↧

SSIS 2016 SP1 published projects scheduled through SQL Agent return a warning about opening global shared memory after upgrade to SP1

November 18, 2016, 12:15 pm
$
0
0

Greetings,

After upgrading our SSIS 2016 Enterprise Edition server to SP1, we have begun receiving the following warning during each package execution to deployed packages, scheduled to run with SQL Agent:

Warning: Could not open global shared memory to communicate with performance DLL; data flow performance counters are not available.  To resolve, run this package as an administrator, or on the system's console.

This behavior did not occur prior to the SP1 installation. Does anyone have any idea how we can resolve?


↧

Getting rid of decimal and 0 in flat file source.

November 18, 2016, 11:23 am
$
0
0

Hi All,

Image 1 shows the record of a column from source file which is a csv file. when i created a flat file connection to read this file in SSIS, the output i get is as shown in image 2. 
when the value is a whole number in source file as 3414, why when i create a flat file connection the output is 3414.0. why SSIS is adding extra decimal  and 0 to the value. The data type is string, when i change this data type to int in advance editor, it works but i dont want to change data type to int. Is there any way we can remove extra 0 and decimals without changing the data type.

Image 1:

Image 2:

 

↧

execution error in ssis package

November 19, 2016, 3:43 am
$
0
0

while transfering data from flat file source(text file) to the excel destination following error has occured:

[Connection manager "Excel Connection Manager"] Error: The requested OLE DB provider Microsoft.Jet.OLEDB.4.0 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode. Error code: 0x00000000.
An OLE DB record is available.  Source: "Microsoft OLE DB Service Components"  Hresult: 0x80040154  Description: "Class not registered".


↧
↧

sysssislog nice OnError message column name and row number

November 8, 2016, 8:39 am
$
0
0

Hi,

I am currently logging error from my flat file source to standard sysssislog table.

The logging is very nice. It includes the column name and row number on which the error occurred, for example An error occurred while processing file "c:\test1.csv" on data row 3. 

Am I able to log the same information when using row redirect?

Thanks,

Ben

Mr Shaw... One day I might know a thing or two about SQL Server!

↧

Send and receive SOAP request through SSIS

November 16, 2016, 9:42 pm
$
0
0

Hi,

  Please find my below requirement and suggest me the best ideas. Before that,I am new to SSIS and Webservice integrations.

  • I have a Webservice URL (.wsdl) that requires input as SOAP request which in return provides response as an XML.
  • I need to read the XML output and store it my DB

   My two query is:

  • I am not able to use Webservice task as the limitation in Input type. Hence decided to use a script comment in DFT. How could I design the logic in script comment?
  • How should I store and read the output XML and parse it in my DB?

  Please let me know if any other better suggestion other than SSIS.

Go Green.Save Earth.



↧

Data Sync - SQL DB 2016 to Azure Datawarehouse

November 16, 2016, 1:32 pm
$
0
0

Hi,

We are planning to build SSIS packages that will sync tables from SQL 2016 DB to Azure DW. Only recently, we found out that MERGE is no longer applicable in Azure so our team is trying to figure out alternative ways to sync the data equivalent to MERGE without impacting the performance of the process. Kindly share your solutions and how we can implement these solutions.

Thanks in advance.

↧
Search

SSIS Package to zip folders

November 14, 2016, 8:18 pm
$
0
0

Hi All,

I have a package that extracts images from my db and saves them in the file system in different folders (determined by user name.) i.e. the package creates folders called 'user name', and then saves the images associated with that user name into the appropriate folder.

The package design is per the below:

I wish to have the package zip the destination folders. how can I achieve this?

Please assist,

Many thanks

Regards

Zimiso

↧
↧

The process exit code was "1" while the expected was "0"

November 4, 2016, 2:10 pm
$
0
0

I have an ssis package that calls an "execute process task" which connects to an sftp site and pulls files. The package works fine when I run it manually; however, it errors (the process exit code was"1" while the expected was "0") when executed from a SQL job.

Francis, MCTS: BI 2008, MCITP: BI 2008 Stay Thirsty My Friends

↧

SSRS Report call in SSIS

November 9, 2016, 12:30 pm
$
0
0

hi how i can generate the report uisng ssis?

↧

OnError Event not firing if executing package from job or SSISDB

November 20, 2016, 5:30 pm
$
0
0

Hi I create a simple package with event handlers : OnPreExecute, OnError and OnPostExecute.

It works fine if I execute the package on SSDT (2013) (all 3 fired, if having error)

But but if I deploy the package on SSISDB and execute it there, the OnError event does not get fired (only Pre and Post fired, even having the error)

The same thing, if I execute the package from a job, the event does not get fired either(only Pre and Post fired, even having the error).

I did try

1 - DelayValidation : true/false - both do not work

2 - DisableEventHandler : true/false - both do not work

Combine all possibilities of 1 and 2 - do not work either

Do you have any idea

Thanks

 

↧

Excel Source, Failed to retrieve long data for column (Unicode Text stream Data Type)

November 20, 2016, 9:58 am
$
0
0
I am Exporting Excel File data(passing the file names dynamically) from a folder into another Excel File. I have done all the proper conversion including handling the long data columns. But at the excel source connection I am having "Failed to retrieve long data for column" I have tried different soon like increasing the size of defaultDefaultMaxRows to 20000 also I have increased the size of DefaultBuffersize, but I am still getting the error, also specified new temp locations for BLOB and buffer. But still having the same error "Failed to retrieve long data for column". is there any thing I am missing Please

↧
↧

how to send SSIS errors through emails.

November 20, 2016, 2:40 am
$
0
0

Hi,

Can any one suggest how to send errors in SSIS via Email using SMTP Server and pls suggest how to create SMTP server also.

Thanks

Ipsita

↧

SSIS -->Multiple clients-->Environment variables-->SQL Agent--> Unable to set for multiple clients

November 13, 2016, 5:54 pm
$
0
0

hello gurus, 

I have a situation where i deploy 1 version of SSIS package in (dev,qa,stage,prod) servers. But i have to run the ssis package for more than 5 clients. each client has different database source connections/servers to pull data from.

I tried to create a environment variable for the ssis package by five different conn strings one for each client. i opened the ssis package and set the references to pick the parameter from the created environment variable for Client A by default.

Client A - JOB

i scheduled ssis package inside the SQL Agent jobs, and assigned the environment variable, i could successfully set for client A which works good

Client B - Job

But problem happens when i create a second sql agent job to schedule the ssis package. i go inside to set the client B, but there are no option for me to choose the environment variable now. it automatically got assigned CLIENT A conn string environment variable. there is no drop down, or multiple values to pick for CLIENT B or client c etc

Can someone help me ? is there any other way to create environment variable and use the SQL jobs?

PS: I dont find a separate forum thread for SQL server Agents, so i narrowed down to this SSIS forum. If the moderator feels, this forum is not the right one, please feel free to move my thread.

Thanks.

Regards, KarthikShanth. "Mark as Answer" if this helps you! <p style=""><a target="_blank" href="http://feeds.feedburner.com/~r/karthikshanth/~6/1"><img src="http://feeds.feedburner.com/karthikshanth.1.gif" alt="Shanth's techwitz" style="border:0"></a></p> <p style=""><a href="http://feedburner.google.com/fb/a/headlineanimator/install?id=91i2ausjr5mqq9fu4im39uvnvo&w=1" target="_blank">↑ Grab this Headline Animator</a></p> Disclaimer: The contents, I write here are my personal views, not the view of my employer and anyone else.

↧
Search

Getting latest file(s) from directory while checking for latest file already imported

November 14, 2016, 6:30 am
$
0
0

Hi all

i am currently in the middle of an SSIS project which involves grabbing the latest files from a directory and processing them accordingly.  The process is as follows:

  • Any files that are imported into the database at the start of the day are logged into a table - detailing name of file and import date
  • SSIS package will then fire via a SQL Agent job which checks for new files every 30 minutes
  • Any new files that have a created date later than the latest import date in the import log table is then imported.
  • When files are imported, they stay where they are - they are not moved to an archive folder


My question is how do I get a list of files to import looking at the created dates of files in a specific directory?

I also need to get the name of the file(s) from the directory so I can import into the log table.

For info, the directory name will be stored in a package-scoped variable - as will the latest file import date for the date check.

Any questions, please let me know.

Thanks.



↧

Create a empty text file using script task C#

November 8, 2016, 8:14 pm
$
0
0

Hi All,

I have a small requirement , I want to create one trigger flat file  with the massage inside file like "Extract generated successfully"

Trigger filename will be like Extract_YYYYMMDD_HHMMSS_trigger.txt

This task will be last step of the package.

do we have any c# script which can be used to achieve above requirement.

Thanks in Advance

Thankx & regards, Vipin jha MCP

↧
↧

Getting errors in report while setting the connection string dynamically.

November 3, 2016, 7:24 am
$
0
0

As a requirement I need to encrypt the connection string for OLE DB connection for Oracle provider.
Since SSIS does not support encryption/decryption for Xml configuration file, I have followed the following steps:
1. Encrypt the connection string in SQL server using Self certificate and Symmetric key (Link)
2. Create variable X and assign the encrypted connection string value to it.
3. Create OLEDB connection manager and use the expression to set the connection string by using X variable.
4. As a first component in the package, use Script component, in which decrypt the connection string and set to the variable Xml

It is working fine from Visual Studio 2015.
When I deploy to SQL Server 2012 (SP3), even though package is running successfully I am getting error in the report log and it says the package failed.

Attached the screen shot for reference.

Is there any other best approach to encrypt/decrypt the connection string.

Please help for the same.

Syed

↧

Migrate SSIS package from SQL Server 2014 to SQL Server 2016

November 2, 2016, 3:43 am
$
0
0

Hi ,

We are migrating SSIS packages from SQL Server 2014 to SQL Server 2016.  I'm using SSISUpgrade.Exe to upgrade my packages.

I have two questions :

1) I see provider name as  'Provider=SQLNCLI11.1;Integrated Security=SSPI' which is same as for SQL Server 2014 . What is correct name for SQL Server 2016 provider name for SSIS and should this be updated in all packages in SSIS.

2) When I open a solution after upgrading the packages in folder using Visual studio 2015 , I see that project opens with name as it is still in 2014 version ( Ex ProjectA(SQL Server 2014) ) . how can I upgrade packages in such a way that when I open a solution file , I should be able to open and see all packages when project in 2016 version .

any help is appreciated.

↧

SSIS job error

October 31, 2016, 2:38 pm
$
0
0
I have a Nightly job running, which fails on n off with a same reason. primary key violation.
I queried source SP to confirm if there is a dup record, no trace of dup records from source.
Source : Sybase table
Destination : SQL table
Mode of connection: Linked server in SQL 2008 SSMS.
Description/Situ - This package is failing in PROD on and off with same reason. BTW this is Truncate and load table package.
What I did was, created a new package with same source and mapped to test env DB. so now I have same package in Prod and test, with different destinations. funny thing happened today was, jobs success in Prod, failed in test with pk violation.

below is the errors, any thoughts pls let me know. Thank you all in advance.

Error Message:Code: 0xC0202009     Source:  Customer_Detail OLE DB Destination [16]     Description: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.  An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 10.0"  Hresult: 0x80004005  Description: "The statement has been terminated.".  An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 10.0"  Hresult: 0x80004005  Description: "Violation of PRIMARY KEY constraint 'PK_Customer_Detail'. Cannot insert duplicate key in object 'dbo.Customer_Detail'. The duplicate key value is (47186).".  End Error  Error: 2016-10-31 00:30:39.32     Code: 0xC0209029     Source:  Customer_Detail OLE DB Destination [16]     Description: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "input"OLE DB Destination Input" (29)" failed because error code 0xC020907B occurred, and the error row disposition on "input "OLE DB Destination Input" (29)" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.  End Error  Error: 2016-10-31 00:30:39.36     Code: 0xC0047022     Source: Customer_Detail SSIS.Pipeline     Description: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "OLE DB Destination" (16) failed with error code 0xC0209029 while processing input "OLE DB Destination Input" (29). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure.  End Error  DTExec: The package execution returned DTSER_FAILURE (1).  Started:  12:00:00 AM  Finished: 12:38:59 AM  Elapsed:  2338.5 seconds.  The package execution failed.  The step failed.

ss

↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>