Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Component Script is only reading 1 row in the oracle data reader

$
0
0

I am using  Foreach ADO enumerator that brings back a small amount claim id's into a data flow. I then run those claims id's in the script component below which is a datasource. Each claim can have from 0 to 5 rows of data. it works find for zero and one row of data. It doesn't work for multiple rows of data. See Script below - I am probably doing something wrong in the CreateNewOutputRows section. 

I get the following error when it runs into a claim with multiple rows. It writes the first row to the distinction - and then stops.

I don't believe this is actual error - since there are no columns set to Int32

   at Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSBuffer100.AddRow(IntPtr ppRowStart)

   at Microsoft.SqlServer.Dts.Pipeline.PipelineBuffer.AddRow()

   at ScriptMain.CreateNewOutputRows()

   at UserComponent.PrimeOutput(Int32 Outputs, Int32[] OutputIDs, PipelineBuffer[] Buffers, OutputNameMap OutputMap)

   at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers)

Script is below

#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Data.SqlClient;
using System.Collections.Generic;
using System.Diagnostics;
using System.Data.OracleClient;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    string Claimid = string.Empty;
    private IDTSConnectionManager100 connMgr;
    private OracleConnection oracleConn;
    private OracleCommand oracleCmd;
    private OracleDataReader oracleRdr;
    public override void AcquireConnections(object Transaction)
    {
        connMgr = this.Connections.ORCL;
        oracleConn = (OracleConnection)connMgr.AcquireConnection(Transaction);
    }
    public override void PreExecute()
    {
        base.PreExecute();
        /*
         * Add your code here
         */
        //string claimid = Convert.ToString(dts.Variables["User::ClaimID"].Value);
        //this.VariableDispenser.LockForRead("User::ClaimID");
        Claimid = this.Variables.ClaimID;
        //MessageBox.Show (Claimid);
               string sql = "SELECT CMC_SBSB_SUBSC.SBSB_ID, CMC_CLED_EDI_DATA.CLED_EXT_REF, CMC_CLED_EDI_DATA.CLED_TRAD_PARTNER, CMC_CLED_EDI_DATA.CLCL_ID, CMC_MECB_COB.MCRE_ID, CMC_MCRE_RELAT_ENT.MCRE_NAME, CMC_MECB_COB.MECB_EFF_DT, CMC_MECB_COB.MECB_TERM_DT, CMC_MECB_COB.MECB_POLICY_ID, cast(CMC_MECB_COB.MEME_CK as nvarchar2(25)) FROM    fac.CMC_CLED_EDI_DATA LEFT JOIN fac.CMC_CLCL_CLAIM ON fac.CMC_CLED_EDI_DATA.CLCL_ID = fac.CMC_CLCL_CLAIM.CLCL_ID LEFT JOIN fac.CMC_SBSB_SUBSC ON fac.CMC_CLCL_CLAIM.SBSB_CK = fac.CMC_SBSB_SUBSC.SBSB_CK LEFT JOIN fac.CMC_MEME_MEMBER ON fac.CMC_SBSB_SUBSC.SBSB_CK = fac.CMC_MEME_MEMBER.SBSB_CK LEFT JOIN fac.CMC_MECB_COB ON fac.CMC_MEME_MEMBER.MEME_CK = fac.CMC_MECB_COB.MEME_CK LEFT JOIN fac.CMC_MCRE_RELAT_ENT ON fac.CMC_MECB_COB.MCRE_ID = fac.CMC_MCRE_RELAT_ENT.MCRE_ID WHERE fac.CMC_MECB_COB.MCRE_ID In ('10542', '11593', '11605', '11610', '11746', '11753', '19055', '19283', '19302', '19977', '20028', '20031', '20211', '20241', '20501', '20636', '20894', '20911', '24336', '24658', '20633', '27759', '27762', '24349')   AND  fac.CMC_CLED_EDI_DATA.CLCL_ID = '" + Claimid + "'";
        //MessageBox.Show(sql);
        oracleCmd = new OracleCommand(sql, oracleConn);
    }
    private string ToString(object sQLCommand)
    {
        throw new NotImplementedException();
    }
    public override void PostExecute()
    {
        base.PostExecute();
        /*
         * Add your code here
          sqlReader.Close(); 
                   
         */
        oracleCmd.Dispose();
    }
    public override void ReleaseConnections()
    {
        connMgr.ReleaseConnection(oracleConn);
    }
    public override void CreateNewOutputRows()
    {
        /*
            Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
            For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
       */
        oracleRdr = oracleCmd.ExecuteReader();
        while (oracleRdr.Read())
        {
            {
                MyOutPutBuffer.AddRow();
                MyOutPutBuffer.SBSBID  = oracleRdr.GetString(0);
                MyOutPutBuffer.CLEDEXTREF = oracleRdr.GetString(1);
                MyOutPutBuffer.CLEDTRADPARTNER = oracleRdr.GetString(2);
                MyOutPutBuffer.CLCLID = oracleRdr.GetString(3);
                MyOutPutBuffer.MCREID = oracleRdr.GetString(4);
                MyOutPutBuffer.MCRENAME = oracleRdr.GetString(5);
                MyOutPutBuffer.MECBEFFDT = oracleRdr.GetDateTime(6);
                MyOutPutBuffer.MECBTERMDT = oracleRdr.GetDateTime(7);
                MyOutPutBuffer.MECBPOLICYID = oracleRdr.GetString(8);
                MyOutPutBuffer.MEMECK = oracleRdr.GetString(9);
            }
            MyOutPutBuffer.SetEndOfRowset();
        }
    }
                   
}


Losing code

$
0
0
We have a couple of projects in the integration services catalog, both containing packages that contain vb script tasks. I did an export of them from the integration services catalog into ispac files and tried to pull them into visual studio 2015.    they both pulled in, but when opening the script task we noticed that it was converted to c# and was just a shell.    After that, I pulled the projects directly from the integration services catalog using visual studio and the integration services import project wizard.    after doing this, I was able to open one of them and the script task was vb and was complete, but the other was still a C# shell as before.      The server instance is sql 2016.     We have the code saved in source control, but we are having a prod issue and were wanting to pull the code from prod and verify what is actually running to make sure it was what we thought it was.    Any idea why it's losing the vb script task and converting to a c# shell?       

if we set file group auto growth to unlimited what is the problem in sql

$
0
0

Hi,

if we set file group auto growth to unlimited what are the problems will raise,what happen my scenario means I got critical alert from SCOM ,it's says that Could not allocate space for object in database because the File group is full.

  When i went DB see that DB auto growth options max size of file 371 MB they restricted , after that i changed toUNLIMITED.Now it's working fine but my doubt is is there any problem will occur moving forward?I hope will 

Regards,

Niranjan B

How best to send table results in an email?

$
0
0

Hi All,

We use SSIS and SQL 2012 for our data warehouse and ETL system.  Most of our SSIS packages include email tasks to let somebody know a process has completed.  I want those emails to include a very small table of any notable data.  (For example, we have validation queries that execute automatically and any with the "result" column = FAIL need to be in this email.) 

What's the best way to do this?  I'd like to build a repeatable process that can be used for any email notification.  I created a sproc that takes a query string as an input parameter and then executes it, putting the results into a temp table, then it steps through sys.columns to figure out how many columns exist, etc., etc.  It's a lot of dynamic SQL being executed and the problem is it would have to exist on every server or else the query couldn't be executed, so that was a time-consuming failure.

I considered that perhaps passing a table variable to a stored procedure would be easier.  That sproc could give the HTML back to SSIS as an output parameter.  That sproc would then only need to exist in one place and could be called from SSIS.  This seems like a viable option, but I haven't tried it yet.  Again, I think it would be a lot of dynamic SQL, not to mention difficult for less-experienced developers to maintain.

I've also considered using SQL's dbmail instead of SSIS mail tasks.  Would you suggest this method over passing a table variable to a sproc that returns HTML?  (Or possibly both together?)  I see pages online that take a query and convert it to an XML format of HTML result, but I can't seem to get this into a varchar.

Thanks,
Eric B.

Build SSIS code

$
0
0

For web or windows apps building the code makes it harder to decrypt the code. I need to be able to send the SSIS 2016 package code for deployment (project type package to deploy to SSISDB). If I send ispac file this can be easily imported in VS, is there any way to build SSIS code in order to upload to catalog by DBA (other than password protected of course).

Thank you,

Visual Studio 2017 SSIS: Orcale as Source

$
0
0

Hi,

I need to develop a SSIS package to load the data from oracle source and post the files on Azure Data lake.

For that, I am using VS 2017. But I am not able to create the successful connection with oracle db. could you please guide what is the correct procedure or is there any tutorial for making successful connection?

Thanks in advance.

Regards,

Gohar

file excel matrice source

$
0
0

hello , please how can i import a matrice from an excel file in ssis if not how can i solve this problem ( my source file is an axcel file witch containt a matrix ) 

thank you 

How to load multiple excel(.xlsx) files with similar but not equal file names into a SQL table using SSIS Package?

$
0
0

Hi All,

Thanks for your help in advance.

I'm wondering if there is a possibility to load multiple files that has something in common in the file name.

All the files has the same columns and I need to upload them into the same table.

However the name is not same in all the files, but the file name contains a word in all the file names


For example:

Hello_1.xlsx

Hello_2.xlsx

Hello_3.xlsx


is it possible to load files where filename contains "Hello"? in the same path

I'm open to hear any other suggestion or idea

Thanks


Who will be announced as the next SQL Server Integration Services Guru? Read more about January 2019 competition!!

$
0
0


What is TechNet Guru Competition?

Each month the TechNet Wiki council organizes a contest of the best articles posted that month. This is your chance to be announced as MICROSOFT TECHNOLOGY GURU OF THE MONTH!

One winner in each category will be selected each month for glory and adoration by the MSDN/TechNet Ninjas and community as a whole. Winners will be announced in dedicated blog post that will be published in Microsoft Wiki Ninjas blog, a tweet from the Wiki Ninjas Twitter account, links will be published at Microsoft TNWiki group on Facebook, and other acknowledgement from the community will follow.

Some of our biggest community voices and many MVPs have passed through these halls on their way to fame and fortune.

If you have already made a contribution in the forums or gallery or you published a nice blog, then you can simply convert it into a shared wiki article, reference the original post, and register the article for the TechNet Guru Competition. The articles must be written in January 2019 and must be in English. However, the original blog or forum content can be from beforeJanuary 2019.

Come and see who is making waves in all your favorite technologies. Maybe it will be you!


Who can join the Competition?

Anyone who has basic knowledge and the desire to share the knowledge is welcome. Articles can appeal to beginners or discusse advanced topics. All you have to do is to add your article to TechNet Wiki from your own specialty category.


How can you win?

  1. Please copy/Write over your Microsoft technical solutions and revelations to TechNetWiki.
  2. Add a link to your new article on THIS WIKI COMPETITION PAGE (so we know you've contributed)
  3. (Optional but recommended) Add a link to your article at the TechNetWiki group on Facebook. The group is very active and people love to help, you can get feedback and even direct improvements in the article before the contest starts.

Do you have any question or want more information?

Feel free to ask any questions below, or Join us at the official MicrosoftTechNet Wiki groups on facebook. Read More about TechNet Guru Awards.

If you win, people will sing your praises online and your name will be raised as Guru of the Month.


PS: Above top banner came from Vimal Kalathil.



Dave Rendon - wikiazure

Foreach Loop - Move and Copy from within the same Container with no scripting

$
0
0

Main question: Can I have two separate File System Tasks within the same Foreach Loop container?

My requirement is: Move n number of files from source directory to an archive (dated) directory. Then copy all files from archive directory to an import directory.  

The first part of the requirement works successfully. When I add second File System task to container in order to perform the copy, the process fails.

I can not change the file names - they must remain the same.

support on ODATA connector in SSIS using Multi factor authentication

$
0
0

how to connect with project online data using the ODATA connector it hasmulti factor authentication..Because I didn't find any option to use the multi factor authentication and it is giving error when I am trying with basic authentication.

Please help me is there any way to connect ODATA connector using multi factor authentication, if not suggest any work around approach to load the Microsoft project online data into Sql server.

looking for a solution that is really dynamic

$
0
0

Hi we run 2016 enterprise. We have an ssis apparatus that thru metadata can load consecutive waves of 9 simultaneous source to destination data loads. As soon as the longest running load of wave x is done, wave x + 1 begins.

I don't like that any one wave blocks the next wave until its longest running load finishes.  I'd rather see a solution that starts up a load when a slot becomes available (we currently limit ourselves to 9 simultaneous slots) and all the prereqs for that task have been satisfied.  I assume 9 is a magical number for one ssis pkg so perhaps there is a solution out there that pushes the server to more than 9 by enlisting multiple pkgs.  Out loads are mostly incremental so too much traffic at one time especially on the same destination db shouldn't be a problem but we may want to limit some loads from running at the same time.  

Has anyone seen such a solution in ssis?  Or even part of such a solution?

Migrate database synced from CDC db to another server

$
0
0

Running 2016 SP1 standard. I have a database that gets synced hourly from a CDC enabled database. I need to move this database to another server/instance. Just to confirm this is the destination database receiving incremental data not the CDC enabled database (which does not change server). This destination database gets refreshed by a SSIS package using standard SSIS CDC tools: control flow, data source, splitter. The SSIS package is deployed to catalog on original ETL/database server.

In order to move this database and ETL to another server/instance, I see:

Before migration: deploy SSIS package to new server catalog, reconfigure environments/parameters, keep job disabled.

Actual Migration: stop hourly agent job on original server, restore destination database to new server and start sync agent job on new server.

Please let me know if you have any guide, hyperlink similar to this.

Thank you,

truncate load data in to target

$
0
0

Hi 

I have a scenario to load data from flat file to target on a daily basis. I would need to truncate and load the data, before loading I need to check if the current data is good then after truncate the target and load the new data. The need for data check is any issues with the data then target table shouldn't be empty

thanks

Using Visual Studio 2015 and TFS for SSIS projects

$
0
0

Hello,

We plan to start using Visual Studio 2015 and TFS soon, both new to our BI group.  Should I expect any problems with SSIS projects or can I expect it to work just like database projects?  It seems like I read somewhere that TFS doesn't handle SSIS projects correctly, but I have no experience one way or the other.

Thanks,
Eric B.


SSIS loading .csv flat files using For Each Loop Container : Execute SQL Task expression to populate the audit table with Row Count

$
0
0

I am loading many .csv files into a SQL Server table using SSIS (Visual Studio 2017).

I am using For Each Loop Container.  I have two user variables - FileName (string) and RowCnt (int)

I have developed an audit table (dbo.FEfiles) to track each file loaded along with the number of records in each file.

Audit table has two columns:

1) FileName(varchar(20)

2) RowCount(int)

After the setting up of the Data Flow Task inside the For Each Loop Container, I have an Execute SQL Task that connects to my audit table.

In this Execute SQL Task, in the Expression tab, under the SQLStatementSource, I have the following expression to get the file namealone:

"INSERT INTO dbo.FEfiles (FileName)  SELECT  '" + @[User::Filename] + "' "

This parses correctly when I evaluate the expression, I get:

INSERT INTO dbo.FEfiles (FileName)  SELECT  'filename.CSV' 

Now, how do I add the variable for the RowCnt, which has integer data type ?

I tried many options such as the one below:

"INSERT INTO dbo.FEfiles (FileName,RowCount)  SELECT  '" + @[User::Filename] + "', + (DT_WSTR, 12) @[User::RowCnt] "

When I evaluate the expression, I get the following:

INSERT INTO dbo.FEfiles (FileName,RowCount)  SELECT  'filename.CSV',+ (DT_WSTR, 12) @[User::RowCnt] 

I need to get INSERT INTO dbo.FEfiles (FileName,RowCount)  SELECT  'filename.CSV',0  instead 

Can you kindly help me in getting the right expression ?

I even watched this video, but not able to figure out:

https://www.youtube.com/watch?v=Um7tDy9jZRs








SSIS source code security

$
0
0

Hi ,

we will be creating and deploying SSIS package in the client environment but the source code should not be viewed by them.How can i we approach for this one.Could you please help me.

Regards,

Niranjan B


SSIS source code security

$
0
0

Hi,

we will be creating and deploying SSIS package in the client environment but the source code should not be viewed by them.

Means they can execute the package only but no need to open designer.

we will do protection level EncryptAllWithPassword but asking password package executing time also   .

My requirement is client can able to execute the package but no need to give permission on open package in designer.

Could you please help for this issue.

Regards,

Niranjan B


اسرع خدمه اصلاح نورج || 0235695244 || اعطال ثلاجات نورج || 01225025360 ||

اصلاح غساله ويرلبول بالضمان @ 0235695244@ صيانة ويرلبول الدوريه @ 01225025360 @

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>