Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live
↧

Can't insert more than 8000 rows to excel via JOB

December 5, 2013, 1:10 am
$
0
0

Hello, 

I have SSIS package where I have two tasks.

File System task copies file (with header names) to destination folder.

Data flow task export data from table to copied Excel file. Select are simple "select * from table" without any filters.

There are about 100 000 rows in the table.

When I run this task through Visual studio everything works fine. Data to Excel file is exported.

When I run this package in JOB (Job Activity Monitor) the file is created but 0 rows is exported.

One interesting thing, that when I use select "select top 7000 * from table" 7000 rows is inserted but when I try to export more than 8000 rows then 0 rows is exported. The JOB generates 0 errors. In fact he tries to export. Because all the time Excel's file "Date modified" is changing.

It seems like data is exported but the final "commit" isn't done. Anybody knows where could be a problem?

↧
Search

Is it okay to use SSIS over Linked Server for 1000 to 2000 records

December 4, 2013, 11:49 pm
$
0
0

Hi All,

I just confused what to use SSIS or Linked Server.I have to transfer around 1000 to 2000 records into remote SQL server.Will it(SSIS) be a big hammer for 1000 to 2000 records (Also it is complex from maintenance perspective) Can you please help me to choose? My destination server is SQL server 2008 while source server is SQL server 2005.Both m/c are 32 bit Windows 2003 servers.

Banky

↧
↧

SSIS Date Conversion Error

December 4, 2013, 6:43 am
$
0
0

Good day,

I'm sure this question has been asked a million times, but googling seems to be failing me at the moment.  I have a date/time field coming from a flat file in the following format:

25-JUN-1992 10:39

When I attempt to run the ETL to import this field into the database, the job fails with the error:

The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".

I'm thinking that I have to use the SUBSTRING function in a derived column expression to put the date in a format that SSIS understands how to convert, but nothing I've tried is working.

Any suggestions?

Greg

Greg Frair

↧

based on Date comparison fetch exact value

December 5, 2013, 1:41 am
$
0
0
Hi Everyone,

My source table Table1 contains columns WELLID,ORIG_REF, LOG_DATE

Sample data:

WELL_IDORIG_REF  LOG_DATE
--------------- --------
1CALIB 11/19/2000
2BRITAN 6/30/2006
3CALGRY NULL
4DENVAR 8/20/2009
5ALBERTA NULL


i have to fetch one column from reference table : R_TABLE,
 columns:[ORIG_REF,END_DATE,ENT_NAME] based on key column Table1.ORIG_REF

from this table i have to fetch ENT_NAME based on WELL_ID,END_DATE

reference table Sample data:


WELL_IDORIG_REF END_DATEENT_NAME--------------- ----------------
1CALIB 11/19/2001X
1CALIB 6/30/2006Y
1CALIB 10/19/2000Z
1CALIB NULLAB
2BRITAN 6/30/2006P
2BRITAN 6/30/2004Q
2BRITAN NULLR
3CALGRY 6/30/2006C
3CALGRY 6/30/2005D
3CALGRY NULLF
5ALBERTA 6/30/2005QA
5ALBERTA 8/30/2005CBZ


my requirement is:

condition #1:

for  a given  orig_ref:CALIB  in reference table contains 4 records, from this i have to fetch  ENT_NAME based on LOG_DATE and END_DATE.
i have to fetch ENT_NAME in such a way that LOG_DATE [ON or BEFORE] END_DATE.

i.e., log_date value : 11/19/2000,  END_DATE values are [11/19/2001,6/30/2006,10/19/2000,NULL] here log_date on or before thatdate  value i have to take from reference table,
 value is 10/19/2000 hence for this record  ENT_NAME: z this value i have to fetch.


condition #2:

for  a given  orig_ref:BRITAN  in reference table contains 3 records, source log_date : 6/30/2006 exactly matches with END_DATE: 6/30/2006 hence 
for this i have to fetch ENT_NAME: P


condition #3:

for  a given  orig_ref:CALGRY  in reference table contains 3 records, source log_date : NULL, Reference table END_DATE also contains NULL value hence 
for this i have to fetch ENT_NAME: F

note : if the  reference table does'nt have value with NULL then  hardcode ENT_NAME:'NOT APPLICABLE'

condition #4:

for  a given  orig_ref:ALBERTA  in reference table contains 2 records,source log_date : NULL,BUT  Reference table END_DATE contains date:6/30/2005,8/30/2005
here there is no NULL date in reference table hence hardcode ENT_NAME:'NOT APPLICABLE'


condition #5:

for  a given  orig_ref:DENVAR  in reference table does't have any records for this hardcode ENT_NAME:'NOT APPLICABLE'


output : 

WELL_IDORIG_REF LOG_DATEENT_NAME---------------  ----------------
1CALIB 11/19/2000Z
2BRITAN 6/30/2006P
3CALGRY NULLF
4DENVAR 8/20/2009NOT APPLICABLE
5ALBERTA NULLNOT APPLICABLE

please help on this.

karteek_miryala


↧

Help with Creating Raw Files dynamically

December 5, 2013, 11:59 am
$
0
0

Hello,

I have a problem with creating text files dynamically in SSIS. I've already read plenty of websites, tutorials and so on and still i can't cope with that!

Here's my flow chart:

Nevermind about two first Execute task components, they do not affect on foreach loop container.

Basically, I would like to do following task:

  • I would like to execute SQL query:
    SELECT distinct([City]), lastname FROM  FROM [AdventureWorks].[HumanResources].[vEmployee]
  • Gather City, assign to the "CITY" variable
  • Gather LastName assign it to "LASTNAME" variable
  • Then, In DataFlow component I would like to create files dynamically. FileName of those files should be as the value
    stored in LastName variable. So it should like this:
    C:\MAIN_DIRECTORY\[CITY_FOLDERS]\[LastName].csv
  • Folders (from City column) are already created and there is no need to create them automatically.

In the DATA FLOW I tried to Use "FLAT FILE DESTINATION",however it doesn't have Expression method to assign new path combined fromlastname and city.
I've noticed that "RAW FILE DESTINATION" doesn't have this option as well, but it's possible to provide "File name from variable". When I tried to provide variable name which I created and it exists in variables menu error was generated with following message: "the file name property is not valid. the filename is a device or contains invalid characters"

Here's how my variables looked like:

MAIN_DIRECTORY = "C:\tests" (tried C:\\tests\\, C:\\tests, "C:\\tests\\")
FULLPATH = @[User::MAIN_DIRECTORY] + "\\" @[User::CITY] + "\\" @[User::LASTNAME] + ".csv"

Variables CITY, LASTNAME are assigned in the FOREACH component.
And getting values, assigning them to the variables works correctly, the only problem is connected with creating the files!

So, I couldn't create RAW FILE DESTINATION because of error, so I tried to trick SSIS and created a script:

        Dim strFolder As String
        Dim lastname As String
        Dim loc As String
        Dim strNewFullPath As String
        strFolder = Dts.Variables("strMainPath").Value.ToString()
        lastname = Dts.Variables("LastName").Value.ToString()
        loc = Dts.Variables("City").Value.ToString()
        strNewFullPath = strFolder & loc & "\\" & lastname & ".txt"
        Dts.Variables("FullPath").Value = strNewFullPath
        Dts.TaskResult = Dts.Results.Success

Additionally to SSIS FullPath variable I assigned some path to file, which really exists. Then configured RAW FILE DESTINATION to use that variable. There was no error, as this file really existed. I could create a component, however,the same error was generated when I executed my solution....

Please help me, I've analysed lots of forum posts, even on this forum.
In many posts which I read there was an information to add Expression.
How SHOULD I DO THAT if Raw File & Flat File don't have this property ?
I know there is an option in the Connection Manager called"NEW FILE CONNECTION"and it has Expressions property. But how to match this connector with FLAT/RAW Destination component?

↧
↧

How do I dynamically get the list of variables and parameters in a package

December 5, 2013, 7:27 am
$
0
0

Hi,
    Is there a task/collection which will return the list of variables and parameters in the package? I want to run a foreach loop and print the variables/packages names and values.

Thanks

↧

Can I call a child package multiple times, synchronously, from a master package?

December 5, 2013, 11:21 am
$
0
0

Hi All,

We need to download 5 files from an FTP server and import them daily.  If any of the 5 files is not available, we want to send an email.  Each file is in a different source folder on the FTP site.  We don't want the unexpected absence of any file to interrupt the download and import of any of the other files that should be available.

This seems like a good time to use a master package that calls a "check and see if the file's available and if so then download it" package for each of the 5 files.  I imagine making 5 simultaneous calls to this child package (by using 5 execute package tasks), passing the source folder name as a variable from the master package to each.

Is this feasible?  Are there any gotchas I need to be aware of?

Thanks,
Eric

↧

text files and extended character sets

December 5, 2013, 12:45 pm
$
0
0

Hi.  I'm setting up a prototype pkg that will consume pipe delimeted text files and load them into untyped staging tables.

Target columns are all nvarchar(?).

Expectation is that some of our foreign biz partners will be feeding unicode into the text files.  I'm assuming regular text files can be saved as unicode and this will not be a problem.  Are there any gotchyas in unicode text to nvarchar etl mapping that I need to know about beforehand? I'm already wondering what the diff is between unicode text files and unicode big endian.

↧
Search

parsing string

December 5, 2013, 11:55 am
$
0
0
I need to parse a string like 'GHTRKDNE Linux File System/dev/tre1mounted on / (local)/NTFS)'.  I need the bold underlined portion of the string. The string vary in length and content but always contain the text in  bold italic. I need to use SSIS expressions. I can get close but my head is sore from banging it on the table. Thanks for the help. Rich
↧
↧

Script Component: Looping through a list for matching strings

December 5, 2013, 10:20 am
$
0
0

Hello my friends:

I have the following Script Components that makes all data in a specific column Title Case.
In addition to that I would like to format certain address suffixes by converting them to upper.

How ever I am getting the following error:

Error: 0xC0202009 at Data Flow Task, SQL Server Destination [302]:
SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80040E14  Description: "Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".".

    public static string StringInput(string input)
    {
        TextInfo tCase = new CultureInfo("en-US", false).TextInfo;
        return tCase.ToTitleCase(Regex.Replace(input.ToLower(), @"\s+", " "));
    }

    public static List<string> AddressSuffix = new List<string>();

    public override void Input0_ProcessInputRow(Input0Buffer Row)
    {

        AddressSuffix.Add("Sw");
        AddressSuffix.Add("Nw");
        AddressSuffix.Add("Ne");
        AddressSuffix.Add("Se");

        if (!Row.STREET1_IsNull)
        {
            foreach (string suffix in AddressSuffix)
            {
                if (StringInput(Row.STREET1) == suffix)
                {
                    Row.STREET1 = StringInput(Row.STREET1).ToUpper();
                }
                else
                {
                    Row.STREET1 = StringInput(Row.STREET1);
                }
            }

        }

Thanks for looking into this:

OG

↧

SQL Server Agent Reporting Job Success Yet Nothing Happens to Underlying Package/Data

December 5, 2013, 2:50 pm
$
0
0

I have SSIS packages developed on 2012 and deployed to file system in a 2008 server. There is a SQL Server Agent job to run these packages. The Agent job is running from SQL 2012. As such, I am using Operating System Command option on SQL Server Agent to execute the packages. The command which I am using for one of the packages is

"C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\DTExec.exe" /FILE "\"C:\C**\SSISPackages\Acute** Datasets\D** **T\SFTP_GetFiles.dtsx\""

-----apologies for asterisks. They replace real words for obvious reasons-----

The SQL Server Agent job runs under a Proxy_Account. The surprising thing to me is that when I kick the job on SQL Server Agent, the job runs with no errors yet no data is manipulated neither does anything happens to the data. The job takes a few seconds and it’s complete with success. On the other hand if I go to the package itself, right click the file, Open it, and run Execute Package Utility – the package executes fully as expected. The command from the Execute Package Utility is

/FILE "\"C:\C**\SSISPackages\Acute** Datasets\D** **T\SFTP_GetFiles.dtsx\"" /CHECKINGPOINT OFF/ REPORTING EWCDI

Can someone please kindly help me why the package is not executing when I run it from SQL Server Agent while the same SQL Server Agent job reports success. On the other hand the same package executes when manually run from the file itself using Execute Package Utility?

Many thanks,

Mpumelelo

↧

BCP error message

December 5, 2013, 10:51 am
$
0
0

I have a BCP Argument that is being passed in from an SSIS package and it is failing saying that I am missing either a -U or a -T but that is included in the statement.  Can someone please tell me why this is failing?

bcp.exe "SELECT (test_date ) FROM archive_test at1 WHERE at1.test_date <= CONVERT(VARCHAR(10), DATEADD(Day,( -1 ),GETDATE()), 120)" queryout "Z:\bcp_test\" -N -T -D "GGG" -S "BBZ600"

↧

DQS Cleansing Component Mismatching fields

December 2, 2013, 10:05 am
$
0
0

Howdy,

I have been working on a SSIS/DQS project that has worked with many fields as far as the basic cleansing that DQS can do.  However, As I have added more fields, the DQS Cleansing Component seems to have an issue that I can not find any other questions/documents.  

I was cleaning a few fields appropriately, however, once I went from say 20 to 100 fields (I didnt figure out what number of domains this changed at)... the Cleansing component seems to be applying rules of unmatched domains to the incoming field. 

When I run the field through the DQS UI ... it seems to work appropriately.  However, with the SSIS transformation, even DataSets of 6 rows, come up with errors because FieldA fails a rule in DomainB.

Any ideas?

- Roger

↧
↧

Data Flow Lookup Join on DT_I8 fails with invalid input parameters

December 5, 2013, 2:03 pm
$
0
0

I have a lookup transformation:

1)  no cache, OLE DB connection

2) connection is SQL query:   Select fld1, fld2, fld3 from table1  --fld1 and fld2 are datatypebigint, fld3 is n/a (it is the field i am adding to data flow

3) join columns:  

  • fld1 (which SSIS datatypes as DT_I8) joins to data flow IncomingField1 (which is a DT_I8) 
  • fld2  (which SSIS datatypes as DT_I8) joins to data flow IncomingField2 (which is DT_I8)
  • fld3 again irrelevant as it just becomes the returned field added to data flow.

4) I run it and it fails: 

Error: 0xC0202009 at Data Flow, Lookup [131]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available.Source: "Microsoft SQL Server Native Client 11.0"Hresult: 0x80040E21 Description: "Invalid input parameter values. Check the status values for detail.". Error: 0xC0208253 at Data Flow, Lookup [131]: OLE DB error occurred while fetching parameterized rowset. Check SQLCommand and SqlCommandParam properties.  Error: 0xC0047022 at Data Flow, SSIS.Pipeline: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Lookup" (131) failed with error code 0xC0208253 while processing input "Lookup Input" (145).

5) I have valid input parameters.  debugger shows 2 rows going down this path. 

  • Data Flow row 1, IncomingField1 has value '2', IncomingField2 has value '2688099'.  
  • Data Flow row 2, IncomingField1 has value '2',  IncomingField2 has value '2688103'. 
  • at this point in dataflow, i am assured not to have nulls because that has all been tested out and handled in prior steps.

6) at this point in data flow, i also know i have rcds in table1 that match the lookup because I am attempting to handle records that fail an insertion step due to duplicate key.

  • Table1, row n/a,  fld1 has value '2',  fld2 has value '2688099'.  
  • Table1, row n/a,  fld1 has value '2',  fld2 has value '2688103'.

7) why does the lookup fail?  everything i have found on the web so far points to datatype problems but i have DT_I8 on both sides of join.

8) I attempted to add a step prior to the lookup, a step to do data conversion or derived column to convert the IncomingField1 and IncomingFiled2 from DT_I8 to DT_WSTR(100) then use lookup SQL that casts fld1 & fld2 to NVARCHAR(100).  but then the data conversion / derived column step fails.  it would seem I can't convert a DT_I8 to a DT_WSTR?  I tried with length 100 and then again with length 4000.  i tried to convert to DT_STR and that fails as well.

9) so I have gone back to the original error and posted here for help.

please help.

↧

create file

December 5, 2013, 4:43 pm
$
0
0

I have a SSIS package that loads csv files into sql server table. 

i want to create a log file that will provide me how may rows per file per day were inserted into table.

in log file i want

no of rows     filename       date

1232a.csv 12-05-2013

1221 b.csv 12-0502013

Can this be done?

↧
Search

Auto run if Script get fails in Loop

December 5, 2013, 11:35 pm
$
0
0

Hi team,

I have used to create the report using script task and run through for each loop container. some times my script task webservice connection strings is getting time out for some times. after some times, it will come to up. so i need solution, even my package get failed for this loop, i don't want to get failure for whole package. i want to give chance to failure request 2 more times .

http://social.technet.microsoft.com/Forums/en-US/9c23f232-652b-48d9-af94-827778c02317/ssis-package-auto-retry-until-succeed

This is link extact solution for me, but its not clear to me how to use this.

↧

SSIS package schedule

December 6, 2013, 1:01 am
$
0
0
Does it is possible to schedule ssis package or task with in SSIS.

no

↧
↧

SQL Agent Job error while trying to execute SSIS package

December 6, 2013, 1:52 am
$
0
0
I keep getting the error "Executed as user: Domain\User. Unable to open Step output file. The step failed." while trying to execute a SSIS package as a SQL Agent Job. The error is thrown almost instantaneously.

I have created a proxy account for executing the job & this account has read/write permissions to the folder where the package is saved.

Also, there is nothing being written to the error output file.

Thanks Girija
↧

configuring ADO.NET connection manager to execute stored procedure

December 4, 2013, 12:50 pm
$
0
0

Hi ,

I'm trying to configure ADO.NET connection manager to execute SP which has 2 parameters  but it errors out with the following error :

[Execute SQL Task] Error: Executing the query " [dbo].[xyz] " failed with the following error: "Could not find stored procedure ''.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.

My stored procedure has 2 input parameters ( @Startdate and @enddate)  which loads a table  based on given dates.

Could any one guide me through process to configure ADO.NET connection manager.

Thank you ,

Vishal.

↧

SQL 2008 R2 SP 1 doesn't fix KB2142496

November 27, 2013, 2:41 pm
$
0
0

KB2142496 indicates the problem is fixed in SQL 2008 R2 SP1.

However, it is not listed in the bugs fixed by SP1 (KB2528583).


I have installed SP2 and am still getting the error described by the symptoms section in KB2142496.


Where can I get a fix for this issue that applies to SQL 2008 R2 SP2?


Thanks.


↧
Viewing all 24688 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>