Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

using sql bulk copy throwing exception -The given value of type String from the data source cannot be converted to type int of the specified target column

$
0
0

Hi All,

I am reading notepads files and inserting data in sql tables from the notepad-

while performing sql bulk copy on this line it throws exception - "bulkcopy.WriteToServer(dt); -"data type related(mentioned in subject )".

Please go through my  logic and tell me what to change to avoid this error -

 public void Main()
        {


            Dts.TaskResult = (int)ScriptResults.Success;




            string[] filePaths = Directory.GetFiles(@"C:\Users\jainruc\Desktop\Sudhanshu\master_db\Archive\test\content_insert\");

            for (int k = 0; k < filePaths.Length; k++)
            {
                string[] lines = System.IO.File.ReadAllLines(filePaths[k]);



                //table name needs to extract after = sign
                string[] pathArr = filePaths[0].Split('\\');
                string tablename = pathArr[9].Split('.')[0];



                DataTable dt = new DataTable(tablename);
              |
                string[] arrColumns = lines[1].Split(new char[] { '|' });



                foreach (string col in arrColumns)
                {

                    dt.Columns.Add(col);
                }


                for (int i = 2; i < lines.Length; i++)
                {
                    {
                        string[] columnsvals = lines[i].Split(new char[] { '|' });



                        DataRow dr = dt.NewRow();
                        for (int j = 0; j < columnsvals.Length; j++)
                        {
                            //Console.Write(columnsvals[j]);
                            if (string.IsNullOrEmpty(columnsvals[j]))
                                dr[j] = DBNull.Value;

                            else
                                dr[j] = columnsvals[j];

                        }
                        dt.Rows.Add(dr);
                    }

                }

                SqlConnection conn = new SqlConnection();

                conn.ConnectionString = "Data Source=UI3DATS009X;" + "Initial Catalog=BHI_CSP_DB;" + "User Id=sa;" + "Password=3pp$erv1ce$4";
                conn.Open();

                SqlBulkCopy bulkcopy = new SqlBulkCopy(conn);

                bulkcopy.DestinationTableName = dt.TableName;
                bulkcopy.WriteToServer(dt);
                conn.Close();

            }



        }


sudhanshu sharma Do good and cast it into river :)


Analysis Service Execute DDL Task throwing error with SourceType Variable

$
0
0

Hi,

I have Configuring Analysis Services Execute DDL Task to use Variable and Process Data(xmla Script) like below:

When I execute this task I get the below error message:

[Analysis Services Execute DDL Task] Error: The -->
text node at line 23, column 3 cannot appear inside the DataSource element (namespace http://schemas.microsoft.com/analysisservices/2003/engine) under Envelope/Body/Execute/Command/Batch/Parallel/Process. This element can only have text nodes containing white-space characters.

Can anyone please let me know how to resolve this.

SSIS Flat File - 27 Column Headers in Row 1, 47 Column Headers in Row 3 - Flat File Connection Manager only returns 27 Columns

$
0
0

Hi all

I have a flat file which contains 10 records. 

Row 1: Master Column Headers. 27 column headers.

Row 2: Master Column Data. Also 27 columns.

Row 3: Detail Column Headers. 47 column headers.

Row 4-10: Detail column data. Also 47 columns.

I want to bring rows 3 onwards from flat file into my destination table. However, the Flat File Connection Manager only ever recognizes 27 columns. 

How do I get it to ignore the first two records and attempt to read column headers from row 3?

I have tried Header rows to skip =2

I have also tried Data rows to skip =2

Both still bring only 27 columns back. Please advise.

Thanks

LP

SSIS Package for Excel to SQL Server Import

$
0
0

Dear All Members,

I have developed an SSIS package to import Excel File to SQL Server Database. I have properly mapped all the fields. But due to some rows the Data Flow Task is getting failed. What I want to do is to ignore those rows (and to keep track) and insert the others in the SQL Server Database. How can I do this? Please help.

Thanks and Regards,

Anujit Karmakar


Anujit Karmakar Sr. Software Engineer

SSIS: Dynamic destination table creation with dynamic input columns

$
0
0

Hi,

I have several CSV files that need to be added to an SQL database, each having its own separate table with the same name as the CSV file. The files do not all have the same columns.

To summarize, what I need to do is:

  1. Get the file name (This gives the table name for my DDL script)
  2. Get column information from file (The Flat File Connection Manager gives both i.e. the column names as well as suggested types for each of the columns. I want to use these in my dynamically generated DDL script)
  3. Create the table using the generated script
  4. Load all data from file to destination table

My main concern is STEP 2. How do I get the column names and their types (as suggested by SSIS) to generate a dynamic DDL for each file that comes in? A script task? If yes, then how exactly?

There is something similar to my concern mentioned here but I'm unable to drive my way towards a solution with what has been suggested there.

TIA!




Parameters cannot be extracted from the SQL command

$
0
0

Hi,

I'm trying to use SSIS OLE <acronym style="border-width:0px 0px 1px;border-bottom-style:dotted;border-bottom-color:#000000;cursor:help;color:#333333;font-family:Verdana, Arial, Tahoma, Calibri, Geneva, sans-serif;font-size:13px;line-height:normal;" title="Database">DB</acronym> Source to get some data using a logical date filter such that it only returns the data that has been added into the the data source since after the last execute time of the SSIS package.

The problem is I have to use a parameter in the query, and it spits out an error as soon as i click on the Parameters to add parameter to it.

here's the error content:

Parameters cannot be extracted from the SQL command. The provider
might not help to parse parameter information from the command. In
that case, use the "SQL command from variable" access mode, in which
the entire SQL command is stored in a variable.

Please i need help on this ASAP.

Thanks.


me

SSIS Lookup Conversion failed when converting date and/or time from character string

$
0
0

Hi all,

I'm trying to do a lookup by passing a date, however (i'm guessing) one of the dates is in a bad format and is failing the component.  How do I find the suspect row?  I set both lookup outputs to flat file but it is just erroring and not outputting anything.  The date being passed in is in datetime format and destination is the same data type.


Data conversion error. Overflowed the specific type.

$
0
0

Hi guys, as usual I am struggling with the data conversion. I have got a column BS as Float in SQL, the usual values is 10.5656445, 899.66552366 etc. I'll try to move in Excel the data but I wish to have only one decimal. I try decimal in the data conversion with precision 1 but it returns me the same value now I am trying

numeric, precision 1 but it returns the error "Conversion failed because the data value overflowed the specified type". Any suggestion? Thanks


Do not error on The input file did not process or has no records

$
0
0

I have a text file with two header rows: 

Hdr0
HdrA, HdrB, HdrC
Data 1.a, Data 1.b, Data 1.c
Data 2.a, Data 2.b, Data 2.c

I wrote a script component to capture this header and data rows into the same row:

Hdr0, HdrA, HdrB, HdrC, Data 1.a, Data 1.b, Data 1.c
Hdr0, HdrA, HdrB, HdrC, Data 2.a, Data 2.b, Data 2.c

When the text file has no data, only the header, the SSIS Data Flow element fails "The input file did not process or has no records".

It is acceptable to have zero records.

How do i keep the data flow task from failing if no rows are processed?

SSIS 2005 Adding a Column to an existing Package

$
0
0

Hello Forum

forgive me but it 's been over 5 years since I touched SSIS.

We have an SSIS package that compiles data and then exports the data to a MS Ecxel Spreadsheet.  We are struggling to add an existing column.

Essentially the package reads data from source, performs a "Data Conversion" so that the Output Columns are named "Output of Field1" etc and then the data is exported to the Excel Spreadsheet.

The developer has succeeded in adding the column to the Data conversion and has manually added the column in the Spreadsheet; however although the converted column appears in the Column Mappings, the corresponding destination Column in the Spreadsheet does not.

Can someone please provide instructions on how we can achieve this?  Many thanks.


Please click "Mark As Answer" if my post helped. Tony C.

SSIS Order by not working

$
0
0

Good Morning Friends,

I'm just running a package that loads data from the source into destination. The requirement is the load should be sorted. I'm using sql command and doing ORDER BY. I went to the advance editor and changed the IsSortable property to TRUE, made the column sort key position 1. When i build the query it works fine. When i load the data into the destination table its sorting by the PRIMARY KEY (AdmissionRate_id). Any ideas what is wrong?

SELECT     AdmissionRate_ID, FacilityHoursType,  Rate
FROM         MYTABLE

ORDER BY FacilityHoursType ASC, Rate DESC


SV

Pass value from SQL agent job step

$
0
0

Hi 

I have created one SSIS package and I am scheduling the same using SQL Agent job. Now I want to pass one value from SQL job agent and want to use that value in SSIS package and then run that package.  Can someone point to the solution.


Aniruddha http://aniruddhathengadi.blogspot.com/

Please help with the error in the Data Flow task -: "Could not continue scan with NOLOCK due to data movement.".

$
0
0

Hi,

I have a simple package where from a source excel, I am loading the data to a  destination table. Duplicate records from source have to be redirected  in a excel file.

In order to eliminate the duplicates, i have written a function and in the table DDL, i have given a check constraint based on the function i created.

But the problem is, when i am loading the data from excel to table, i am getting the error as 'Could not continue scan with NOLOCK due to data movement'.

I spent more than 4 hours for debugging but i could not find the reason.

The function is working fine. So, no issues with the function.

Also, one of my counterpart in different country is successfully able to load the data and also able to redirect  the duplicate records from the same function i created and also used the same constraint as me.

He used everything like me but i am getting the error 'Could not continue scan with NOLOCK due to data movement' - Can you please help me on this ?

Thanks much in advance for your help!

Load sharepoint list data using Odata connection manager dynamically

$
0
0

Hello,

I am trying to load sharepoint list data to sql table using Odata. However I would like to load all the collections (list) from the sharepoint site. But there is no option to dynamically select the collection name. It works fine if we select collection (list), but there is no option to write a expresiion for this to select the collection dynamically.

Please let me know in case of any thoughts.

Thanks!
Shamsuddeen

Importing and manipulating data in a relational database

$
0
0

Hi

First Questing : When importing data into a relational DB, eg, inserting or updating data into table A does it influence Table B ?

Second Question : Am tasked to import extracts in which the header or column names repeat after every 1000th record, how do i go about that, write a program? insert it as is then fix it instead? 


Usability of Connection Managers and Packages across different development users

$
0
0

Hi All,

I am new to SSIS. Currently we are doing the development in SSIS 2012 using the SQL Server Data Tools and there are two different Windows users who login to the server to work on SQL Server Data Tools.

We have created one SSIS project and under that project we have created two different OLE DB Connection Managers to connect to the same database. We had to create two different Connection Managers as the Windows users were not able to connect to the DB using the connection manager created by the other user and users are also not able to execute packages created by the other user. The packages were developed with Protection Level set as "EncryptSensitiveWithUserKey". 

Now at the end of development and before moving onto the testing phase, we have to integrate the packages that should run on a single Connection Manager. The Windows user should be able to run the SSIS packages using a single connection manager and also be able to execute the packages created by the other user.

My question may sound silly. But since I am relatively new to SSIS, I need your help on above and also the steps for configuration and settings for deployment.

Thanks in advance!

Abhishek


Fuzzy Lookup Formula/Logic

$
0
0
Could someone from the dev team please share as much details as you're comfortable with regarding the formula or logic used to calculate the similarity for a fuzzy lookup match?

Error 6522 - deploy_project_internal

$
0
0

Hi,

I have installed SSIS on a server and when I deploy a package, I have this error:

Error Number: 6522
Severity: 16
State: 1
Procedure: deploy_project_internal

I read this blog and have changed permissions but I have ever again the error message:

http://blog.oraylis.de/2013/09/fixing-sql-2012-ssis-deployment-error-6522-a-required-privilege-is-not-held-by-the-client/

Version of SQL: 11.0.3460

SQL Server Engine has a Domain account but SSIS is running with "NT Service\MsDtsServer110".

I suspect that I need to use a domain account for SSIS.

Have you suggestions to analyse or resolve this problem?

Thanks in advanced for your help

Regards,

STH


Best regards, Mit freundlichen Grüssen, Avec mes meilleures salutations, Stéphane Haby

Output Column does not have a matching source column in the schema

$
0
0

Hi Everyone,

                       Can you please help me in this. Already an ETL is loading the data from AS400 TO SQL server. I need to add a new column. In the Source (OLEDB source) am pulling the data from AS400 using ADO.NET connection manager.

New column is added in AS400. To bring the new column from AS400, I added column name in the query in OLEDB source. Test connection is good. In the Source editor, when I click on preview it gives me the values of the new column and if I click on build new query also it gives me values for the new column.

After mapping the columns and refreshing the data in OLEDB destination, I don't see any errors or warnings.But when I start executing the package it throws me the below error.

 Error: The output column  (4659) does not have a matching source column in the schema.

[SSIS.Pipeline] Error: "component  (2861)" failed validation and returned validation status "VS_NEEDSNEWMETADATA".

I need suggestion on this.

Thanks

Veeresh

Metadata could not be determined

$
0
0

Hi,

I am using temp table to load data from SQL to it and later transfer data from it to SQL table.

Here loading to temp table is not issue as of now. But i get struck with loading data from temp table to SQL main table.

The error I am getting is as follows,

"The metadata could not be determined because statement 'Select * from ##tempOINV' uses a temp table.".

...

this works when execute the query for loading to temp table from SSMS, but its not working when the loading takes place inside the same package.

Can anyone please help-?

Thank you!


--------------------------- Radhai Krish | Golden Age is no more far | --------------------------

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>