Evan Johnson

Trying to load Flat File to SQL Server. Number of columns in Flat file

ProductID - two-byte signed integer [DT_I2]
Name -Unicode string [DT_WSTR](50)
ProductNumber-Unicode string [DT_WSTR](25)
MakeFlag-Boolean [DT_BOOL]

Getting issue with last column 'MakeFlag'.

I have attached sample Flat file if some body wants to test it

Error: 0xC02020A1 at Data Flow Task, Flat File Source [1]: Data conversion failed. The data conversion for column "MakeFlag" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
Error: 0xC0209029 at Data Flow Task, Flat File Source [1]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "output column "MakeFlag" (22)" failed because error code 0xC0209084 occurred, and the error row disposition on "output column "MakeFlag" (22)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
Error: 0xC0202092 at Data Flow Task, Flat File Source [1]: An error occurred while processing file "D:\Users\Administrator\Desktop\New Folder\IndiaIndia.txt" on data row 211.
Error: 0xC0047038 at Data Flow Task, SSIS.Pipeline: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "Flat File Source" (1) returned error code 0xC0202092. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.

Need to convert the boolean value to a 0 or 1 before importing it

Used derived column to convert the boolean value to a 0 or 1

Expressions what i used in derived column for two columns MakeFlag and FinishedGoodsFlag

MakeFlag ? (DT_BOOL)1 : (DT_BOOL)0

FinishedGoodsFlag ? (DT_BOOL)1 : (DT_BOOL)0

I get error messages as below.Any thing wrong with the expression i am using here.Please let me know.Please send me the correct expression

Error: 0xC02020A1 at Data Flow Task, Flat File Source [1]: Data conversion failed. The data conversion for column "MakeFlag" returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
Error: 0xC0209029 at Data Flow Task, Flat File Source [1]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "output column "MakeFlag" (22)" failed because error code 0xC0209084 occurred, and the error row disposition on "output column "MakeFlag" (22)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.

First, I have read through the numerous posts on this topic and now of the provided solutions have worked.  So I post my specidifc issue.

Using VSTA2008 from SQL2008 STD edition.

Using local machine for development.  I am the local and domain admin.

The path that I am denied access to is the Local C:\TEMP directory.

In BId, I now have two scripts that run.  the first script sets the file level permissions to full control under the context of the task that is being run.

The secong task performs a PGP decryption of the files.  This is were I get the error.  I can set the file permissions with out an issue, yet as soon as I caaess the same files to decrypt them, I get the error.  The error is being thrown as soon as I access the files.  The PGP decryption routine will run when attached to a windows forms project on the same machine.  This is how I was able to test and debug the code prior to adding it to the SSIS Task. 

In total the job picks up a file from an FTP site, moves it to the local machine, decrypts file, then pipes it along for further processing. The only point of failure is in the one script task.  and I am at a loss.

Let me add this in case it is relevant.  in the affected script, I reference a third party dll (bouncycastle.crypto) for the PGP encrypt/decrypt utilities.  A second code (class) file exists that performs all of the logic, and i call it from the script main() passing in the validated arguments.

If anyone has any ideas, I am open to any and all suggestions.

Thanks in advance.

Hi,

we want to access data via a SQL Anywhere ADO.net driver. The issue is that this driver has slightly different versions on different servers and developer machines, while the package should be the same across the servers.

The version info, as found in machine.config is e. g. as follows:

<add name="SQL Anywhere 12 Data Provider" invariant="iAnywhere.Data.SQLAnywhere" description=".Net Framework Data Provider for SQL Anywhere 12" type="iAnywhere.Data.SQLAnywhere.SAFactory, iAnywhere.Data.SQLAnywhere.v3.5, Version=12.0.1.31523, Culture=neutral, PublicKeyToken=f222fc4333e0d400" />

<add name="SQL Anywhere 12 Data Provider" invariant="iAnywhere.Data.SQLAnywhere" description=".Net Framework Data Provider for SQL Anywhere 12" type="iAnywhere.Data.SQLAnywhere.SAFactory, iAnywhere.Data.SQLAnywhere.v3.5, Version=12.0.1.34573, Culture=neutral, PublicKeyToken=f222fc4333e0d400" />

It seems that the ADO.net connection manager references the exact version of the driver in its "Qualifier" property, which is e. g.

iAnywhere.Data.SQLAnywhere.SAConnection, iAnywhere.Data.SQLAnywhere.v3.5, Version=12.0.1.3473, Culture=neutral, PublicKeyToken=f222fc4333e0d400

This property is not available for configurations.

I tried to edit the machine.config file to contain:

  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="SQL Anywhere 12 Data Provider" publicKeyToken="f222fc4333e0d400" culture="neutral" />
        <bindingRedirect oldVersion="12.0.1.0-12.0.1.65534" newVersion="12.0.1.34573" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>

where the newVersion value matches the version installed on the machine. But this does not seem to be used, and I get the same error as without this entry if the version in the Qualifier property does not exactly match the version installed on the machine:

Error: Microsoft.SqlServer.Dts.Runtime.DtsCouldNotCreateManagedConnectionException: Could not create a managed connection manager.

This error does not occur if the version info exactly matches the version of the driver installed.

Any idea how I could get this running?

Frank

Hi all,

I need to send data from SQL to Teradata destination. TPT load is not working properly on my machine. I have 32 bit ODBC driver for Tera data on my machine. SO can any one please let me know the procedure like what kind of destination i should use and how to set properties.

Thanks in advance,

Jyo

my dataflow task in SSIS 2008 is failing with PK violation error, when run as job. I checked the source data and couldn't find any duplicate . The destination is always truncated before the dataflow task runs. But when I run the same package from visual studio, I executes successfully. there is around 13mil rows. Any idea what could be causing this?

Thanks

I created an SSIS package that will grab CSV files in a folder and import them into the SQL Server.  I've done imports in the past with Excel and experienced no problems.  However, I'm having trouble in the deployed environment whereas the test environment is working without any problems.

The ForEach loop has a variable under expressions for the directory and it seeks CSV files (by name and extension).  The import file is also listed under variable mappings.  The process should check the folder for ANY CSV file and import them into SQL Server.  In the test environment, this works.  However, in the deployed environment, it does not grab the files (the directories are the same, test file names are the same).

Does anyone know what the problem might be?

(Note: in the deployed environment, SSIS will show that it successfully completed with no debug errors, however, no data were imported).

MS SQL 2008

Prior to recently converting our MS Access 2003 BE to SQL 2008 I ran a scheduled task from my local PC to run a MS Access FE module that would run select Reports and send them as email attachments.

This is a very painful process for my local PC as it is opening the MS Access FE across our company WAN (Wide Area Network).  IT will not give me access to scheduled tasks control panel on our Citrix Server Farm. 

Now that we have coverted our BE to SQL 2008. I would like to move this process of sending weekly/daily reports to SSIS package is possible.

So the question is.   Can I attach a MS Access Report to the Send Mail Task in SSIS?  If I can,  how? 

If not hopefully there is an alternative via stored procedures.

Hi All,

I have requirement is as below

1. I MDX query which pulls 108736 record in all my requirements is if the records are more than 65K then record should go to next sheet and so on

2. I have excel template with some header and Columns heading and the data insertion shoulod start from row 6

Thanks

Hope this will help you !!!
Sanjeewan

Hi, i have an SSIS package having two Sequence Containers connected to Execute sql task in Parallel.

I have an expression in between execute sql and both sequence containers if @Test = 1. @Test gets value from execute sql task.

I want to fail the package if any of the two sequence container fails. So i set FailpackageOnFailure to TRUE for both containers but even though one of them fails, the package wont fail and other one still runs as i have an expression for both.

Any ideal how do i solve this?

This is how the package looks like:

Package fails when importing data from flat to SQL Server

Number of columns in Flat File

ProductID,Name,ProductNumber,MakeFlag,FinishedGoodsFlag,Color,SafetyStockLevel,ReorderPoint,StandardCost

Table structure in SQL Server

    [ProductID] [int] NULL,
    [Name] [nvarchar](50) NULL,
    [ProductNumber] [nvarchar](25) NULL,
    [MakeFlag] [bit] NULL,
    [FinishedGoodsFlag] [bit] NULL,
    [Color] [nvarchar](15) NULL,
    [SafetyStockLevel] [smallint] NULL

Used derived column to convert the data types

MakeFlag1        <add as new column>        MakeFlag == "False" ? "0" : "1"        Unicode string [DT_WSTR]        1  

FinishedGoodsFlag1        <add as new column>        FinishedGoodsFlag == "False" ? "0" : "1"        Unicode string [DT_WSTR]        1      

SafetyStockLevel1        <add as new column>        (DT_I2)SafetyStockLevel        two-byte signed integer [DT_I2]              

Mapped the columns correctly  with the newly created derived columns in OLE DB destination.I am getting error message in the case of    'SafetyStockLevel' . As far i know everything has been done properly here what could be the issue.Please correct me if i am going wrong any where. Please find the error message as below          
                 

Error: 0xC0049064 at Data Flow Task, Derived Column [58]: An error occurred while attempting to perform a type cast.
Error: 0xC0209029 at Data Flow Task, Derived Column [58]: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR.  The "component "Derived Column" (58)" failed because error code 0xC0049064 occurred, and the error row disposition on "output column "SafetyStockLevel1" (83)" specifies failure on error. An error occurred on the specified object of the specified component.  There may be error messages posted before this with more information about the failure.
Error: 0xC0047022 at Data Flow Task, SSIS.Pipeline: SSIS Error Code DTS_E_PROCESSINPUTFAILED.  The ProcessInput method on component "Derived Column" (58) failed with error code 0xC0209029 while processing input "Derived Column Input" (59). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.  There may be error messages posted before this with more information about the failure                  

Smash126

HI

  I create a package in SSIS,in that package i want to load from excel to sql server db..in that excel sheet i am having column called order in that i am having

3

3.1

3.1.1

3.1.2

3.1.2.1

like this order is in Excel sheet..My table contain that Order as string ..But when i try to load that excel sheet to database it takes only 3,3.1 values

i think that problem is in Excel .if i change that columns string its takes only

3.1.1

3.1.2

3.1.2.1 these value to load

if i gave number format in Excel

it takes

3

3.1 these value only..How to solve this issue in SSIS

I am trying to post the file to ftp site, file is posted as empty whenever i run the package. but when i run the task individually the file is posted in the ftp site, could you please tell what could be the error. i am using same code to post another file in the same location it works perfect....

i am using following code

Hi,

today my dts package freeze during the loading of my data.

I have multiple dataflows in my package, and the freeze occurs in the dataflow 5 or 6, never on the first dataflows.

the 5 is pretty simple, I get data from SQL server to fill a cache transform.

but the freeze could occurs here, even if the execution normaly takes 5 to 10 sec.

and if the freeze is not in the DF 5, its in the 6 which is the biggest one.

yesterday everything works fine!!!

I try to execute inside VS and using DTExec, same behavior.

on the SQL Server side, I see that the network IO wait task is at 1000ms and no more activity.

so why this could occurs?

why today and not yesterday? (I try this 10 times and always the same behavior occurs at different point in time)

why never on my first dataflows and only the DF 5 or 6?

I have a build a SSIS to import data from an Excel file (big file) to SQl Server table.

I have a composite_Key (OrderDate+OrderNo) to do unique record in the database

Firstly I've a simple Data Flow Task to import Excel Source to OLE DB source table

Secondly, I made a solution as Andy Leonard suggest, using a lookup and conditional split, but this is doesn't work for me.

When I create a relation between Lookup and conditional Split I've to choose "Input Output Selection" and I got 2 options

"Lookup Match Output" or "Lookup No Match Output"  , that mean that the conditional split will not work for both.

If I choose ""Lookup Match Output" and setup New Rows in the conditional split when I run doesn't insert any new row. Where is my problem?

I hope someone can help me 

Eliana 

I am trying to connect to SAP BW and access a cube that they have created. SAP people have created RFC connection and have given me the details.
I have installed Microsoft Connector 1.0 and Added SAPBI connection, SAP Source, SAP Destination to my DataFlow components.

SAP Connection works fine. (Checked using Test connection Button)
Then I dragged a SAP Source into a Data Flow Task, and gave all info required (OHS destination, Execution mode as Wait for Notify, Gateway host, Gatway service ,Program ID)

1. When I do a Preview, it always gives me a time out error with following details.
An internal exception occurred.
Microsoft.SqlServer.Dts.SapBw.Connectivity.ERPException: The RFCServer object could not be registered. Check your network configuration.
   at Microsoft.SqlServer.Dts.SapBw.Connectivity.RFCServer.Accept()
   at Microsoft.SqlServer.Dts.SapBw.Connectivity.RFCServer.Run()

2. When I just give OK (without preview) and connect the SAP source with DataReaderDestination, it gives the below error:
[SAP BI Source [1]] Error: Microsoft.SqlServer.Dts.SapBw.Connectivity.ERPException: Value for ID 'PASSWD' not found
   at Microsoft.SqlServer.Dts.SapBw.Connectivity.R3Connection.Open(String ConnectionString)
   at Microsoft.SqlServer.Dts.SapBw.ConnectionManager.SapBwConnectionManager.AcquireConnection(Object txn)
   at Microsoft.SqlServer.Dts.Runtime.ManagedWrapper.AcquireConnection(Object txn)
   at Microsoft.SqlServer.Dts.Runtime.ConnectionManager.AcquireConnection(Object txn)
   at Microsoft.SqlServer.Dts.SapBw.Components.SapBwPipelineComponent.AcquireConnections(Object transaction)
   at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostAcquireConnections(IDTSManagedComponentWrapper100 wrapper, Object transaction)

Does anybody have any ides on this? Pls help if possible.

Thanks in advance.

When opening an package (.dtsx) the toolbox no longer shows the Toolbox groups and items as before...

how do I reset the groups and items?

Have tried reinstalling SQL Server, and are able to manually add groups and items, but I'm looking for the correct procedure to get them back.

.LP

Hello,

I am in the process of defining a SQL authentication mode model for SSIS package development.  I am proposing to eliminate SQL mixed mode based authentication COMPLETELY and to solely utilize Windows authentication (Windows Integrated Security) as our SQL Server authentication type.

Are any of you working at SSIS heavy shops where you only utilize Windows based authentication for SQL Server access?

From a security standpoint, utilizing solely Windows authentication within a SQL environment has many advantages.

I am now in the process of identifying the impact this security model would have on the development, maintainability, scalability, and performance of SSIS packages.

Particularly, I am concerned that with the elimination of SQL based authentication, overall development time and complexity of SSIS packages will greatly increase.

Consider the following example.   I develop highly N-Tier conforming SSIS packages in which one main SSIS package would call several data flow packages in order to segregate units of functionality.  This chain dependency of SSIS packages allows for easy maintenance and testing within a production environment.

A typical SSIS package I develop can be viewed in the following manner:

[Main Package 1] ---> [Business Logic Layer Package] --->  [Individual Data Flow Package]

As one can see above, one SSIS package calls another SSIS package, which finally calls another.

Now, as anyone would, I would schedule SSIS package execution using SQL Server Agent.  This being said, using the example above, if I had 5 data flow packages being executed within my main SSIS package that reference 5 different data sources, for security adherence, I could create 5 separate SQL authentication based users within SQL Server so that only the needed permissions are granted to each user used with each of the 5 data flows.

Now, if I were to eliminate SQL Server based authentication, the process becomes difficult as I would only be able to use Windows Integrated Security for authentication.

Now, being that I will continue to desire to assign a custom set of SQL based permissions to each user utilized within the 5 data flows above in our example, the only way I can see being able to assign a custom user to each SSIS data flow using the Windows authentication model for SQL Server would be to use "Credential" and "Proxy: objects as discussed here:

However, this solution presents a problem in that I cannot assign a unique Windows user to execute each of my 5 SSIS packages as only one SSIS package is actually going to get scheduled by SQL Server Agent. 

Being that one SSIS package calls other SSIS packages, unless I am mistaken (please correct me if I'm wrong) I can only set the "run as" Windows user at the SQL Server Agent job level and not at the SSIS package level like I want to.  This would limit my SSIS package being executed as one user, which is not what I desire security-wise. 

Is there any way you know of to be able to specify the exact Windows user each of the 5 SSIS packages in my example would be executed as?

If not, please consider my proposed idea to create a "hybrid" approach to this security issue:

1. SQL Server would be configured to support "mixed mode" authentication.
2. One Windows authentication based user would be created on SQL server that would be used as the "Run As" user when scheduling the main SSIS package via SQL Server Agent.
3. 5 seperate SQL Server authentication based users would be created on SQL Server for each of the data flow packages.
4. Each of the 5 SSIS packages would be developed to store the individual SQL Server authentication based connection strings on a centrally located SQL Server table.
5. Only the "Run As" Windows authentication based user that the entire set of SSIS packages are running as would be granted select only access to this "SSIS package parameters" table.
6. During execution time, each of the 5 SSIS data flow designated packages would authentication to SQL Server FIRST via Windows authentication in order to access the SSIS SQL "parameters table".
7. Each of the 5 SSIS data flow designated packages would then access the parameter table and grab its unique SQL authentication based connection string and apply this connection string to its corresponding data flow task objects.
8. All ETL operations would then be performed by the data flow designated SSIS packages using the uniquely assigned SQL authentication based user within each data flow package.

This solution above has the following benefits:

1. No connection information including user names and passwords are stored in plain text within the SSIS environment via XML config files.
2. Each SSIS package can only access SQL connection string information from the "parameter table" ONLY if the Windows user the SSIS package is currently running as is granted sufficient permissions.
3. SSIS package development is not overly bloated and cumbersome as individual SSIS data flow designated packages can still use package configurations to easily configure data flow tasks, not adding unnecessary development efforts to existing development workflows.

What do you think about my solution?  Is this a feasible solution for an enterprise environment?  Do you think this approach adheres to a solid set of good security practices?

I am open to all suggestions on how I can tackle this problem with security and ease of development being number one priority.

I would greatly appreciate your input on this!