i understand that microsoft is aware of SSIS flat file connection manager bug:
SQL Server 2008 R2 SSIS is not capable of correctly importing RFC 4180 compliant csv files
The behaviour that we are encoutering is: the flat file connection manager correctly identifies the columns of a comma delimited file where the data in each column is simple one character data.
However give it a real scenario of a flat file where one column holds up to 7000 chars and is enclosed in "" which is defined as the text qualifier in the conection manager properties .......
And the result is that the flat file connection manager does not recognize the columns following the large text field, but instead merges them comma delimiters and all into one column.
My workaround is to subsequently use a derived column task to separate out the merged columns so that the source fields data can be mapped to the destination fields.
is this the recommended way to deal with the merged columns interoduced by using the flat file connection manager? has anyone encountered the same issue? is there a more resilient way to import flat files using SSIS?