SSIS pakage failing to redirect error rows (SQL Server 2016 / Visual Studio 2015)

I have a set of 40,000 files that I am importing into a SQL Server 2016 database using an SSIS package I am building in Visual Studio 2015. Each file consists of several hundred lines of data that are delimited by a space, with text data qualified by double quotes ("). This will result in a table of over a million rows of data.

The problem arises when a string of text contains a double quote that then abnormally ends the text string and generates a "column delimiter for column XXX was not found" error.

Normally, I deal with these types of rows by redirecting them to a failures table, but in this case the package is failing on the error without generating any error rows and does not subsequently complete its run.

Can someone help me understand why the rows are not being redirected?

Alternately, can someone help me deal with text strings that contain text qualifier characters within the string?

Here are the errors I receive:

[ELB Log Source [11]] Error: The column delimiter for column "userAgent" was not found.

[ELB Log Source [11]] Error: An error occurred while processing file "D:\mypath\myfile.txt" on data row 2.

[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on ELB Log Source returned error code 0xC0202092.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.

In this case, I have verified that row 2 of myfile.txt does contain a text string that includes a text qualifier character ("), which breaks up the expected column patterns.


I imagine this has something to do with "the error is fatal and the pipeline stopped executing" part of the error message, but I haven't run into this type of behavior until upgrading to SQL Server 2016 and Visual Studio 2015.