I load SQL server table with data coming from a tab delimited txt file using SQL Server 2005 SSIS pkgA, the data flow component script task is NOT able to parse the UTF8 file (no error reported). Used another pkgB with a flat file source(UTF8) & destination (Unicode) to convert input tab delimited UTF8 file to comma delimited Unicode with one-to-one column mapping between source/destination, Unicode file worked with pkgA.
The problem is input file columncount varies and I cannot use a script task in pkgB as the script task is NOT able to parse through UTF8 input file as in (http://social.msdn.microsoft.com/Forums/en/sqlintegrationservices/thread/a4150a3a-0b97-4236-864d-b81be6ef69fe);
I also tried mapping the entire row as one column data into destination file without doublequotes and no delimeter change - but this file does NOT work with pkgA (no error reported).
How can I make pkgB to accept varying columncount UTF8 input file and convert to Unicode?
{pkgA is flexible to use different delimeter (| , ~), also pkgA can accept columns with or without doublequotes}