Hi,
I'm new to ETL and I'm experiencing something that I don't understand. I did a lot of research without success.
I have a MySQL source from which a use the following query to get data :
select d.ID, ifnull((select group_concat(SKU) from MyTable where TABLE_ID = d.ID group by TABLE_ID ),'') as SKU from MyTable d group by d.ID order by d.ID
When I used the query like that, to get every row from the table, the ETL succeed. BUT, the original data length is 559, but in the target column, in a MSSQL table, the data length is only 341. Becareful, I'm not talking about the column length. If a run the query on a mysql server, the result is fine and every data of the SKU column is there, so this is not a problem on the MySQL server side. Now, if a add a parameter to the query like that :
select d.ID, ifnull((select group_concat(SKU) from MyTable where TABLE_ID = d.ID group by TABLE_ID ),'') as SKU from MyTable dwhere d.ID = 1337212 group by d.ID order by d.ID
Like that, I'm targeting only one row, and now I'm having a full SKU of 559 length. I have to admit that I don't understand why I'm having this trouble. Is there some sort of buffer somewhere that need to be set ?
My column in MSSQL is of type varchar(4000) in the ETL I have an ADO.NET source + a Data conversion that convert my SKU to a DT_STR of 4000 + an ADO.NET destination. Everything run fine whitout even a warning.
Any help will be welcome :)
Thank you.
Daniel