Oke, I have a nice question and I wonder if any of you can give me a good answer. Personally I have tried everything and concluded to use a script task and a raw file as solution.
Problem is:
I have a .csv file having a matrix-format like this:
time, code1, code2, code3...
date, value,value,value
date, value,value,value
date, value,value,value
...
Now I want to transform the matrix so I can store it easily in the DB so in this format:
time code value,
date, code1, value
date,code2, value
date,code, value
...
What is the best way to perform this action. There is one additional note: 'The number of codes is not static. This will increase over the years til 10000+ codes. ( so a lot of columns)'
I have tried:
First solution: Forloop, but takes way to long.
Second solution: Pivot the data, however this takes a large amount of work to create a decent pivot and is not doable for 10000 codes.
Final solution: Transform the matrix using a script task, store the data in a raw file and use the raw file for the data flow. (Possibly the fastest way?)
I am curious about other advices/solutions.
Best regards,