Hi -
I am really hoping someone can help me or point me in the right direction... I have been seaching online for a while to no avail:
I have developed an ETL that will pull in 50+ .csv files, import them in to a SQL server DB, do some data quality checks and then move the good data to a final table in the SQL Server DB. The data then needs to be put back into a .csv file and moved to a shared folder (so far, no problem). My original plan was to just create a flat file destination and dump all of the data into a generic file to be moved to the shared folder. HOWEVER, I have now been told by the business that I need to retain the original file name, rather than all data dumped into a single generic file.
Now I know I can re-write my ETL and do this by looping through each file, doing my checks and as long as everything passes, just move the original/good files to the desired folder. But I've been working on this ETL for quite some time, and have only this final piece to complete, I'd like to try to recreate the file using the origina file name. I have retained the original file name in a column in the final table, but my question is: How can I create flat file destination using values that come from a field in a table. Again, there are multiple files to be created, so I can't just use one variable and call it a day.
Any thoughts or guidance would be greatly greatly appreciated!