SSIS 2008 – Read roughly 50 CSV files from a folder, create SQL table from them dynamically, and dump data.

Hello everyone,

I’ve been assigned one requirement wherein I would like to read around 50 CSV files from a specified folder.
In step 1 I would like to create schema for this files, meaning take the CSV file one by one and create SQL table for it, if it does not exist at destination.
In step 2 I would like to append the data of these 50 CSV files into respective table.
In step 3 I would like to purge data older than a given date.

Please note, the data in these CSV files would be very bulky, I would like to know the best way to insert bulky data into SQL table.
Also, in some of the CSV files, there will be 4 rows at the top of the file which have the header details/header rows.

According to my knowledge I would be asked to implement this on SSIS 2008 but I’m not 100% sure for it.
So, please feel free to provide multiple approaches if we can achieve these requirements elegantly in newer versions like SSIS 2012.

Any help would be much appreciated.

Thanks,
Ankit

---

Thanks, <b>Ankit Shah</b> <hr> Inkey Solutions, India. <hr> Microsoft Certified Business Management Solutions Professionals <hr> http://ankit.inkeysolutions.com