Hello everyone,
I am currently working on creation of SP to automate the process of importing flat data into SQL Server.
I want you to give me some useful advice on the best way how to realise, taking into account things I need:
- File path and name, delimeter sign, table name as variables.
- Logging process of importing in one file for every import task (errors and output results like in bcp)
- Process flat files with different column order
- Check if data we are trying to import already exists in database (don't replicate)
- Batch import
- Error cathing
Okay, now my thoughts. I would use either bcp or BULK INSERT.
Declare delimeter, source file, table as variables.
To identify column order I think we should parse first row with names and use intermediate table from which we then INSERT...SELECT into destination table in the right order.
Check if data already EXISTS before importing.
Still don't know about logging in one file with timeline. Any other errors that may occur?
Any advice would be appreciated. Thank you in advance.