I am just not getting it - how to loop through flat files and get them into a database table. I can do it in the BIDS environment, but I cannot get them to work consistently using package configurations, especially when the end user wants to change the source folder, etc. I have the following sample situation:
Folder: (on my workstation): C:\MyFolder\
MyFiles: MyFile1.txt, MyFile2.txt, MyFile3.txt, MyFile4.txt. The contents, at this point doesn't matter.
All I want to do is import them all to MyTable and then create a package configuration file to allow the user to change the foldername primarily.
I don't know what values I need in the configuration file. Right now, I have the login and password to the database, the package password, and a variable I created called strFilePath with a value in the configuraion file of C:\MyFolder\MyFile1.txt. (but I don't know how to configure that either)
Either I get no files found, or 1 file loaded, or some kind of error. I have spent many hours trying everything. Often I think it is working and it is only importing the files from my desktop instead of what is in the configuration file. When I get it to use the configuration file somehow, I end up with either nothing, 1 file, or an error.
If someone can walk me through
1) Creating the simple package
2) Creating the right variables for the source folder and files and the correct default values for each.
3) Adding the For Each Loop container and configuring the Folder and Files field in the Collection tab and the Expressions field at top, if necessary.
4) Creating the proper file mask to allow it to get all 4 of those files, 1, 2, 3, 4 respectively
5) Create a package configuration that will allow the user to change the folder, but not necessarily the file mask
6) Properly configuring the source connection manager primarily
7) Anything else vital that I didn't think of.
Any help here would be greatly appreciated. I know and have found how to do most of this. I am just having a disconnect someplace and can't get it.