Hi,
We are trying to build a SSIS framework in my company. We have 200+ feeds written in C++, C#, Java and we want this framework to migrate all these feeds. All feeds are launched by Appworx and follow similar pattern. Step 1: Get FTP credentials from database. FTP Download the files to the file system. Step 2(optional): XSLT transformation, Step 3: Parse and Load the file into a table. We are planning to create 8-10 child packages (2-3 packages to support different FTP tasks, 5-6 packages to support different file formats). And then we want to create a Project (Parent package which maintains the variables) for each feed which calls these child packages plus the file to table mappings.. Each Project will have a package configuration as xml file or DB config.. Appworx will kick off each Project one by one.
Is this design feasible? Or is there a better approach or a better tool to do this? The below is the list of design decisions we are facing.
- Is it recommended that the parent package references the child packages as Project reference or External reference? We prefer external reference because if a child package is upgraded we just overwrite the old package in one place on the network. If we use Project reference then each child package is shipped along with the parent package and we need to deploy all projects which use the child package.
- The parent package cannot have both parameters and variables. Variables are preferred because Child packages will be able to set parent variables but not its parameters. Is there a need to use parameters? Also, how can we force a parent package to define variables needed in the child package? Like a reverse interface implementation concept.
- Is deploying projects as SSIS services or on File system better? We prefer file system as its easier to update.
- We want to be able to turn on/off execution of child packages. This can be achieved only if the Project – Child packages mapping is configurable in the xml file or database. Example: lets say my project has Child1,2 and 3 where Child 1 is a FTP. The first time the project runs, child 1 is complete but 2 fails. I want to be able to re-run the project with just 2 and 3.
Your insights are much appreciated.
BTW: I referred to Andy's blogs about SSIS framework. http://sqlblog.com/blogs/andy_leonard/archive/2011/09/16/designing-an-ssis-framework.aspx
But his examples dont include passing variables between parents and children which he admits is complex,.