So I know the SSIS Solution works.
Please allow me to explain.
- Allow for a @DateParmOverride Variable in the SSIS Package. If False, then derive the @GlobalStartDate Variable to be the previous Saturday and derive the @GlobalEndDate to be the previous Friday when this is run through SQL Server Job Agent at 4AM on Saturday
- We then CREATE and INSERT the results of our SQL Server Stored Procedure using the @GlobalStartDate and @GlobalEndDate Parameters into a Global ##Temporary Table in order to determine if we have a result set for the indicated period
- We then do a SELECT COUNT(*) against that created and inserted Global ##Temporary Table in order to determine if we need to process for the previous week or simply send an Email indicating that there are No Records for that week
Like I have stated, we know this solution works because when we override and set the variable @DateParmOverride to "True" and provide the @GlobalStartDate and @GlobalEndDate variables in the SQL Server Agent Job, the job determines there is a result set and processes accordingly.
Similarly, if we run the Job in SQL Server Agent One Time and during the day using @DateParmOverride being "False", it derives the @GlobalStartDate and @GlobalEndDate variables correctly and processes accordingly.
The only time it seems to choke is when it runs on its normal schedule...on Saturday mornings at 4AM. We get an erroneous Email which states that there are No records to process for the derived prior week dates when we know there are.
Now the queries are going across a linked server utilizing the SSIS Connection Manager and my first thought is maybe the job is running too fast in the middle of the night...trying to determine the COUNT(*) of the Global ##Temporary Table before the CREATE and INSERT were done. I did add a
WAITFOR DELAY '00:00:05'
;
in the Execute SQL Task that creates and inserts into the Global ##Temporary Table. But last week it seemed to rip right through again without producing a file when data exists for the derived week.
Has anyone ever experienced anything like this? I hate like hell to re-work through the solution when I know it works and potentially derive the @StartDate and @EndDate parameters within the SQL Server Stored Procedure and handle it that way.
Does anyone have any explanations...work arounds...I know back in the COBOL days I could put a READY TRACE and DISPLAYS to try and de-bug. Does anything like that exists in SSIS and running a SSIS Package through SQL Server Job Agent?
Thanks soooo much for your review and am hopeful for a reply.