Quantcast
Channel: SQL Server Integration Services forum
Viewing all 24688 articles
Browse latest View live

Deleting files from different folders (multiple locations) which are older than 10 days

$
0
0

Hi,

I am very new to MSBI and learning it. I have been trying to create a SSIS package which helps me to delete the text files which are older than 30 days from different folders. For this initially, I have tried to create a package which helps me to delete text files which are older than 30 days only from one folder, but I was facing some issues while running the package. I have used, foreach loop container, script task and file system task in it. I have done the coding using Micrososft Visual Studio 2008.

  1. Initially I have created 3 variables which are shown in the below screen shot

I have created three Variables:

Name :File , DataType : String, Value:

Name: FileDate, Data Type: DataTime, Value:

Name: FileSource, Data Type: String, Value: path of the folder

 2. After that I have taken foreach loop container and have given the below settings

In Collection tab:

Enumerator: Foreach File Enumerator

Folder: path of the folder

Files : *.txt

Retrieve File Name: Fully Qualified

In Variable Mappings:

Variable: User::File

Index  : 0

 3. I have dragged a script task into the ForEach loop Container and have given the following settings

ReadOnlyVariables: User::FileSource

ReadWriteVariables: User::File,User::FileDate

In the scipt task i have presented this in the code section

added Namespace System.IO

        public void Main()
        {
            // Create a variables 'container' to store variables            
            Variables vars = null;

            // Lock SSIS variables          
            Dts.VariableDispenser.LockForRead("User::File");
            Dts.VariableDispenser.LockForWrite("User::FileDate");

            // Add variables from the VariableDispenser to the variables 'container'
            Dts.VariableDispenser.GetVariables(ref vars);

            // Variable for file information            
            FileInfo fileInfo;

            // Fill fileInfo variable with file information            
            fileInfo = new FileInfo(vars["User::File"].Value.ToString());

            // Choose between creation date and lastmodified date            
            // Fill SSIS variable with last modified date            
            vars["User::FileDate"].Value = fileInfo.CreationTime;

            // Release the locks            
            vars.Unlock();

            // TODO: Add your code here
            Dts.TaskResult = (int)ScriptResults.Success;
        }
    }
}

4. After doing this, I have linked Script task to a file system task inside the ForEach loop Container. The settings given in the foreach loop is given below

In General Tab:

Operation : Delete File

IsSourcePathVariable: True

SourceVarilable: User::File

 5. I have given a condition for the files deletion in the precedent constraint editor:

Evaluate operation:   Expression and Constraint

Value                 :   Success

Expression          :     DateDiff("dd",@[User::FileDate],@[System::StartTime])>30, 

CheckedLogical AND. All constraint must evaluate to true.

After giving all the conditons it fails :(

 6. When I am executing the code I am getting the below error:

Error: 0xC001405D at Script Task: A deadlock was detected while trying to lock variables "User::File" for read access and variables "User::FileDate" for read/write access. A lock cannot be acquired after 16 attempts. The locks timed out.
Error: 0x1 at Script Task: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.SqlServer.Dts.Runtime.DtsRuntimeException: A deadlock was detected while trying to lock variables "User::File" for read access and variables "User::FileDate" for read/write access. A lock cannot be acquired after 16 attempts. The locks timed out.
 ---> System.Runtime.InteropServices.COMException (0xC001404D): A deadlock was detected while trying to lock variables "User::File" for read access and variables "User::FileDate" for read/write access. A lock cannot be acquired after 16 attempts. The locks timed out.

   at Microsoft.SqlServer.Dts.Runtime.Wrapper.IDTSVariableDispenser100.GetVariables(IDTSVariables100& ppVariables)
   at Microsoft.SqlServer.Dts.Runtime.VariableDispenser.GetVariables(Variables& variables)
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.Dts.Runtime.VariableDispenser.GetVariables(Variables& variables)
   at ST_7927eea4463a495eaa6e3a951b0450e7.csproj.ScriptMain.Main()
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.RuntimeType.InvokeMember(String name, BindingFlags bindingFlags, Binder binder, Object target, Object[] providedArgs, ParameterModifier[] modifiers, CultureInfo culture, String[] namedParams)
   at Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTATaskScriptingEngine.ExecuteScript()
Task failed: Script Task
Warning: 0x80019002 at Delete Archived Files: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The Execution method succeeded, but the number of errors raised (3) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
Warning: 0x80019002 at Package: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED.  The Execution method succeeded, but the number of errors raised (3) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.

  • I am littel bitconfused with the connection Manager, What should i mention in the connection manager, Is it required for this?

I need some help on this and correct my mistakes which i have done.

Also suggest, how to delete text files which are older than 30 days from multiple folders.


move file with date and time

$
0
0

hi,

i m usinf file system task to move file , using following expression,

@[User::ArchivePath] + SUBSTRING( @[User::myfilevalue] , 1 , FINDSTRING( @[User::myfilevalue],".",1) - 1 ) + "-" + (DT_STR, 2, 1252) Month( @[System::StartTime] )+

(DT_STR, 2, 1252) Day( @[System::StartTime] )+

 (DT_STR, 4, 1252) Year( @[System::StartTime] )+ "- "

 + RIGHT("0" + (DT_WSTR,2)DATEPART("hh",  @[System::StartTime]), 2) + ":"
    + RIGHT("0" + (DT_WSTR,2)DATEPART("mi",  @[System::StartTime]), 2) + ":"
    + RIGHT("0" + (DT_WSTR,2)DATEPART("ss",  @[System::StartTime]), 2)+

 SUBSTRING( @[User::myfilevalue] , FINDSTRING( @[User::myfilevalue],".",1) , LEN( @[User::myfilevalue] ) )

when i run package it shows error like "n error occurred with the following error message: "The given path's format is not supported.".

if i remove 

 + RIGHT("0" + (DT_WSTR,2)DATEPART("hh",  @[System::StartTime]), 2) + ":"
    + RIGHT("0" + (DT_WSTR,2)DATEPART("mi",  @[System::StartTime]), 2) + ":"
    + RIGHT("0" + (DT_WSTR,2)DATEPART("ss",  @[System::StartTime]), 2)+

its working fine,but i want time also attached with my file

XML package Configuration

$
0
0

Hi,

I have packages that use XML config file as package configurations. Currently, the test packages are on test server share and their XML config files are also on the share. The packages are scheduled as job through SQL agent and they work fine.

My ques is that when I will deploy the packages to prod, and lets say I load the packages the prod share, how would I deal with their XML config files? Will I just simply copy the XML config files and paste on prod share? Can someone please provide me step by step instructions in this regard?


Thanks, hsbal

SSIS appears to have lost my control flow items....

$
0
0

I had a bunch of task that I had been working on for weeks. THere were a group that I wanted to test so I put all the ones that I didn't want to test into a sequence container and disabled them all. When I try to take the task back out of the container I get the error message: The element cannot be found in a collection. What?! I can open the task and see all the data flow items and configurations but SSIS refuses to let me take the task back out of the container. Additionally some of the task were connected but now for some reason after running the package all the prescident constraints have disappeared. How do I save literally weeks of work w/o painstakingly reconstructing all of my task?!

.xlsx file

$
0
0

Hi, I'd like to export my SQL 2012 results to a .xlsx file. 

I've already tried the SQL Import and Export Wizard - it doesn't give me any Excel options.  I have Excel 2010 on my computer.

Can someone please provide information on how to do this, other than copying and pasting from SQL? 

Thanks


I hear and I forget. I see and I remember. I do and I understand.


how to capture the string(trailoer record) in flat file using SSIS?

$
0
0

Hello everyone, i have a question on how to capture the trailor record in flat file? file looks like thi below; report format looks ike below

i have to capture the mem records input in to my table through ssis ( file looks exacty like this, i just placed the format not the exact values).

I appreciate your help

Thanks

john

                                                                vendor_partc_return_file

vxppoi                                                                  page1                                                                                acknowledge receipt

date                                                        time                                                                   eligibility report

-------------------------------------------------------------

provider:     MA8888

Account:   *ALL

Group:      *ALL

------------------------------------------------

IDENTIFIER: MA8888                  version: 62

reformat: ackrefel88 Libname:Phhaham

------------------------------------------------------

Name             y

address           y

date                y       00/00/00

   member records input : 100                                   group records input : 0

****end of report ****

DTS migration wizard, where does it put migrated version?

$
0
0

From SQL Server Management Studio I right click on Management/Legacy/DTS and select Migration Wizard and enter a SQL Server Source and Destination. A list of DTS packages from the source server is displayed and I select one of the DTS packages and give the destination a unique name. The wizard runs and says it's successful. I can't find the new SSIS version of the DTS package with the new name that I've given it, however. Where does it put the new migrated copy of the DTS package.

Thanks,

John

.DTSX Job Failing in SQL Server

$
0
0

Hello Community,

I am trying to run a .dtsx file from SQL Server.  When I run the job it returns an error "Execution of job 'Fact Table Monitoring' failed. See the history log for details.()

I am not sure where this log is located and I am able to run that .dtsx file from command line.


CSV file source from Vendor with commas in data

$
0
0

We are getting a flat file from one of our vendors for reporting purposes.  It is a csv file.  The issue is they have an extra comma in the date field but no text qualifier so the source editor in SSIS reads the date as 2 separate fields.  

Here is an example:

,21-Feb, 2013 18:02:59,21-Feb, 2013 00:02:00,

So instead of reading them as 2 date fields it reads it as 4 fields.  Any thoughts on how to correctly parse that out?  Fixed width won't work because of the other column lengths.

ADO Net Source

$
0
0

I'm using the ADO Net Source with a SQL Command for the Data Access Mode. I need to use one parameter. I can use the expression editor to set up the sql query with the parameter, but then what would I enter for the SQL Command Text where the SQL Query goes? I tried entering the query with the ? for the parameter but that doesn't work. Do I need to enter the query here or should I be entering somethign else since I have the query setup in the Expressions of the Data flow in the control flow tab?

Thanks

Multiple Selects results to single flat file via SSIS

$
0
0

SQL Server 2008 R2 - Windows 2008 R2.

I am trying to get a single output file from about 20 different select statements with no luck. Here is what happened. The developers have ran a query with about 20 select statements in Query Analyzer in SQL Server 2000 and save the result and sent it to client. Client approved the result and asked us to send the result files daily. I am trying to automate this and can't find a way around it in SSIS.

The query starts with

declare

@packages_changed_in_X_days int

set

@packages_changed_in_X_days = 1

This applies to most of the 20 queries lke one of them,

.....................INNER

JOIN dbo.packages pkg on b.oid= pkg.oidWHERE

pkg.lastchanged>=dateadd(d,-(@packages_changed_in_X_days),getdate())................

How can I get the result of all these 20 selects into one flat file ?. I have created a stored procedure with all the 20 statements and try to execute that from SSIS but still no luck.

I have created an Execute SQL task and added "EXEC sp_procedure" into "SQL Statement" window, choose "Full Result Set" and create a object type variable. It executes without any errors but how can I send the whole result (Execution of the stored procedure) into a flat file (Muliple rows result for each Select statements) ?.

 

Thanks for any pointers.

SSIS

$
0
0
How can package variables be referenced within SSIS?

A very small question on excel

$
0
0

Hi,

I am trying to dump data from sql server to excel.

My question: Every time i run it is appending the data to the excel. I wanted it to create a new excel every time it runs.

I know its a very simple...but not able to do this...

Any help ?


Mark as answer if the post help you... Regards, Indraneel A

Execute Stored Procedure in ADO.NET Source

$
0
0

I am wondering if there is a way to  execute a stored procedure in ADO.NET and return a result set to a different database. I would prefer not to use a script task if possible. 

I have been trying to do this in a Dataflow task:

I set it the execute statement in the expressions in the data flow task properties. And the query appears in the command task:

And I even get results when I preview is but an error is still there and when I try to map it to the destination it says there are no available input columns. The error on the ADO.NET Source states that the component needs to be disconnect connected before it can be connected when I click on the source the error: the component is not in a valid state pops up.

Any suggestions?

Excel Refresh fails under SQLAGENT account.

$
0
0

Hi ,

As I posted before in below thread,

http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/c7f52891-7920-480c-999e-183d8f1b6387

I have a SSIS package which uses a Script task to refresh an Excel report with Pivot tables and queries in it.

When I schedule that package as to run as job, it fails when runs under Domain\SQLAGENT.

But , it runs fine under a Proxy created using my windows login (Domain\MywindowsLogin).

I tried different options as I mentioned in the above link.

Can somebody advise me what else can be the problem.

I am in bad need of help.

Thanks in advance.

-Madhuri


Packageformat field in sysssispackages table

$
0
0

Hi All,

I have created an SSIS package using BIDS 2008 and deployed it on SSIS 2008 server. When querying the sysssispackages table in msdb database I am getting value for column"packageformat" as 0 for the deployed package. However, as as my package is "SSIS 2008 package", I was expecting 3 (Packageformat). Any idea what does 0 signifies?





Pivot transformation

$
0
0

Hi all,

below is my original source

when i am pivoting this in sql server am getting correct output,see below screen shot

custBEERMILKSODAVEG
FRED24 3NULL NULL
KATE12 165


but when am trying same thing in ssis using Pivot transformation am getting following output

am expecting only one row for each customer but it is giving me two rows for customer "KATE"

so how can i overcome this without grouping the data

thanks,

kishor

Merge Join Problem

$
0
0

Hi,

I've a problem with multiple packages with a merge join (full outer join) with a conditional split resulting in new, update and delete paths.

I've a merge join which combines data from a staging table and a destination table. I'm using a merge join with a key (but other packages with mmultiple keys give the same problem.. An initial load is working properly but when i rerun the package it seems that the keys are NOT matched properly. What could be the problem?

Greetz,

Hennie

NAme to divide in firstname lastname

$
0
0

Hi

I have one column in Source table called as NAME

Let say Ashley James Lee

I need its output in destination column

First name  Lastname..

Ashley        James Lee

Kindly help

How can i do this uisng SSIS.

Thanks With Regards

SSIS Master Package taking long duration / gap between two sub package execution.

$
0
0

Hi All,

I have a master package which runs other sub packages and its schedule to run in back end, in master package each Execute package task transfers control to other on "Success", we have used logging in each sub packages to report start time and end time of the package execution in sysssislog table.

The problem is if i query sysssislog table there is huge gap between end time of package A and start time of package B. I don't understand what is happening in between.

If i run the same in front end from IDE this doesn't happen at all, I don't understand how to debug this.

Please help me.

Regards,

Harshit Vasa

Viewing all 24688 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>