Frends.FTP.DownloadFiles / 2.1.0

Summary

Transfers files off an FTP or FTPS server, and decides per file what happens to the source and to a name collision at the target. Counts and errors come back per file, so a partly successful run is visible as such.

Typical use — scheduled collection from a partner, where moving or renaming the source is what prevents a second run refetching it

Remarks

Transfer overview

The file transfer progress has the following steps:

Initialize

Initializes the transfer and opens the source connection.
ListFiles

Get a list of files from the source endpoint according to the filename/mask. If there are no files to transfer, the source connections are closed, and the transfer finishes. The result of the task will then depend on what the option Source.NotFoundAction was set to. If it was set to Error, either the #result.Success property will be set to false, or if the ThrowErrorOnFailure option was also set, an exception is thrown. If the Source.NotFoundAction was set to Info or Ignore, the #result.Success will be set to true and #result.ActionSkipped also set to true.
Transfer files

If there are files to transfer, they are then transferred individually. For every file in the list returned from the source endpoint, the following process is repeated:
1. GetFile
  
  Get a file from the source endpoint to the local work directory. If the parameter RenameSourceFileBeforeTransfer is set to true, the file is first renamed with a temporary filename before transfer.
2. Rename or move the source file.
  
  This is done before transferring the file to the destination, this means that possible errors in the renaming or moving that would cause the transfer to fail will happen as early as possible - before we actually try to transfer files onward.
3. Download the file.
  
  If destination file already exists, depending on the parameter Destination.Action either an exception is thrown, the destination file is overwritten or the source file is appended to the destination file.
  
  If the parameter RenameDestinationFileDuringTransfer is true, the file is first transferred with a temporary file name and afterwards renamed to intended filename, otherwise the file is transferred with the intended filename. The intended filename has its possible file masks expanded.
4. Do the source file operation.
  
  Perform the operation defined by the SourceOperation.
Finish

Close the source and destination endpoint connections.

If the transfer is cancelled (e.g. by calling Terminate on the process instance), the files that are currently being transferred will be processed until finished, but no new files will be transferred. The cancelled transfer end result will be Failed.

Macro reference

Macros can be used to dynamically configure source directory, destination directory or destination file name for a file transfer.

Generally the following rules apply for macros:

Macros are case insensitive.
You can use any number of macros in all of the cases.
Dates and times are formatted with leading zeros.

The following macros can be used with all of dynamically configurable locations for file transfer:

%Ticks% = will be replace with the current time as Ticks.
%DateTime% = will be replaced with date and time in format: "yyyy-MM-dd-HH-mm-ss".
%DateTimeMs% = will be replace with date and time in format: "yyyy-MM-dd-HH-mm-ss-fff".
%Date% = will be replaced with date in format: "yyyy-MM-dd".
%Time% = will be replaced with time in format: "HH-mm-ss".
%Year% = will be replaced with current year.
%Month% = will be replaced with current month.
%Day% = will be replaced with current day.
%Hour% = will be replaced with current hour.
%Minute% = will be replaced with current minute.
%Second% = will be replaced with current second.
%Millisecond% = will be replaced with current millisecond.
%WeekDay% = will be replaced with a number of weekday, ranging from 1 (monday) to 7 (sunday).
%Guid% = will be replaced with a new unique identifier.
%TransferId% = will be replaced with the transfer id.
%TransferName% = will be replaced with TransferName parameter specified in Connection point schema.
%TransferGroupName% = will be replaced with TransferGroupName parameter specified in routine's task arguments.

Destination file name has two additional macros that can be used for dynamically creating destination file name.

%SourceFileName% = will be replaced with source file name without extension.
%SourceFileExtension% = will be replaced with source file's extension, with the dot '.' included, i.e. if the source file is named 'foo.txt', the %SourceFileExtension% will be expanded as '.txt'. If the source file name does not have an extension, the macro result will be empty, i.e. for original file name "foo", "bar%SourceFileExtension%" will result in "bar"

Details

Frends version: 5.7 and newer
Agent: Crossplatform
Target framework: net8.0

Input parameters for the FTP download task, combining source, destination, and transfer info.

Name	Type	Default	Example	Description
Source	`Source`	–	`new Source { Directory = "/files", FileName = "*.txt" }`	Source file location and transfer options.
Directory	`String`	–	`/directory`	Directory on the server.
FileName	`String`	–	`example*.csv`	File name or file mask of the files to be fetched.
NotFoundAction	`SourceNotFoundAction`	`Error`	`SourceNotFoundAction.Error`	What to do if source file is not found. Error = alarm and fail, Info = alarm info and quit with success status, Ignore = quit with success status. Possible values: `Error`: Will log an error in case source files are missing. `Info`: Will log a notification in case source files are missing. `Ignore`: Will not log anything and continue execution without any additional actions.
Operation	`SourceOperation`	`Delete`	`SourceOperation.Delete`	What to do with the source file after transfer. Possible values: `Delete`: Will delete files after downloading. `Rename`: Will rename the files after downloading. `Move`: Will move the files after downloading. `Nothing`: Will do nothing to source files after downloading.
FileNameAfterTransfer	`String`	–	`%SourceFileName%%SourceFileExtension%`	Parameter for Rename operation. Set the file name for the source file.
DirectoryToMoveAfterTransfer	`String`	–	`/movedFiles`	Parameter for Move operation. Sets the directory to which source files will be moved after transfer.
Destination	`Destination`	–	`new Destination { Directory = "C:\\downloads", Action = DestinationAction.Overwrite }`	Destination directory location and overwrite options.
Directory	`String`	–	`/somedir`	Directory on the server.
FileName	`String`	–	`myFile%Year%.txt`	File name of the destination file with possible macros.
Action	`DestinationAction`	`Error`	`DestinationAction.Error`	Operation to determine what to do if destination file exists. Possible values: `Append`: Will either create the file(s) or append content to existing file(s). `Overwrite`: Will overwrite existing file(s). `Error`: Will throw an error if destination file(s) already exist.
Info	`Info`	–	`new Info { TransferName = "MyTransfer" }`	Optional transfer info such as transfer name, working directory, and process context.
TransferName	`String`	`"FTPDownload"`	`FTP Download`	Optional descriptive name for the transfer. Will be included in the file transfer log entries and also in all error messages in the event log.
WorkDir	`String`	–	`c:\workDir`	Directory where temporary files are stored during transfer.
ProcessUri	`String`	`#process.uri`	`#process.uri`	The process URI, use #process.uri
TaskExecutionID	`String`	`#process.executionid`	`#process.executionid`	Reference to the Task execution id, use #process.executionid