Frends.FTP.DownloadFiles / 2.1.0
Summary
Transfers files off an FTP or FTPS server, and decides per file what happens to the source and to a name collision at the target. Counts and errors come back per file, so a partly successful run is visible as such.
Typical use — scheduled collection from a partner, where moving or renaming the source is what prevents a second run refetching it
Remarks
Transfer overview
The file transfer progress has the following steps:
Initialize
Initializes the transfer and opens the source connection.
ListFiles
Get a list of files from the source endpoint according to the filename/mask. If there are no files to transfer, the source connections are closed, and the transfer finishes. The result of the task will then depend on what the option
Source.NotFoundActionwas set to. If it was set to Error, either the #result.Success property will be set to false, or if theThrowErrorOnFailureoption was also set, an exception is thrown. If theSource.NotFoundActionwas set toInfoorIgnore, the#result.Successwill be set to true and#result.ActionSkippedalso set to true.Transfer files
If there are files to transfer, they are then transferred individually. For every file in the list returned from the source endpoint, the following process is repeated:
GetFile
Get a file from the source endpoint to the local work directory. If the parameter
RenameSourceFileBeforeTransferis set totrue, the file is first renamed with a temporary filename before transfer.Rename or move the source file.
This is done before transferring the file to the destination, this means that possible errors in the renaming or moving that would cause the transfer to fail will happen as early as possible - before we actually try to transfer files onward.
Download the file.
If destination file already exists, depending on the parameter
Destination.Actioneither an exception is thrown, the destination file is overwritten or the source file is appended to the destination file.If the parameter
RenameDestinationFileDuringTransferistrue, the file is first transferred with a temporary file name and afterwards renamed to intended filename, otherwise the file is transferred with the intended filename. The intended filename has its possible file masks expanded.Do the source file operation.
Perform the operation defined by the
SourceOperation.
Finish
Close the source and destination endpoint connections.
If the transfer is cancelled (e.g. by calling Terminate on the process instance), the files that are currently being transferred will be processed until finished, but no new files will be transferred. The cancelled transfer end result will be Failed.
Macro reference
Macros can be used to dynamically configure source directory, destination directory or destination file name for a file transfer.
Generally the following rules apply for macros:
- Macros are case insensitive.
- You can use any number of macros in all of the cases.
- Dates and times are formatted with leading zeros.
The following macros can be used with all of dynamically configurable locations for file transfer:
- %Ticks% = will be replace with the current time as Ticks.
- %DateTime% = will be replaced with date and time in format: "yyyy-MM-dd-HH-mm-ss".
- %DateTimeMs% = will be replace with date and time in format: "yyyy-MM-dd-HH-mm-ss-fff".
- %Date% = will be replaced with date in format: "yyyy-MM-dd".
- %Time% = will be replaced with time in format: "HH-mm-ss".
- %Year% = will be replaced with current year.
- %Month% = will be replaced with current month.
- %Day% = will be replaced with current day.
- %Hour% = will be replaced with current hour.
- %Minute% = will be replaced with current minute.
- %Second% = will be replaced with current second.
- %Millisecond% = will be replaced with current millisecond.
- %WeekDay% = will be replaced with a number of weekday, ranging from 1 (monday) to 7 (sunday).
- %Guid% = will be replaced with a new unique identifier.
- %TransferId% = will be replaced with the transfer id.
- %TransferName% = will be replaced with TransferName parameter specified in Connection point schema.
- %TransferGroupName% = will be replaced with TransferGroupName parameter specified in routine's task arguments.
Destination file name has two additional macros that can be used for dynamically creating destination file name.
- %SourceFileName% = will be replaced with source file name without extension.
- %SourceFileExtension% = will be replaced with source file's extension, with the dot '.' included, i.e. if the source file is named 'foo.txt', the %SourceFileExtension% will be expanded as '.txt'. If the source file name does not have an extension, the macro result will be empty, i.e. for original file name "foo", "bar%SourceFileExtension%" will result in "bar"
- Fixed XML documentation: corrected
<returns>tag to match actualResultclass properties, fixed incorrect parameter references in<frendsdocs>and improved<param>descriptions.
- Breaking The task method signature has been updated to consolidate the Source, Destination, and Info parameters into a single
Inputparameter tab. Existing flows using this task will need to be updated to use the newInputparameter structure. - Upgraded target framework from net6.0/netstandard2.0/net471 to net8.0.
- Renamed
ThrowErrorOnFailoption toThrowErrorOnFailurefor consistency with Frends standards. - Added
ErrorMessageOnFailureoption for a custom error message when the task fails. - Added
Errorproperty to the task result, containing error details when the transfer fails andThrowErrorOnFailureis disabled. - Updated package metadata: added Company, Product, and updated PackageProjectUrl to https://frends.com/.
- Fixed: Added missing changelog and restored redirection to source code.
- Updated the Newtonsoft.Json package to version 13.0.3.
- Fixed bug when using wildcard filemask and no source files were found.
Improved
- Improved Operations log by adding more logging steps.
- Added setup for FtpClient.ReadTimeout, FtpClient.DataConnectionConnectTimeout and FtpClient.DataConnectionReadTimeout which were all defaulting to 15 seconds.
- Added search for local certificates from machine certification store.
- Breaking Added parameters ClientCertificationName and ClientCertificationThumbprint for exclusive search of client certification.
- Moved client.EncryptionMode setting to be done if UseFTPS is enabled.
- Connection.BufferSize documentation update. Value is set as bytes instead of KBs.
- Removed Connection.ClientCertificatePath's default value.
- Fixed RestoreModified method to use the whole path of the destination file and not just the name.
- Fixed logging of successful transfer to use the name of the file and not reference of FileItem class.
- Added DisplayFormat annotation to the Connection ClientCertificatePath input field.
- Initial implementation