Frends.AzureDataLake.UploadFiles / 2.0.0
Summary
Copies files from the agent's file system into a container, optionally walking subdirectories, and can leave the target file open for appending.
Typical use — delivering a batch a Process produced into a lake layout
Input parameters.
| Name | Type | Default | Example | Description |
|---|---|---|---|---|
| Source | Source |
– | { SourceDirectory = "c:\temp", SourceFilePattern = "*.csv" } |
Specify source files parameters |
| SourceDirectory | String |
– | c:\temp |
Upload all files from the given directory. |
| SourceFilePattern | String |
– | *.*, Search*.*, *.xml, foobar.txt |
The search string is used to match against the names of files in the path. This parameter can contain a combination of valid literal path and wildcard (* and ?) characters, but it doesn't support regular expressions |
| Destination | Destination |
– | { ConnectionMethod = ConnectionMethod.ConnectionString, ContainerName = "examplecontainer", DestinationFolderName = "uploads" } |
Specify destination Data Lake parameters |
| ConnectionMethod | ConnectionMethod |
ConnectionString |
ConnectionMethod.ConnectionString |
Connection method to be used to connect to Azure Data Lake. Possible values:
|
| ContainerName | String |
– | examplecontainer |
Name of the Azure Data Lake container. Task will convert all letters to lowercase. |
| ConnectionStringSecret | String |
– | DefaultEndpointsProtocol=https;AccountName=accountname;AccountKey=Pdlrxyz==;EndpointSuffix=core.windows.net |
Connection string for Azure Data Lake. |
| StorageAccountName | String |
– | storage |
Name of the Azure storage account. |
| ApplicationID | String |
– | Y6b1hf2a-80e2-xyz2-qwer3h-3a7c3a8as4b7f |
Application (Client) ID of Azure AD Application. |
| TenantID | String |
– | Y6b1hf2a-80e2-xyz2-qwer3h-3a7c3a8as4b7f |
Tenant ID of Azure Tenant. |
| ClientSecretSecret | String |
– | Password! |
Client Secret of Azure AD Application. |
| DestinationFolderName | String |
– | ExampleDir |
Name of the blob folder. If left empty, file will be uploaded directly to container. |
| Overwrite | Boolean |
False |
true |
How the existing file will be handled. |
| UploadFilesRecursively | Boolean |
True |
true |
True: Upload all files matching pattern, even in nested directories. False: Upload files matching pattern, which are only directly in Source Directory |
| Close | Boolean |
True |
true |
Specifies whether the uploaded file should be finalized (closed) after upload. When true, it triggers a "final update" event in Azure Storage Events. |