Configure hotfolder actions

Hotfolder actions configure automated commands that handle files that are transferred to the censhare server. Hotfolder actions are specific for each module.

Purpose

For some modules in censhare, hotfolder actions are available to process files that are transferred to the censhare Server. Usually, hotfolder actions are an automated variant of the respective manual action.

Context

Hotfolder actions are configured in the censhare Admin Client in the respective Configuration/Modules configuration.

Prerequisites

You must be familiar with the Module, for which you want to configure a Hotfolder action, and with the node's schema of the configuration.

Introduction

Besides the manual file upload in censhare, the censhare Server can import and process files that are uploaded to defined directories on the file system (aka hotfolders). The in directory of a hotfolder action can be connected to third-party systems or accessed via FTP.

If a hotfolder action is configured for a module, the censhare Server scans the corresponding in directory. If one or multiple files are uploaded to that directory, the hotfolder actions process the files, creates or updates the respective assets, and imports the files into the filesystem.

In some special use cases, a hotfolder action accesses the data in the uploaded files and writes these data into the target assets. In this case, no assets are created from the uploaded file. This type of hotfolder action is used for example in the TMX importer module of the Translation with Memory application.

Hotfolder configuration dialogs

The configuration dialogs of hotfolder actions in the censhare Admin Client show the necessary fields to configure the respective action. Therefore, dialogs in different hotfolder actions can show different fields and options. However, the underlying node schema is the same for all hotfolder actions.

The General setup of hotfolder actions is similar to the general setup of other modules in censhare. For a reference of this part of the setup, see the General setup section below.

The Filesystem setup is required for all hotfolder actions. For a reference of this part of the setup, see the Filesystem setup section below.

For other configuration fields than the ones referred in the General setup and Filesystem setup sections, see the Node schema and Attributes section.

General setup

In the General setup area, configure the hotfolder action as follows:

Field	Default	Description
Server Name	-	In a censhare cluster or master server configuration, you can restrict the server action to one server or create different configurations on different servers. By default, the action is enabled on all servers.
Enabled	false	Must be enabled to.
Title	-	If a resource placeholder exists, do not change. For custom hotfolder actions without resource placeholders, enter a name.
Description	-	If a resource placeholder exists, do not change. For custom hotfolder actions without resource placeholders, enter a description.
Version	-	For internal use only.
Interval	30 seconds	See <listen-events/>.
Process user	-	Optional. To process different kinds of data/files, you can create multiple configurations of the same hotfolder action. In this case, we recommend to select a process user.

Filesystem setup

Field	File system	Path
Input dir	The file system in which files are uploaded. See scan-dir-filesystem in <scan/>.	The relative path to uploaded files. See scan-dir-relpath in <scan/>.
Working dir	The file system in which files are moved when they are processed. See work-dir-filesystem in <scan/>.	The relative path to files when they are processed. See work-dir-relpath in <scan/>.
Output dir	The file system in which processed files are moved after they were successfully processed. See completed-dir-filesystem in <import-manager/>.	The relative path of files when they are successfully imported. See completed-dir-relpath in <import-manager/>.
Error dir	The file system in which processed files are moved if they could not be processed. See error-dir-filesystem in <import-manager/>.	The relative path to files that could not be processed. See error-dir-relpath in <import-manager/>.

Node schema

The underlying node schema of all hotfolder actions is as follows. For a reference of the [ATTRIBUTES] that are available in each node, see the Attributes section below.

$stringEscapeUtils.escapeHtml($body)

XML

Attributes

<listen-events/>
- <timer/>
- <cron/>
<scan/>
- <match/>
  - <data/>
    - <asset-duplicate-check/>
    - <attribute/>
<import-manager/>

<listen-events/>

The event listenter is a standard method that is required for the automatic execution of commands. In this node, use either a <timer/> node for intervals, or a <cron/> node for cron expressions to control the execution of the command.

<timer/>

The time node sets a fixed intervall (in seconds) for the execution of the hotfolder command.

Attribute	Description
delta-sec	Scans the in directory every x seconds. Default is 30.

<cron/>

Attribute	Description
pattern	Cron expression for the automatic execution of the hotfolder command.

<scan/>

This method creates a command that processes files that are stored in the defined in directory.

Attribute	Description
group-result	If true, results are grouped by <match/> nodes and sub-directories.
group-export-key	Default is file-export-group. Must be identical with the group-export-key in the <import-manager/> node (cross reference).
fork	If true, the command creates a copy of itself for each process, and the original command remains in waiting state. If false, only one command is used to process all files. Default is true.
scan-dir-filesystem	The file system where uploaded files are stored. This file system must be accessible from external locations, for example via FTP. The default is interfaces. File systems are configured in the Master data/File systems table in the censhare Admin Client.
scan-dir-relpath	The relative path that points to the directory that the hotfolder action scans for incoming files. Use the pattern file:[hotfolder]/in (replace [hotfolder] by the name of the hotfolder action).
work-dir-filesystem	Moves files to the working directory before a command is executed. Recommended to avoid that files are queued a second time when the in directory is scanned again before the last invoked command is finished. Default is interfaces.
work-dir-relpath	The relative path to the working directory. Default is file:[hotfolder]/work (replace [hotfolder] by the name of the hotfolder action).
out-filesystem-key	XPath expression to retrieve the filesystem key. Default is file@filesystem.
out-relpath-key	XPath expression to retrieve the file URL on the filesystem. Default is file@url.
out-url-key	Optional. The attribute key where the file URL is placed in the executed command template.
context-url	Optional. The context url for relative urls in the dir and work-dir attributes. If context-url itself is relative, then the path is relative to the ImportManager context.
max-files	The maximum number of files that are moved from the in directory to the work directory.
max-queue-size	The maximum number of commands that can run in parallel, regardless of the command state. If fork = false, the max-queue-size is always 1. If max-files = 1, the max-queue-size is ignored.
create-unique-url	If set to true, files that are moved to the working directory, cannot be overwritten accidentally. Default is true.
last-modification-delay-sec	Only processes a file when its last modification date is older than the delay (in seconds). The delay ensures that the file upload is complete before a command starts processing the file. This attribute is necessary for UNIX systems because it is not possible to determine whether a file transfer is finished or still in progress.
scan-subdirectories	If true, files to be processed can be placed in nested directories.
include-base-path	Optional. If scan-subdirectories is set to true, this attribute can be used to specify a regular expression that matches sub-directories including their relative path from the scan-dir-relpath attribute.
include-directory-regex-pattern	If scan-subdirectories is set to true, this attribute can be used to specify a regular expression for sub-directories to be included. Default is .* (meaning, any sub-directory is included).
include-directory-glob-pattern	Optional. If scan-subdirectories is set to true, this attribute can be used to specify a wildcard expression for sub-directories to be included.
exclude-directory-regex-pattern	Optional. If scan-subdirectories is set to true, this attribute can be used to specify a regular expression for sub-directories to be skipped.
exclude-directory-glob-pattern	Optional. If scan-subdirectories is set to true, this attribute can be used to specify a wildcard expression for sub-directories to be skipped.
ignore-non-matching-files	If true, files that do not match the include-file-regex-pattern or exclude-file-regex-pattern file pattern, are skipped. Otherwise, non-matching files are moved to the error directory.
add-data-node	Optional. If true, adds a <data/> node to each <match/> node for each executed command.

<match/>

In the match node, you can define criteria in regular expressions that must be matched to process a file, or criteria that must be matched to skip a file.

Attribute	Description
include-file-regex-pattern	Only files that match this regular expression are processed. Either include-file-regex-pattern or include-file-glob-pattern must be defined.
include-file-glob-pattern	Only files that match this wildcard expression are processed. Either include-file-regex-pattern or include-file-glob-pattern must be defined.
exclude-file-regex-pattern	Optional. Files that match this regular expression are skipped.
exclude-file-glob-pattern	Optional. Files that match this wildcard expression are skipped.

<data/>

In the data node, you can write attributes that are applied to each file that is processed. The in the <scan/> node, the add-data-node attribute must be true for this method.

Attribute	Description
create-assets	Set to true to create assets from imported files. In this case, you must also set the asset type to be created. If you do not want to create assets (for example: TMX import), you can set the attribute to false, or simply remove the <data/> element.
use-duplicate-check	Set to true to perform a duplicate check. If true, set also the perform-update attribute.
perform-update	If use-duplicate-check=true, this attribute defines how to handle duplicates: Set to true to force updating the storage item, set to false to use the existing storage item.
checkout-checkin-options	Set to outin to perform a check-out and check-in of assets. This creates a new asset version. Set to update, to update the storage item. This does not create a new version.
replace-files	Set to true to replace existing storage items (non-master files).
type	If create-assets=true, set the asset type to be created with this attribute.
map-metadata	Optional. If true, file metadata are written into asset properties. Note: This requires metadata mapping.
application	Optional. Sets an application to edit the file in censhare. If no specific application is available, set to default.

<asset-duplicate-check/>

This method defines, which attributes/values are used for the duplicate check. The <asset-duplicate-check/> node is only required if use-duplicate-check is set to true in the <data/> node.

Attribute	Description
key	The asset attribute against which the duplicate check is performed. Default is @id_extern.
src-pattern	Regular expression that retrieves the string used for the duplicate check against the key attribute.
replacement	Optional replacement pattern to transform the src-pattern string.

<asset-attribute/>

This method can be used to retrieve any value from the processed file and write it into an asset attribute.

Attribute	Description
key	The asset attribute key. For example: @name.
src-pattern	A regular expression that retrieves the desired string value from the source file.
replacement	Optional. A replacement pattern to transform the string retrieved with src-pattern.
use-path	Optional. If true, the context path to the source is included.

<import-manager/>

The <import-manager/> method defines how to handle processed files.

Attribute	Description
group-export-key	Default is file-export-group. Must be identical with the group-export-key in the <scan/> node (cross reference).
completed-dir-filesystem	The file system in which successfully processed files are stored. Default is interfaces.
completed-dir-relpath	The relative URL to successfully processed files. Default is file:[hotfolder]/work (replace [hotfolder] by the name of the hotfolder action).
error-dir-filesystem	The file system in which files that caused an error during processing are stored. Default is interfaces.
error-dir-relpath	The relative URL to files in the error directory. Default is file:[hotfolder]/error (replace [hotfolder] by the name of the hotfolder action).
transaction-group-name	Value is hotfolder-rollback-transaction-group - do not change!

<group-asset-importer/>

The <group-asset-importer/> node contains the configuration for the processing of sub-folders. Optionally, you can add one or multiple <asset-attribute/> nodes inside this node.

Attribute	Description
group-asset-setup	If true, imported assets from sub-folders are grouped with the corresponding Group assets.
include-group-regex-pattern	Enter a regular expression that matches sub-folders to be included.
exclude-file-group-pattern	Enter a regular expression that matches sub-folders to be excluded.
create-asset-tree	If true, assets from sub-folders are assigned as child assets.
checkout-checkin-options	see the respective attributes in the <scan/> node.
replace-files
create assets
perform update

Result

The censhare Server runs the hotfolder action and processes the files according to the configuration. Updated and newly created assets are available in the system.