Pipeline
Concept
The connector crawls items from a source system and feeds these items into a target system through a pipeline. To adjust or process each item before it gets ingested, it is possible to provide custom stages for this pipeline. These stages are software modules provided by us which can be uploaded and configured via the UI.
How to upload and use stages
To upload a stage open the pipeline view:
Then click on the + symbol between Source System and Target System and the Select Pipeline Stage view pops up:
Here you can see stages which are already uploaded to the connector. To upload a new stage, drag and drop the respective software module or click on Drag file to upload or browse to navigate to and select the module.
Once the module is uploaded it is listed in the available stages where it can be selected. After selecting the stage it is shown as step in the pipeline where it can be configured:
After completing the configuration of this single stage, it has to be saved by pressing SAVE. It is possible to set up and configure multiple custom stages in any order. To apply stages this setup of stages has to be validated and then saved by pressing the respective buttons. After saving the setup, the connector has to be restarted so that the pipeline stages are applied in the next traversals. This can be done by pressing RESTART CONNECTOR on the popped up message:
Default packaged stages
The connector is packaged with pipeline stages which are delivered but not applied by default:
-
File Item Type Enricher Stage
-
Icon Assigner Stage
-
Metadata Assigner Stage
-
Metadata Drop Stage
-
Metadata Mapper Stage
-
Root and Parent Breadcrumbs Stage
-
Metadata Regular Expression Stage
File Item Type Enricher Stage
The File Item Type Enricher Stage maps Item Types based on the File Extension of a given Item.
Mappings
As the Stage replaces the Item Type of items based on their File Extension, in the table below you can see which File Extensions map to which Item Type.
| File Extension | Nice Item Type |
|---|---|
7z |
ZIP Archive |
acm |
ACM Audio |
aspx |
HTML Document |
avi |
AVI Video |
bat |
Batch Script |
bmp |
BMP Image |
doc |
Microsoft Word |
docx |
Microsoft Word |
dot |
Microsoft Word Template |
dotx |
Microsoft Word Template |
eml |
Microsoft Outlook |
gif |
GIF Image |
htm |
HTML Document |
html |
HTML Document |
img |
Disk Image |
jpeg |
JPEG Image |
jpg |
JPEG Image |
mov |
QuickTime Video |
mp3 |
MP3 Audio |
mp4 |
MPEG-4 Video |
mpeg |
MPEG Video |
mpg |
MPEG Video |
msg |
Microsoft Outlook |
ods |
OpenDocument Spreadsheet |
one |
Microsoft OneNote |
odt |
OpenDocument Text |
Adobe PDF |
|
php |
HTML Document |
png |
PNG Image |
pot |
Microsoft PowerPoint Template |
potx |
Microsoft PowerPoint Template |
ppt |
Microsoft PowerPoint |
pptx |
Microsoft PowerPoint |
ps1 |
PowerShell Script |
rar |
RAR Archive |
smil |
SMIL Presentation |
swf |
Shockwave Flash |
tif |
TIFF Image |
tiff |
TIFF Image |
txt |
Plain Text |
wav |
WAV Audio |
xls |
Microsoft Excel |
xlsx |
Microsoft Excel |
xlt |
Microsoft Excel Template |
xltx |
Microsoft Excel Template |
xml |
XML Document |
zip |
ZIP Archive |
Configuration
| Property | Description |
|---|---|
Enable Item Type Enrichment |
Toggles the enrichment of the Item Type of items based on the respective File Extension. Specifically, this will set a display-friendly Item Type for any well-known File Extensions. e.g. the Item Type will be set to 'Microsoft Word' for any documents with the File Extension 'doc'. |
Enable Default to File Extension |
If enabled, the Item Type will be set to the File Extension suffixed with '<file extension> File' for any unknown File Extensions. Otherwise, the Item Type remains unchanged. |
Icon Assigner Stage
The Icon Assigner Stage sets the Item’s Icon URL based on the Item’s File Extension and Item Type.
Configuration
| Property | Description |
|---|---|
Item Types |
The Icon will only be assigned if one of the Item Types matches. If none are defined, all Item Types will be accepted. |
File Extensions |
The Icon will only be assigned if one of the File Extensions matches. If none are defined, all File Extensions will be accepted. |
Icon URL |
The Icon URL that will be added to the item if it matches both an Item Type and a File Extension. |
Icon Metadata Key |
The Key under which the Icon will be added to the metadata. |
Metadata Mapper Stage
The Metadata Mapper Stage copies the values from the source key into target key.
Metadata Regular Expression Stage
The Metadata Regular Expression Stage is used to manipulate a metadata field by replacing all occurrences of a regular expression with a configured value.