...
- Windows/Samba shares
- harvester's local filesystem
- Amazon S3
...
File Types
The File Extractor supports the following file types
- Office documents (Word, Powerpoint etc.)
- text-based documents (emails)
- CSV
- XML and JSON
- Infinit.e shares
- The results of Infinit.e plugins
...
Connecting to File Locations
...
"file://<path including leading '/'>"
| Noteinfo |
|---|
"file://" sources can only be run by administrators if secure mode is enabled (harvest.secure_mode=true in the configuration). Local filesystem usage is mostly intended for testing, debugging, and "micro installations". The "tomcat" user must have read access to the directories and files on the path. |
...
The source must share at least one community with the custom plugin in order to be processed.
...
Windows/Samba
To connect to a Windows/Samba share, the following url format must be used:
"smb://server:port/path"
...
Amazon S3
To connect to an Amazon S3 location, the following url format must be used:
...
The files in the S3 bucket should be readable by the account specified by the access key.
| Noteinfo |
|---|
S3 is not supported by default, the AWS SDK JAR must be copied into the classpath as described here. |
...
For S3, the Access ID should be entered into the "username", and the Secret Key into the "password"
| Noteinfo |
|---|
It is recommended for security that you create a separate AWS user with no permissions other than S3 read/list on the directories. |
...
This field can be left blank for Amazon S3 environments.
...
File Types
This section describes the configurations for the various supported file types.
...
For .sv files you can use the XmlSourcename parameter to build the document url.
| Noteinfo |
|---|
XmlRootLevelValues must be set. |
...
You can use XmlPrimaryKey to help identify whether a record is new or previously harvested. This requires tat that the parameter XmlRootLevelValues has been set.
...
Office Files
You can use the XmlRootlevelValues parameter to configure Apache Tika for parsing of Office-type files.
...
Example: "application/pdf:{'setEnableAutoSpace':false}" ... will call PDFParser.setEnableAutoSpace(false)
...
JSON/XML
For JSON files the parameter XmlIgnoreValues is not applicable.
...
You can usethe parameter XmlPrimaryKey to help identify whether a record is new or previously harvested.
| Noteinfo |
|---|
...
CSV Files
| Noteinfo |
|---|
IN PROGRESS
Legacy documentation:
...