Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

JSON format

Note that there is a separate overview of how to use the File Harvester. This page is mostly reference information.

...

Code Block
titleSource.file object
{
	"username" : "string", // Username for file share authentication,
	"password" : "string", // Password for file share authentication,
	"domain" : "string", // Domain location of the file share, 

    "pathInclude": "string", // Optional - regex, only files with complete paths matching the regular expression are processed further
    "pathExclude": "string', // Optional - regex, files with complete paths matching the regular expression are ignored (and matching directories are not traversed)
	"renameAfterParse" "string", // Optional, renames files after they have been ingested - the substitution variables "$name" and "$path" are supported; or "" deletes the file
									// (eg "$path/processed/$name")
 
	"type": "string", // One of "json", "xml", "tika", "*sv", or null to auto decide
 
	"XmlRootLevelValues" : [ "string" ], // The root level value of XML to which parsing should begin 
										// also currently used as an optional field for JSON, if present will create a document each time that field is encountered
										// (if left blank for JSON, assumes the file consists of a list of concatenated JSON objects and creates a document from each one)
									// (Also reused with completely different meaning for CSV)
	"XmlIgnoreValues" : [ "string" ], // XML values that, when parsed, will be ignored - child elements will still be part of the document metadata, just promoted to the parent level. 
										// (Also reused with completely different meaning for CSV)
	"XmlSourceName" : "string", // If present, and a primary key specified below is also found then the URL gets built as XmlSourceName + xml[XmlPrimaryKey], Also supported for JSON and CSV.
	"XmlPrimaryKey" : "string", // Parent to XmlRootLevelValues. This key is used to build the URL as described above. Also supported for JSON and CSV.
}

...