...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Format
Code Block | ||
---|---|---|
| ||
{
"display": string,
"file":
{
"username" : "string", // Username for file share authentication,
"password" : "string", // Password for file share authentication,
"domain" : "string", // Domain location of the file share,
"pathInclude": "string", // Optional - regex, only files with complete paths matching the regular expression are processed further
"pathExclude": "string', // Optional - regex, files with complete paths matching the regular expression are ignored (and matching directories are not traversed)
"renameAfterParse" "string", // Optional, renames files after they have been ingested - the substitution variables "$name" and "$path" are supported; or "" or "." deletes the file
// (eg "$path/processed/$name")
"type": "string", // One of "json", "xml", "tika", "*sv", or null to auto decide
"mode": "string", // "normal" (defaults if mode not present), "streaming", see below
"XmlRootLevelValues" : [ "string" ], // The root level value of XML to which parsing should begin
// also currently used as an optional field for JSON, if present will create a document each time that field is encountered
// (if left blank for JSON, assumes the file consists of a list of concatenated JSON objects and creates a document from each one)
// (Also reused with completely different meaning for CSV - see below)
// (In office mode, can be used to configure Tika - see below)
"XmlIgnoreValues" : [ "string" ], // XML values that, when parsed, will be ignored - child elements will still be part of the document metadata, just promoted to the parent level.
// (Also reused with completely different meaning for CSV)
"XmlSourceName" : "string", // If present, and a primary key specified below is also found then the URL gets built as XmlSourceName + xml[XmlPrimaryKey], Also supported for JSON and CSV.
"XmlPrimaryKey" : "string", // Parent to XmlRootLevelValues. This key is used to build the URL as described above. Also supported for JSON and CSV.
"XmlPreserveCase": boolean, // default false, converts everything to lower case
"XmlAttributePrefix": "string", // default: null - if enabled, attributes are converted into tags with this prefix
}
} |
Infinit.e Jobs
...
...
...
...
...
...
...
...
By default, the matching portion of the line (eg "#" in the example above) is removed.
To not remove it then simple place the value in quotes (using the specified quote char).
eg. assuming the quote char is ', then "`#`" in the above example would return 3 fields: "#field1", "field2" and "field3"
In the example log file below, the header row is prefixed by '#'.
Code Block |
---|
#Date,Device,SrcIP,dstIP,Alert,Country
SCANNER_1,2012-01-01T13:43:00,10.0.0.1,66.66.66.66,DUMMY_ALERT_TYPE_1,United States |
In the example source below, XmlIgnoreValues
automatically identifies the header using "#". This also identifies the field names using the separator ",".
...
...
...
...
Header Not Prefixed by String:
In the case where the first header is not prefixed by a string, it is still necessary to identify it as the header row.
For example, consider a header row formatted as follows:
"field1,field2,field3"
In this case, XmlIgnorevalues
should be set to the following: [ "\"field1\"" ] (ie the first header wrapped in whatever the quote character is, " by default)
...
Panel |
---|
Footnotes:
Legacy documentation: |
Legacy documentation:
...
Using the File Harvester