...

Creates records using the very powerful and flexible Logstash platform. This is currently the only extractor capable of generating records, all the others create documents. 

Format

The Infinit.e side format is very simple:

{
	"display": string,
	"logstash": {
	    "config": string, // contains XXXthe complex
 		"streaming": boolean // defaults to true - data is stored only for a rolling 30 days, if false data is stored forever (useful for demos, should be used with care)
	} 
}

Description

...

Obviously the most significant element of the Logstash configuration is the "config" string field. This contains the Domain Specific Language described here, pointing to the various elements listed here.

Whenever manually creating the Logstash configuration, the source editor should be used - there is an "LS" editor window (to the left immediately below the main source form, whenever the "extractor type" is specified) a standard code editing experience. (The indentation doesn't quite work because the Logstash syntax doesn't map onto any existing languages!). There is also a "Logstash" template available from the "New Source" window.

The Infinit.e implementation provides a number of limitations:

• For non-admin users, only the following input elements can be used:
  • collectd, drupal_dblog, gelf, gemfire, imap, irc, lumberjack, s3, snmptrap, sqs, syslog, twitter, udp, xmpp, zenoss
• For all users, only a single input element can be specified
• For non-admin users, only the following filter elements can be specified:
  • advisor, alter, anonymize, checksum, cidr, cipher, clone, collate, csv, date, dns, drop, elapsed, extractnumbers, fingerprint, geoip, gelfify, grep, grok, grokdiscovery, l18n, json, json_encode, kv, metaevent, metrics, multiline, mutate, noop, prune, punct, railsparallelrequest, range, sleep, split, sumnumbers, syslog_pri, throttle, translate, unique, urldecode, useragent, uuid, wms, wmts, xml
• For all users, no output element can be specified, one is inserted automatically by Infinit.e

Info
System administrators can configure the set of allowed inputs via the 2 configuration parameters: "harvest.logstash.allowed_inputs" and "harvest.logstash.allowed_filters". This can be used either to ban elements considered unsafe, or to add new or custom elements.

Panel
Footnotes: External documentation: Logstash

...