...
- It takes in records and documents primarily
- Also external files and results from already-run custom jobs
- It applies different types of processing to the data:
- XXX
...
- Generic templated business logic using Java or scripting languages (currently JS, Python is on the roadmap)
- A growing set of built-in configurable functions
- Generic high speed aggregation
- Generic high speed filtering and joining on other data sources
- Data format exploration
- Roadmap: real-time analytics using Storm-over-Hadoop
- The output of the processing is (with a few exceptions) objects representing things like:
- New documents or records (alerts, groups of documents eg tweets)
- Lookup tables
- Flat reports
- (exceptions: export to file, modify existing documents)
The remainder of this section categorizes the different elements that can be used to build up functionality:
- Custom Inputs: Which data to bring in, what filtering and transforms to apply
- Custom Control: Scheduling and other controls
- Custom Processing: Generic scripting or customizable templates, single or chained (roadmap)
- Custom Outputs: Where the results of the data end up (documents, records, custom tables)
Custom Inputs
- TODO
Custom Control
- TODO
Custom Processing
- TODO
Custom Outputs
- TODO