Table of Contents
Javascript Prototype Engine - basic operation
The Infinit.e plugin manager comprises a built-in processing module, the "Javascript Prototype Engine". This module runs javascript entered into the "user arguments" text field.
...
| Info |
|---|
Note that the combine function is normally present for performance and can be discarded without loss of functionality. If the reduce function is ignored then the output of the map is written directly to the output collection (/HDFS directory) - if multiple objects with the same key are emitted from the same map function, only one is preserved. Removing either of the combine/reduce functions must be accompanied by commenting out the corresponding Combiner/Reducer class in the plugin manager GUI. |
Javascript Prototype Engine - advanced
Accessing the Hadoop context object
...
| Code Block |
|---|
// For just the date
var date = new java.text.SimpleDateFormat('yyyy-MM-dd').parse(val.publishedDate["$date"]);
// (or val.created or val.modified, or val.associations[x].time_start, etc)
// For date time:
var datetime = new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss").parse(val.publishedDate["$date"]); |
Writing javascript Date type to Mongo as Mongo ISODate
Due to the slightly non-standard date storage in MongoDB, together with the limitiations of the built-in Date function in Rhino (the javascript engine built into Java), javascript dates must be prepared for being written into Infinit.e/MongoDB in a slightly convoluted way:
| Code Block |
|---|
//current datetime
var dateString = new Date().toISOString();
var mongoDate = {};
mongoDate['$date'] = dateString;
//now it will be stored as proper Mongo ISODate when emitted:
emit(key, {..., 'publishedDate':mongoDate,...}); |
Logging
Messages can be logged and are stored in the status message as follows (note unless run in debug mode, only ERROR category messages are retrieved).
...