Versions Compared

Old Version 3

changes.mady.by.user andrew johnston

Saved on

compared with

New Version 4

changes.mady.by.user andrew johnston

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Plugin Manager is displayed when clicking on Source editor from the Manager interface.

Image Modified

 

Description:

Use the Plugin Manager as a simple web-based user interface for uploading new and updated map reduce plugins and saved queries to the system, and for sharing them across the different communities.

 

FieldDescriptionNotes
Upload New Plugin  
Title  
Next scheduled time  
Frequency  
Input collection  
   
Query

The "query" field has a few noteworthy points:

  • Some additional control fields can be supplied, as described here.
    • The most common control fields can be specifically added with their default value by pressing the "Add Options" button
  • The aforementioned query must be a one of: 
  • Press the "Check" button next to the "Query" field to validate the query JSON.
  • (From March 2014) You can paste a saved workspace link into the query field (eg from the GUI) instead of typing out the JSON to generate an Infinit.e style query. You can include JSON below that to add query qualifiers, as described here.
  • Note that MongoDB uses some JSON extensions that must be used in queries from the command line:
    • When querying the ObjectId type (eg "_id"), it should be queried as the object '{ "$oid": "<object id string>"' }'
    • When querying a Date type, it should be queried as the object '{ "$date": "<date in Java time, ie milliseconds since 01 Jan 1970" }'
    • The full list is here: http://docs.mongodb.org/manual/reference/mongodb-extended-json/
 
 Communities  
 Mapper Class Note: You can temporarily remove a combiner or reducer by putting "#" in front of it. Only the mapper is mandatory (others can be set to "none"), though normally at least the mapper and reducer are set. 
Combiner Class   
Reducer ClassNote: You can temporarily remove a combiner or reducer by putting "#" in front of it. Only the mapper is mandatory (others can be set to "none"), though normally at least the mapper and reducer are set. 
Output key Class  
Output Value Class  
Export to HDFS  
   
Append ResultsIf you set append results to false, there is no need to set an age out. 
Job dependenciesIf you don't want your job to depend on another jobs completion, do not select any job dependencies (you can CTRL-click to remove selected options if necessary). 
JAR file  
User arguments

the "user arguments" field can be any string, it is interpreted by the code in the Hadoop JAR. For custom plugin developers: see this tutorial for a description of how to incorporate user arguments in the code (under advanced topics). Since the user arguments will normally be JSON or javascript (see info box below), a "Check" button has been provided that will validate either of those 2 formats.

The "user arguments" field can be any string, it is interpreted by the code in the Hadoop JAR. For custom plugin developers: see this tutorial for a description of how to incorporate user arguments in the code (under advanced topics). Since the user arguments will normally be JSON or javascript (see info box below), a "Check" button has been provided that will validate either of those 2 formats.

Info

In particular, the built-in "HadoopJavascriptTemplate" template job uses the "user arguments" to hold the javascript code that gets executed in Hadoop.

 
SubmitSubmit will save the task (or update it if it already exists). If the frequency is not "Never" and the "Next Scheduled Time" is now or in the past, then job is immediately scheduled. The page refreshes immediately (unlike "QuickRun" below) and the progress can be monitored as described under "Following a job's progress" below. 
QuickRun

"QuickRun" will set the frequency to "Once Only" and the time to "ASAP" (as soon as possible) and then will do 2 things:

  • Submit as above 
  • It will wait for the job to complete before refreshing the page (all the "action" buttons are disabled in the meantime). You can't see the progress (see below) in the meantime, so this is best used on smaller jobs.
 
Check  
Add Options  
Save and Debug  

 

 

Panel

Related Procedural Documentation:

Plugin Manager

 

 

 

Panel

In this section:

Table of Contents