Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Suspended sources can be re-activated as described below.

Re-

...

activate suspended sources

Re-activating a suspended source is the same operation: the source will have a "Enable Source" button in the same place, and clicking on it and confirming will perform the following:

...

In other cases, particularly as your activities move from data investigation to data science, it becomes necessary either to apply either more complex algorithms (for example graph theory or social network analysis), or to calculate standard statistics in domain-specific (a very simple example of this would be aggregating sentiment geographically).

TODO screenshots (sentiment + geo -> custom geo)Image Added

In order to support these sorts of operations, Infinit.e provides the ability to plug-in analytic modules that can run over any subset of the data (including all of it). The general topic of building plug-in modules and scheduling and running them is beyond the scope of this documentation; this section will provide links to the Infinit.e documentation and describe the aspects most relevant to Datasift. 

In particular, we have provided 3 sample jobs that illustrate a few different types of analytic and demonstrate how to access the document objects, and interpret the results (in practice the Infinit.e-specific bits like this are very easy, the difficulty is typically in building the algorithms themselves, as it should be).

Example 1 - XXX

TODO

Example 2 - XXX

TODO

...

Info

Note that the examples below assume that you have an instance into which you have loaded some sources and collected some data, using the techniques described in previous sections.

Example 1 - Aggregate sentiment by geo

This sample analytic can be seen in the screenshot above. It simply creates a 10 degree x 10 degree grid and aggregates the sentiment associated via geo-tagged entities with those grid squares.

The sequence of the screenshots below shows how to access the example (starting from the manager webapp, eg press the "MANAGER" link in the top right of the main GUI):

Image AddedImage AddedImage Added

TODO The key screenshot above is the middle one, which shows the "scary-at-first-glance" plugin manager. This page TODO

Example 2 - Aggregate sentiment by gender

TODO

Example 3 - Show top co-references

TODO

Creating new Javascript plug-ins

TODO

Creating new Hadoop plug-ins

TODO

Run-time options for plug-ins

TODO

Visualizing the output of plug-ins

...