...
Suspending and deleting sources
| Widget Connector | ||
|---|---|---|
|
As outlined in the video above, once Datasift sources have been created, as detailed here (video and documentation), then they can be controlled from the source editor.
The following operations can be performed:
...
To suspend an active source, simply navigate to the source editor tab of the manager, select the desired source from the list on the left, and select the "Disable Source" button (clicking "OK" to confirm publishing the source when prompted).
...
To delete a source follow the steps from the source editor documentation. Deleting a source will automatically delete the corresponding Datasift push subscription.
...
If it is necessary to make minor modifications to the JCSDL then this can be done without deleting and re-creating the source. Instead, suspend the source, wait two minutes (to be sure the existing Datasift push subscription has been terminated), modify the JCSDL in the subscription (this is one of the fields in the source editor), and then activate it (which will automatically publish the change).
| Info |
|---|
If it is requested enough then we will add a graphical editor into the Datasift connector widget so as to be able to modify sources more formally. |
Further reading:
Creating aliases and discarding unwanted entities
| Widget Connector | ||
|---|---|---|
|
...
| Info |
|---|
One downside to this is that there is no way within the widget of transferring aliases from the sandbox to a real community (eg once you are happy with them). However the sub-section "Manually setting alias configurations" explains how this can be achieved easily using the File Uploader page instead. |
Creating synthetic alias masters
...
As with all Infinit.e GUI functionality, the Entity Alias Builder widget is just an interface to our open API.
Aliases are stored in Infinit.e as JSON shares of type "infinite-entity-alias". Their format is described here. They can be manually uploaded and shared between communities using the File Uploader manager page.
This can be useful for 2 purposes:
- Where there are large numbers of aliases to be generated, it would not be much fun to use the GUI for each one. Instead you can programmatically generate (eg with a script) a JSON file containing the aliases and then upload it.
- This is a bit beyond the scope of this documentation, but you can also create a plugin (eg using the Javascript scripting engine) and then create a share with type "infinite-entity-alias" that points to the custom plugin results (this is described in the File Uploader documentation).
- So as an example if you have a word document that lists lots of social media handle mappings, then you could upload that as a share, then import that share as a source (this is discussed further below under "Importing other sources"), then write a Javascript plugin (see below under "More complex analytics") that parses the document into the right format, and then finally point a share to that!
- This would have the nice feature that it would automatically update itself whenever the document was re-uploaded.
- So as an example if you have a word document that lists lots of social media handle mappings, then you could upload that as a share, then import that share as a source (this is discussed further below under "Importing other sources"), then write a Javascript plugin (see below under "More complex analytics") that parses the document into the right format, and then finally point a share to that!
...
| Info |
|---|
A few things to note when using aliases in multi-user environments:
|
...
- An IKANOW blog post discussing an operational use of aliasing
- File Uploader documentation
- Community Manager documentation
- The alias API
More complex analytics and visualization
...
- Custom Viewer - Map: Finds fields in the record that "look like" lat/long points and plots them on a map (MapQuest) colored according to a score defined by a numeric field in the same record (see below for more details)
- The rules for "looks" like lat/long are as follows:
- Is a top-level object called "geo", "geotag", "latlong", "latlon" AND consists of 2 numeric field (or strings representing numbers) with names "lat" and "lon" or "latitude" and "longitude"
- Has two top level numeric fields (or strings representing numbers) with names names "lat" and "lon" or "latitude" and "longitude"
- The rules for "looks" like lat/long are as follows:
- Custom Viewer - Bar Graph: Uses any field from the record as a key, and plots a bar of height defined by a numeric field in the same record (see example 2 for more details).
...
TODO jsfiddle
Further reading:
- Plugin manager documentation
- Information about the built-in Javascript engine
- Developer information about building Java Hadoop plugins
- An IKANOW blog post discussing using jsfiddle to visualize custom analytics
- (contains links to some other relevant blog posts about running analytics on Infinit.e datasets, including this one about doing temporal/sentiment analytics on emails)
...
An important feature of the Infinit.e platform is that it wants data to be open: our User Interfaces and applications use our open RESTful API, so any other client can get the same data.
The primary method of getting at the data is via the query API call, and that linked page shows some examples of making the call in javascript and actionscript. In addition we have a beta (ie undocumented!) Java driver here (that we use internally, so is well supported). There are more, general, examples of using the API in different languages here.
...
Harvesting in Infinit.e is controlled by JSON documents called sources. These sources can be tested by POSTing to the Config - Source - Test REST endpoint, and activated/updated ("published") by POSTing to the Config - Source - Save REST endpoint.
In practice the Source Manager GUI can be used to perform these activities in a more visual intuitive way. It still requires building the source JSON with limited development support - as can be seen from the documentation here, this requires some javascript skills and some effort. The source manager provides some templates to get up and running on simpler types of ingest, and there is a source gallery with some real world examples of various complexities.
...
Creating new users can be performed from the Person Manager GUI.
Creating new communities can be performed from the Community Manager GUI.
A few points to note:
- After a community is created, only the owner is initially added. Other users can be added (or removed) by selecting the "Add New Members" (or "Edit/Remove Members") at the bottom of the right pane for the selected community.
- For users to be able to add new sources, they must either be system administrators, or be added as "Content Publisher" or better in their community role (right hand pane after selecting a user from the "Edit/Remove Members" page).
- (In secure mode, see below, users must be administrators to create new sources)
...