Lookup Tables
- AlexI (Unlicensed)
- andrew johnston (Unlicensed)
Overview
With Community Edition (CE), you can use Javascript to perform transformations of data for the creation of metadata, entities or associations. For a more detailed account, see Using Javascript
When using Javascript with CE, it is possible to use Lookup tables, in order to access a set of global variables loaded at harvest time based on JSON shares, custom tables, or document collections. This can be used to help to simplify the process of generating metadata for the ingested content, and the lookup tables can be used in all Javascript scripts.
In order to use Lookup tables in your source configuration, you need to specify a lookupTables
object that includes a descriptive name, and the appropriate ID.
Not currently supported in the scripts in Follow Web Links pipeline elements.
This page has been organized into the following sections for ease of localization.
Format
{ "display":string, // for informational use only "lookupTables": { "string":string, // for each lookup table, a string that describes the lookup and then the id of the share containing the JSON object // eg "geoLookup": "525831c31723dff58fbf0eef" } }
Description
Using lookup tables you can access global variables from JSON shares, custom tables, or document collections
JSON shares:
To create a lookup table for a JSON share use the following format
MyShare: share_id
where share_id is the ID created by CE when the JSON share was added to the system.
Custom tables:
To create a lookup table for a custom table use the following format
MyShare: job_id
where job_id is the ID created by CE for the custom table.
Document collection:
To create a lookup table for a document collection use the following format
MyShare: source_id
where source_id is the ID created by CE when the document collection was created.
See examples below.
Accessing the Shares
Once the shares have been brought into the CE runtime environment, they can be accessed using the following formats.
JSON shares:
JSON shares are accessed via "_cache['NAME']"
where eg. "NAME" is the key specified in the lookup table
Custom or Document Objects:
Custom and document objects are accessed via "_custom['NAME'].get(KEY)",
where KEY is the URL (for sources) or the custom key
In this case only Text keys or BSONWritable keys with a single string parameter are supported.
See examples below.
Examples
Specifying the Lookup Table
To use a lookup table, you need to specify the appropriate ID and local name for the lookup table, as in the following example source.
{ "display": string, // for informational use only "lookupTables": { "myLookupTable": "4e0c7e99eb5af0fbdcfbf697"
Accessing the Lookup Table
Within any script, you can access the JSON object by indexing "_lookupTables" with the local name specified as above.
Consider the following uploaded JSON share.
{ //... "US": "United States", "USA", "United States of America", "UK": "United Kingdom", "Great Britain", "GB", //... }
The uploaded JSON share could be accessed by calling myLookupTable
within the javascript, as in the code block below.
where myLookuptable
represents the name of the lookup table specified using the lookuptables
object.
"entities": [ //... { "iterateOver": "geo.countries", "disambiguatedName": "$SCRIPT( return _lookupTable['myLookupTable'][ _value ];)", "type": "Country" } ], //... } }
In this section:
Footnotes:
Legacy documentation:
Legacy documentation:
- StructuredAnalysis object (replaces caches)
- UnstructuredAnalysis object (replaces caches)