Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Overview

This toolkit element enables the generation of one or more types of entity based on the document or content metadata. The expressions default to replacement strings, or $SCRIPT(...) can be used to return a string using javascript.

Format

{
	"display": string,
	"entities": [
	{
	    "iterateOver" : "string", // OPTIONAL: If specified, a metadata field (nesting supported using dot notation) which is looped over to generate calls with _value/_iterator/_index
	    "disambiguated_name" : "string", // MANDATORY: String/script, the disambiguated name of the entity
	    "actual_name" : "string", // OPTIONAL: String/script, the actual name of the entity if different to the disambiguated name
	    "dimension" : "string", // MANDATORY: String/script: Must be/return one of "Who", "What", "Where"
	    "type" : "string", // MANDATORY: String/script: It is recommended to use a type from the 
    			            // OpenCyc, AlchemyAPI, or OpenCalais ontologies, for compatibility with future Infinit.e features
 
	    "linkdata": "string", // OPTIONAL: if present should return a comma-separated list of URLs (commas should be URL-encoded)
	    "relevance" : "string", // OPTIONAL: String/script: Must specify/return a double/string-parsable-into-a-double
	    "sentiment" : "string", // OPTIONAL: String/script: Must specify/return a double/string-parsable-into-a-double, by convention this is between -1.0 and 1.0.
	    "frequency" : "string", // OPTIONAL: Must specify/return a long/string-parsable-into-a-long
	    "geotag" : { // OPTIONAL: Format is identical to the docGeo format specified above
	        "lat": "string", "lon": "string,
    	    "city": "string", "stateProvince": "string, "country": "string", "countryCode": "string
	    },
	    "ontology_type": "string", // OPTIONAL: String/script: Only used if geotag is specified: 
	                    // allows specification of the scale of the geographic entity (see below for useful link), defaults to "point"
	    "useDocGeo": "boolean", // OPTIONAL: If true, uses any lat/long generated from the top level "docGeo" specification, defaults to false
	    "creationCriteriaScript" : "string", // OPTIONAL: script: If populated, runs a user script function and if return value is false doesn't create the object
	}
	]
}

 

Description

Entities are the who, what, and where's contained within a record (i.e. people, places, and things).

The following table describes the parameters of the manual entities configuration.

ParameterDescription
iterateOver

OPTIONAL: If specified, a metadata field (nesting supported using dot notation) which is looped over to generate calls with _value/_iterator/_index

disambiguated_name

MANDATORY: String/script, the disambiguated name of the entity

actual_name

OPTIONAL: String/script, the actual name of the entity if different to the disambiguated name

dimension

MANDATORY: String/script: Must be/return one of "Who", "What", "Where"

type

MANDATORY: String/script: It is recommended to use a type from the // OpenCyc, AlchemyAPI, or OpenCalais ontologies, for compatibility with future Infinit.e features

linkdata

OPTIONAL: if present should return a comma-separated list of URLs (commas should be URL-encoded)

relevance

OPTIONAL: String/script: Must specify/return a double/string-parsable-into-a-double

sentiment

OPTIONAL: String/script: Must specify/return a double/string-parsable-into-a-double, by convention this is between -1.0 and 1.0.

frequency

OPTIONAL: Must specify/return a long/string-parsable-into-a-long

geotag

OPTIONAL: Format is identical to the docGeo format specified above "lat": "string", "lon": "string, "city": "string", "stateProvince": "string, "country": "string", "countryCode": "string

ontology_type

OPTIONAL: String/script: Only used if geotag is specified: // allows specification of the scale of the geographic entity (see below for useful link), defaults to "point"

useDocGeo

OPTIONAL: If true, uses any lat/long generated from the top level "docGeo" specification, defaults to false

creationCriteriaScript

OPTIONAL: script: If populated, runs a user script function and if return value is false doesn't create the object

 

Examples

Entities

In the example source, the entities block has been configured to set actual_name as a value which will be provided by the document metadata.  Entities will expect a JSON object which is a representation of the Twitter user's display name.

Data is extracted from the source using the $ operator.

    },        {
            "entities": [
                {
                    "actual_name": "$metadata.json.actor.displayName",
                    "dimension": "Who",
                    "disambiguated_name": "$metadata.json.actor.preferredUsername",
                    "linkdata": "$metadata.json.actor.link",
                    "type": "TwitterHandle"
                },

Sample Output

In the output we see that an entity has been created based on the Twitter user's Twitter handle.

 {            "actual_name": "CRM Buddy",
            "dimension": "Who",
            "disambiguated_name": "FocalCRM",
            "doccount": 0,
            "frequency": 1,
            "index": "focalcrm/twitterhandle",
            "linkdata": ["http://www.twitter.com/FocalCRM"],
            "relevance": 0,
            "totalfrequency": -1,
            "type": "TwitterHandle"
        },

 

Metadata

The metadata that document metadata used to create the JSON object.

 ],    "mediaType": ["Social"],
    "metadata": {"json": [{
        "actor": {
            "displayName": "CRM Buddy",
            "followersCount": "245",
            "friendsCount": "0",
            "id": "id:twitter.com:835627776",
            "image": "http://a0.twimg.com/profile_images/2630355549/8cad59efaddd57283dbb159332336744_normal.jpeg",
            "languages": ["en"],
            "link": "http://www.twitter.com/FocalCRM",
            "links": [{"rel": "me"}],
            "listedCount": "6",
            "objectType": "person",
            "postedTime": "2012-09-20T13:59:56.000Z",
            "preferredUsername": "FocalCRM",
            "statusesCount": "3688",
            "summary": "",
            "verified": "false"
        },

 

Extracting Entities From Arrays

You can use manual entities with iterateOver to create an entity for each object in the array, when the document metadata contains an array for a specific metadata field. 

Source

In the example below, the source for entities has been configured to iterateOver the document metadata "victims."

  },
                {
                    "dimension": "Who",
                    "disambiguated_name": "$FUNC( getVictim(); )",
                    "frequency": "$FUNC( getVictimCount(); )",
                    "type": "VictimType",
                    "useDocGeo": false,
                    "iterateOver": "victim"
                },
                {
                    "dimension": "Who",
                    "disambiguated_name": "$FUNC( getVictim(); )",
                    "frequency": "$hostagecount",
                    "type": "HostageType",
                    "useDocGeo": false,
                    "iterateOver": "victim"
                }
            ]
        },

 

Sample Output

In the sample output you can see some of the entities outputted based on the example source.  The entities have been created based on the array that was returned for the metadata field "victim."

  },
        {
            "actual_name": "Targeted, Civilian, Adult from Afghanistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Civilian, Adult from Afghanistan",
            "doccount": 0,
            "frequency": 5,
            "index": "targeted, civilian, adult from afghanistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },
        {
            "actual_name": "Targeted, Civilian, Child from Afghanistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Civilian, Child from Afghanistan",
            "doccount": 0,
            "frequency": 2,
            "index": "targeted, civilian, child from afghanistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },

 

Metadata

The metadata was used by Manual entities to create the two entities for metadata value "victim."

"victim": [
            {
                "child": "No",
                "combatant": "No",
                "deadcount": "1",
                "definingcharacteristic": "Unknown",
                "hostagecount": "0",
                "indicator": "Targeted",
                "nationality": "Afghanistan",
                "targetedcharacteristic": "Unknown",
                "victimtype": "Civilian",
                "woundedcount": "4"
            },
            {
                "child": "Yes",
                "combatant": "No",
                "deadcount": "2",
                "definingcharacteristic": "Unknown",
                "hostagecount": "0",
                "indicator": "Targeted",
                "nationality": "Afghanistan",
                "targetedcharacteristic": "Unknown",
                "victimtype": "Civilian",
                "woundedcount": "0"
            }
        ],

 

Specifying Entity Location

In the example source, the entity block is configured to output an entity using the dimension "where."  The location is determined using javascript, and the scripts are calling on function methods defined in the javascript Globals section.

 },                {
                    "dimension": "Where",
                    "disambiguated_name": "$metadata.json.actor.location.displayName",
                    "geotag": {
                        "city": "$SCRIPT( return getAddressVal( _doc.metadata.json[0].actor.location.displayName, 0 ) )",
                        "stateProvince": "$SCRIPT( return getRegion(getAddressVal( _doc.metadata.json[0].actor.location.displayName, 1 )) )",
                        "countryCode": "US",
                        "alternatives": [
                            {
                                "stateProvince": "$SCRIPT( return getRegion(getAddressVal( _doc.metadata.json[0].actor.location.displayName, 1 )) )",
                                "countryCode": "US"
                            }
                        ]
                    },

 

The example source will return output with the dimension of "Where" if the location can be determined by the metadata.

In some cases, it will not be clear what geographical type a field is (eg a freeform field that might be city, state, or country). The geographical specification allows you to specify alternatives.

The alternatives are tried in order until one of them works or there are no more to try.

 

Footnotes:

Legacy documentation:

Legacy documentation:

 

 

  • No labels