Associations can be "something that happens or is regarded as happening; an occurrence, especially one of some importance", "the outcome, issue, or result of anything", or "something that occurs in a certain place during a particular interval of time" (Definitions found here: http://dictionary.reference.com/browse/event). Within Infinit.e events are typically a combination of entities assembled in the form of Noun - Verb - Noun, e.g. "a car crashed into a building", "the plane flew to San Diego". In addition to the Noun - Verb - Noun form events can include geographic information (i.e., where an event happened) as well as a start and/or end time for an event.

iterateOver

iterateOver can be used in more advanced cases.

Iterate Over a Metadata Field:

Specify a metadata field, in order to generate associations.

eg. "iterateOver": "json.twitter_entities.user_mentions"

This is the same as iterating over a metadata array to obtain entities using Manual entities.

See detailed example below

Iterate Over a Single Entity Type:

Use "dummy" to iterate over only one entity type,

eg "entity1/dummy" or "entity2,dummy"

See detailed example below

Multiplicative:

Create one association for every combination of entities of specified types

eg. "iterateOver": "entity1/entity2/geo_index",

See detailed example below

Associative:

Create one association for every pair (/set) of entities of specified types

See detailed example below

creationCriteriaScripts

...

Field	Description	Data Type
`iterateOver`
`entity`	A free form text field containing information about the event "subject", i.e. an entity's disambiguous name.
`entity1_index`	If present this is the "index" field of the entity matching the entity1 disambiguous name above.
`verb`	A free form text field describing the event "verb"
`verb_category`	Also a free form text field describing the event "verb", but intended to group related verbs together (eg "travel" for verbs: "flew", "drove")

IN PROGRESS

iterateOver

iterateOver can be used in more advanced cases.

Iterate Over a Metadata Field:

Specify a metadata field, in order to generate associations.

eg. "iterateOver": "json.twitter_entities.user_mentions"

Metadata field nesting is also supported using dot notation, in which case they are looped over to generate calls with _value/_iterator/_index

This is the same as iterating over a metadata array to obtain entities using Manual entities.

For more information concerning metadata fields and support for nested fields, see section Manual entities.

See detailed example below

Iterate Over a Single Entity Type:

A less common scenario is when you want to iterate over a single entity type. You can do this by using "dummy."

eg "entity1/dummy" or "entity2,dummy"

See detailed example below

Multiplicative:

Create one association for every combination of entities of specified types. Also referred to as combinatorial association.

eg. "iterateOver": "entity1/entity2/geo_index", an _iterator object with the following fields is available in the Javascript: "_iterator.entity1_index", "_iterator.entity2_index", "_iterator.geo_index".

These fields can be usefully used together with "creationCriteriaScript" scriptlets to filter out unwanted associations, eg when looping over entity1 and entity2 with the same entity type, the following script would ensure the association didn't involve the same entity:

"creationCriteriaScript": "$SCRIPT( return _iterator.entity1_index != _iterator.entity2_index

See detailed example below

Associative:

Create one association for every pair (/set) of entities of specified types. Also referred to as lock-step or additive association.

eg. "iterateOver": "entity1,entity2"

See detailed example below

...

creationCriteriaScripts

Association fields generated from the entity loop are placed in "_iterator". For example, for "iterateOver": "entity1/entity2/geo_index", an _iterator object with the following fields is available in the Javascript: "_iterator.entity1_index", "_iterator.entity2_index", "_iterator.geo_index".

These fields can be usefully used together with "creationCriteriaScript" scriptlets to filter out unwanted associations, eg when looping over entity1 and entity2 with the same entity type, the following script would ensure the association didn't involve the same entity:

"creationCriteriaScript": "$SCRIPT( return _iterator.entity1_index != _iterator.entity2_index; )", "iterateOver": "entity1/entity2", "entity1": "EmailAddress", "entity2": "EmailAddress", //etc

The creationCriteriaScript runs before the association is generated (so can be safely used to remove items that would return errors).

See detailed example below

Examples

Basic Association

...

The resulting output displays the details of the date and time, and the specific type of offense reported.

Code Block

 "associations": [
        {
            "entity1": "theft,2",
            "entity1_index": "theft,2/criminalactivity",
            "verb": "reported",
            "verb_category": "crime",
            "time_start": "2011-01-29T00:00:00",
            "geotag": {
                "lat": 38.9099278028729,
                "lon": -77.0436067765966
            },
            "assoc_type": "Summary"
        }

...

iterateOver

Anchor
metadata
metadata
iterateOver a Metadata Field

...

Code Block

{
            "associations": [
                {
                    "assoc_type": "Event",
                    "entity1": "$SCRIPT( return _doc.metadata._FILE_METADATA_[0].metadata.Author[0];)",
                    "entity2": "$SCRIPT(return _value;)",
                    "iterateOver": "email_meta.Message-To",
                    "time_start": "$SCRIPT( return _doc.publishedDate;)",
                    "verb": "emailed",
                    "verb_category": "emailed/communicated"
                }
            ]
        }

Output

Metadata:

The output displays the association between the sender and receiver in the email correspondenceemail metadata "Message-To" used to generate the associations is included in the source block below.

Code Block

         ],
    "associations    "email_meta": [
        {    [
         "entity1": "cara.semperger@enron.com",      {
      "entity1_index": "cara.semperger@enron.com/account",             "verbCreation-Date": "emailed",[
            "verb_category": "emailed/communicated",             "entity2": "will.smith@enron.com",2001-07-09T18:33:32Z"
             "entity2_index": "will.smith@enron.com/account",      ],
      "time_start": "2001-07-09T14:33:32",             "assoc_type": "EventMessage-To": [
                        "will.smith@enron.com"
        }            ],

...

Multiplicative association are associations that are created by "multiplying" a combination of entities, locations, and times together to determine the number of associations to extract from the source data.

In the example, a perpetrator (Sunni Islamic Extremist) attacked multiple types of victims (an adult and a child) in Sri Lanka.

The association specification uses the multiplicative format to create events using the following math to determine the total number of associations: Entity1 (Person Perpetrator) * Entity2 (Victim Type) * Geo_index (Location) = Total Number of Associations.

Code Block

},
                {
                    "creationCriteriaScript": "$FUNC( isOrganizationSpecified(); )",
       
                    "Content-Type": [
                        "message/rfc822"
                    ],
                    "subject": [
                        "RE: Testing Preschedule workspace"
                    ],
                    "date": [
                        "2001-07-09T18:33:32Z"
                    ],
                    "Author": [
                        "cara.semperger@enron.com"
                    ],
                    "Message-From": [
                        "cara.semperger@enron.com"
                    ]
                }
            ]
        ]

Output:

The output displays the association between the sender and receiver in the email correspondence.

Code Block

    "associations": [
        {
            "entity1": "cara.semperger@enron.com",
            "entity1_index": "cara.semperger@enron.com/account",
            "verb": "emailed",
            "verb_category": "emailed/communicated",
            "entity2": "will.smith@enron.com",
            "entity2_index": "will.smith@enron.com/account",
            "time_start": "2001-07-09T14:33:32",
            "assoc_type": "Event"
        }
    ],

Anchor
single
single
iterateOver a single entity type

In some cases you will only want to iterate over a single entity type, rather than two or three, which is more common. For these cases you can use "dummy" for iterateOver.

In the following source, iterateOver is setup to only iterate over the entity type: "VictimType."

Code Block
{ "assoc_type": "Event", "entity1": "$SCRIPT( return _doc.metadata.eventtype[0]; )", "entity2": "VictimType", "iterateOver": "entity2/dummy", "verb_category": "suffered" },

The code acts on the following entities in this example

Code Block

    "entities": [
        {
            "actual_name": "Targeted, Paramilitary/Private Security, Adult, Tribal from Pakistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Paramilitary/Private Security, Adult, Tribal from Pakistan",
            "doccount": 0,
            "frequency": 3,
            "index": "targeted, paramilitary/private security, adult, tribal from pakistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },
        {
            "actual_name": "Targeted, Non Official Public Figure, Adult, Tribal from Pakistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Non Official Public Figure, Adult, Tribal from Pakistan",
            "doccount": 0,
            "frequency": 1,
            "index": "targeted, non official public figure, adult, tribal from pakistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },
        {
            "actual_name": "Targeted, Civilian, Adult, Tribal from Pakistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Civilian, Adult, Tribal from Pakistan",
            "doccount": 0,
            "frequency": 2,
            "index": "targeted, civilian, adult, tribal from pakistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },

Output:

In the example output you can see that associations are only created based on entities of entity type: "VictimType." Three associations are created, for each of the Victim Type entities specified above.

Code Block

{
    "associations": [
        {
            "assoc_type": "Event",
            "entity1": "Armed Attack",
            "entity1_index": "armed attack/eventtype",
            "entity2": "targeted, paramilitary/private security, adult, tribal from pakistan",
            "entity2_index": "targeted, paramilitary/private security, adult, tribal from pakistan/victimtype",
            "verb": "suffered",
            "verb_category": "suffered"
        },
        {
            "assoc_type": "Event",
            "entity1": "Armed Attack",
            "entity1_index": "armed attack/eventtype",
            "entity2": "targeted, non official public figure, adult, tribal from pakistan",
            "entity2_index": "targeted, non official public figure, adult, tribal from pakistan/victimtype",
            "verb": "suffered",
            "verb_category": "suffered"
        },
        {
            "assoc_type": "Event",
            "entity1": "Armed Attack",
            "entity1_index": "armed attack/eventtype",
            "entity2": "targeted, civilian, adult, tribal from pakistan",
            "entity2_index": "targeted, civilian, adult, tribal from pakistan/victimtype",
            "verb": "suffered",
            "verb_category": "suffered"
        },

Anchor
multi
multi
Multiplicative

Multiplicative association are associations that are created by "multiplying" a combination of entities, locations, and times together to determine the number of associations to extract from the source data.

In the example, a perpetrator (Sunni Islamic Extremist) attacked multiple types of victims (an adult and a child) in Sri Lanka.

The association specification uses the multiplicative format to create events using the following math to determine the total number of associations: Entity1 (Person Perpetrator) * Entity2 (Victim Type) * Geo_index (Location) = Total Number of Associations.

Code Block

            {
                "creationCriteriaScript": "$FUNC( isOrganizationSpecified(); )",
                "entity1": "Organization",
                "entity2": "VictimType",
                "iterateOver": "entity1/entity2",
                "time_start": "$SCRIPT( return _doc.metadata.incidentdate[0]; )",
                "verb": "attacked",
                "verb_category": "assault/attack"
            },

On the following set of entities:

Code Block

    "entities": [
        {
            "actual_name": "Taliban",
            "dimension": "Who",
            "disambiguated_name": "Taliban",
            "doccount": 0,
            "frequency": 1,
            "index": "taliban/organization",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "Organization"
        },
        {
            "actual_name": "Targeted, Civilian, Adult from Afghanistan",
            "dimension": "Who",
            "disambiguated_name": "Targeted, Civilian, Adult from Afghanistan",
            "doccount": 0,
            "frequency": 1,
            "index": "targeted, civilian, adult from afghanistan/victimtype",
            "relevance": 0,
            "totalfrequency": -1,
            "type": "VictimType"
        },

Sample Output:

Code Block

{
    "associations": [
        {
            "assoc_type": "Event",
            "entity1": "taliban",
            "entity1_index": "taliban/organization",
            "entity2": "targeted, civilian, adult from afghanistan",
            "entity2_index": "targeted, civilian, adult from afghanistan/victimtype",
            "time_start": "2009-07-01T00:00:00",
            "verb": "attacked",
            "verb_category": "assault/attack"
        }

Anchor
associative
associative
Associative

Additive associations cover the less common case where (eg) 2 entity types have the same number of elements and are ordered "in lock step". For example:

Code Block
"entities": [ { "index": "alex/person", "index": "craig/person", "index": "baltimore/city", "index": "washington dc/city", } ]

In this case the additive association specification:

Code Block
{ "iterateOver": "entity1,entity2", // note "," instead of "/" "entity1": "Person", "entity2": "City", "verb_category": "lives in", }

Would generate the 2 associations depicted in the sample output below

Code Block

],
    "associations": [
        {
            "entity1": "OrganizationAlex/person",
   
                "entity2": "FacilityType",
      verb": "lives in",
             "geoverb_indexcategory": "Locationresidence",
 
                  "iterateOver"entity2": "entity1/entity2/geo_indexBaltimore/city",
            "assoc_type": "Fact"
       "time_start },
		{
			"entity1": "$SCRIPT( return _doc.metadata.incidentdate[0]; )","Craig/person",
            "verb": "lives in",
             "verb_category": "attackedresidence",
            "entity2": "Washington dc/city",
     "verb_category": "assault/attack"
                },
           "assoc_type": "Fact"
		}

 ],

Anchor
creation
creation
creationCriteriaScript

The associations code block below creates associations based on a twitter user(actor).

The creationCriteriaScript establishes conditions for the creation of associations. For objects returned that do not match, associations are not created.

Code Block

  },
           {
                    "creationCriteriaScript"associations": "$FUNC( isOrganizationSpecified(); )",[
                {
     "entity1": "Organization",                     "entity2"assoc_type": "VictimTypeEvent",
                    "geo_indexcreationCriteriaScript": "Location$SCRIPT( return (null != _doc.metadata.json[0].object.actor); )",
                    "iterateOverentity1_index": "entity1/entity2/geo_index$SCRIPT( return _doc.metadata.json[0].actor.preferredUsername + '/twitterhandle';)",
                    "timeentity2_startindex": "$SCRIPT( return _doc.metadata.incidentdate[0]; json[0].object.actor.preferredUsername + '/twitterhandle';)",
                    "verb": "attackedretweets",
                    "verb_category": "assault/attackretweets"
                },
                {
                    "creationCriteriaScriptassoc_type": "$FUNC( isOrganizationSpecified(); )",Event",
                    "creationCriteriaScript": "$SCRIPT(     "entity1": "Organization"return (null != _doc.metadata.json[0].object.actor) && (null != _doc.metadata.json[0].object.actor.location); )",
                    "entity2entity1_index": "HostageType$SCRIPT( return _doc.metadata.json[0].object.actor.preferredUsername + '/twitterhandle';)",
                    "geoentity2_index": "Location$SCRIPT( return _doc.metadata.json[0].object.actor.location.displayName+ '/location';)",
                    "iterateOververb": "entity1/entity2/geo_indextwitter_location",
                    "timeverb_startcategory": "$SCRIPT( return _doc.metadata.incidentdate[0]; )",twitter_location"
                },
             "verb": "took hostage",   {
                    "verbassoc_categorytype": "assault/attackEvent"
,
               },     "entity1_index": "$SCRIPT(      return _doc.metadata.json[0].actor.preferredUsername + '/twitterhandle';)",
    {                     "creationCriteriaScript"entity2_index": "$SCRIPT( if (isOrganizationSpecified() == false) return truereturn _iterator.text + '/hashtag'; )",
                    "entity1iterateOver": "PersonPerpetratorjson.twitter_entities.hashtags",
                    "entity2verb": "FacilityTypetweets_about",
                    "geoverb_indexcategory": "Location",
           tweets_about"
        "iterateOver": "entity1/entity2/geo_index",       },
             "time_start": "$SCRIPT( return _doc.metadata.incidentdate[0]; )", {
                    "verbassoc_type": "attackedEvent",
                    "verbentity1_categoryindex": "assault/attack"
    $SCRIPT( return _doc.metadata.json[0].actor.preferredUsername + '/twitterhandle';)",
           },           "entity2_index": "$SCRIPT( return _iterator.screen_name +  {'/twitterhandle'; )",
                    "creationCriteriaScriptiterateOver": "$SCRIPT( if (isOrganizationSpecified() == false) return true; )json.twitter_entities.user_mentions",
                    "entity1verb": "PersonPerpetratortweets_to",
                    "entity2verb_category": "VictimType",tweets_to"
                },
               "geo_index": "Location", {
                    "iterateOverassoc_type": "entity1/entity2/geo_indexEvent",
                    "timeentity1_startindex": "$SCRIPT( return _doc.metadata.incidentdatejson[0]; .actor.preferredUsername + '/twitterhandle';)",
                    "verbentity2_index": "attacked$SCRIPT( return _iterator.expanded_url + '/url'; )",
                    "verb_categoryiterateOver": "assault/attack" "json.gnip.urls",
                     }"verb": "tweets_link",
                {    "verb_category": "tweets_link"
               "creationCriteriaScript": "$SCRIPT( if (isOrganizationSpecified() == false) return true; )", }
            ]
         },

Output:

The following associations are output, based on the entity types and the creationcriteriaScript.

Code Block

{
    "entity1associations": [
"PersonPerpetrator",        {
            "entity2assoc_type": "HostageTypeEvent",
            "entity1": "focalcrm",
      "geo      "entity1_index": "Locationfocalcrm/twitterhandle",
                    "iterateOver"entity2": "entity1/entity2/geo_indexcrm",
                    "time_start"entity2_index": "$SCRIPT( return _doc.metadata.incidentdate[0]; )","crm/hashtag",
            "verb": "tweets_about",
            "verb_category": "took hostage",tweets_about"
        },
        {
  "verb_category": "assault/attack"         "assoc_type": "Event",
      }

Sample Output:

TODO add sample output.

...

Additive associations cover the less common case where (eg) 2 entity types have the same number of elements and are ordered "in lock step". For example:

Code Block

"entities": [
	{ "      "entity1": "focalcrm",
            "entity1_index": "alexfocalcrm/persontwitterhandle",
            "entity2": "http://www... },
	{ "crmbuyer.com/rsstory/76578.html",
            "entity2_index": "craig/person", ... },
	{ "index": "baltimore/city", ... },
	{ "index": "washington dc/city", ...},
	...
]

In this case the additive association specification:

Code Block

{
	"iterateOver": "entity1,entity2", // note "," instead of "/"
	"entity1": "Person",
	"entity2": "City",
	http://www.crmbuyer.com/rsstory/76578.html/url",
            "verb": "tweets_link",
            "verb_category": "lives in",
	...
}

...

tweets_link"
        }

Panel

Footnotes:

Legacy documentation:

Specifying Associations

Legacy documentation:

StructuredAnalysis object (under "Associations")

...

Versions Compared

Old Version 15

New Version Current

Key

iterateOver

creationCriteriaScripts

iterateOver

creationCriteriaScripts

Examples

Basic Association

iterateOver

Anchor
metadata
metadata
iterateOver a Metadata Field

Anchor
single
single
iterateOver a single entity type

Anchor
multi
multi
Multiplicative

Anchor
associative
associative
Associative

Anchor
creation
creation
creationCriteriaScript

Page Comparison

Versions Compared

Old Version 15

New Version Current

Key

iterateOver

creationCriteriaScripts

iterateOver

creationCriteriaScripts

Examples

Basic Association

iterateOver

AnchormetadatametadataiterateOver a Metadata Field

AnchorsinglesingleiterateOver a single entity type

AnchormultimultiMultiplicative

AnchorassociativeassociativeAssociative

AnchorcreationcreationcreationCriteriaScript

Anchor
metadata
metadata
iterateOver a Metadata Field

Anchor
single
single
iterateOver a single entity type

Anchor
multi
multi
Multiplicative

Anchor
associative
associative
Associative

Anchor
creation
creation
creationCriteriaScript