Data Sources

The Data Sources section of Manage is where you can create your Data Sources using the Source Builder.

Authorization Requirements: You must have administrative permissions in order to use the Manager.

Some of the visualizations that ship with Information Security Analytics require additional data processing jobs to be executed from the platform. An IKANOW resource is required to execute these map reduce jobs before your data will appear in the visualizations.

Source	Source Name/Link	Description
	Logstash Import Source	A lighter weight IKANOW object format for storing logs, term/record volumes, or statistics. A very common use case for logs/records is in Dev. Ops environments where log files need to be filtered and appropriately analyzed. Using the IKANOW record format it is easy to filter log files, define column names, and determine geo ip information for example. This record data can then be analyzed along with other IKANOW documents for log analysis and big data cyber analytics within one platform.
	RSS with NLP	You can use the Information Security Analytics Manager to connect to an RSS feed as an input source.
	Web pages with NLP	Extracts documents from lists of URLs.
	Yahoo Search API	You can use the Information Security Analytics Manager to connect to the Yahoo Search API. Provides a rich set of premium data APIs and tools that developers and entrepreneurs can use to build custom search engines and innovative experiences.
	Datasift from S3	Datasift is an aggregation service that streams and enriches tweets, posts, blogs, and news from a variety of social media and other Internet sources.
	Twitter Search API	The Twitter Search API is part of Twitter’s REST API. It allows queries against the indices of recent or popular Tweets and behaves similarly to, but not exactly like the Search feature available in Twitter mobile or web clients.
	Lookup table builder	You can use the Information Security Analytics Manager to build a lookup table in the Information Security Analytics platform. Building a lookup table describes the process of indicating the JSON share where the lookup table is located, and specifying the Key Field and Header Fields in your lookup table. Once the lookup table is built it can be loaded at harvest time to specify a global set of variables that can be used to simplify the process of generating metadata using Javascripting.
	Lookup table applier	You can use the Information Security Analytics Manager to apply a previously created lookup table. Applying a lookup table describes the process of indicating the lookup table name to apply, as well as the Data Group to apply the lookup table, and the record type (eg. Apache).
	Advanced Source Builder	You can use the Advanced source builder to edit the raw JSON of the Source Pipeline Elements directly in the browser.

About Configuring Data Sources

During Data Source configuration you are prompted to specify source configuration settings. These settings are applicable to every source type/template.

Source Configuration Setting	Description
Source Name	Source Name
Data Group	Data Group the Data Source is associated
Media Type	News, Social, report, Record etc.
Data Origin	Internal or External. Internal: Data internal to your organization. eg. logs, records, reports External: Data from external threat feeds, RSS feeds etc.
Frequency	Frequency at which ISA will process the data source to account for changes