You can use the Threat Analytics Manager to import Logstash records into Data Sources.
Using the wizard, it is easy to specify header rows, field names, as well as quote and separator characters.
Importing the CSV File
To import the CSV record
From the Treat Analytics Dashboard, click on Data Sources (top right).
Under "What kind of source would you like to create?" specify "CSV Record Import Source."
Click on Next.
Configure the fields as described in the table below.
Field
Description
Note
File Path
Local file path where the CSV file can be located.
support for urls?
Date
Specific date that you would like to associate with the uploaded CSV file
What is the benefit of this, especially for files where each record has different date/time?
Time
Specific time that you would like to associate with the imported CSV file
Quote Character
Default: "
These should not require manual input, and should take defaults if user specifies nothing (currently throws an error). Default values should be viewable on GUI
Separator
Default: ,
Escape
Default: \
Shouldn't escape also be on the GUI?
Column Headers
Manually specify each column header
Shouldn't have to do this. Eg. If each header field starts with '#' they should be able to only specify this. Or leave blank for platform to do it automatically.
Advanced Options
The following fields are also available for more advanced CSV import settings.
Field
Description
Note
Geo IP
Input the "src_ip" (source IP address) that you want associated to this record. It will be used to geocode the CSV field to latitude and longitude information.
Type
Not sure what this is referring to.
Configuring and Testing
Once you have made the input settings, you will need to perform additional configuration and testing.
To configure and test
Provide a name for the source.
Select the previously created Data Group. todo link to data group.
Specify the frequency at which the source should be harvested (eg. Once per day)
Click on Test.
About Testing
If the source has been configured properly testing with return test results, and you will be able to move forward with Publishing the new source. Otherwise, a failure message is generated which can be used for troubleshooting (currently it only says FAIL). You can always Save your source and come back to fix any testing errors later.
Saving or Publishing
Saving
To save the source after testing
Click on Save.
The source is saved and you are re-directed to the Source Manager.
Publishing
To publish the source after testing
Click on Publish.
The source is published and you are re-directed to the list Source Manager.