Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Code Block
languagejava
{
	"display": string, 
	"harvest": {
		"searchCycle_secs":integer, // How often to run the harvester (copied to SourcePojo when published)
		"duplicateExistingUrls":Boolean,// If false (defaults to true) then documents matching the URL of any existing document in the community is ignored (copied to SourcePojo when published)
		
		"maxDocs_global":Integer, // If specified, limits the total number of documents that can be harvested for a given source - when new documents are harvested exceeding this limit, older documents are deleted to maintain the size
		"throttleDocs_perCycle":Integer, // If specified, limits the number of documents that can be harvested for a given source (state moves to SUCCESS_ITERATION ie the next harvest cycle, the harvester will pick up again, as above)
		"maxDocs_perCycle":Integer, // If specified, limits the number of documents that can be harvested for a given source (state moves to SUCCESS - ie this+searchCycle_secs limits document ingest rate)
		
		"distributionFactor":Integer, // (EXPERIMENTAL) If specified, attempts to distribute the source across many threads
	}

}

 

Description

The following table describes the parameters of the harvest control settings configuration.

...

Panel

Footnotes:

Legacy documentation: