...
Code Block | ||
---|---|---|
| ||
{
"display": string,
"harvest": {
"searchCycle_secs":integer, // How often to run the harvester (copied to SourcePojo when published)
"duplicateExistingUrls":Boolean,// If false (defaults to true) then documents matching the URL of any existing document in the community is ignored (copied to SourcePojo when published)
"maxDocs_global":Integer, // If specified, limits the total number of documents that can be harvested for a given source - when new documents are harvested exceeding this limit, older documents are deleted to maintain the size
"throttleDocs_perCycle":Integer, // If specified, limits the number of documents that can be harvested for a given source (state moves to SUCCESS_ITERATION ie the next harvest cycle, the harvester will pick up again, as above)
"maxDocs_perCycle":Integer, // If specified, limits the number of documents that can be harvested for a given source (state moves to SUCCESS - ie this+searchCycle_secs limits document ingest rate)
"distributionFactor":Integer, // (EXPERIMENTAL) If specified, attempts to distribute the source across many threads
}
} |
Description
The following table describes the parameters of the harvest control settings configuration.
...
Panel |
---|
Footnotes: Legacy documentation: |