...
- In order to iterate through pages, the Follow Web links element can be used (eg in the "links.script" you would "var json = eval('('+text +')')", observe that "json.meta.offset + json.meta.limit < json.meta.total_count" and then push "{ url: _doc.url + "?limit="+json.limit, /* etc */, spiderOut: true}" onto the return array
- (You can also just set a large "numPages" and then set "stopPaginatingOnDuplicate" togther with "pageChangeRegex" and "pageChangeReplace") to avoid needing any scripting (in some cases the API reply doesn't have a limit/offset schema but instead each JSON object contains the URL of the next call - in this case scripting is currently needed)
- Headers and content data (ie POST) can be used. This is discussed in Web extractor.
- To use secure credentials, upload a JSON share containing them to your personal community (using the 5. File Uploader) and then use the substitution format "#IKANOW{ID.FIELD}" (Where ID is the "_id" (hex string) of the share, and FIELD is the fieldname).
- Multi-step authentication is harder - again the Follow Web links can be used.
...