API - Examples
- Caleb Burch (Unlicensed)
- andrew johnston (Unlicensed)
- Christopher Morgan (Unlicensed)
- AlexI (Unlicensed)
This page shows examples of how to use the API in to perform simple tasks in Java, Actionscript and curl. Any language that can send HTTP request can consume the API, these are just some examples showing some common languages.
Note that there is a "beta" Java driver in the data model JAR (github link). It is beta in the sense that it is incomplete (ie some REST calls don't have Java functions) and that it is (currently) undocumented. All functions that are present are well-tested (they have been used in our own internal applications).
Example 0: Hashing a password
If we look at the login page documentation, we see that the password needs to be SHA-256 encoded and a base-64 string.
Because logging in is a vital function, here is a Java, Actionscript, and Web example of getting your hashed password:
*If you are doing this correctly the password "12345" will hash to "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U="
//In this example we use the apache commons codec library to convert our hashed password to base 64 //the library can be found at: http://commons.apache.org/codec/download_codec.cgi import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import org.apache.commons.codec.binary.Base64; public class hashpword { public static void main(String[] args) throws NoSuchAlgorithmException { String password = "12345"; MessageDigest digest = MessageDigest.getInstance("SHA-256"); byte[] hashbytes = digest.digest(password.getBytes()); String hashpassword = Base64.encodeBase64String(hashbytes); System.out.println(hashpassword); } }
//The Crypto class can be found in the as3crypto library //the library can be found at: http://code.google.com/p/as3crypto/ var password:String = "12345"; var c:Crypto = new Crypto(); var cipher:IHash = Crypto.getHash( "sha256" ); var data:ByteArray = Hex.toArray( Hex.fromString( password ) ); var hashed:String = Base64.encodeByteArray( cipher.hash( data ) ); Alert.show( hashed );
Web Browser - Hashing a Password
Example 1: Sending an HTTP request and sending a request to Login
Because our API is REST based, the most important thing to do in any language is understand how to send HTTP Requests so you are able to call our system. Many languages may have multiple ways to do this so we will show an example of how you can send requests in Java, Actionscript, curl, and your browser.
//set up url String username = "sterling_archer@ikanow.com"; String hashedpassword = "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments String address = "http://infinite.ikanow.com/api/auth/login/" + username + "/" + hashedpassword; //send request URL url = new URL(address); URLConnection urlConnection = url.openConnection(); ((HttpURLConnection)urlConnection).setRequestMethod("GET"); //read back result BufferedReader inStream = new BufferedReader(new InputStreamReader(urlConnection.getInputStream())); StringBuilder strBuilder = new StringBuilder(); String buffer; while ( (buffer = inStream.readLine()) != null ) { strBuilder.append(buffer); } inStream.close(); //print out response json System.out.println(strBuilder.toString());
protected function button_clickHandler( event:MouseEvent ):void { //set up url var username:String = "sterling_archer@ikanow.com"; var hashedpassword:String = "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments var address:String = "http://infinite.ikanow.com/api/auth/login/" + username + "/" + hashedpassword; //send request var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = address; httpService.send(); } protected function httpFaultHandler( event:FaultEvent ):void { Alert.show( "Http Request had an error: " + event.message ); } protected function httpResultHandler( event:ResultEvent ):void { //print out response json Alert.show( event.result.toString() ); }
curl http://infinite.ikanow.com/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D
1. In the url bar of your favorite web browser just type the url you want to navigate to and push go: http://infinite.ikanow.com/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D
Once you have mastered how to send http requests, you can attempt to send requests to the API. To login to the system you simply need to send an http request to http://infinite.ikanow.com/api/auth/login with username and password. On a successful login, you will receive a cookie that will be active for 30 minutes from your last command. This cookie will need to be passed back anytime you want to perform an API call that needs authentication.
In the next section we show examples of how to send requests with cookies.
Example 2: Getting a cookie, and sending the cookie in a subsequent request (Login, Get Person, Logout).
To make requests to many of our services we require you to send a cookie along with the HTTP Request so we can verify who we are getting the information for. Below are examples of how to receive cookies from a Login call, send the cookies to another call, and log out.
public static void main(String[] args) throws Exception { //don't forget to URLEncode your arguments String address = "http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; String loginresult = sendRequest(address); //Our data objects can be used by importing infinit.e.data_model.jar and gson.jar ResponsePojo response = ResponsePojo.fromApi(loginresult, ResponsePojo.class); if ( response.getResponse().isSuccess()) { //send next request String personAddress = "http://infinite.ikanow.com/api/social/person/get"; String personresult = sendRequest(personAddress); //We need to convert the result object into a response with a person object in it response = ResponsePojo.fromApi(personresult, ResponsePojo.class, PersonPojo.class, new PersonPojoApiMap()); if ( response.getResponse().isSuccess() ) { PersonPojo personResult = (PersonPojo)response.getData(); System.out.println(personResult.getEmail()); } } else { System.out.println("error logging in: " + response.getResponse().getMessage()); } //logout when we are done, this will deactivate our cookie sendRequest("http://infinite.ikanow.com/api/auth/logout"); } private static String cookie = null; public static String sendRequest(String urlAddress ) throws Exception { URL url = new URL(urlAddress); URLConnection urlConnection = url.openConnection(); if ( cookie != null ) //add cookie to request if we have one urlConnection.setRequestProperty("Cookie", cookie); ((HttpURLConnection)urlConnection).setRequestMethod("GET"); //read back result BufferedReader inStream = new BufferedReader(new InputStreamReader(urlConnection.getInputStream())); StringBuilder strBuilder = new StringBuilder(); String buffer; while ( (buffer = inStream.readLine()) != null ) { strBuilder.append(buffer); } inStream.close(); //save cookie if cookie is null if ( cookie == null ) { String headername; for ( int i = 1; (headername = urlConnection.getHeaderFieldKey(i)) != null; i++ ) { if ( headername.equals("Set-Cookie") ) { cookie = urlConnection.getHeaderField(i); break; } } } return strBuilder.toString(); }
protected function button_clickHandler( event:MouseEvent ):void { //set up url var username:String = "sterling_archer@ikanow.com"; var hashedpassword:String = "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments var address:String = "http://infinite.ikanow.com/api/auth/login/" + username + "/" + hashedpassword; //send login request var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = address; httpService.send(); } protected function httpResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //successfully logged in, cookies are automatically stored in flash so no need to handle them //send request for our person object (will print out in httpPersonResultHandler) var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpPersonResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = "http://infinite.ikanow.com/api/social/person/get"; httpService.send(); } } protected function httpPersonResultHandler( event:ResultEvent ):void { //just print out person result json Alert.show( event.result.toString() ); //logout var httpService:HTTPService = new HTTPService(); httpService.url = "http://infinite.ikanow.com/api/auth/logout"; httpService.send(); } protected function httpFaultHandler( event:FaultEvent ):void { Alert.show( "Http Request had an error: " + event.message ); }
curl -c cookies.txt http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D curl -b cookies.txt http://infinite.ikanow.com/api/social/person/get curl -b cookies.txt http://infinite.ikanow.com/api/auth/logout
1. In the url bar of your favorite web browser just type the url you want to navigate to and push go: http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D 2. In the url bar of your favorite web browser just type the url you want to navigate to and push go: http://infinite.ikanow.com/api/social/person/get 3. In the url bar of your favorite web browser just type the url you want to navigate to and push go: http://infinite.ikanow.com/api/auth/logout
Beyond this point the examples are just showing added functionality. As a result only a single language will be used to demonstrate each examples. Using the above principles in example 2 you can extend any language to do the tasks below.
Example 3: Login, Get Communities, Perform a Query, Logout
Performing a query adds a final bit of complexity in which we must send a POST request with a json object in the body.
<?xml version="1.0" encoding="utf-8"?> <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600"> <fx:Script> <![CDATA[ import com.adobe.serialization.json.JSON; import mx.controls.Alert; import mx.rpc.events.FaultEvent; import mx.rpc.events.ResultEvent; import mx.rpc.http.HTTPService; protected function button1_clickHandler( event:MouseEvent ):void { //LOGIN //set up url var username:String = "sterling_archer@ikanow.com"; var hashedpassword:String = "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments var address:String = "http://infinite.ikanow.com/api/auth/login/" + username + "/" + hashedpassword; //send login request var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpLoginResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = address; httpService.send(); } protected function httpFaultHandler( event:FaultEvent ):void { Alert.show( "Http Request had an error: " + event.message ); } protected function httpLoginResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //successfully logged in, cookies are automatically stored in flash so no need to handle them //send request for our person object var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpPersonResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = "http://infinite.ikanow.com/api/social/person/get"; httpService.send(); } } protected function httpPersonResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //here we will grab the communities so we can send them in a query var communities:Array = jsonObject.data.communities; //now we need to turn the community ids into a comma deliminated string //you do not need to send all communities, just the ones you want to query on var communityString:String = communities[ 0 ]._id; for ( var i:int = 1; i < communities.length; i++ ) { communityString += "," + communities[ i ]._id; } //create a json object for the query, actionscript objects //can be converted into json so we can just create an anonymous object var queryData:Object = new Object(); queryData.qt = new Array(); var queryTerm:Object = new Object(); queryTerm[ "ftext" ] = "sterling archer"; queryData.qt.push( queryTerm ); //send request for query var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpQueryResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = "http://infinite.ikanow.com/api/knowledge/document/query/" + communityString; //to send a post we just put the data in the send handler httpService.send( JSON.encode( queryData ) ); } } protected function httpQueryResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //now you can do something with the results of query //say get every title? var titles:String = ""; for each ( var doc:Object in jsonObject.data ) { titles += doc.title.toString() + "\n"; } Alert.show( titles ); //dont forget to logout var httpService:HTTPService = new HTTPService(); httpService.url = "http://infinite.ikanow.com/api/auth/logout"; httpService.send(); } } ]]> </fx:Script> <fx:Declarations> <!-- Place non-visual elements (e.g., services, value objects) here --> </fx:Declarations> <fx:Script> <![CDATA[ ]]> </fx:Script> <s:Button label="Push Me" click="button1_clickHandler(event)" /> </s:Application>
Example 4: Login, Create a Community, Invite User, Logout
import java.io.BufferedReader; import java.io.InputStreamReader; import java.net.HttpURLConnection; import java.net.URL; import java.net.URLConnection; import java.net.URLEncoder; import com.ikanow.infinit.e.data_model.api.ResponsePojo; import com.ikanow.infinit.e.data_model.api.social.community.CommunityPojoApiMap; import com.ikanow.infinit.e.data_model.store.social.community.CommunityPojo; public class test { /** * @param args * @throws Exception */ public static void main(String[] args) throws Exception { //login String address = "http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments String loginresult = sendRequest(address); ResponsePojo response = ResponsePojo.fromApi(loginresult, ResponsePojo.class); if ( response.getResponse().isSuccess()) { //create a community String communityName = "testcommunity4"; String communityDesc = "some long desc"; String communityTags = "testtag1,testtag2"; String createCommunityAddress = "http://infinite.ikanow.com/api/social/community/add/" + URLEncoder.encode(communityName,"UTF-8") + "/" + URLEncoder.encode(communityDesc,"UTF-8") + "/" + URLEncoder.encode(communityTags,"UTF-8"); String communityresult = sendRequest(createCommunityAddress); //We need to convert the result object into a response with a person object in it response = ResponsePojo.fromApi(communityresult, ResponsePojo.class, CommunityPojo.class, new CommunityPojoApiMap()); if ( response.getResponse().isSuccess() ) { CommunityPojo communityResult = (CommunityPojo)response.getData(); String personID = "abcde12345"; //some other users id that you want to invite //now invite a user //to use ids, you need to import a mongo.jar available at https://github.com/mongodb/mongo-java-driver/downloads String inviteUserAddress = "http://infinite.ikanow.com/api/social/community/member/invite/" + communityResult.getId().toString() + "/" + personID; String inviteresult = sendRequest(inviteUserAddress); response = ResponsePojo.fromApi(inviteresult, ResponsePojo.class); if ( response.getResponse().isSuccess()) { //invited user successfully! now we are done } } } else { System.out.println("error logging in: " + response.getResponse().getMessage()); } //logout when we are done, this will deactivate our cookie sendRequest("http://infinite.ikanow.com/api/auth/logout"); } private static String cookie = null; public static String sendRequest(String urlAddress ) throws Exception { URL url = new URL(urlAddress); URLConnection urlConnection = url.openConnection(); if ( cookie != null ) urlConnection.setRequestProperty("Cookie", cookie); ((HttpURLConnection)urlConnection).setRequestMethod("GET"); //read back result BufferedReader inStream = new BufferedReader(new InputStreamReader(urlConnection.getInputStream())); StringBuilder strBuilder = new StringBuilder(); String buffer; while ( (buffer = inStream.readLine()) != null ) { strBuilder.append(buffer); } inStream.close(); //save cookie if cookie is null if ( cookie == null ) { String headername; for ( int i = 1; (headername = urlConnection.getHeaderFieldKey(i)) != null; i++ ) { if ( headername.equals("Set-Cookie") ) { cookie = urlConnection.getHeaderField(i); break; } } } return strBuilder.toString(); } }
Note that the Java example above includes an example of how to use the Data Model library to deserialize from JSON strings into Infinit.e data store (or API) objects:
// Single object: ResponsePojo response = ResponsePojo.fromApi(communityresult, ResponsePojo.class, CommunityPojo.class, new CommunityPojoApiMap()); // (use null instead of "new CommunityPojoApiMap()" for objects derived from BaseApiPojo) CommunityPojo community = (CommunityPojo)response.getData(); // Multiple objects: ResponsePojo response = ResponsePojo.listFromApi(communityresult, ResponsePojo.class, CommunityPojo.listType(), new CommunityPojoApiMap()); List<CommunityPojo> communities = (List<CommunityPojo>)response.getData();
Example 5: Login, Get Public Communities, Request to Join Community, Logout
curl -c cookies.txt 'http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D' > response.txt curl -b cookies.txt 'http://infinite.ikanow.com/api/social/community/getpublic' > response.txt //From the response of getpublic communities we can copy one of the ids to what community we wish to join curl -b cookies.txt 'http://infinite.ikanow.com/api/social/community/member/join/4c927585d591d31d7c37097b' > response.txt //response will say if you have been successfully added, or awaiting owner permission curl -b cookies.txt 'http://infinite.ikanow.com/api/auth/logout' > response.txt *note: All of these curl calls are pushing their output to a local file named response.txt, you can remove this if you want to just print out the result in console
Example 6: Login, Get a Source, Edit the Source, Test the new Source, Save the new Source, Logout
<?xml version="1.0" encoding="utf-8"?> <s:Application xmlns:fx="http://ns.adobe.com/mxml/2009" xmlns:s="library://ns.adobe.com/flex/spark" xmlns:mx="library://ns.adobe.com/flex/mx" minWidth="955" minHeight="600"> <fx:Script> <![CDATA[ import com.adobe.serialization.json.JSON; import mx.controls.Alert; import mx.rpc.events.FaultEvent; import mx.rpc.events.ResultEvent; import mx.rpc.http.HTTPService; private var source:Object; protected function button1_clickHandler( event:MouseEvent ):void { //LOGIN //set up url var username:String = "sterling_archer@ikanow.com"; var hashedpassword:String = "WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments var address:String = "http://infinite.ikanow.com/api/auth/login/" + username + "/" + hashedpassword; //send login request var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpLoginResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = address; httpService.send(); } protected function httpFaultHandler( event:FaultEvent ):void { Alert.show( "Http Request had an error: " + event.message ); } protected function httpLoginResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //this is a random microsoft source //you can get a source from config/source/good var sourceID:String = "4ef498756a068f1eee17fac4"; //successfully logged in, cookies are automatically stored in flash so no need to handle them //send request for our person object var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpSourceResultHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.url = "http://infinite.ikanow.com/api/config/source/get/" + sourceID; httpService.send(); } } protected function httpSourceResultHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //here we will edit the source to try something new source = jsonObject.data; source.url = "http://rss.cnn.com/rss/cnn_topstories.rss"; source.title = "CNN top stories"; source.description = "CNNs top news stories around the world"; //lets removes some of the fields we dont need anymore delete source._id; delete source.created; delete source.modified; delete source.harvest; delete source.shah256Hash; delete source.ownerId; delete source.key; delete source.tags; delete source.harvestBadSource; //now we will test our new source to see if the docs it brings back are what we want var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpSourceTestHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.method = "POST"; httpService.url = "http://infinite.ikanow.com/api/config/source/test?numReturn=1&returnFullText=false"; //to send a post we just put the data in the send handler //we are posting the modified source we created and set the content type httpService.contentType = "application/json"; httpService.send( JSON.encode( source ) ); } } protected function httpSourceSaveHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //dont forget to logout var httpService:HTTPService = new HTTPService(); httpService.url = "http://infinite.ikanow.com/api/auth/logout"; httpService.send(); } } protected function httpSourceTestHandler( event:ResultEvent ):void { //convert result json string to an as3 object using the as3corelib library //available at: https://github.com/mikechambers/as3corelib var jsonObject:Object = JSON.decode( event.result.toString(), true ); if ( jsonObject.response.success == true ) { //if we like how the source was harvest in jsonObject.data //we can save our source back to the server //you can get community ids from person/get var communityID:String = "4c927585d591d31d7b37097a"; var httpService:HTTPService = new HTTPService(); httpService.addEventListener( ResultEvent.RESULT, httpSourceSaveHandler ); httpService.addEventListener( FaultEvent.FAULT, httpFaultHandler ); httpService.method = "POST"; httpService.url = "http://infinite.ikanow.com/api/config/source/save/" + communityID; //to send a post we just put the data in the send handler //we are posting the modified source we created and set the content type httpService.contentType = "application/json"; httpService.send( JSON.encode( source ) ); } } ]]> </fx:Script> <fx:Declarations> <!-- Place non-visual elements (e.g., services, value objects) here --> </fx:Declarations> <fx:Script> <![CDATA[ ]]> </fx:Script> <s:Button label="Push Me" click="button1_clickHandler(event)" /> </s:Application>
Example 7: Login, share a hadoop Jar file, schedule a map reduce job with jar, Logout
import java.io.BufferedReader; import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.OutputStream; import java.net.HttpURLConnection; import java.net.URL; import java.net.URLConnection; import java.net.URLEncoder; import com.ikanow.infinit.e.data_model.api.ResponsePojo; public class test { /** * @param args * @throws Exception */ public static void main(String[] args) throws Exception { //login String address = "http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D"; //don't forget to URLEncode your arguments String loginresult = sendRequest(address, null); ResponsePojo response = ResponsePojo.fromApi(loginresult, ResponsePojo.class); if ( response.getResponse().isSuccess()) { //get map reduce file bytes, send in POST to add binary call String mapreduceFile = "C:/Users/Burch/Desktop/mapreducefile.jar"; byte[] fileBytes = getBytesFromFile(new File(mapreduceFile)); String shareTitle = "Archer Sum Map Reduce"; String shareDesc = "Sums up the sources by community, by Sterling Archer"; String createCommunityAddress = "http://infinite.ikanow.com/api/social/share/add/binary/" + URLEncoder.encode(shareTitle,"UTF-8") + "/" + URLEncoder.encode(shareDesc,"UTF-8"); String communityresult = sendRequest(createCommunityAddress, fileBytes); //The response returns only the id of the share response = ResponsePojo.fromApi(communityresult, ResponsePojo.class); if ( response.getResponse().isSuccess() ) { String shareID = (String)response.getData(); String jobTitle = "Test Job"; String jobDesc = "testing a mr job"; String communityIds = "4c927585d591d31d7b37097a"; //can get communities from social/person/get String jarURL = "$infinite/share/get/" + shareID; //the web url of the share we just added $infinite will be converted to the local api serverside String timeToRun = "0"; //if we want this job to run in the future we will put the ms time to run at String frequencyToRun = "NONE"; //see api page for available enums, we only want it to run once so we put NONE String mapperClass = "com.ikanow.infinit.e.core.mapreduce.examplejars.Test%24TokenizerMapper"; //package name and class of mappper String reducerClass = "com.ikanow.infinit.e.core.mapreduce.examplejars.Test%24IntSumReducer"; //package name and class of reducer String combinerClass = "com.ikanow.infinit.e.core.mapreduce.examplejars.Test%24IntSumReducer"; //this can be the same as reducer usually String query = "null"; //we want everything so we can put null, otherwise you can put json mongodb query String inputcollection = "DOC_METADATA"; //see API for what is available String outputKey = "org.apache.hadoop.io.Text"; String outputValue = "org.apache.hadoop.io.IntWritable"; //now we want to schedule a map reduce job with this jar String scheduleJobAddress = "http://infinite.ikanow.com/api/custom/mapreduce/schedulejob/" + URLEncoder.encode(jobTitle,"UTF-8") + "/" + URLEncoder.encode(jobDesc,"UTF-8") + "/" + URLEncoder.encode(communityIds,"UTF-8") + "/" + URLEncoder.encode(jarURL,"UTF-8") + "/" + URLEncoder.encode(timeToRun,"UTF-8") + "/" + URLEncoder.encode(frequencyToRun,"UTF-8") + "/" + URLEncoder.encode(mapperClass,"UTF-8") + "/" + URLEncoder.encode(reducerClass,"UTF-8") + "/" + URLEncoder.encode(combinerClass,"UTF-8") + "/" + URLEncoder.encode(query,"UTF-8") + "/" + URLEncoder.encode(inputcollection,"UTF-8") + "/" + URLEncoder.encode(outputKey,"UTF-8") + "/" + URLEncoder.encode(outputValue,"UTF-8"); //to use ids, you need to import a mongo.jar available at https://github.com/mongodb/mongo-java-driver/downloads String schedulejobresult = sendRequest(scheduleJobAddress, null); response = ResponsePojo.fromApi(schedulejobresult, ResponsePojo.class); if ( response.getResponse().isSuccess()) { //scheduled job successfully, get results ID, will be available once job has finished running String mapreduceResultID = (String) response.getData(); System.out.println(mapreduceResultID); } } } else { System.out.println("error logging in: " + response.getResponse().getMessage()); } //logout when we are done, this will deactivate our cookie sendRequest("http://infinite.ikanow.com/api/auth/logout", null); } private static String cookie = null; public static String sendRequest(String urlAddress, byte[] postData ) throws Exception { URL url = new URL(urlAddress); URLConnection urlConnection = url.openConnection(); if ( cookie != null ) urlConnection.setRequestProperty("Cookie", cookie); if ( postData != null ) { urlConnection.setDoOutput(true); urlConnection.setRequestProperty("Accept-Charset", "UTF-8"); urlConnection.setRequestProperty("Content-Type", "application/java-archive" + ";charset=" + "UTF-8"); ((HttpURLConnection)urlConnection).setRequestMethod("POST"); OutputStream output = urlConnection.getOutputStream(); output.write(postData); output.close(); } else { ((HttpURLConnection)urlConnection).setRequestMethod("GET"); } //read back result BufferedReader inStream = new BufferedReader(new InputStreamReader(urlConnection.getInputStream())); StringBuilder strBuilder = new StringBuilder(); String buffer; while ( (buffer = inStream.readLine()) != null ) { strBuilder.append(buffer); } inStream.close(); //save cookie if cookie is null if ( cookie == null ) { String headername; for ( int i = 1; (headername = urlConnection.getHeaderFieldKey(i)) != null; i++ ) { if ( headername.equals("Set-Cookie") ) { cookie = urlConnection.getHeaderField(i); break; } } } return strBuilder.toString(); } /** * from: http://www.exampledepot.com/egs/java.io/file2bytearray.html * * @param file * @return * @throws IOException */ public static byte[] getBytesFromFile(File file) throws IOException { InputStream is = new FileInputStream(file); // Get the size of the file long length = file.length(); // You cannot create an array using a long type. // It needs to be an int type. // Before converting to an int type, check // to ensure that file is not larger than Integer.MAX_VALUE. if (length > Integer.MAX_VALUE) { // File is too large } // Create the byte array to hold the data byte[] bytes = new byte[(int)length]; // Read in the bytes int offset = 0; int numRead = 0; while (offset < bytes.length && (numRead=is.read(bytes, offset, bytes.length-offset)) >= 0) { offset += numRead; } // Ensure all the bytes have been read in if (offset < bytes.length) { throw new IOException("Could not completely read file "+file.getName()); } // Close the input stream and return bytes is.close(); return bytes; } }
Note that unlike example 4, this Java example uses the simpler ResponsePojo "fromApi" call for when the "data" field from the API call is just a string:
ResponsePojo response = ResponsePojo.fromApi(schedulejobresult, ResponsePojo.class); String responseData = (String) response.getData();
Example 8: Login, Get Map Reduce Jobs, Get Reults of a map reduce job, Logout
curl -c cookies.txt 'http://infinite.ikanow.com/api/auth/login/sterling_archer@ikanow.com/WZRHGrsBESr8wYFZ9sx0tPURuZgG2lmzyvWpwXPKz8U%3D' > response.txt curl -b cookies.txt 'http://infinite.ikanow.com/api/custom/mapreduce/getjobs' > response.txt //copy over the job id that we want to see results for curl -b cookies.txt 'http://infinite.ikanow.com/api/custom/mapreduce/getresults/abcde12345' > response.txt //the results of the map reduce job will be in the data field of the response curl -b cookies.txt 'http://infinite.ikanow.com/api/auth/logout' > response.txt