| Table of Contents |
|---|
Overview
Version 0.5 of IKANOW requires that CDH5 be installed. This will enable applications such as Storm and Spark to be run on the YARN framework.
Unfortunately CDH5 and CDH3 are very non-backwards compatible with one another (*) - therefore CDH5 must be installed before IKANOW v0.5. This is enforced by requiring that hadoop-installer-v0.5 be present before infinit.e-processing-engine-v0.5 can be updated (and hence also infinit.e-interface-engine-v0.5).
(*) Except that the MapReduce JARs themselves do not need to be recompiled.
This page describes the steps necessary to do this. If installing CDH5 on a new IKANOW install, the "Upgrading from CDH3" step can be ignored.
...
| Warning |
|---|
It is not possible to upgrade from CDH3. Instead CDH3 must be removed and then CDH5 installed. All jobs must be stopped before this occurs - create the empty file "/opt/infinite-home/bin/STOP_CUSTOM" and wait for all jobs to complete in the jobtracker ("ROOT:8090/monitoring/<jobtracker-hostname>/50030/jobtracker.jsp") before continuing.
All files in HDFS that you don't want to lose must be copied off (eg to S3) and then copied back again when the upgrade is complete. You should check whether you have any scripts that import data into HDFS and turn them off temporarily. Then go to the Cloudera Manager and shut down all services. (It is not necessary to remove hosts because this cluster database will be destroyed as part of these steps). |
Once the above steps have been taken to stop Hadoop on the cluster, install/upgrade the latest "infinit.e-hadoop-installer" RPM (must be v0.5 or higher - note this also means the machines must have JDK8 installed).
- (eg "sh infinite_deploy_rpm_el6.sh <CLUSTER> ./infinit.e-hadoop-installer.online-v0.5-* <HOSTS>" for enterprise users)
Next run the script "/opt/hadoop-infinite/scripts/uninstall_cdh3.sh" on each node
- (eg 'sh infinite_run_script_el6.sh <CLUSTER> "sh /opt/hadoop-infinite/scripts/uninstall_cdh3.sh" <HOSTS>' for enterprise users)
Then
First step
To uninstall CDH3 TODO
Installing CDH5 (MRv1)
Command line phase
If you did not already do as a step under "Upgrading from CDH3", install/upgrade the latest "infinit.e-hadoop-installer" RPM (must be v0.5 or higher).
TODO
User interface phase
TODO
Installing CDH5 (YARN)
TODOIKANOW is not yet integrated with CDH5 running YARN. Once it is, this section will explain how to:
- Install CDH5 running YARN from scratch
- Move from CDH5 MRv1 to CDH5 YARN.