Versions Compared

Old Version 15

changes.mady.by.user AlexI

Saved on

compared with

New Version 16

changes.mady.by.user AlexI

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This guide is meant to guide and give suggestions as to how to setup and configure the Hadoop installation once you have completed the RPM installation instructions found here: Infinit.e-hadoop-installer.

...

Warning

The following items must be changed for the full functionality of Infinit.e to be available:

  • Set "MapReduce Service Environment Safety Valve" to 'HADOOP_TASKTRACKER_OPTS="-Djava.security.policy=/opt/infinite-home/config/security.policy $HADOOP_TASKTRACKER_OPTS"'
  • Set "" to "-Djava.security.policy=/opt/infinite-home/config/security.policy"
    • (these 2 shouldn't both be necessary, however empirically across different OS configurations sometimes one would work, sometimes the other)
  • Set "Map Tasks Speculative Execution" and " " to be false

We recommend changing these items:

  1. Number of tasks to run per JVM (mapred.job.reuse.jvm.num.tasks) to -1
  2. Use Compression on Map Outputs (mapred.compress.map.output) to false (uncheck the box)
  3. Maximum Number of Simultaneous Map Tasks to 2
  4. Maximum Number of Simultaneous Reduce Tasks to 1
  5. (On systems with large amount of RAM available: increase the size of "MapReduce Child Java Maximum Heap Size")

After making these changes, navigate to Instances (from the toolbar at the top of the page), the task trackers will show as having "outdated instances". Select them all and restart them.

...