Hadoop installation 2.7.2 Guide

Hadoop 2.7.2 installation Guide latest one by somappa Srinivasan

Step1 : Install Vmworkstion or oracle virtual Box in your machine(computer)

Download link for vmworkstation

http://filehippo.com/download_vmware-workstation-pro/

Download Oracle Virtual Box

https://www.virtualbox.org/wiki/Downloads

Step 2: Install Ubuntu os

Download Link for Ubuntu Os

https://www.ubuntu.com/download/desktop

Step 3: Update Ubuntu packages

Command : Sudo apt-get update

step 4 : Install Java 1.7 or 1.8

Command : sudo apt-get install openjdk-7-jdk

Step 5: Check whether java installed or not

Command : java -version

Step 6 : Check the Java path where JAVA installed in Ubuntu :

command : cd /usr/lib/jvm/java-1.7.0-openjdk-amd64

Step 7 : Set JAVA path in .bashrc file

Command : Sudo gedit .bashrc

Step 8 : set Java path in .bashrc file

Command :

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64

export PATH=$PATH:$JAVA_HOME/bin

Step 9 : update the .bashrc file

Command : Source .bashrc

Step 9 : install SSH

Step 10 : Download Hadoop latest tar file hadoop 2.7.2 in apache.org website

url : https://archive.apache.org/dist/hadoop/core/hadoop-2.7.2/

SStep 11: List of out the tar file in hadoop file

Step 12: Give Permission to hadoop tar file

Step 13: Extract tar file in ubntu terminal

command : sudo tar -xvf hadoop-2.7.2.tar.gz

Step 14: List out the Hadoop Configuration files

Command :cd hadoop/etc/hadoop

Step 15 : List out bin directory files in hadoop

command : cd hadoop/sbin

Step 16 : Edit Hadoop Configuration Files

Step 17 : Create log directory in hadoop

Step 18 : Edit hadoop-env.sh file

Code :

# The java implementation to use.

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-amd64

Step 18 : Create mapred-site.xml file & Edit mapred-site.xml

Code :

<configuration>

<property>

<name>mapred.job.tracker</name>

<value>localhost:54311</value>

<description>The host and port that the MapReduce job tracker runs

at. If “local”, then jobs are run in-process as a single map

and reduce task.

</description>

</property>

</configuration>

Step 20 :Edit yarn-site.xml

Code :

<configuration>

<property>

<name>yarn.nodemanager.aux-services</name>

<value>mapreduce_shuffle</value>

</property>

<property>

<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>

<value>org.apache.hadoop.mapred.ShuffleHandler</value>

</property>

</configuration>

Step 21 : edit hdfs site.xml before editing hdfs-site.xml

Create two empty directory for namenode and datanode

Step 22 : Edit hdfs-site.xml

Code :

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

<description>Default block replication.

The actual number of replications can be specified when the file is created.

The default is used if replication is not specified in create time.

</description>

</property>

<property>

<name>dfs.namenode.name.dir</name>

<value>file:/home/somu/namenode</value>

</property>

<property>

<name>dfs.datanode.data.dir</name>

<value>file:/home/somu/datanode</value>

</property>

</configuration>

Step 23 : Hadoop namenode format

Step 24 : Start-all .sh .. to start all deamons

command : ./start-all.sh

Step 25: to stop all deamonds

command : ./stop-all.sh

Step 26 : BROWSER UI

Namoenoe : localhost:50070

BIGDATA & HADOOP SOFTWARES

Download Link for Bigdata and Hadoop softwares list :
Download Hadoop 2.7.1 tar file

Download Hadoop 2.7.1 tar

Download Ubuntu OS

Download Ubuntu Os

Download eclipse

Download Eclipse Indigo
Download File Zilla

Download FileZilla
Cloudera 3 download Link

Download Cloudera 3

Cloudera 4

Download Clouder 4 Quick Start

Download Hadoop 1.2.0 tar

Download Hadoop 1.2.0
Download WinScp

Download Winscp Softwares
mysql Connector

Download FileZilla
Download Mysql software
https://drive.google.com/file/d/0B2-rlCGKD40NangwRGdLUXg2REE/edit