Single node cluster

Hadoop Architecture:

(Its shared nothing disk architecture)

Tag Line Of Hadoop:

(Write Once Read More)

2 Hardware Components

  1. Master node
  2. Slave node

2 Layers:

  1. HDFS Layer(for storing data)
  2. Map Reduce Layer(For processing the data)

5 Components:

  1. Name node
  2. Secondary Name node
  3. Job Tracker
  4. Data Node
  5. Task Tracker

Hadoop Installation Steps;

Step1: Install VM Ware work software

Step2: Open the VM ware player

Step3: go to Hadoop software

Step4: Hadoop By sasi (Open vm ware player)

Step5: Login into root

Password: root

Step6: open F: /Hadoop /basicsw

Step7: Install winscp setup

Step8: Open winscp software

Step9: open root linex terminal

Step10: Type IP address: 192.168.83.128

Copy IP address and paste it in winscp

Hostname: 192.168.83.128

Username: root

Password: sasi

Login

Step11: copy file /Duag files from

F:/Hadoop /basicsw to /root / Documents

Copy software paste it /root/documents

1. jdk-6u39-linex-drag

2. vm player.50.8

3. winscp.exe

Step12: go to root [root ~ ]# cd documents

[root ~] # ls –ltu

Step13: you can see file name JDK-6U39-linex-1586-rpm

Step14: Its shows you red color means you need to give permissions to the file/software.

Step15: using chmod command for giving permission chmod 777 Jdk-6u39-linex.rpm

Step16: Execute the file using tar command

Tar –xzvf jdk-6u39-linex.bin

./Jdk-6u39-Linex-1589-linex.rpm

o/p: Java(TM) SE Development kit 6

successfully Installed

press Enter for continue

step17:

stop Java (Jdk)

Go to [root] # cat/proc/sys/net/ipv6/conf/all/disable-IPV6

Step18: Add users use this command

[#root]#user add hadoop

#password: hadoop

Step19: Login into user as a hadoop

Username: Hadoop

Password: hadoop

Set password means if your not using means you need to give three times password.

[hadoop@~] $ssh-keygen –t rsa –P” “

…………….. Generating public /private

[hadoop@~] /home/hadoop/.ssh/id_rsa

[hadoop@~] /home/hadoop/.ssh/id_rsa.pub

[hadoop@~] cd .ssh

[hadoop@~.ssh] $ ls –ltr

………………id-rsa.pub

[hadoop@ .ssh] cat id-rsa.pub> /home/hadoop/.ssh/authorized-keys

[hadoop@.ssh] $ ls_ltr

……………….id_rsa.pub

Authorized-keys

[hadoop@.ssh] $ chmod 777 authorized_keys

[hadoop@.shh] ssh localhost

(yes/no) = yes

Step20: Login to winscp

Hostname: 1192.168.83.128

Uname: Hadoop

Password: hadoop

Step21: copy Hadoop software

F:/Hadoop/home/hadoop

Step22: [hadoop@ localhost]$ ls –ltr

Step23: [h@~] $ chmod 777 hadoop-1.0.3-bin.tar.gz

Step24: [h@~]$ tar – xzvf hadoop-1.0.3-bin.tar.gz

  1. Bashrc file always will be in linex machine
  2. It set the shortcurt commands

Step25: copy 4 lines of hadoop settings files

Step26: view .bashrc.

Step27: $ source .bashrc (update the .bashrc)

Step28: [h@ L]$ cd $HADOOP_HOME

Step29: $[l @ L] cd conf

Step30: [l @ hadoop- 1.3.0/conf] ls –ltr

Step31: Display all setting files /Hadoop setting files

Mapred-site.xml

Hdfs-site.xml

Hadoop-env.sh

Core-site.xml

Step32: After finish all works on configuration files

[h @ L conf] $ cd…

[hadoop-1.0.3]$ bin/hadoop namenode –format

[hadoop-1.0.3] $ bin/start-all.sh

[hadoop-1.0.3] $ bin/stop.sh

[hadoop-1.0.3] $ jps

Hadoop configuration files and settings:

  1. Hadoop-env.sh:

Set hadoop-env.sh variables(javapath).

  1. Core-site.xml:

Set core-site.xml to configure HDFS and physical locations (metadata $ HDFS path)

  1. Hdfs-site.xml:

To set replication factors

  1. Mapred-site.xml:

Set run programs like job tracker.

Flow chart for installation hadoop:

 

 

  1. Hadoop/conf/$ gedit hadoop-env.sh

Copy export java path

  1. $ gedit core-site.xml

<configuration>

<property>

<name>hadoop.tmp.dir</name>

<value>/hpme/sajith/hadoop-test</value>

</property>

<property>

<name> fs.default-name</name>

<value>hdfs://localhost:54310</value>

</property>

</configuration>

 

  1. Mapred-dite.xml

<configuration>

<property>

<name> mapred.job.tracker</name>

<value>localhost:5431</value>

</property>

</configuration>

  1. Hdfs-site.xml

<configuration>

<property>

<name> dfs.rep</name>

<value>1</value>

</property>

</configuration>

  1. 1-sudo mode
  2. 3-cluster
  3. 8-maximum

Cd …<–

Hadoop-1.0.3 $ bin/hadoop namenode –format

Its is used for structured properly run correctly

[h @ hadoop] $ jps

[h @ h] bin/start-all.sh

[h @ h] jps

Namenode

Secondaryname node

Job tracker

Task tracker

Data node

[h @ h] bin/stop-all.sh

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s