Skip to content

Latest commit

 

History

History
28 lines (21 loc) · 923 Bytes

README.md

File metadata and controls

28 lines (21 loc) · 923 Bytes
Raw

vagrant_hadoop

Introduction

Vagrant project to create a cluster of 4 virtual machines with Hadoop v2.4.1 and Spark v1.0.1.

  1. node1 : HDFS NameNode + Spark Master
  2. node2 : YARN ResourceManager + JobHistoryServer + ProxyServer
  3. node3 : HDFS DataNode + YARN NodeManager + Spark Slave
  4. node4 : HDFS DataNode + YARN NodeManager + Spark Slave

Requeriments

  1. VirtualBox
  2. Vagrant
  3. Git clone this project,
  4. Run vagrant up to create the VM.
  5. Run vagrant ssh to get into your VM.
  6. Run vagrant destroy when you want to destroy and get rid of the VM.

Web Interface ( Monitoring )

You can check the following URLs to monitor the Hadoop daemons.

  1. [NameNode] (http://10.211.55.101:50070/dfshealth.html)
  2. [ResourceManager] (http://10.211.55.102:8088/cluster)
  3. [JobHistory] (http://10.211.55.102:19888/jobhistory)
  4. [Spark] (http://10.211.55.101:8080)