Cloudera Manager Administration

Question

I newbie to Cloudera Hadoop,i want to learn cloudera hadoop administration myself,i am having 12 GB ram installed on windows laptop, for virtual machine around 9 GB available to use, do anyone know how to install and use Cloudera manager in virtual environment to practice cloudera administration.is there any way to work on cloudera manager with only 8 gb ram available?

Answer 1

So basically first u need to install a Virtual machine with OS i generally prefer CentOS but check for the supported OS by cloudera before installing it and for creating VM you can follow links from internet here is one

http://teaching.idallen.com/cst8207/14f/notes/000_centos_virtualbox_install.html

Then once VM is up and you have internet connectivity on it configure the preriqusites for cloudera mention over here

http://www.cloudera.com/documentation/enterprise/5-4-x/topics/cm_ig_cm_requirements.html

You will have two option manual or automated by cloudera manager proceed with your preferred choice.

http://www.cloudera.com/documentation/enterprise/5-4-x/topics/installation_installation.html

I hope this will help you and others.

Answer 2

You could download cloudera quickstart. Its a vm with many components of cloudera, and requirements is 4gb of ram.

Link:

http://www.cloudera.com/documentation/enterprise/5-3-x/topics/cloudera_quickstart_vm.html

Answer 3

You can use Vagrant which is a command line utility for managing the lifecycle of virtual machines.

In particular you can follow these instructions: http://blog.cloudera.com/blog/2014/06/how-to-install-a-virtual-apache-hadoop-cluster-with-vagrant-and-cloudera-manager/

Since you don't have a lot of RAM available if you choose to do so, you have to modify the project Vagrantfile to fit your RAM requirements and you may not be able to install all the cloudera services on you nodes.

The default project will spin a cluster of 4 virtual machines; one master node with 4GB of RAM and 3 slaves with 2GB of RAM each.

You can delete one slave from the Vagrantfile, e.g. remove slave3 by deleting the following lines:

config.vm.define :slave3 do |slave3|

    slave3.vm.box = "precise64"

    slave3.vm.provider :virtualbox do |v|
       v.name = "vm-cluster-node4"
       v.customize ["modifyvm", :id, "--memory", "2048"]
    end

    slave3.vm.network :private_network, ip: "10.211.55.103"
    slave3.vm.hostname = "vm-cluster-node4"
    slave3.vm.provision :shell, :inline => $hosts_script
    slave3.vm.provision :hostmanager

end

So you will have a cluster of 3 nodes; one master with 4GB of RAM ans 2 slaves with 2GB of RAM.

Answer 4

You can install

• Install CentOS 6.x or 7.2 from the Centos DVD and choosing "Minimal Server" or "Minimal Workstation" depending if you want a graphical interface
• Install Cloudera Manager from scratch using the Cloudera Documentation, selecting Path B with external PostgreSQL. It's more representative of the administration tasks than Path A. Path C is just a nightmare to begin with CM.

With 9GB RAM, your cluster can run a little slowly if you had an eclipse instance to develop some proof of concept, but it's acceptable. When your cluster is up, you can stop Cloudera Manager and use Hadoop services as well.