Friday, September 27, 2013

CDH - Hadoop - Hbase - Adding a host to an existing cluster

1. Install CentOS version matches to the existing hadoop clusters. 
2. Configure static Ip, DNS, gateway and hostnames.
3. Enable ssh and disable firewall. 
4. In /etc/selinux/config file disable the SELINUX
5. In /etc/hosts file add the distributed cluster machines
6. In /etc/fstab hadoop partition add noatime with default on ext4 http://www.howtoforge.com/reducing-disk-io-by-mounting-partitions-with-noatime)
7. Install "yum install ntp" and start the service and add the service to chkconfig ntpd on. Sync the time with server time pool "ntpdate pool.ntp.org"
8. Login into cloudera Click 'Hosts'
9. Add Hosts
10. Type the hostname and search
11. select and click 'Install CDH on machine' 
12. Before you select which version to install check in master CDH version and do the same. (go to master and type hbase shell). In hbase start it will show hbase and CDH version.
13. Copy the .bashrc file settings from other server to the new server
14. Go to each and every service (TaskTracker, DataNodes and RegionServer and add the new server). Make sure Master is not checked while adding regionServer
15. Copy the library of (hadoop_lib_jars from svn into $HADOOP_HOME/lib) and restart MapReduce in CDH.
16. Restart client like azkaban once you modified the zookeeper quorum adding this host cluster

No comments :

// Below script tag for SyntaxHighLighter