Deleting Hosts

You can stop Cloudera Manager from managing a host and the Hadoop daemons on the host.

First, make sure there are no roles running on the Host; you can decommission the host to ensure all roles are stopped.

Second, you must stop the Cloudera Manager Agent from running on the host; if you don't stop the Agent, it will send heartbeats to the Cloudera Manager Server and show up again in the list of hosts.

To delete a host:

1. Decommission the host to ensure all roles on the host have been stopped. For instruction, see Decommissioning a Host.
2. Stop the Agent on the host. For instructions, see Stopping and Restarting Cloudera Manager Agents.
3. In the Cloudera Manager Admin Console, click the Hosts tab.
4. Select the host in the Hosts tab.
5. From the Actions for Selected menu, select Delete.

Reference : http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Free/4.5.1/Cloudera-Manager-Free-Edition-User-Guide/cmfeug_topic_6_7.html

Note : Atleast 1 hbase master should run while decommission  So, the region data will be swap to another region server. So, make sure hbase and other services running. Cloudera will take care of adding into excluding list safely

Also make sure the agents on cloudera machines were stopped else the host will be displayed automatically on cloudera

GitHub - Source Code Control

Why GitHub ? 

Github is basically a version control system same as SVN (SubVersion) or CVS to maintain your code in the web UI. Very popular openSource component to keep our code. It has 2 feature Public or Private. Public projects will be viewed by anyone. You will find most of the openSource code published in GitHub. You can integrate gitHub easily with standard editors "Eclipse, Netbeans, IntelliJ.."

Latest Eclipse (Luno) release comes with GitHub plugin as in-build. You can checkout your source and check-in easily.

GitHub in Command Line In Interface on Ubuntu

Install Git in Ubuntu

$ sudo apt-get install git

How to Checkout from Git?

$ git clone https://github.com//.git

How to Check status on your directory with git master ?

$ cd ~/
$ git status -u origin master

Pull your changes from GitHub

$ cd ~/
$ git pull -u origin master

Push your changes into GitHub

$ git add 
$ git commit -m "Adding  into Git"
$ git push -u origin master

Add your existing project into Git

Login to your gitHub in web UI

Create a Repository in your github UI 

Push project from Terminal into your created Git Repository

$ cd ~/
$ git init
$ git add 
$ git commit -m "Adding projects into git" .
$ git remote add origin https://github.com/username/projectName.git
$ git pull -u origin master 
$ git push -u origin master

Check out GitHub project repository into Eclipse

• Open Eclipse with workspace
• Go to Window > Open Perspective > Other > Git > Ok
• Right click and select 'Paste Respository Path or URI'
• Copy and paste the Git URI from Git Web UI > HTTPS Clone URL
• Move on with Next and finish to checkout repository into your workspace
• Select Git Repository > Right Click 'Import Project' > Select your project into working directory > Finish

Convert Regular checkout project into Java Project in Eclipse

• Select your project > right click > Properties > Project Facets > Click 'Convert to faceted form..'
• Select 'Java' and 'Version' change in dropdown if you have multiple version
• Click 'Apply' > Ok

Create sourceFolder (src/main/java) for your project in Eclipse

• Select your project > right Click > Source Folder > FolderName 'src/main/java' > Finish 
• Select your project > right Click > Source Folder > FolderName 'src/main/test' > Finish 

Push your changes into Git Repository through Eclipse

• Open Eclipse with Workspace had checkout project from git. 
• Select your project > right Click > Team > Add To Index (Need to add first)
• Select your project > right Click > Team > Commit > provide commit message > commit and push > ok

Pull your changes into workspace from Git Repository through Eclipse 

• Open Eclipse with Workspace had checkout project from git.  
• Select your project > right Click > Team > Pull

Note : You can do synchronize with Repository through Team > Synchronize Workspace to sync and verify the pull or push changes before you do.

Try : Familiar yourself with Git in 5 mins

How to get remote ip address when data get post through LB (Load Balancer)

In tomcat under conf/server.xml. Update below code

           prefix="localhost_access_log." suffix=".txt"
           pattern="%{X-Forwarded-For}i %h %l %u %t "%r" %s %b"    
resolveHosts="false" />

         

Java Code

The following Java code extracts the originating IP address of an HttpServletRequest object.

public final class HTTPUtils {

    private static final String HEADER_X_FORWARDED_FOR =

        "X-FORWARDED-FOR";

    public static String remoteAddr(HttpServletRequest request) {

        String remoteAddr = request.getRemoteAddr();

        String x;

        if ((x = request.getHeader(HEADER_X_FORWARDED_FOR)) != null) {

            remoteAddr = x;

            int idx = remoteAddr.indexOf(',');

            if (idx > -1) {

                remoteAddr = remoteAddr.substring(0, idx);

            }

        }

        return remoteAddr;

    }

}

Squirrel - Java SQL Client

Why Squirrel ?

SQuirreL SQL is an open-source Java SQL Client program for any JDBC compliant database. Easy configurable by loading jars and can connect to various SQL Servers (MSSQL, MySQL, Sybase...)

You can learn more about Squirrel.

Explore Squirrel

1) Download squirrel-sql-3.4.0-install.jar

2) Run / Execute the jar from console : java -jar squirrel-sql-3.4.0-install.jar

3) Squirrel will get install in your desktop as squirrel plugin under $HOME directory. You  have uninstall script under the same directory to do so.

4) Open the Squirrel client go to Drivers and select the driver, Click modify and load the (Jars for Mysql or MsSQL)  under External class path. 

For MySQL :
mysql-connector.jar - class name : com.mysql.jdbc.Driver

For SQL Server 
sqljdbc4.jar - class name : com.microsoft.sqlserver.jdbc.SQLServerDriver

5) Go to alias and add your 'new alias' with and show your driver. You will be connected to the server and enjoy playing with your squirrel client on connecting to different database servers.

How to change your browser User Agent ?

user Agent switcher for chrome

What's user agent ?

Your browser sends its user agent to every website you connect to. Based on the browser and platform some websites serves the content. Always user agent information will send to web servers to notify.

http://www.howtogeek.com/114937/htg-explains-whats-a-browser-user-agent/

Cloudera Manager (SCM - Service and Configuration Management for BigData)

How to change the ips in cloudera manager

1) stop all the cloudera manager agents in all the clusters
2) stop the cloudera manager server and server db service in the scm server (namenode server)
3) Modify /etc/cloudera-scm-agent/config.ini to point to the new scm server ip (namenode server ip)
4) In /etc/hosts file add your new ip address and hostnames
5) Start all the services 1 by 1 from server-db, server, agent and you can see the ips got changed in the Host on cloudera

Adding a new role instances to your cloudera manager or adding a new host into your existing cluster.

https://ccp.cloudera.com/display/ENT/Adding+Role+Instances

Tools to be installed in windows

MobaxTerm - MobaxTerm is the excellent tool to connect to linux as SSH.