Saturday, February 21, 2015

Oozie Examples

Setting Up the Examples
The examples/ directory must be extracted from oozie and copied to the user HOME directory in HDFS:
cd /usr/local/oozie
tar -xvzf oozie-examples.tar.gz
chown -R hduser:hadoop examples/
hadoop fs -put /usr/local/oozie/examples/ /user/hduser/examples/
NOTE: If an examples directory already exists in HDFS, it must be deleted before copying it again. Otherwise files may not be copied.

Running the Examples
Add Oozie bin/ to the environment PATH in .bashrc or /etc/profiles
export OOZIE_HOME=/usr/local/oozie
export PATH=$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin:$PIG_HOME/bin:$SQOOP_HOME/bin:$OOZIE_HOME/bin

Execute job from Terminal (hduser)
oozie job -oozie http://localhost:11000/oozie -config /usr/local/oozie/examples/apps/map-reduce/job.properties -run

NOTE: The job.properties file needs to be a local file during submissions, and not a HDFS path. Modify job.properties for namenode & jobtracker url.

Check Oozie URL to track the oozie workflow job status : http://localhost:11000/oozie

Note : The example applications are under the $OOZIE_HOME/examples/app directory, one directory per example. The directory contains the application XML file (workflow, or worklfow and coordinator), the job.properties file to submit the job and any JAR files the example may need. Go through each workflow to run hive, java, sqoop, streaming examples and try

Reference : Oozie Example
Oozie workflow with Pig and Hive

No comments :

// Below script tag for SyntaxHighLighter