Running Hadoop as a Batch Job

Overview

Teaching: 0 min
Exercises: 0 min
Questions
  • Do we have to be in interactive mode?

Objectives
  • Know how to use the template to develop Hadoop-based batch jobs

Integrating Hadoop job into Palmetto workflow. You need to be on login001.

$ cd
$ cat -n ~/myhadoop/codes/movieAnalyzer.pbs
$ qsub ~/myhadoop/codes/movieAnalyzer.pbs
$ qstat -anu $USER
$ 

Key Points

  • You can deploy Hadoop as part of your workflow.