Running Hadoop as a Batch Job


Teaching: 0 min
Exercises: 0 min
  • Do we have to be in interactive mode?

  • Know how to use the template to develop Hadoop-based batch jobs

Integrating Hadoop job into Palmetto workflow. You need to be on login001.

$ cd
$ cat -n ~/myhadoop/codes/movieAnalyzer.pbs
$ qsub ~/myhadoop/codes/movieAnalyzer.pbs
$ qstat -anu $USER

Key Points

  • You can deploy Hadoop as part of your workflow.