SMP launcher

This script will manage a queue jobs list on a SMP (or not, it’s up to you) computer. Simple::Proc module has to be installed on your computer. You can run smp_launcher.pl with different options:

Usage: /Users/lecerf/bin/smp_launcher.pl [options]
Options:
	verbose: -v, verbose (optionnal)
	data   : -d [data], file list of data file
	file   : -f [file], command file
	repeat : -r [nb], repeat analysis X times
	cpu    : -c [nb], number of cpu
	unique : -u, launch unique analysis from command file

You must define a command file, which contains the command line to run and you must specify the number of simultaneous processes that you wish to run (i.e. the number of cpu). For all the jobs, the STDERR and the STDOUT will be redirect to a file (job_XX.stderr and job_XX.stdout, where XX is the job number). Then, you can choose between the 3 analysis mode :

  • many different analysis: the different commands have to be written in the command file:
    command1
    command2
    …
    commandn

    Then, you can run SMP launcher as follows:

    smp_launcher.pl –f commandfile –u –c 4
  • same analysis with different data files: The unique command has still to be written in the command file:
    /usr/bin/soft –various_options –-input

    Note that the input file option must be the last.
    The filenames of datafile have to be written in another file (data_list, for instance), one datafilename per line. Then, run smp_launcher as follows:

    smp_launcher.pl –f commandfile –d data_list –c 4
  • same analysis many times: in this case, you want to run for simulation purpose (for instance) the same program many times (the variations are directly implemented in the program). You just have to write in the command file the command line to run your program:
    /usr/bin/soft –various_options –-input datafile

    Then, to run smp_launcher 100 times, you just have to type the following command:

    smp_launcher.pl –f commandfile –r 100 –c 4

Typically, you would have that sort of output screen:

smp_launcher.pl -u -f command_unique -c 8
----------------------------------------------------------------
smp_launcher.pl 1.0 (2010)
----------------------------------------------------------------
14 command(s) loaded from file
setting up jobs in UNIQUE mode...
  - preparing 14 job processes

Launching job processes in background...
  ++ job No. 1 launched (pid 23804)
  ++ job No. 2 launched (pid 23805)
  ++ job No. 3 launched (pid 23806)
  ++ job No. 4 launched (pid 23807)
  ++ job No. 5 launched (pid 23808)
  ++ job No. 6 launched (pid 23809)
  ++ job No. 7 launched (pid 23810)
  ++ job No. 8 launched (pid 23811)
  -- job No 2 terminated (pid=23805)
  -- job No 4 terminated (pid=23807)
  ++ job No. 9 launched (pid 23812)
  -- job No 6 terminated (pid=23809)
  -- job No 3 terminated (pid=23806)
  -- job No 8 terminated (pid=23811)
  -- job No 1 terminated (pid=23804)
  -- job No 5 terminated (pid=23808)
  ++ job No. 10 launched (pid 23813)
  -- job No 7 terminated (pid=23810)
  ++ job No. 11 launched (pid 23814)
  -- job No 10 terminated (pid=23813)
  ++ job No. 12 launched (pid 23815)
  -- job No 11 terminated (pid=23814)
  ++ job No. 13 launched (pid 23816)
  ++ job No. 14 launched (pid 23817)
  -- job No 9 terminated (pid=23812)
  -- job No 12 terminated (pid=23815)
  -- job No 13 terminated (pid=23816)
  -- job No 14 terminated (pid=23817)

ALL jobs performed.
--
Execution time: 0.05 seconds CPU user-time

I hope this script will be useful to you (it was for me ;-) !). Feel free to add comments to this article. I’m sure there is some others smarter ways to manage that kind of multitasking job control.

source code of smp_launcher.pl

Submit a Comment

Spam Protection by WP-SpamFree