Python execute subprocess.call inside map
Hi I have the following code:
import sys
import multiprocessing as mp
def create_parser():
from argparse import ArgumentParser, FileType
ap = ArgumentParser()
ap.add_argument('infile', type=FileType('r'),
help="file with shell commands to execute")
ap.add_argument('-n', '--ncpu', type=int, default=0,
help="Number of CPUs to use (default: %(default)s: all CPUs)")
return ap
def cpus_to_use(ncpu):
return ncpu if ncpu else mp.cpu_count()
if __name__ == '__main__':
from subprocess import call
ap = create_parser()
args = ap.parse_args(sys.argv[1:])
ncpu = cpus_to_use(args.ncpu)
if args.infile:
# Read commands from already open file and close
commands = [c for c in args.infile.read().split('n') if c]
args.infile.close()
# Create a pool and map run_cmd to the shell commands
pool = mp.Pool(processes=ncpu)
pool.map(call, commands)
I'm basically importing a text file from the command line that has for each line a specific command to execute (that I'm trying to parallelize). I'm using Python 2.7.12 and the output from print(command) looks fine.
I suspect there is an error in the last line line syntax as I'm getting: File "run_parallel.py", line 47, in pool.map(call, commands) File "/home/ect/anaconda2/lib/python2.7/multiprocessing/pool.py", line 251, in map return self.map_async(func, iterable, chunksize).get() File "/home/ect/anaconda2/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value
Thank you
commands = [c.split() for c in args.infile.read().split('n') if c]
(这是从http://sburns.org/2014/05/03/cortical-tractography-recipe.html,对吧?我只是有同样的问题:)它似乎工作分裂每个条目的命令在白色空间,以便池能够正确地解析每个函数调用的参数。
链接地址: http://www.djcxy.com/p/55292.html上一篇: python 3.x