python debug tools for multiprocessing

I have a python script that works with threads, processes, and connections to a database. When I run my script, python crashes.

I cannot explicitly detect the case in which this happens.

Now I am looking for tools to get more information when python crashes, or a viewer to see all my created processes/connections.

---

I created a module RemoteException.py that shows the full traceback of a exception in a process. Python2. Download it and add this to your code:

import RemoteException

@RemoteException.showError
def go():
    raise Exception('Error!')

if __name__ == '__main__':
    import multiprocessing
    p = multiprocessing.Pool(processes = 1)
    r = p.apply(go) # full traceback is shown here

OLD ANSWER

I had the problem, too.

this is what i did... a RemoteException to debug multiprocessing calls

RemoteException.py

copy the source and remove line 19: file.write('nin %s ' % (Process.thisProcess,)) and the line import Process

The problem is: multiprocessing only transfers the exception but looses the traceback. The code below creates an Exception object that saves the traceback. And prints it in the calling process.

In your script you can do something like this:

import RemoteException

def f():
    try:
        # here is code that fails but you do know not where
        pass
    except:
        ty, err, tb = RemoteException.exc_info() # like sys.exc_info but with better message
        raise ty, err, tb

# here follows your multiprocessing call to f

---

I don't know your case, but if you use threads or multiprocessing then your code is applicable for parallel processing (usually). In difficult cases I do everything just calling a function without pool, catch error and then go to pools again.

---

There is a graphical debugger called WinPDB which gives you the option to follow a particular Python process. You can step through and see all the variables at different locations in the call stack.

It gives you the option of which process to follow at fork.

It will even capture the final exception and allow you to see where it originated.

http://winpdb.org/

链接地址: http://www.djcxy.com/p/66482.html

上一篇: Android 4.x中的SSL客户端身份验证

下一篇: 用于多处理的python调试工具