Debugging asynchronous code

You have written a robust asynchronous program and tested it thoroughly, and it still has an error. How do you debug it? Like with testing, threads may run at different speeds, making it difficult to know the order in which things happen. Having a robust code with test in place should give you more confidence in the individual functions, even if there may be a problem with the coordination between them.

The standard Python Debugger, pdb, will work with threads. I use PyCharm as my integrated development environment. It makes debugging very easy, and I wouldn’t want to write any complex code without it. In debug mode, when you post your code, it shows you the stack trace for all running threads. I strongly recommend you use your IDE for debugging. It makes it so much easier. There are some standalone debuggers, such as gdb and Winpdb, which support threads. If you don’t use an IDE, it may be worth checking them out.

It can be difficult to know what has already happened by the time your program hits a bug. The more information you have, the better. Use the standard Python logging library to log relevant events. You can configure the logger to automatically include the thread name in the log messages using the format option. As you may have noticed, print statements are not thread safe. Print output from different threads can get mixed up. The logging module is thread safe. You will only see complete logging messages. They won’t be mixed up.

In Python, unhandled exceptions are printed out using cis.excepthook. You can create your own function for this, for instance to write the arrow details to a log file. However, this only covers the main thread. If you want to include unhandled exceptions from threads, use threading.excepthook. This was introduced in Python 3.8. Keep track of threads starting and stopping and of locks being acquired and released.

Deadlocks can be tricky to fix. Your code gets stuck, and you have no idea what is going on. To understand what is happening, create a background thread, which dumps the core stacks of the threads on a regular basis, say, every 5 or 10 seconds. It will show you what each thread is doing at the time or, in the case of a deadlock, where they are stuck. The dump traceback function in the fold handle library dumps the tracebacks of all threads into a file. This is part of the standard library. There are some sample scripts online, which you can find by searching for Python debug deadlock.