Every now and then I need to profile parts of my code (as we all do). For me this used to happen with a frequency low enough to prevent remembering what one does with the output of the profiler, i.e. how to display results in an interpretable way. So every time I had to measure code performance I googled the cProfile module and looked up the examples on how to display, sort and interpret results. This was a minor annoyance.

Fortunately, it is no more thanks to the snakeviz package. All you have to do in order to use it is run

pip install snakeviz

inside your virtualenv. Then proceed as usual – in your code import cProfile and replace direct call to the main() function (or any other function you wish to use as starting point for profiling) with

cProfile.run('main()', "path_to_stats.prof_file")

After your program terminates just point snakeviz to the profiler output file

snakeviz path_to_stats.prof_file

This should pop up a browser window with a nice and meaningful visualization of results. And that’s it – you got your results presented in a clean and ready to consume way.

In order to show snakeviz in action I have created a very simple mock-up script:

import time
import cProfile

def read_data():
    time.sleep(1.5)

def clean_data():
    time.sleep(2.7)

def fit():
    time.sleep(4.2)

def build_model():
    clean_data()
    fit()

def main():
    read_data()
    build_model()

if __name__ == "__main__":
    cProfile.run('main()', "stats.prof")

Results are shown below. After launching snakeviz you can select one of two display styles – Sunburst or Icicle. The later one seems to be more informative and it’s shown in the image below (click on image for a full-size version):

Visualizing cProfile results with snakeviz.

Snakeviz provides two ways to explore profiler data. You can choose the sorting criterion in the output table (e.g. the number of given function calls or cumulative time) or select a subset of the output graph to display.

If you work iteratively (i.e. “change code-rerun the profiler” multiple times), snakeviz will also nicely fit this pattern. In order to display updated results, you don’t have to go through the command line and restart it. Assuming the profiler output went to the same file, just go to the browser, hit refresh (i.e. F5 in most browsers) and voila – a new set of results appears.

Snakeviz is a very well thought tool. Not overdone, with a “just right” set of features for the job it aims to do.

Final notes:

  • For more info on snakeviz go visit the project homepage.
  • Another notable tool for the job is RunSnakeRun. Unfortunately, it seems to be no longer maintained (never actually tried it, so your mileage may vary).
  • Sometime cProfle is not enough, i.e. you may need more detailed information. The line_profiler package is another great utility module, allowing to see a line by line execution time of the profiled function. I’m going to cover this module on the blog soon.

Multiprocessing and exceptions - some batteries not included Pickle performance bottlenecks when using multiprocessing

Leave a Reply

Your email address will not be published.