When Guido van Rossum mentioned that one of our Google Highly Open Participation Contest™ (GHOP) students, Benjamin Peterson, was interested in doing a more extensive project on Python testing, we were happy to help. Benjamin Peterson recently finished his work on testing the CPython core and was kind enough to send us this report.
Benjamin writes:
With sponsorship from Google, I've just spent the summer working on the tests for the CPython core.
I spent a lot of time working on the test driver, regrtest.py, in order to make implementing features easier in the future. One of the ugliest parts of regrtest.py is that it kept a list of all the tests that could be expected to be skipped on a platform, so I moved this logic to the tests themselves. In the end, I wasn't able to fully remove Python core related logic from it, but the removal of expected skip lists simplified the program greatly.
I made command line use of regrtest easier in several ways. When specifying tests, one no longer needs to give the test_ prefix. I also implemented test selectors which allow a range of tests to be specified. For these new features, I added unittests. Another major feature is that regrtest automatically writes a list of the tests run, with failures and skips noted, after each testing session. The session can be rerun exactly the same way on another computer by reading the file with the -f option.
Finely grained test skipping is another important feature. New decorators in test_support allow whole test cases and individual methods to be skipped. Decorators range from the generic skip_if, which skips when the condition passed to it is true, to skip_on_platform, which skips if any of the listed platforms are in sys.platform. I hope that some form of these decorators can be included in unittest eventually.
I visited each of the 361 tests to upgrade it to use my new features. I was able to consolidate some tests with each other. A few tests that escaped earlier sprints were converted to unittest.
I really had a fantastic experience this summer. I learned an incredible amount from nice, patient people and look forward to next year! Many thanks to Georg Brandl, my awesome mentor, Guido van Rossum and the rest of the Python core development team, and Google's Open Source Programs Office for making this possible.
And in other news Python and GHOP, Zachary Voase, another Python GHOP student, recently attended Campus Party in Valencia, Spain, and spoke about his experiences in the contest. You can check out the video of Zack, along with some of our Google Summer of Code™ students, on YouTube. (Campus Party site and video are both in Spanish.)
