PyConFr 2014

SCons (www.scons.org) is a build system, entirely written in Python. It offers cross-platform support for languages like C/C++, D and Fortran, and uses content-based change detection (MD5) over the traditional timestamp approach. Despite all these obvious advantages over the classic "make", many people reject SCons as a build system because it is said to get very slow in large projects (10000+ files). That's why the SCons team stepped up and closely investigated some of the benchmarks on the internet that helped to spread the bad rumors. The results we found by very intensive and large-array profilings, boil down to the following point: It's actually not SCons that is slow, but the underlying process spawning mechanism (fork) that's causing the problems. The major slowdown of the running time can affect any other application (Python, or even a plain C program) that spawns single processes sequentially, while the memory usage of the parent process grows.

Against this unwanted effect, we developed a simple wrapper which redirects the standard calls of the subprocess module to posix_spawn() directly. It's going to be integrated to our source base in the upcoming release (Posix systems only), and will reduce the runtime for clean builds by a significant amount.

With this presentation we'd like to make our results and findings publicly available to a broader audience. We'd also like to get in touch with users and projects that experienced similar problems and discuss our and their solutions with them.

Voir le support de présentation