cogito ergo vagus

February 15, 2008

Source-level parallel build in Visual Studio 2005

Filed under: software — vagus @ 10:28 am
Tags: , ,

I regularly work with Microsoft Visual Studio 2005 with SP1 on my Pentium 4 single-core non-hyperthreading based PC and occasionally run some builds on an Athlon 64 X2 dual core based PC. I noticed the compiler (cl.exe) is a single process with a single thread running, even on the dual core.

I did some digging around to see if I could enable parallel (multi processor) builds to take advantage of the extra idle core to speed up my builds and came across the following links:

Visual Studio 2005 tour and MSDN multiprocessor project builds

I was glad to find the /Mx  option where x is the maximum number of project builds you want to run in parallel. I dug around the codebase and realised the build script already had /M8 specified as a parameter to the call to vcbuild.exe. I even opened up the project in Visual Studio 2005 and the setting was at 2 (since this is a dual core)  as expected.

But the build still seemed to be running a single thread and not using the extra core :-(

I then re-read the links above more carefully and saw that the option is for parallel project builds so it would only work for multiple projects and not for many source files in a single project.

I did the customary STFW/JFGI and found the new /MP option which is a part of the new Visual Studio 2008 and this option enables source level multiprocessor parallel builds.

But what about VS 2005? I did some more searching and, lo and behold, found this blog entry which says the /MP  option is also available, but undocumented, in Visual Studio 2005!

Adding it to the build script’s parameter list for vcbuild.exe didn’t work, but adding it in the “additional options” window in the Visual Studio 2005 project worked and now doing a build would launch three processes of cl.exe, one which had 45 threads but took very little CPU cycles, and the other two were single threaded and utilised a CPU core each to the fullest! That was my Homer Simpson “woohoo” moment of the day :-)

Advertisements

9 Comments »

  1. Woohoo!

    Thanks…

    no,
    THANKS!

    Comment by lev — March 1, 2008 @ 6:36 pm | Reply

  2. I assume this is a C++ only option? I can’t seem to find any such thing for C#/VB projects. :\

    Comment by dan — March 25, 2008 @ 3:38 pm | Reply

  3. Excellent! Thanks.

    Comment by mfz — May 15, 2008 @ 4:46 pm | Reply

  4. Did you come across any problem with the /Mx and /MP options? We’ve experienced some trouble where the last couple of projects in the build don’t get built completely. It’s as if when the first thread completes, it shuts down the remaining ones as well. I’ve asked MS for info, but they haven’t gotten back to me, and the project we build is rather large, so I haven’t been able to setup a minimal repro case. Without /Mx and /MP it always builds correctly.

    Comment by Andreas Axelsson — June 25, 2008 @ 12:40 pm | Reply

  5. Did you come across any problem with the /Mx and /MP options? We’ve experienced some trouble where the last couple of projects in the build don’t get built completely. It’s as if when the first thread completes, it shuts down the remaining ones as well. I’ve asked MS for info, but they haven’t gotten back to me, and the project we build is rather large, so I haven’t been able to setup a minimal repro case. Without /Mx and /MP it always builds correctly.

    Comment by Andreas Axelsson — June 25, 2008 @ 12:42 pm | Reply

  6. I’ve not been using this regularly or for project builds, just did a few “medium sized” project builds with this option and it has worked. Well, by “worked” I mean the build succeeded and a few tests passed but nothing further than that :-)

    Comment by vagus — June 25, 2008 @ 1:03 pm | Reply

  7. […] However, my Win32 builds of our game engine have been only compiling on a single core until I found this posting today. Finally, the native Win32 build is faster than the PS3 build, which makes much more […]

    Pingback by Visual Studio 2005 source level parallel builds « Daily Game Development — July 1, 2008 @ 8:55 pm | Reply

  8. […] between price and performance. Multiple cores are useful for running VMs and compiling (apparently Visual Studio 2005 requires a tweak to use multiple cores for compilation). Quiet fans would be nice. I’m not interested in overclocking as I worry about the effect […]

    Pingback by Speccing my dream development PC « Successful Software — December 14, 2010 @ 6:04 pm | Reply

  9. thanks!
    i have an eight core cpu and the improvement is amazing!

    Comment by jazzprogramming@gmail.com — July 1, 2012 @ 1:08 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: