Huh, just created a job in Jenkins to run msbuild over some solutions. The job accepted a parameter called "Configuration" which was just some simple JSON.

Everytime I ran the job, msbuild failed.

Then I realised that the Debug/Release info for a .NET Solution gets put into an environment variable called "Configuration" that msbuild creates and relies on! >.<