I have wrestled with a problem for days The problem was that I didn’t get coverage for all assemblies that were tested. No errors, no exceptions, no problem whatsoever in the builds and tests. But no matter what I did, the tests did not produce the expected coverage. NCover  and NUnit-console just spits out a coverage report with large holes in it.

As it turns out, all boils down to how NUnit treats the assemblies and the program database files (PDB) while testing. And avoiding shadow copies while executing tests on the build server.

Some background

A couple of weeks ago we made the move from CC.NET to TeamCity. At the same time we restructured the solution folders, having problem with dependencies on builds and that scripts took forever to finish. So to speed up the feedback loop for the day-to-day development we needed the new structure.

Setting up the new structure took about one day of work and creating the initial build and test loop for development another couple of days. This without any build scripts at all. The deployment took a bit longer and involved a few MSBuild scripts and it works well now. Also in this is a setup for executing selenium tests, this took some more scripting but finally it’s in place as well.

The problem

The final piece of script was the coverage. Unfortunately there are no integrated support for NCover in TeamCity so you have to revert to good (or bad) old MSBuild scripting. As there a few long running tests in this soup we opted for running coverage at nightly builds. The dev coverage can be executed on the developer machine so there's no problem with this.

So after setting up a converge build for each of the solutions I found that it never produced the same result for all solutions. With no apparent problem, assemblies was omitted from the result. And sometimes it worked as expected. After more analysis it turns out that when one solution has been built it affects the coverage of other builds. More than was explained by the interface points between the solutions.

It got even worse, I did a build for project A, checked the coverage for A, then built project B and the result for project A changed. Checking the files used for instrumenting project A I could not see one single difference down to the timestamp and version for each and every file of the A build.

The solution

Finally I sat down with FileMon from SysInternals and did a check on one specific assembly used in the test to get a grip on what was tested and where it came from. Hmm, what is all this shadow copy used by NUnit? What happens here?

Well it turns out that NUnit copies assemblies to a temporary folder while executing the unit tests, this to allow the user to continue develop while the tests are executed. Well, this is not the problem on the build server. In the coverage run that I created the assemblies built are copied to an analysis folder and the tests are executed there. And the simple and working solution is to add the /noshadow option to the NUnit command. After checking with FileMon and the resulting coverage I finally got this last nagging problem solved.

After the various coverage results are merged with coverage explorer, it is now published with the working builds on our demo server. Automatic build configuration finally done. And less copying of files while testing might even shave a few seconds of build times.

By the way, now that I know exactly the problem and the solution, it was easy enough to find a blog post concerning NCover problems, that explains it all.

// Håkan Reis

A college of mine recently added a WPF showcase on Codeplex. We all need inspiration and this is a great place  to start. Here is not just a showcase of what you can do with WPF, there are panel, custom controls and shader effects. The collection is created along the way to try out various intrinsic of WPF as well as educational. Some has in one form or another made it into working applications.

So go grab it and start experimenting

// Haqwin

This is a rant. As much as apple has had its focus mainly on the user they are now heading in a new direction. The new iTunes addiction is sales, the new incarnation of iTunes is only about how to get the next fix, closing the next deal.

The sad thing is that as with many other companies that has reach world domination - it cripples innovation. There hasn’t been much innovation in the user interface since it was introduced years ago. The few is focused on one thing, selling stuff.

To start with, what is iTunes?

Well this is one problem, it has become a beast, to the user it is foremost a media player, or at least tries to be. It’s not among the better media players but it gets the job done. As a media player there has been added a few extra features like burning CDs, music sync to iPod and oh, it let’s you buy new music.

But later on they have added more and more bloat; iPhone calendar and contact sync, application sales, pushing media and stuff you already own, podcast, video downloads. Oh, and the updater doubles as a portal for pushing safari as well.

The iTunes beast of today

iPhone sync - Why on earth do I want my iPhone to start up a (now) bloated media player just to sync contacts, calendar details and some music? Why not a blazing fast slimmed down sync application that use the database from iTunes to get the playlists, media track and application that are stored there?

The pusher – For a long time now iTunes ha added a little arrow to each and every track in the library, clicking it took me to iTunes music store, so that I could easily - buy the track I already own. Not a big problem as you could just remove it with a little setting. But not anymore, you can solve it with a hack but how many user are able to do that? We are going to sell more whether you like it or not! And it gets worse.

The genius - Masquerading as smart playlist manager genius gives Apple a way to display even more buttons with the word BUY in them. Now they have taken up 10% more of my screen real-estate with lists of music to buy. The playlists are not that bad, but I guess using genres, artists and some randomness would create about the same result. 

Grid view – The latest “innovation” is the ability to show albums in a grid…wow. And the cover flow before that, that turns out to be, not that useful. I love Eye candy and think it can serves a purpose but adding a black blob with a reflection in the middle of the database grid that represent all your tracks doesn’t help much.

What they could have added

Folder watching – I tend to move my music around, as there are not enough space on my laptop to have all my music and there are no easy way to have a central place for all tracks I have a subset on my machine, about 10GiB of music. So why not just add some smart folder watching? Thankfully there are people out there that get it.

Re-sync of statistics – If I play my music on my iPhone it would be so easy to count and then sync that back, as well as what tunes I skip etc. that would be excellent statistics for that genius playlists…

Native OS support – I’m a windows vista user, I actually like vista, and applications should behave. This is not a new problem with iTunes. But when are you going to address it? Not in one single place have they added right click support. You know, we use it a lot in windows. Or even looked at the Visa UX guidelines. But then again, why would they add functionality to us windows user that would be missing for the Mac OS crowd?

There; now I can get back to syncing photos, buying ringtones clips of music I already own and let iTunes steal GDI and memory resources from my system.

// Haqwin

I have said it before and I will continue to push my theory that the next big problem area will be in configurations. The environment variables are an old habit, used extensively by the java framework but it also makes ripples in the Microsoft build environment.

I just spent some time introducing TeamCity continuous integration and build system to a company. It’s an excellent product from JetBrains that I can truly recommend. Comparing Cruise Control with TeamCity is like comparing notepad with Visual Studio. All can be done but not without inflicting pain.

Normally, setting up a new project build and test is a child’s play in TeamCity, but not this time. Apart from a few hurdles regarding subversion access and some cumbersome handling of references in the solutions there was a more evil and nasty problem.

At each build we were getting a MSB4126: “Release|BWS” configuration invalid. I searched sifted through tons of config files, project and solutions with no luck. Finally I fond the problem, it was in the environment variable Platform. For some reason this variable was set to BWS. I still have no explanation on to what BWS means (other than Beer Whine and Spirituous) and why it was set. I removed it and the system runs without a problem. After that little fix the builds executed like a charm.

The root of this problem is actually that MSBuild tries to be smart, looking at the Platform environment variable and adding what’s in it to the build. But why should it try this, what other configuration sources should I be aware of that can inflict strange behavior on my set-up? And in the name of flexibility and choice we add to this mess of configuration, every part of the system should be configurable, its always in one of the constricts of every project to day. Never hard code, always provide flexibility.

The problem is that leads paralysis as in Paradox of choice, and the way out of this is to step back, look around and start setting up some boundaries for choice. That we offer flexibility with a great sense of responsibility, and stick to a well established default behavior.