In the first part I wrote about how to get the version number that is available in TeamCity into the assemblies that are built and also how to maintain the dynamic unit testing feedback. I will now continue to explain how to get the complete chain working up until naming the finished package.

Propagating the version number

I left out one bit of MSBuild script that is vital for the next step. How can you transfer a version number from one project to another? Well, it’s actually quite simple, just save it as a text file among your artifacts and you can read it back up in a property in your MSBuild files. This is what it looks like in the MSBuild script:

<WriteLinesToFile File="SoftwareVersion.txt" 
                  Lines="$(BUILD_NUMBER)" 
                  Overwrite="true" />

This gives you a file named SoftwareVersion.txt that you can save as an artifact and use in the following build projects, how simple some solutions are.

File renaming and creating a package

The good thing about about MSBuild is all the extra things around just executing builds, in combination to the WriteLinesToFile there is of course a ReadLinesFromFile, and as you guessed this will read lines from files. These lines can be stored in an item collection (for many lines) or a property (that’s what we are after) like this:

<ReadLinesFromFile File="$(ReleaseDir)\SoftwareVersion.txt">
  <Output TaskParameter="Lines" 
          PropertyName="Version" />
</ReadLinesFromFile>

Now we have what we need to get the version that was built in one project to control the name of a file in another project. The renaming part is easy.Actually there are no rename task but you can of course execute a command directly with the Exec task so renaming will look like this:

<Exec Command="rename $(ReleaseDir)\SoftwareServiceSetup.msi
               SoftwareServiceSetup-$(Version).msi" />

Transferring TeamCity versions to Subversion tags

Last is to get the labels created back into subversion (as it is that I’m currently working with, it is what I know well). It is easy to just add the labels in TeamCity. The settings are placed under Version control settings in Build Configuration step two.

However, there might be a mix-up on where you want to save your tags, this is set under the VCS root configuration. The default is set to trunk=>tags but this hasn’t worked for the setup I have. Instead I have to set this to /trunk=>/tags, and remember, don’t checkout the trunk locally it will be a big checkout…

So there you have it. Version numbers that follows every build from development, through testing and ending up on the subversion labels and the package names. Traceability that will make a configuration manager cry with joy.

// Håkan Reis

Last time I posted on MSBuild and TeamCity I showed how you could use it to deploy remotely in Deploy with MSBuild and custom targets Part 1 and Part 2. There are one issue that’s always a part of professional projects - version handling. I will show you a way to let the version get into the assemblies and trickle through all build steps all the way to the production package. In the first part I will concentrate on getting the version number into the assemblies while keeping unit test feedback, where currently running tests are directly visible in TeamCity.

Setting the version in TeamCity

First off is to set the version and build number in TeamCity. This is one of the easiest parts. Just go into the General settings for the project and set up the version number format and the build counter setting. like this:

Now we have something to base our versioning on, the first part 1.0.2. are static and you update that manually as you start a new release, etc. The last {0} will give you a build number, you get the idea.

Getting the version number to the assemblies

Next step is to make sure the version number from TeamCity is used to build the assemblies and added instead of the Assembly, File and Product Version set in the visual studio project properties. But first some background.

When the assemblies are built version values are automatically set from AssemblyInfo.cs files. If nothing is set, it will get the value of 0.0.0.0, file version will get the same version as in Assembly Version and application version will follow the file version. More about this in Assembly version, File version, Product version. The basics is that if we want to control the version we must make sure the AssemblyVersion attribute  is in there. I would suggest setting it to 0.0.0.0 to make it easy to identify if anything fails to update. You could even have safeguards for it in code, aborting execution if the version is zeroed.

Now you have to make sure you change this number after the checkout but before the build has started. This calls for a shift from solution building to using MSBuild. But for the basic solution to build you only need one line of MSBuild script:

<MSBuild Projects="Software.sln" 
  Targets="Rebuild" 
  Properties="Configuration=Release;" /> 

That’s easy enough. Moving on to the version, we first collect the AssemblyInfo.cs files in the solution and then rewrite the version number with the TeamCity number. As it happens, there are a build property available containing the TeamCity number: $(BUILD_NUMBER). To get the build number into the assembly info we can use a task called FileUpdate that is part of a collection of open source MSBuild tasks, it will look like this:

<!— Collect the Assembly info files in the solution –>
<ItemGroup>
  <AssemblyInfoFiles 
    Include="**/Properties/**/AssemblyInfo.cs;" />
</ItemGroup> 
 
<!—Identify current AssemblyVersion and replace -–>
<FileUpdate Files="@(AssemblyInfoFiles)" 
  Regex=’AssemblyVersion\(“.*”\)\]’
  ReplacementText=’AssemblyVersion(“$(BUILD_NUMBER)”)]’ />

This will get the desired result, with all assemblies carrying the correct version number. However, it will also give as a new problem; we now loose the unit test feedback in TeamCity, but we can solve that in the next step.

Unit testing with feedback

When you move from solution based execution to MSBuild based, the first issue you come across is that the unit tests are missing. The natural step is to make the MSBuild script execute them. Normally you would call nunit-console or maybe the NUnit task. But that will only give you a file to display as an artifact. Not the direct and continuous feedback you get from TeamCity. Fortunately TeamCity will provide a unit test runner task that can be used instead. The following lines need to be in the MSBuild script:

<!-- Integreate with TC's nunit runner –>
<UsingTask TaskName="NUnit" 
  AssemblyFile="$(teamcity_dotnet_nunitlauncher_msbuild_task)" />
 
<!-- Select the assemblies to test –>
<ItemGroup>
  <TestAssemblies Include="**/release/*UnitTest*.dll;" 
    Exclude="**\obj\**;" />
</ItemGroup>
 
<!-- Execute the tests –>
<NUnitTeamCity Assemblies="@(TestAssemblies)" />

Read more about the NUnitTeamCity task on the TeamCity wiki, where you find information on how to execute it in 32bit, categories, on specific versions of NUnit, etc. Now we have all the parts to handle versions and still get the feedback. This also opens up some more opportunities as we now execute the build from MSBuild. We can do some pre- and post-processing around the build like zipping up files or exchange files or data after checkout but before build.

Next step is to get the version numbers across to the next project and use it for packaging as well as labeling our builds in subversion.

// Håkan Reis

From the previous post we have these custom targets we want to create; MSI package building, Database deploy, Service installation, and Web deploy. I will go through each of them and give you some hints on how the targets were created.

MSI package building

For this part we need to know the solution name, the project containing the setup and where VisualStudio is installed. Remember the registry thing from the last post? Here it really comes in handy.

<PropertyGroup>
  <VS>$(registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\
    VisualStudio\9.0\@InstallDir)\devenv</VS>
</PropertyGroup>

<Target Name="BuildMsi">
  <Message Text="Building package $(Vdproj)" />
  <Exec Command='"$(VS)" "$(Sln)" /Build "Release|Any CPU"
    /project "$(Vdproj)" 
    /ProjectConfig "Release"'/>
</Target>

Never mind the extra line breaks, it’s for readability. But note the single quote that is used on the Command part, this makes it possible to use double quote in the command text for readability. The other option is to use &quote; but that really makes it very hard to read the scripts.

Database deploy

When we deploy a database you typically have five parameters involved; where, who and what. Oh, the who part is user and password and where includes both server and database, a total of five. Now, let us start with a simple target and a magic database task.

<Target Name="DeployDatabase">
  <Message Text="Deploying $(DBContent) to $(DB) on $(Server)" />
  <MagicTask DataPath="$(DBContent)"
    ConnectionString="Database=$(DB);Server=$(Server);
                      UID=$(User);PWD=$(Pass)"
    DropDatabase="true" />
  <Message Text="Database deployed on $(Server)" />
</Target>

The MagicTask is depending on how you deploy the database, we are using NHibernate and have written custom tasks to handle the deployment. There are a set of build tasks on CodePlex, created by a college of mine, that can help you with the specific tasks for database deployment.

Service installation

When it comes to service installation you have to get hold of a little bag of tricks to get all of it working. Locally there is not as big issue . Just call the MSI installer and execute it silently both for uninstall and install.

<Exec Command='msiexec /i "$(Msi)" /qn' />

$(Msi) holds the full path of the installer package. When it’s installed, it should be started and for that we use net start with the service name as parameter.

<Exec Command="net start $(Service)" />

But how do you do it remotely? Here’s where the bag of tricks come in handy. First you need a place on the remote system where you can transfer the files. Make sure the agent has access right to copy the files there. Then, use some magic from Sysinternals, named PsExec.exe. This little tool makes it possible to remotely execute net start and msiexec. his target we need the MSI, the Server, user and password and the name of the Service that is installed. You need to touch up the scripts with some of your own paths to make sure you have all the files in the right place to start with.

<PropertyGroup>
  <!-- The sysinternals tool -->
  <PsExec>$(MSBuildStartupDirectory)\Tools\psexec.exe</PsExec>
  <!-- Here is the new MSI package is placed -->
  <NewMsi>$(MSBuildStartupDirectory)\msi</NewMsi>
  <!-- Here is the currently installed MSI package placed -->
  <SharedMsi>$(server)\msi</SharedMsi>
</PropertyGroup>

<Target Name="InstallMsi">
  <Message Text="Installing and starting $(msi)/>

  <!-- First Uninstall the old service if the config is there -->
  <Exec Command='$(PsExec) -accepteula $(Server) 
        -u $(User) -p $(Pass) -w "$(OldMsi)" -n 60 
        msiexec /x "$(SharedMsi)\$(msi)" /quiet' />

  <!-- Copy and install the new service -->
  <Copy SourceFiles="$(NewMsi)\$(msi)" 
    DestinationFolder="$(OldMsi)" />
  <Exec Command='$(PsExec) -accepteula $(Server) 
        -u $(User) -p $(Pass) msiexec /i 
        "$(SharedMsi)\$(msi)" /qn' />

  <!-- and start the service -->
  <Exec Command='$(PsExec) $(Server) -u 
        $(User) -p $(Pass) -d net start $(Service)' />

  <Message Text="$(Service) installed and started" />
 </Target>

Hopefully this will give you the basic idea. For all the switches on PsExec, look into Sysinternals. But one that can be a real timeserver (often hangs the build) is the –accepteula switch, it auto accepts the EULA splash screen.

Web deploy

Maybe the simplest deployment is the web deployment. Most of the time there is nothing more to it than copy the files and you are up and running. However, you may need to set some configuration etc depending on how your environments are setup. But for now a simple target like this will do the trick for web deployment:

<ItemGroup>
  <WebContent Include="$(ContentPath)\**\*.*" />
</ItemGroup>
  
<Target Name="DeployWeb">
  <Message Text="Installing web on $(Server)" />
  <Copy SourceFiles="@(WebContent)" 
    DestinationFolder="$(Server)\$(WebFolder)\%(RecursiveDir)" />
</Target>

The ItemGroup here will produce a recursive list of files and folders from the ContentPath and it get placed in WebContent item group. The files will then be copied to the WebFolder on the Server. So here we have three properties to set for this target. Also make sure you have the correct access levels on the folders and you can do this remote as well.

Calling the targets

Now that we have the targets in place, how do we get all the properties into them? Well, custom targets can be accessed in a couple of different ways. You can import them and then call them as usual and all the properties you set in the calling script are set as if the target was in your current script.

<Import Project="CustomTargets.proj" />

<PropertyGroup>
  <Server>Srv01</Server>
  <User>Kalle</User>
  <Pass>Kula</Pass>
  <Msi>setup.msi</Msi>
</PropertyGroup>

<Target Name="default">
  <CallTarget Targets="InstallMsi" />
</Target>

But in this case I want to pass in more than one set of properties to the target to make it more flexible. And in that case it’s easier to use the MSBuild task. This will execute the target as if you were calling it from command line.

<MSBuild Projects="CustomTargets.proj" 
  Targets="DeployDatabase"
  Properties="Server=DbSrv01;User=sa;Pass=sa;DB=Base;DBContent=Data" />

All together now

All this is, in our case, trigged from TeamCity, but could of course be triggered from other build systems as well. Adding some initial parameters, like where to deploy things, makes it very easy for me to add more servers later on. For now I just execute MSBuild with the needed properties.

MSBuild deploy.msbuild /property:DB=DbSrv01;DeployOn=Srv01;Web=WwwRoot

You have to make a lot of choices where to trigger and store the properties to maximize flexibility and readability. Additionally, some smart naming of files and projects will help in minimizing the number of properties. All this is of course depending on the project you are deploying.

And one more thing, make sure the build agents are executing with the right access rights, or you will not be able to execute some of the commands. This is especially important when it comes to accessing the other systems through PsExec. It took me a while to fix that problem :)

// Håkan Reis

We use TeamCity not only for the development builds and test execution, but also deployment to test and demo environments. The ultimate goal is to make it all the way into the stage environment. If we ever will get there I cannot tell, there are a number of obstacles like firewalls, port restrictions and good old access rights in the way but there are ideas on the drawing table how to solve that. As I was going through the deployment scripts I found that it needed an overview so I too the time to extract a few custom targets.

Background

The application is depending on three mechanisms: web, services and database. Web is deployed into IIS, a number of services are installed using MSI, and a database that are deployed using custom MSBuild tasks. This concoction is installed to three different machines for test and demo. Additionally some prerequisites around this are:

• As TeamCity uses its own internal checkout paths, if not specified, all paths should be relative. This will ensure that its really simple to add more agents later on.
• Naturally the scripts work for both 32bit and 64bit environments.
• The artifact handling in TeamCity should be used as much as possible.

Go for absolute relativity

You should always use relative paths, absolute paths always make a mess of things. Making it hard to move script or folder structure. But it means you sometimes need to be able to find the absolute path at a given moment. for example command line often need the current working directory. A few useful reserved properties can take care of that, and two of them are:

• $(MSBuildStartupDirectory) – the directory where the MSBuild execution was started
• $(MSBuildProjectDirectory) – the directory where the current executing script is placed

About the 32/64 bit problem; on 64bit VisualStudio is in Program Files (x86) but on 32bit its in Program Files. This is a problem as we currently use VisualStudio in command line for building MSI packages

Well, an easiest way to get the path right is the registry to find out. And we are in luck, there are registry access in MSBuild 3.5. So to get the path use this syntax:

$(registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\
VisualStudio\9.0\@InstallDir)

Artifact handling in TeamCity we know about from a previous post so I will not go into detail around that. However, one problem I run into is that using VisualStudio to build the install packages, means that we are not using TeamCity artifacts all the way. We are rebuilding the assemblies in this step. But hopefully WIX can help out here.

Stay on target

So with this background the following targets surface:

• MSI package building
• Database deploy
• Service installation
• Web deploy

There reason for the selection of these parts was not only to avoid repetition. This will also give much more readable and maintainability of scripts. And if we move to WIX or change database deployment we only have one place to update the scripts.

In the title I promised you custom targets and deployment, and I will give you that, but not until the next part.

// Håkan Reis

It was time, the old layout and colors wasn’t working but I haven’t gotten around to fix it, especially the comment section looked really bad. Then I found out about blogengine.net. After experimenting with it for a while I found that it had what I liked:

• The code was readable and easy to get into for modification
• It wasn’t depending on a database for storage (how many bloggers are really creating that much information that calls for a full fledge database?). But it could be added.
• It had a clear separation of the view and presentation that makes it easy to build and expand themes.
• I found a good looking basic view that I could modify for my needs.
• It has easy export/import functionality through the standardized BlogML format (it handles posts, comments and images just right, really powerful).

So after tweaking the view a bit (more is coming) I decided to make the switch. And as you can se it’s up and running. Hope you like the new look.

All that happen to subscribe should really shift to the feedburner link instead of the old web link. But for those who are still using the old one I found an easy solution. I used built in syndication http handler and some configuration.

Normally, blogengine is configured so that a call to syndication.axd is fed to a HttpHandler called SyndicationHandler. So I just added the old SyndicationService.asmx (without the extra GetRSS part) to point to the same piece of code. In the web.config I added the following line to the <httpHandlers> section of web.config:

<add verb="*"
path="SyndicationService.asmx"
type="BlogEngine.Core.Web.HttpHandlers.SyndicationHandler,
    BlogEngine.Core"
validate="false" />

And that was all I had to do. Now I just have to get going with some more CSS and images to spice things up a a bit…

// Håkan Reis

Version 4 of TeamCity has been out for a few weeks now and there are a lot of improvements in this release. A few of them are:

• Improved build runners now with of FxCop an new versions of NUnit
• Test reordering (problematic and failed tests are executed first to give even faster feedback)
• Build agent improvements
• Artifact dependencies

There are a lot more and not all are .Net features of course. I’m going to focus on the last one here that are a big improvement in my current project.

We have a setup of development builds and a few nightly test and analysis builds. To top it of we have deployment builds to testing environments. The development builds are easy to maintain. But the analysis and deployment scripts were giving us a lot of problems. In the scripts we were copying files from the development builds to deployment and analysis but this was not a stable solution. If the development builds were running the deploy and analysis builds would fail. Artifact dependencies to the rescue.

Artifacts

Whenever a build is completed I can direct files to a special TeamCity repository. Here it gets saved in a structure I define. So for example when a development build of our web project is done I store the files for deploying to the web, unit and function testing, and a few shared assemblies. the shared files are stored as build artifacts using the following artifact paths (Build Configuration Step 1 – General settings):

So for each successful build we get a copy of the produced shared assemblies (and all other supporting files in the two release folders). So the artifact structure for each web build will look like the following image:

Here we have the Web files, the shared Common and Core assemblies and unit test projects.

Artifact dependencies

Moving on to the deployment builds, we want to start out with Common and Core assemblies already in place but from the last successful development build. This is really easy with the new artifact dependencies. Going into the deployment build configuration we set up this artifact dependency (Build Configuration Step 5 - Dependencies) like this:

Now the assemblies in Shared/Common/ and Shared/Core/ folders in TeamCity will be placed in my deployment build checkout folder before the subversion checkout is done. This dependency can be modified to only work on pinned builds and even specific build numbers. And you can add more artifact paths and of course artifacts from multiple build configurations. So if you haven’t upgraded to TeamCity 4.0.1 go ahead now.

// Håkan Reis

With all sorts of code metric analysis, unit tests, code coverage, selenium tests, and automatic deployment scripts, you should think that you got most of it covered. But there are other ways to check you code and one of the best is probably gut feeling and common sense reviewing and reading code.

But just because you find code smells and code that start to fall apart doesn’t mean you have the time or want to break your flow with a big restructuring or even a smaller refactoring. You want to  mark it and later get back to it. So you place some comment there, like a TODO or NEEDS WORK and what happens?

Two weeks later you find the TODO but still don’t have the time to do the job properly or you cannot even remember why you put it there in the first place. And when is there time to take care of this code.

In the current team we try to collect bigger restructuring and refactoring to refactor Friday. We are not allowed to add new features during this day, only clean up and clarify the code. So we try to find the TODOs and comments or even collect some comments on our scrum board but we need more structure and a better way to handle code smells.

So we came up with a few ideas for this. The first part was to add a .NET attribute that we can add to classes and methods that we find need work. The found code smells are now marked with an attribute as follows:

[SmellyCat(LevelOfSmell = "This one reeks", 
    YourCodeSmellsBecause = "It’s unreadable", 
    IFeltTheSmell = "Sniffy")]

The attribute includes a way to add details on why and how bad the smell is to be able to handle it later on and maybe give it priority.

The next step is doing something with all these attributes. A custom task to MS build looks like a good start, this is on it’s way and will produce an output that we can include in our code analysis section in TeamCity, currently it produces an XML file so the next step is to add some XSLT and produce a lovely HTML file to include as artifact in TeamCity. The goal here is to find a good level for this build task and simply fail the build if we have to many code smells in the code. Forcing us to take care of the code smells before it reaches unbearable levels of stench.

// Håkan Reis