blog.reis.se

It's all about looks

Seminar in Gothenburg cancelled

Unfortunately I have to announce that the session Continuation of continuous integration has been cancelled for Gothenburg. I will still be in Malmoe, Linkoping and Stockholm, so now changes there.

I would love to give it in Gothenburg as well, but it’s out of my control. But who knows, I might show up in another context… :)

// Håkan Reis


TC-Logo_3I will be speaking  in Malmö, Göteborg, Linköping, and Stockholm in March/April. The subject will be build automation, some MSBuild and all done in TeamCity. The following abstract is in Swedish as the session will be in Swedish:

Continuation of continuous integration

Vad är nästa steg för dina automatiska byggen?

2009 års seminarier inleds med en närmare titt på hur man kan vidareutveckla automatiska bygge (Continuous Integration). I dag är de flesta medvetna om hur viktigt ett versionshanteringssystem som Subversion eller Team Foundation Server är. Oftast har man också insett hur viktigt det är att alltid ha byggbar kod, så man har gått vidare med periodiska, eller ännu hellre kontinuerliga byggen. Men i förlängningen vill man ständigt ha en leverans redo.

Vi tittar på hur man kan hantera kodanalys, integrationstester, release och automatisk installation. Fokus kommer att ligga på att utföra detta i en lösning från JetBrains - TeamCity. Men vi kommer inte ensidigt titta på TeamCity utan även på verktygen runt detta samt hur man kan utnyttja MSBuild för mer än att bara kompilera koden.

Bland annat kommer vi gå in på:

  • Vad kommer efter kontinuerliga byggen
  • Artifakthantering och beroende mellan byggen
  • Tricks MSBuild och custom tasks

If there are any other topics that are of interest I can still make room for changes in my material so feel free to contact me.

You can also register for any of the events, and don’t forget to mention where and date.

// Håkan Reis


From the previous post we have these custom targets we want to create; MSI package building, Database deploy, Service installation, and Web deploy. I will go through each of them and give you some hints on how the targets were created.

MSI package building

For this part we need to know the solution name, the project containing the setup and where VisualStudio is installed. Remember the registry thing from the last post? Here it really comes in handy.

<PropertyGroup>
  <VS>$(registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\
    VisualStudio\9.0\@InstallDir)\devenv</VS>
</PropertyGroup>

<Target Name="BuildMsi">
  <Message Text="Building package $(Vdproj)" />
  <Exec Command='"$(VS)" "$(Sln)" /Build "Release|Any CPU"
    /project "$(Vdproj)" 
    /ProjectConfig "Release"'/>
</Target>

Never mind the extra line breaks, it’s for readability. But note the single quote that is used on the Command part, this makes it possible to use double quote in the command text for readability. The other option is to use &quote; but that really makes it very hard to read the scripts.

Database deploy

When we deploy a database you typically have five parameters involved; where, who and what. Oh, the who part is user and password and where includes both server and database, a total of five. Now, let us start with a simple target and a magic database task.

<Target Name="DeployDatabase">
  <Message Text="Deploying $(DBContent) to $(DB) on $(Server)" />
  <MagicTask DataPath="$(DBContent)"
    ConnectionString="Database=$(DB);Server=$(Server);
UID=$(User);PWD=$(Pass)"
DropDatabase="true" /> <Message Text="Database deployed on $(Server)" /> </Target>

The MagicTask is depending on how you deploy the database, we are using NHibernate and have written custom tasks to handle the deployment. There are a set of build tasks on CodePlex, created by a college of mine, that can help you with the specific tasks for database deployment.

Service installation

When it comes to service installation you have to get hold of a little bag of tricks to get all of it working. Locally there is not as big issue . Just call the MSI installer and execute it silently both for uninstall and install.

<Exec Command='msiexec /i "$(Msi)" /qn' />

$(Msi) holds the full path of the installer package. When it’s installed, it should be started and for that we use net start with the service name as parameter.

<Exec Command="net start $(Service)" />

But how do you do it remotely? Here’s where the bag of tricks come in handy. First you need a place on the remote system where you can transfer the files. Make sure the agent has access right to copy the files there. Then, use some magic from Sysinternals, named PsExec.exe. This little tool makes it possible to remotely execute net start and msiexec. his target we need the MSI, the Server, user and password and the name of the Service that is installed. You need to touch up the scripts with some of your own paths to make sure you have all the files in the right place to start with.

<PropertyGroup>
  <!-- The sysinternals tool -->
  <PsExec>$(MSBuildStartupDirectory)\Tools\psexec.exe</PsExec>
  <!-- Here is the new MSI package is placed -->
  <NewMsi>$(MSBuildStartupDirectory)\msi</NewMsi>
  <!-- Here is the currently installed MSI package placed -->
  <SharedMsi>$(server)\msi</SharedMsi>
</PropertyGroup>

<Target Name="InstallMsi">
  <Message Text="Installing and starting $(msi)/>

  <!-- First Uninstall the old service if the config is there -->
  <Exec Command='$(PsExec) -accepteula $(Server) 
        -u $(User) -p $(Pass) -w "$(OldMsi)" -n 60 
        msiexec /x "$(SharedMsi)\$(msi)" /quiet' />

  <!-- Copy and install the new service -->
  <Copy SourceFiles="$(NewMsi)\$(msi)" 
    DestinationFolder="$(OldMsi)" />
  <Exec Command='$(PsExec) -accepteula $(Server) 
        -u $(User) -p $(Pass) msiexec /i 
        "$(SharedMsi)\$(msi)" /qn' />

  <!-- and start the service -->
  <Exec Command='$(PsExec) $(Server) -u 
        $(User) -p $(Pass) -d net start $(Service)' />

  <Message Text="$(Service) installed and started" />
 </Target>

Hopefully this will give you the basic idea. For all the switches on PsExec, look into Sysinternals. But one that can be a real timeserver (often hangs the build) is the –accepteula switch, it auto accepts the EULA splash screen.

Web deploy

Maybe the simplest deployment is the web deployment. Most of the time there is nothing more to it than copy the files and you are up and running. However, you may need to set some configuration etc depending on how your environments are setup. But for now a simple target like this will do the trick for web deployment:

<ItemGroup>
  <WebContent Include="$(ContentPath)\**\*.*" />
</ItemGroup>
  
<Target Name="DeployWeb">
  <Message Text="Installing web on $(Server)" />
  <Copy SourceFiles="@(WebContent)" 
    DestinationFolder="$(Server)\$(WebFolder)\%(RecursiveDir)" />
</Target>

The ItemGroup here will produce a recursive list of files and folders from the ContentPath and it get placed in WebContent item group. The files will then be copied to the WebFolder on the Server. So here we have three properties to set for this target. Also make sure you have the correct access levels on the folders and you can do this remote as well.

Calling the targets

Now that we have the targets in place, how do we get all the properties into them? Well, custom targets can be accessed in a couple of different ways. You can import them and then call them as usual and all the properties you set in the calling script are set as if the target was in your current script.

<Import Project="CustomTargets.proj" />

<PropertyGroup>
  <Server>Srv01</Server>
  <User>Kalle</User>
  <Pass>Kula</Pass>
  <Msi>setup.msi</Msi>
</PropertyGroup>

<Target Name="default">
  <CallTarget Targets="InstallMsi" />
</Target>

But in this case I want to pass in more than one set of properties to the target to make it more flexible. And in that case it’s easier to use the MSBuild task. This will execute the target as if you were calling it from command line.

<MSBuild Projects="CustomTargets.proj" 
  Targets="DeployDatabase"
  Properties="Server=DbSrv01;User=sa;Pass=sa;DB=Base;DBContent=Data" />

All together now

All this is, in our case, trigged from TeamCity, but could of course be triggered from other build systems as well. Adding some initial parameters, like where to deploy things, makes it very easy for me to add more servers later on. For now I just execute MSBuild with the needed properties.

MSBuild deploy.msbuild /property:DB=DbSrv01;DeployOn=Srv01;Web=WwwRoot

You have to make a lot of choices where to trigger and store the properties to maximize flexibility and readability. Additionally, some smart naming of files and projects will help in minimizing the number of properties. All this is of course depending on the project you are deploying.

And one more thing, make sure the build agents are executing with the right access rights, or you will not be able to execute some of the commands. This is especially important when it comes to accessing the other systems through PsExec. It took me a while to fix that problem :)

// Håkan Reis


We use TeamCity not only for the development builds and test execution, but also deployment to test and demo environments. The ultimate goal is to make it all the way into the stage environment. If we ever will get there I cannot tell, there are a number of obstacles like firewalls, port restrictions and good old access rights in the way but there are ideas on the drawing table how to solve that. As I was going through the deployment scripts I found that it needed an overview so I too the time to extract a few custom targets.

Background

The application is depending on three mechanisms: web, services and database. Web is deployed into IIS, a number of services are installed using MSI, and a database that are deployed using custom MSBuild tasks. This concoction is installed to three different machines for test and demo. Additionally some prerequisites around this are:

  • As TeamCity uses its own internal checkout paths, if not specified, all paths should be relative. This will ensure that its really simple to add more agents later on.
  • Naturally the scripts work for both 32bit and 64bit environments.
  • The artifact handling in TeamCity should be used as much as possible.

Go for absolute relativity

You should always use relative paths, absolute paths always make a mess of things. Making it hard to move script or folder structure. But it means you sometimes need to be able to find the absolute path at a given moment. for example command line often need the current working directory. A few useful reserved properties can take care of that, and two of them are:

  • $(MSBuildStartupDirectory) – the directory where the MSBuild execution was started
  • $(MSBuildProjectDirectory) – the directory where the current executing script is placed

About the 32/64 bit problem; on 64bit VisualStudio is in Program Files (x86) but on 32bit its in Program Files. This is a problem as we currently use VisualStudio in command line for building MSI packages

Well, an easiest way to get the path right is the registry to find out. And we are in luck, there are registry access in MSBuild 3.5. So to get the path use this syntax:

$(registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\
VisualStudio\9.0\@InstallDir)

Artifact handling in TeamCity we know about from a previous post so I will not go into detail around that. However, one problem I run into is that using VisualStudio to build the install packages, means that we are not using TeamCity artifacts all the way. We are rebuilding the assemblies in this step. But hopefully WIX can help out here.

Stay on target

So with this background the following targets surface:

  • MSI package building
  • Database deploy
  • Service installation
  • Web deploy

There reason for the selection of these parts was not only to avoid repetition. This will also give much more readable and maintainability of scripts. And if we move to WIX or change database deployment we only have one place to update the scripts.

In the title I promised you custom targets and deployment, and I will give you that, but not until the next part.

// Håkan Reis