Tagged: teamcity Toggle Comment Threads | Keyboard Shortcuts

  • danielsaidi 1:03 pm on February 22, 2012 Permalink | Reply
    Tags: assembly version, boo, , nextra, , nuget package explorer, phantom, teamcity   

    Use Phantom/Boo to automatically build, test, analyze and publish to NuGet and GitHub 

    When developing my NExtra .NET library hobby project, I used to handle the release process manually. Since a release involved executing unit tests, bundling all files, zipping and uploading the bundle to GitHub, creating new git tags etc. the process was quite time-consuming and error-prone.

    But things did not end there. After adding NExtra to NuGet, every release also involved refreshing and publishing six NuGet packages. Since I used the NuGet Package Explorer, I had to refresh the file and dependency specs for each package. It took time, and the error risk was quite high.

    Since releasing new versions involved so many steps, I used to release NExtra quite seldom.

    Laziness was killing it.

    The solution

    I realized that something had to be done. Unlike at work, where we use TeamCity for all solutions, I found a build server to be a bit overkill. However, maybe I could use a build script for automating the build and release process?

    So with this conclusion, I defined what the script must be able to help me out with:

    • Build and test all projects in the solution
    • Automatically extract the resulting version
    • Create a release folder or zip with all files
    • Create a new release tag and push it to GitHub
    • Create a NuGet package for each project and publish to NuGet

    The only piece of the release process not covered by this process was to upload the release zip to GitHub, but that would be a walk in the park once the build script generated a release zip.

    The biggest step was not developing the build script. In fact, it is quite a simple creation. Instead, the biggest step was to come to the conclusion that I needed one.

    Selecting a build system

    In order to handle my release process, I needed a build system. I decided to go with Phantom, since I use it at work as well. It is a convenient tool (although a new, official version would be nice) that works well, but it left me with an annoying problem, which I will describe further down.

    So, I simply added Phantom 0.3 to a sub folder under the solution root. No config is needed – the build.bat and build.boo (read on) files take care of everything.

    The build.bat file

    build.bat is the file that I use to trigger a build, build a .zip or perform a full publish from the command prompt. I placed it in
    the solution root, and it looks like this.

    @echo off
    
    :: Change to the directory that this batch file is in
    for /f %%i in ("%0") do set curpath=%%~dpi
    cd /d %curpath%
    
    :: Fetch input parameters
    set target=%1
    set config=%2
    
    :: Set default target and config if needed
    if "%target%"=="" set target=default
    if "%config%"=="" set config=release
    
    :: Execute the boo script with input params - accessible with env("x")
    resources\phantom\phantom.exe -f:build.boo %target% -a:config=%config%

     

    Those of you who read Joel Abrahamsson’s blog, probably recognize the first part. It will move to the folder that contains the .bat file, so that everything is launched from there.

    The second section fetches any input parameters. The target param determines the operation to launch (build, deploy, zip or publish) and config what kind of build config to use (debug, release etc.)

    The third section handles param fallback in case I did not define some of the input parameters. This means that if I only provide a target, config will fall back to “release”. If I define no params at all, target will fall back to “default”.

    Finally, the bat file calls phantom.exe, using the build.boo file. It tells build.boo to launch the provided “target” and also sends “config” as an environment variable (the -a:config part).

    All in all, the build.bat file is really simple. It sets a target and config and uses the values to trigger the build script.

    The build.boo file

    The build.boo build script file is a lot bigger than the .bat file. It is also located in the solution root and looks like this:

    import System.IO
    
    project_name = "NExtra"
    assembly_file = "SharedAssemblyInfo.cs"
    
    build_folder = "_tmpbuild_/"
    build_version = ""
    build_config = env('config')
    
    test_assemblies = (
     "${project_name}.Tests/bin/${build_config}/${project_name}.Tests.dll",
     "${project_name}.Web.Tests/bin/${build_config}/${project_name}.Web.Tests.dll",
     "${project_name}.Mvc.Tests/bin/${build_config}/${project_name}.Mvc.Tests.dll",
     "${project_name}.WPF.Tests/bin/${build_config}/${project_name}.WPF.Tests.dll",
     "${project_name}.WebForms.Tests/bin/${build_config}/${project_name}.WebForms.Tests.dll",
     "${project_name}.WinForms.Tests/bin/${build_config}/${project_name}.WinForms.Tests.dll",
    )
     
    
    target default, (compile, test):
     pass
    
    target zip, (compile, test, copy):
     zip("${build_folder}", "${project_name}.${build_version}.zip")
     rmdir(build_folder)
    
    target deploy, (compile, test, copy):
     with FileList(build_folder):
     .Include("**/**")
     .ForEach def(file):
     file.CopyToDirectory("{project_name}.${build_version}")
     rmdir(build_folder)
    
    target publish, (zip, publish_nuget, publish_github):
     pass
     
    
    target compile:
     msbuild(file: "${project_name}.sln", configuration: build_config, version: "4")
    
     //Probably a really crappy way to retrieve assembly
     //version, but I cannot use System.Reflection since
     //Phantom is old and if I recompile Phantom it does
     //not work. Also, since Phantom is old, it does not
     //find my plugin that can get new assembly versions.
     content = File.ReadAllText("${assembly_file}")
     start_index = content.IndexOf("AssemblyVersion(") + 17
     content = content.Substring(start_index)
     end_index = content.IndexOf("\"")
     build_version = content.Substring(0, end_index)
    
    target test:
     nunit(assemblies: test_assemblies, enableTeamCity: true, toolPath: "resources/phantom/lib/nunit/nunit-console.exe", teamCityArgs: "v4.0 x86 NUnit-2.5.5")
     exec("del TestResult.xml")
    
    target copy:
     rmdir(build_folder)
     mkdir(build_folder)
    
     File.Copy("README.md", "${build_folder}/README.txt", true)
     File.Copy("Release-notes.md", "${build_folder}/Release-notes.txt", true)
    
     with FileList(""):
     .Include("**/bin/${build_config}/*.dll")
     .Include("**/bin/${build_config}/*.pdb")
     .Include("**/bin/${build_config}/*.xml")
     .Exclude("**/bin/${build_config}/*.Tests.*")
     .Exclude("**/bin/${build_config}/nunit.framework.*")
     .Exclude("**/bin/${build_config}/nsubstitute.*")
     .ForEach def(file):
     File.Copy(file.FullName, "${build_folder}/${file.Name}", true)
    
    target publish_nuget:
     File.Copy("README.md", "Resources\\README.txt", true)
     File.Copy("Release-notes.md", "Resources\\Release-notes.txt", true)
    
     exec("nuget" , "pack ${project_name}\\${project_name}.csproj -prop configuration=release")
     exec("nuget" , "pack ${project_name}.web\\${project_name}.web.csproj -prop configuration=release")
     exec("nuget" , "pack ${project_name}.mvc\\${project_name}.mvc.csproj -prop configuration=release")
     exec("nuget" , "pack ${project_name}.wpf\\${project_name}.wpf.csproj -prop configuration=release")
     exec("nuget" , "pack ${project_name}.webforms\\${project_name}.webforms.csproj -prop configuration=release")
     exec("nuget" , "pack ${project_name}.winforms\\${project_name}.winforms.csproj -prop configuration=release")
    
     exec("nuget push ${project_name}.${build_version}.nupkg")
     exec("nuget push ${project_name}.web.${build_version}.nupkg")
     exec("nuget push ${project_name}.mvc.${build_version}.nupkg")
     exec("nuget push ${project_name}.wpf.${build_version}.nupkg")
     exec("nuget push ${project_name}.webforms.${build_version}.nupkg")
     exec("nuget push ${project_name}.winforms.${build_version}.nupkg")
    
     exec("del *.nupkg")
     exec("del Resources\\README.txt")
     exec("del Resources\\Release-notes.txt")
    
    target publish_github:
     exec("git add .")
     exec('git commit . -m "Publishing ${project_name} ' + "${build_version}" + '"')
     exec("git tag ${build_version}")
     exec("git push origin master")
     exec("git push origin ${build_version}")
    

    Topmost, we see a system import. This will allow us to use System.IO for file operations. After that, I define some variables and a list of test assemblies that I want to test.

    Two variables worth mentioning is the build_version, which is set in the compile step, as well as build_config, which is set by the input parameter defined in build.bat.

    The next section of the file defines all public targets, that are intended to be callable by the user. These map directly to target in build.bat.

    Of course, all targets further down can be called as well – there are no such thing as public or private targets. Still, that would probably not be a very good idea.

    If we look at the public targets, we have:

    • default – Executes “compile” and “test”
    • zip – Executes “compile” and “test”, then creates a zip file
    • deploy – Executes “compile” and “test” then creates a folder
    • publish – Executes “zip”, then publishes to NuGet and GitHub

    If we look at the private targets (that do the real work) we have:

    • compile – Compiles the solution and extract the version number
    • test – Runs the NUnit builtin with the .NExtra test assemblies
    • copy – Copies all relevant files to the temporary build_folder
    • publish_nuget – Pack and publish each .NExtra project to NuGet
    • publish_github – Commit all changes, create a tag then push it

    It is not that complicated, but it is rather much. You could take the bat and boo file and tweak it, and it would probably work for your projects as well.

    However, read on for some hacks that I had to do to get the build process working as smoothly as it does.

    One assembly file to rule them all

    A while ago, I decided to extract common information from each of the .NExtra projects into a shared assembly file.

    The shared assembly file looks like this:

    using System.Reflection;
    
    // General Information about an assembly is controlled through the following
    // set of attributes. Change these attribute values to modify the information
    // associated with an assembly.
    [assembly: AssemblyCompany("Daniel Saidi")]
    [assembly: AssemblyProduct("NExtra")]
    [assembly: AssemblyCopyright("Copyright © Daniel Saidi 2009-2012")]
    [assembly: AssemblyTrademark("")]
    
    // Make it easy to distinguish Debug and Release (i.e. Retail) builds;
    // for example, through the file properties window.
    #if DEBUG
    [assembly: AssemblyConfiguration("Debug")]
    #else
    [assembly: AssemblyConfiguration("Retail")]
    #endif
    
    // Version information for an assembly consists of the following four values:
    //
    // Major Version
    // Minor Version
    // Build Number
    // Revision
    //
    // You can specify all the values or you can default the Build and Revision Numbers
    // by using the '*' as shown below:
    [assembly: AssemblyVersion("2.6.3.4")]
    [assembly: AssemblyFileVersion("2.6.3.4")]

    The file defines shared assembly information like version, to let me specify this once for all projects. I link this file into each project and then remove the information from the project specific assembly info file.

    Since the .NExtra version management is a manual process (the way I want it to be), I manage the .NExtra version here and parse the file during the build process to retrieve the version number. The best way would be to use System.Reflection to analyze the library files, but this does not work, since Phantom uses .NET 3.5.

    I tried re-compiling Phantom to solve this, but then other things started to crash. So…the file parse approach is ugly, but works.

    Tweaking NuGet

    After installing NuGet, typing “nuget” in the command prompt will still cause a warning message to appear, since “nuget” is unknown.

    To solve this, either add the NuGet executable path to PATH or be lazy and use the nuget.exe command line bootstrapper, which finds NuGet for you. You can download it from CodePlex or grab it from the .NExtra Resources root folder.

    Regarding each project’s nuspec file, they were easily created by calling “nuget spec x” where x is the path to the project file. A nuspec file is then generated. I then added some information that cannot be extracted from the assembly, like project URL, icon etc. for each of these generated spec files.

    Conclusion

    This post became a rather long, but I hope that it did explain my way of handling the .NExtra release process.

    Using the build script, I can now call build.bat in the following ways:

    • build – build and test the solution
    • build zip – build and test the solution and generate a nextra.<version>.zip file
    • build deploy – build and test the solution and generate a nextra.<version>.zip folder
    • build publish – the same as build zip, but also publishes to NuGet and GitHub.

    The build script has saved me immense amount of work. It saves me time, increases quality by reducing the amount of manual work and makes releasing new versions of .NExtra a breeze.

    I still have to upload the zip to the GitHub download area, but I find this to be a minimum task compared to all other steps. Maybe I’ll automate this one day as well, but it will do for now.

    I strongly recommend all projects to use a build script, even for small projects where a build server is a bit overkill. Automating the release process is a ticket to heaven.

    Or very close to that.

     
    • Markus Johansson 10:07 am on July 16, 2013 Permalink | Reply

      Great post! Looks as a really nice release script! Thanks for sharing your experiance!

      • danielsaidi 8:33 am on July 20, 2013 Permalink | Reply

        Thanks! 🙂 Once all pieces are in place, publishing new releases is a breeze.

  • danielsaidi 3:06 pm on September 8, 2011 Permalink | Reply
    Tags: , teamcity   

    TeamCity 6.5.1 does not play well with NServiceBus 

    I am currently moving projects from an old TeamCity 5.1.2 server to a new 6.5.1 server.

    Everything has been going great, until I tried moving a project that uses NServiceBus test framework. When TeamCity tries to launch these tests, it fails with the following error message (one of many):

    [xxx.Test_Name] Test(s) failed. System.InvalidOperationException : Could not load C:\TeamCity\BuildAgent3\work\c53933ba5b3b087f\xxx\bin\release\xxx.XmlSerializers.dll. Consider using ‘Configure.With(AllAssemblies.Except(“xxx.XmlSerializers.dll”))’ to tell NServiceBus not to load this file. —-> System.BadImageFormatException : Could not load file or assembly ‘file:///C:\TeamCity\BuildAgent3\work\c53933ba5b3b087f\xxx.Tests\bin\release\xxx.XmlSerializers.dll’ or one of its dependencies. This assembly is built by a runtime newer than the currently loaded runtime and cannot be loaded. at NServiceBus.Configure.GetAssembliesInDirectoryWithExtension(String path, String extension, String[] assembliesToSkip) in d:\BuildAgent-02\work\672d81652eaca4e1\src\config\NServiceBus.Config\Configure.cs:line 213 at NServiceBus.Configure.<GetAssembliesInDirectory>d__3.MoveNext() in d:\BuildAgent-02\work\672d81652eaca4e1\src\config\NServiceBus.Config\Configure.cs:line 190 at System.Linq.Buffer`1..ctor(IEnumerable`1 source) at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source) at NServiceBus.Configure.With(String probeDirectory) in d:\BuildAgent-02\work\672d81652eaca4e1\src\config\NServiceBus.Config\Configure.cs:line 101 at NServiceBus.Configure.With() in d:\BuildAgent-02\work\672d81652eaca4e1\src\config\NServiceBus.Config\Configure.cs:line 75 at NServiceBus.Testing.Test.Initialize() in d:\BuildAgent-02\work\672d81652eaca4e1\src\testing\Test.cs:line 20 at xxx.Setup() in c:\TeamCity\BuildAgent3\work\c53933ba5b3b087f\xxx.Tests\xxxTests.cs:line 17 –BadImageFormatException at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) at System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection) at System.Reflection.Assembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, StackCrawlMark& stackMark) at System.Reflection.Assembly.LoadFrom(String assemblyFile) at NServiceBus.Configure.GetAssembliesInDirectoryWithExtension(String path, String extension, String[] assembliesToSkip) in d:\BuildAgent-02\work\672d81652eaca4e1\src\config\NServiceBus.Config\Configure.cs:line 204

    I have replaced secret component names with xxx – do not let it confuse you.

    Now, I am really wondering where d:\BuildAgent-02 comes from, since it does not exist on the old server and not on the new one either. After reading up a bit, it seems that it is an NServiceBus issue.

    The problem resambles the issue being described here:

    http://tech.groups.yahoo.com/group/nservicebus/message/6790

    However, I have installed VS2010 and SP1 and it still does not work. I am out of ideas. I have tried bending the laws of physics, but all I am left with is a NServiceBus test suite that just won’t work.

    Any ideas out there?

     
    • Eugene Petrenko 4:49 pm on September 8, 2011 Permalink | Reply

      Check you specified NUnit assemblies wildcard in the way **/obj/** folders are excluded. MSBuild used to put there assemblies without dependent assemblies.

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel