# Thursday, 19 February 2009

I’m going to be teaching two new one-day courses on setting up a Continuous Integration process, one using freely available tools, and one using Microsoft Team Foundation Server.  For more details and to register, see our registration page.

Both classes will be held at the Westside campus of OIT

Course Description:

This one day course will provide a hands on look at setting up a Continuous Integration process using freely available tools from beginning to end. Students will learn the theory and practice behind establishing a Continuous Integration process, including setting up a build script, running unit tests, setting up an automated build/test server, and capturing reporting information for the whole process.

Course Technology

This course uses Microsoft Windows XP, the Microsoft .NET 3.5 Framework, NAnt, NUnit, and CruiseControl.NET.

At the end of the course you will be able to:

  • Create a build script using NAnt or MSBuild
  • Create and run unit tests using NUnit
  • Set up and run an automated build using CruiseControl.NET
  • Capture reporting data from the automated CI process

Course Outline
    Continuous Integration in Theory
    • Why CI is important
    • How CI fits into the Software Development process
    Creating a build script
    • What goes in a build script?
    • How does NAnt work?
    • How does MSBuild work?
    • Creating build scripts for NAnt or MSBuild
    • Running an automated build
    Adding unit tests to your build
    • Writing NUnit tests
    • Running tests as part of a build
    • Capturing test results
    Continuous Integration with CruiseControl.NET
    • Installing and configuring CC.NET
    • Adding projects to a build server
    • Reporting with CC.NET
    • Running multiple builds on the same server
        Dependencies between builds

Prerequisites

This class is intended for experienced .NET 2.0 software developers.

Thursday, 19 February 2009 12:37:26 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  | 
# Friday, 25 July 2008
I'll be speaking on Continuous Integration at the Software Association of Oregon's Developer SIG meeting in September.  We'll look at how to implement CI across different parts of your organization, using techniques and tools for both .NET and Java.

Friday, 25 July 2008 13:37:41 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, 14 May 2008
Looks like I'll be speaking on Continuous Integration in theory and practice at next month's meeting of the Software Process Improvement Network.  Details are here.  If you are interested in CI, come on down.

Wednesday, 14 May 2008 10:18:08 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Friday, 25 April 2008
My new book Code Leader: Using People, Tools, and Processes to Build Successful Software is now available and in stock at Amazon and hopefully soon at stores near you.  The initial outline for the book came from a post I did a while back, This I Believe, the developer edition.  Writing up that post got me thinking about a lot of things, and I've continued to expand on them both in the book and in presentations since then. 

There is a lot of content in the book about working with tools like SCC, static analysis, and unit test frameworks to build better software more efficiently.  I think if I were to sum up the focus, it is that we as developers owe it to our project sponsors to work efficiently and effectively, and there are lots of incremental changes that can be made to the way we work, both in terms of tools and processes to make that happen. 

The reality of modern software development (mostly) is that since the physical constraints like CPU time and memory/disk space have largely fallen away, the goal of any development project (from a technical standpoint) should be optimized for developer productivity.  That includes a wide array of factors from readability of design, how clear developer intent is to future readers, and how resillient our software is to future change to reducing barriers to developer productivity like implementing a good CI process.  Working together as a team across disciplinary and physical boundaries is at least as big a challenge as writing good code. 

I agree very much with Scott that the why and how of software development are often just as important (and harder to get right) than just the what.  I hope that I have something to contribute to the discussion as we all figure it out together...

Friday, 25 April 2008 11:31:59 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [2]  | 
# Thursday, 10 April 2008

First, there was a few ideas rattling around in my head, then there was a blog post, and now you can buy the book. :-)


The most difficult challenges facing most of us these days are not about making code work.  That's relatively easy, compared to making code that you can work on and maintain with other people.  I've tried to bring together a bunch of issues and practical solutions for writing code with a team, including defining contracts, creating good tracing and error handling, and building a set of automatic tests that you can rely on.  In addition, there's a bunch in here about how to work with other developers using source control, continuous integration, etc. to make everyone's lives easier and more productive.
Thursday, 10 April 2008 10:41:33 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [1]  | 
# Tuesday, 25 March 2008
I'll be speaking at next month's InnoTech Oregon conference, at the convention center April 16th and 17th.  The session is entitled "The Code is the Easy Part".  I'll be speaking on practical steps you can take to improve your software project, specifically around making it easier and more effective for developers to work together as a team.  Specific topics include starting a Continuous Integration process, effective use of source control, and using tests as a means of expressing developer intent.

It looks like there's a little something for everyone at the conference, including some stuff on development practices and "agile" methodologies.  Should be fun.

Tuesday, 25 March 2008 09:34:33 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  | 
# Friday, 14 March 2008
Once again I'll be teaching at the Oregon Institue of Technology this coming Spring term.  This time around the topic is building an application using C# in Visual Studio 2008.  We'll be covering an entire application lifecycle, including design, coding, test, build and deploy.  We'll go into detail about effective use of source control, how to set up a Continuous Integration process, how to write good unit tests, etc.  Along the way we'll be covering some of the new .NET 3.5 features like LINQ and all that entails.

Spread the word, and come on down if you are interested.

The official course title is .NET Tools with C# (CST407) and it will be Wednesday nights at the Hillsboro campus.

Friday, 14 March 2008 10:02:43 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  | 
# Thursday, 06 December 2007
I happened to meet a woman the other day who grew up speaking Swiss German, and she said that one of the hardest bits of American slang for her to get the hang of was our use of "whatever" (usually with an !) to mean "that's the stupidist thing I've ever heard, but it's too trivial for me to take the time arguing with you about it".  That translation is mine, not hers.  Just look at how many works are saved with that simple but idiomatic usage.

So when I find that someone has changed the name of one of my variables (in several places) from "nextButton" to "nextBtn" causing me two separate merge conflicts, I'm glad we have that little word.

Whatever!

Thursday, 06 December 2007 14:56:06 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [1]  | 
# Monday, 03 December 2007
My SoftSource compatriot Leo makes some good points about the overuse, or at least overreliance on unit testing.  I absolutely agree that you have to be clear about what the point of unit testing is.  Unit testing is for exercising all the possible paths through any given code, and verifying that it does what the author of that code thought that it should do.  That doesn't mean that it does what it is supposed to do in the context of a larger application.  That's why we have integration or functional tests.  It's up to whoever writes your functional tests (hopefully professional testers with specs in hand) to verify that the code does what the user wants it to do.  Integration tests will tell you if your code does what the developer in the cube next to you thinks it should. 

It's all about context.  It is folly to think that running a bunch of unit tests guarantees that the application will do what it should.  In fact, I'm working on a web project right now, and it is entirely possible for every unit test to pass with raging success and still not have the ASP.NET application start up.  That's a pretty clear case of the application not even running, even though all the tests pass.  The tests are still useful for what they provide, but there's a lot more to even automated testing than just validating that the code does what the author intended.

Monday, 03 December 2007 16:48:27 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [1]  | 
If you are following a continuous integration strategy, breaking the build should be a big deal.  Most of the time, if the build is broken it's because someone forgot to check in a new file, or because they didn't update before committing their stuff.  Those are both pretty easily avoided.  One strategy that helps with missing files is to do your work in a task-specific branch, get everything checked in the way you think it should be, and then check out that branch to a separate location on your local system.  If it builds there too, then you probably aren't missing any file.  If you are, you'll know about it before you break the integration branch. 

As far as not updating before committing, there are a couple reasons why that might happen more than you would like.  If the build is constantly breaking, then people will be less likely to want to update before committing, since if the build is broken they won't be able to validate their changes, and will have to put off committing.  If your build process takes too long (more than 10 minutes including unit tests) then updating and building before committing will be painful, and people won't do it.  Of couse, sometimes it's simple laziness, but that just needs to be beaten out of people. :)  The other things are fixable with technology, which is easier than getting people to do the right thing most of the time.

To make the process easier for everyone, don't commit any changes after the build breaks unless those changes are directly related to fixing the build.  If they aren't, then those additional commits will also cause failing builds, but now the waters have been muddied and it is harder to track down which changes are causing failures.  If the build breaks, then work on fixing it rather than committing unrelated changes that make it harder to fix the real problem. 

Monday, 03 December 2007 14:44:06 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [1]  | 
# Wednesday, 17 October 2007
Refactoring is cool.  Really cool.  It also means something very specific.  There's a book and everything.  If you haven't read it, you should. (Hint: it's by Martin Fowler)

Refactoring means reducing code smells by making small, incremental, orderly, and very specific changes to existing code.  Make such changes (one at a time) allows one to improve the structure of existing code without changing its functionality.  Always a good thing.  Refactorings have very specific names like "extract method" or "encapsulate field" or "push member up".  Again, there's a book.  You can look them up.

Where is this going?, you might ask.  I already know this, you say.  Cool.  Then let's move on to what refactoring (Refactoring?) isn't.

Refactoring doesn't mean changing code because you think your way would have been better.  It doesn't mean rewriting things from scratch because you have a different opinion.  It doesn't mean starting over again and again in pursuit of the perfect solution to every coding problem. 

Those other things have names (which I won't mention here for the sake of any children reading this), but "Refactoring" isn't among them.  There's a tie-in here with another term we all love, "Agile".  Refactoring fits into an "agile" process after you've made everything work they way it should (i.e. passes the tests) to make it easier to work with the code on the next story/backlog item/iteration.  The point of agile development (IMHO) is to write as little code a possible to meet your requirements.  It doesn't mean redoing things until you end up with the least possible amount of code, measured in lines.  Again, that has a different name. 

Sometimes code needs to be fixed.  More often than we'd like, in fact.  But if you are (re)writing code in pursuit of the most bestest, don't call it Refactoring.  It confuses the n00bs. 

Wednesday, 17 October 2007 12:22:00 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Thursday, 06 September 2007
The project I'm working on right now uses SourceGear Vault for SCC, and I must admit, I'm not digging it.  It's like using a slightly more performant version of VSS.  Except that you can set it in "CVS mode" which means it doesn't do exclusive locks.  It seems very difficult to deal with adding new files (if you aren't integrating with VS.NET, which we aren't) and updates seem unreasonably slow, although that may just be me.  It is a pretty big project.  Doing a "Show history" on the whole tree takes over a minute to return any results at all.  It does do change sets, at least, so that's a big plus over VSS. 
In short, I can see tha market here (for people that are comfortable with VSS, but don't want Access databases for their backend) but I'd rather be using Subversion.  I guess I'm just spoiled.

Thursday, 06 September 2007 15:26:07 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [1]  | 
# Wednesday, 18 July 2007

I decided a while back that I had too much extraneous junk running on my development machine and it was time to scale back.  Fist to go were Google Desktop, and Microsoft Desktop Search, both of which had been competing to see who could eat the most cycles and thrash my drives the worst.  Next on the list was TortoiseSVN.  Tortoise's cache seemed to be eating an awful lot of CPU time.  "How ever will I manage to deal with SVN without it?", I found myself wondering.  The reply came back "have you become completely feeble in your old age? Once you lived by PVCS, before there was such a thing as a decadent SCC GUI!". 

So I figured I'd give it a go.  The results have been pretty good so far, although it has taken me a while to get used to the svn.exe command set.  On the up side, it seems much faster than Tortoise, with updates in particular taking much less time.  The thing I find myself missing the most if the commit dialog, where you can uncheck stuff you don't want to commit right now.  Living without that makes for some extra typing, but overall it's still worth it.

The one thing that's a major hassle is browsing the repository.  If I have to figure out where someone shoved something in which repository, I've been falling back on another machine with Tortoise installed.  Lame?  Possibly.  I couldn't find anything approximating a non-Tortoise GUI SVN repository browser.  Maybe there is one, and I just haven't heard. 

So overall, I'm not quite ready to go back to edlin for text files, but the svn command line is treating me pretty well.  We'll see if I ever come to some watershed event where I decide to go back to the GUI.  That day may well come, but it's not on the horizon just yet.

Wednesday, 18 July 2007 10:57:58 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [10]  | 
# Monday, 09 July 2007

Lately we've been experiencing one of the downsides of all the tooling we've put into our continuous integration practice.  It's fragile.  Trying to get all your tools lined up with versions that all get along is tricky, and hard to keep balanced.  Particularly when two of them (NCover and TypeMock) both try to register themselves as profiles, and have to be chained to keep the tests running while gathering coverage information.

I think we've just about got everybody moved up to the right versions, but there's been lots of "it works on my machine" the last week or so. 

The price you pay, I guess.

Monday, 09 July 2007 14:15:03 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, 23 May 2007

I finally got around to building some docs for our latest project, and decided to go with Sandcastle rather than NDoc for the time being.  I got the March CTP of Sandcastle and started trying to figure out how to get it working with our build. 

While the resulting CHM file came out looking really nice, I'd have to say that Sandcastle is WAY not ready for prime time.  There is little to no documentation, so figuring out how to customize things is largely an exercise for the user.  I was willing to exercise, but many aren't. :-)  The build itself is very complicated, with 10-12 separate steps required to produce the CHM file, and lots of rough edges, like the fact that the MRefBuilder tool won't look for dependencies in the GAC, so I had to copy a bunch of core BCL assemblies out of the GAC and into my build directory to get the docs to build.  Lame. 

As it stands now, Sandcastle is useful only to serious build weenies, since it's too cumbersome for the average developer to deal with.  This is too bad, since it's being touted (at least in some circles) as the "right" solution for building docs going forward.  It smacks of something that the VS 2005 team built to create there docs, then decided to make public.  It's not quite there yet.

Wednesday, 23 May 2007 14:01:33 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [3]  | 
# Tuesday, 01 May 2007

I was trying to come up with a set of guiding principles as we've been using them on my current project, with an eye toward educating new developers being assigned to the project, so I wrote up a PPT deck for the purpose.  Scott thought it would make a good, general purpose manifesto for any project, so here goes...

These are my opinions based on experience and preference, YMMV.  Much of this is common sense, some of it is arbitrary, but it's all stuff that I think is true.

  • Our "guiding principles"
    • Test Driven Development
      • As much as possible, all of our code should be developed according to the TDD methodology.  If we stick with it, we will always have confidence in the current build (i.e. that it really works).
    • Continuous Integration
      • We want our build to produce something that looks as much as possible like a production deployment.  To that end, our build deploys a real live system and runs "integration" tests against it. 
  • Unit tests
    • Two kinds of tests
      • Unit tests have no dependencies, or only depend on other assemblies owned by us, which can be mocked.  This means our unit tests should run on anyone's box without having to set up things like SQL, ADAM, etc.
      • Integration tests depend on code not owned by us, like SQL, ADAM, IIS, etc. 
    • Automation is equally possible for both sets of tests
      • NUnit/MbUnit for all of the tests
      • MbUnit allows for test "categories" like "unit" and "integration" and can run one at a time
      • Unit tests run on every build, integration tests run at night since they take much longer to run
    • All UI development should follow the MVP pattern for ease of testing
      • This includes Win/WebForms, MMC snapins, etc.
  • Test coverage
    • 90%+ is the goal
      • This is largely achievable using proper mocking
    • NCover runs as part of the build, and reports are generated
  • Buy, not build
    • Take full advantage of the platform, even if it only solves the 80% case
      • every line of code we don't have to write and maintain saves us time and money
    • Don't write a single line of code you don't have to
    • Take full advantage of .NET 3.0, SQL 2005, Windows 2003 Server, plan for- and test on Longhorn
    • Don't invent new solutions to solved problems
      • Even if you know how to write a better hashing algorithm than the ones in the BCL, it doesn't add value to your product, so it's not worth your time
  • Limit compile time dependencies on code you don't own
    • Everything not owned by us should be behind an interface and a factory method
      • Easier to mock
      • Easier to replace
    • ILoggingService could use log4net today, EnterpriseLibrary tomorrow, but clients are insulated from the change at compile time
  • Define your data contracts in C# (think "active record")
    • All persistent storage should be abstracted using logical interfaces
      • IUserProfileService takes a user profile data object, rather than methods that take parameter lists
      • IUserProfileService knows how to store and retrieve User Profile objects in ways important to your application
      • How a User Profile object gets persisted across databases/tables/XML/whatever is only interesting to the DBA, not consumers of the service
      • This means database implementations can change, or be optimized by DBAs without affecting the app
      • The application doesn't care about how database file groups or clustered indexes work, so define the contract, and leave that bit to a DBA
  • Fewer assemblies is better
    • There should be a VERY good reason for creating a new assembly
    • The assembly is the smallest deployable unit, so it's only worth creating a new assembly if it means NOT shipping something else
    • Namespace != assembly name.  Roll up many namespaces into one physical assembly if they all must be deployed together.
  • Only the public interface should be public
    • Only make classes and interfaces public if absolutely necessary
    • Test code should take advantage of InternalsVisibleTo attributes
    • VS 2005 defaults to creating internal, not public classes for a reason
    • If it's public, you have to support it for ever
  • Windows authentication (good)
    • Just say no to connection strings
    • Windows authentication should be used to talk to SQL, ADAM, the file system, etc.
    • You can take advantage of impersonation without impersonating end users
      • Rather than impersonating end users all the way back to the DB, which is expensive and hard to manage, pick 3-4 windows accounts like "low privilege", "authenticated user", and "admin" and assign physical database access to tables/views/stored procs for those windows accounts
      • When users are authenticated, impersonate one of those accounts (low priv for anonymous users, authenticated user for same, and admin for customer service reps, for example)
  • Tracing
    • Think long and hard about trace levels
      • Critical is for problems that mean you can't continue (app domain is going away)
      • Error means anything that broke a contract (more on that later)
      • Warning is for problems that don't break the contract
      • Info is for a notable event related to the application
      • Verbose is for "got here" debugging, not application issues
    • Use formatted resource strings everywhere for localization
    • For critical, error, or warning, your audience is not a developer
      • Make error messages actionable (they should tell the recipient what to do or check for to solve the problem)
  • Error handling
    • Method names are verbs
      • the verb is a contract, e.g. Transfer() should cause a transfer to occur
    • If anything breaks the contract, throw an exception
      • if Transfer() can't transfer, for whatever reason, throw
      • Exceptions aren't just for "exceptional" conditions.  If they were really exceptional, we wouldn't be able to plan for them.
    • Never catch Exception
      • If it's not a problem you know about, delegate up the call stack, because you don't know what to do about it anyway
    • Wrap implementation specific exceptions in more meaningful Exceptions
      • e.g. if the config subsystem can't find the XML file it's looking for, it should throw ConfigurationInitializationException, not FileNotFoundException, because the caller doesn't care about the implementation.  If you swap out XML files for the database, the caller won't have to start catching DatabaseConnectionException instead of FileNotFoundException because of the wrapper.
      • Anytime you catch anything, log it
        • give details about the error, and make them actionable if appropriate
      • Rethrow if appropriate, or wrap and use InnerException, so you don't lose the call stack
  • The definition of "done" (or, how do I know when a task is ready for QA?)
    • Any significant design decisions have been discussed and approved by the team
    • For each MyClass, there is a corresponding MyClassFixture in the corresponding test assembly
    • MyClassFixture exercises all of the functionality of MyClass (and nothing else)
    • Code coverage of MyClass is >=90%, excluding only lines you are confident are unreasonable to test
      • yes this is hard, but it's SOOOOO worth the effort
    • No compiler warnings are generated by the new code
      • we compile with warnings as errors.  if it's really not a valid or important warning, use #pragma warning disable/enable
    • Before committing anything to source control, update to get all recent changes, and make sure all unit and integration tests pass
    • FxCop should generate no new warnings for your new code
    • Compiling with warnings as errors will flush out any place you forgot documentation comments, which must be included for any new code
  • There WAS a second shooter on the grassy knoll
    • possibly one on the railway overpass
    • (that's for anyone who bothered to read this far...)
Tuesday, 01 May 2007 13:49:18 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [10]  | 
# Monday, 18 December 2006

I spent about a day and a half this week trying to get a CC.NET server working with a complex combination of MSBuild (running the build script) TypeMock (the solution we're using for "mock objects") and NCover (for code coverage analysis) that proved tricky to get right.

In a typical build script, you'd create a task to run your unit tests that depends on your compile task.  And you might want to run a "clean" task first so you know you're starting from ground zero. 

Bringing a profiler into the mix changes the equation a bit.  Or in this case, two profilers that have to play nice together.  A "profiler" in this case means something that registers itself as a .NET profiler, meaning that it uses the underlying COM API in the CLR to get under the covers of the runtime.  This was originally envisioned as enabling profilers that track things like performance or memory usage that have to run outside the context of the CLR.  NCover uses this capability to do code coverage analysis, so that we can get reports of which lines of code in our platform are not being touched by our unit tests.

TypeMock is also a profiler, which causes some interesting interaction.  TypeMock uses the profiler API to insert itself between calling code and the ultimate target of that call in order to "mock" or pretend to be the target.  We use this to reduce the dependencies that our unit test code requires.  Rather than installing SQL Server on our build box, we can use TypeMock to "mock" the calls to the database, and have them return the results we expect without actually calling SQL. 

So...the problem all this interaction led to re: my build was that in order to make everything work, the profiler(s) have to be active before any of the assemblies you want to test/profile are loaded.  I had my UnitTest target set up like this:

 

<Target Name="UnitTestCoverage" DependsOnTargets="Compile;RemoveTestResults">

 

    <CreateItem Include="$(BuildDir)\\**\*.MBUnit.dll" >

      <Output

              TaskParameter="Include"

              ItemName="TargetFiles"/>

    </CreateItem>

 

    <TypeMockStart Link="NCover" />

 

    <Exec Command="&quot;$(NCoverConsole)&quot; $(MBUnitConsole) @(TargetFiles->'%(FullPath)', ' ') /rt:xml /rf:$(TestResultsDir) /fc:unit //l $(TestResultsDir)\NCover.log //x $(TestResultsDir)\NCoverage.xml //a $(AssembliesToProfile)" ContinueOnError="true"/>

 

    <TypeMockStop/>

    ...

</Target>

with the dependencies set to that the compile and "RemoveTestResults" targets would be evaluated first.  Unfortunately, this caused the whole build to hang for upwards of an hour when this target started, but after the compile and "remove" targets had run.  I'm theorizing (but haven't confirmed) that is is because the compiler loads the assemblies in question during the build process, and they don't get unloaded by the time we get to starting TypeMock.  That apparently means a whole bunch of overhead to attach the profilers (or something).  What finally ended up working was moving the compile target inside the bounds of the TypeMock activation, using CallTarget instead of the DependsOnTargets attribute.

 

    <TypeMockStart Link="NCover" />

    <CallTarget Targets="CleanCompile;RemoveTestResults"/>

 

    <Exec Command="&quot;$(NCoverConsole)&quot; $(MBUnitConsole) @(TargetFiles->'%(FullPath)', ' ') /rt:xml /rf:$(TestResultsDir) /fc:unit //l $(TestResultsDir)\NCover.log //x $(TestResultsDir)\NCoverage.xml //a $(AssembliesToProfile)" ContinueOnError="true"/>

 

    <TypeMockStop/>

This works just fine, and doesn't cause the 1 hour delay, which makes things much easier. :-)
Monday, 18 December 2006 12:27:20 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  | 
# Tuesday, 11 July 2006

One of the things the has irked my about using SVN with VisualStudio.NET is trying to set up a new project.  You’ve got some new thing that you just cooked up, and now you need to get it into Subversion so it doesn’t get lost.  Unfortunatley that means you have to “import” it into SVN, being careful not to include any of the unversionable VS.NET junk files, then check it out again, probably some place else, since Tortoise doesn’t seem to dig checking out over the existing location.  Big pain.

Along comes Ankh to the rescue.  I’ve been using it off and on for a while (version .6 built from the source) but now I’m hooked.  It adds the traditional “Add to source control” menu item in VS.NET, and it totally did the right thing.  Imported, checked out to the same location (in place) and skipped all the junk files.  Worked like a charm.  I’m definitely a believer now.

Tuesday, 11 July 2006 10:46:49 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [3]  | 
# Wednesday, 05 July 2006

So at TechEd, Scott captured Jeff and I talking in the friendliest of fashions about the relative merits of Team System (which Jeff evangelizes) and the “OSS solution” which I helped craft at Corillian involving Subversion, CruiseControl.NET, NUnit, NAnt, et. al. 

Since then, I did a bit for PADNUG about the wonders of source control, and it caused me to refine my positions a bit. 

I think that in buying a system like Team System / TFS (or Rational’s suite, etc.) you are really paying for not just process, but process that can be enforced.  We have developed a number or processes around our “OSS” solution, including integrating Subversion with Rational ClearQuest so that we can relate change sets in SVN with issues tracked in ClearQuest, and a similar integration with our Scrum project management tool, VersionOne.  However, those are policies which are based upon convention, and which we thus can’t enforce.  For example, by convention, we create task branches in SVN named for ClearQuest issues (e.g. cq110011 for a task branch to resolve ClearQuest issue #110011), and we use a similar convention to identify backlog items or tasks in VersionOne.  The rub is that the system depends upon developers doing the right thing.  And sometimes they don’t. 

With an integrated product suite like TFS, you not only get processes, but you get the means to enforce them.  In a previous job, we used ClearQuest and ClearCase together, and no developer could check in a change set without documenting it in ClearQuest.  Period.  It was not left up to the developer to do the right thing, because the tools made sure that they did.  Annoying?  Sure.  Effective?  Absolutely.  Everyone resented the processes until the first time we really needed the information about a change set, which we already had waiting for us in ClearQuest. 

Is that level of integration necessary?  We’ve decided (at least for now) that it’s not, and that we are willing to rely on devs doing the right thing.  You may decide that you do want that level of assurance that your corporate development processes are being followed.  All it takes is money. 

What that means (to me at least) is that the big win in choosing an integrated tool is the integration part.  Is the source control tool that comes with TFS a good one?  I haven’t used it personally, but I’m sure that it is.  Is it worth the price if all you’re looking for is a source control system?  Not in my opinion.  You can get equally capable SCC packages for far less (out of pocket) cost.  It’s worth spending the money if you are going to take advantage of the integration across the suite, since it allows you to not only set, but enforce policy. 

I’m sure that if you choose to purchase just the SCC part of TFS, or just Rational’s ClearQuest, you’ll end up with a great source control tool.  But you could get an equally great source control tool for a lot less money if that’s the only part you are interested in. 

The other thing to keep in mind is that the integrated suites tend to come with some administration burden.  Again, I can’t speak from experience about TFS, but in my prior experience with Rational, it took a full-time adminstrator to keep the tools running properly and to make sure they were configured correctly.  When the company faced some layoffs and we lost our Rational administrator, we switched overnight to using CVS instead, because we couldn’t afford to eat the overhead of maintaning the ClearQuest/ClearCase tools, and none of us had been through the training in any case.  I’ve heard reports that TFS is much easier to administer, but make sure you plan for the fact that it’s still non-zero.

So, in summary, if you already have a process that works for you, you probably don’t need to invest is a big integrated tools suite.  If you don’t have a process in place (or at least enough process in place) or you find that you are having a hard time getting developers to comply, then it may well be worth the money and the administrative overhead.

Wednesday, 05 July 2006 15:54:38 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [2]  | 
# Thursday, 22 June 2006

Next Wednesday evening, I’ll be talking about the wonders of source code control at this month’s PADNUG meeting.  If you’re currently using SCC, you might get some tips for making better use of it.  If you aren’t, you’ll find out why you should be, even if you work alone.  I’ll be focusing on the role of SCC in agile development and continuous integration.  I’ll also talk about some of the many version control systems available, and which one(s) may be right for your environment (as well as which ones you really shouln’t be using:  you know who you are…). 

And there’s pizza!

Thursday, 22 June 2006 15:33:23 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Monday, 23 May 2005

Several times recently I’ve been bitten by “code-freeze”.  Granted, I’ve been bitten because I wasn’t tracking the “freeze” closely enough, my bad.  But there are alternatives to this (in my mind) counter productive process.  Developers need to be free to check in changes.  They fix bugs, or add new features, and need to get those things checked in.  Their dev machine could collapse catastrophically, they could forget to check them in, or the fixes could become irrelevant if they wait too long.  Plus, the farther you get away from the point where you made the changes, the more likely you are to get merge conflicts.  If the “freeze” lasts too long, multiple people may make changes to the same files, so that when the freeze is lifted, merge conflicts ensue, and that’s not fun for anyone.  Not good for continuous integration.  Code freezes pretty much take the continuous right out of it.

The solution, you ask?  Labels.  I think most people don’t realize how many groovy things you can do with good labeling policy.  Part of this stems from the widespread use of Visual SourceSafe.  If you are still using VSS, stop.  VSS and continuous integration just don’t belong in the same sentence.  Labeling doesn’t work right, branches are hard and have lasting consequences, etc.  The common excuse is “it comes for ‘free’ with my development environment”.  Of course, it’s not really “free” but that’s another issue.  Even if it were, there are several (much better) free alternatives.  CVS and Subversion top the list.  Much better choices, since they do labeling and branching right, and they discourage the exclusive check-out, which is pure evil anyway.  (Can you tell I have an opinion or two on this topic?)

What I have done (quite successfully) in the past is to use labels to “freeze” the tree rather than actually preventing people from making check-ins.  It runs something like this…  Every developer is responsible for checking in changes, and for deciding when those changes are stable.  It’s a common mistake to assume that the tip or HEAD of your source tree must always be pristine.  That keeps people from checking things in when they should.  So, decide on a label (like “BUILDABLE”) that represents that stable state.  That way developers can continue to check in changes, even potentially destabilizing ones, on the tip without causing havoc.  When the developer decides that his/her changes are stable, he or she can move the label out to encompass those changes. 

How does this work with continuous integration?  Your build server always builds from the label you’ve chosen to represent stability.  Not from the tip.  That way you should always get the latest stable build, even if that doesn’t represent the latest changes.  When it’s done successfully building, the build server should label whatever it just built with something like “BUILT” and the time/date (in another label).  Remember, each revision can have as many labels as you need, so extras don’t hurt. 

Another big benefit of this process is that if someone is just joining the team, or needs to build the software for the first time, they can pull from the BUILT label, and be assured that they are getting the most recent version that is known to be build-able. 

The end result is that you have two labels that move, leapfrogging each other as new things get marked as stable and then built, and some that don’t move indicating what was successfully built for each date/time/version.  That’s good information to have.  It’s also good for developers to be able to freely check in changes without having to worry about “freezes” or breaking the build.

Give it a shot.  It’s work up front, and requires a little training and discipline, but you’ll be glad you made the effort.

Monday, 23 May 2005 10:58:15 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Wednesday, 27 April 2005

Just in case you haven’t heard yet, Scott Hanselman and I will be at tonight’s PADNUG meeting rapping about continuous integration and how we do our builds at work.  (That’s probably rapping in the 70’s fireside chat sense, rather than the Eminem kind, but you never know what could happen.)

The meeting’s at:

Portland Community College Auditorium
CAPITAL Center, Room 1508
18640 NW Walker Rd.
Beaverton, OR 97006
Directions

Wednesday, 27 April 2005 11:48:28 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Tuesday, 30 November 2004

We’ve recently switched to the latest version of CruiseControl.NET (0.7) and my favorite new feature is the ability of ccnet to deal with CVS directly.  Previously we had to include code in our NAnt build file to do a CVS update at the beginning of the build, then do the CVS tag (so we can tag all the files with the build version) at the end of the build if it was successful. 

The new ccnet will do the update and the label for us, but…

It only supports one format for the labels, which it to allow you to specify a prefix, like “1.0.” and it will increment a number and append it, so you get “ver-1.0.1”, “ver-1.0.2”, etc.  That number resets to 1 every time you restart the ccnet executable.  Hmmm.  What we wanted was to use our previous scheme, which involved the version number we use for our .NET executables (e.g. 1.0.222.3333).  We used the version task from NAntContrib to create that version number on the formula (x.y.monthday.secondssincemidnight). 

Luckily, ccnet .7 provides an interface for the labeling, so you can write your own scheme.  Ours now looks like this…

    [ReflectorType("ourlabeller")]

    public class OurLabeller : ILabeller

    {

        public OurLabeller()

        {

        }

 

        private string majorMinor = "2.0";

        [ReflectorProperty("majorminor", Required=false)]

        public string MajorMinor

        {

            get

            {

                return majorMinor;

            }

            set

            {

                majorMinor = value;

            }

        }

        #region ILabeller Members

 

        public string Generate(IIntegrationResult previousLabel)

        {

            string ver = string.Format("{0}.{1}.{2}",majorMinor,getMonthDay(),calculateSecondsSinceMidnight());

            return ver.Replace(".","_");//it's a label, so no dots...

        }

 

        #endregion

 

        #region ITask Members

 

        public void Run(IIntegrationResult result)

        {

            result.Label = Generate(result);

        }

 

        #endregion

        private int calculateSecondsSinceMidnight()

        {

            DateTime today = DateTime.Now;

            return (today.Hour * 3600 + today.Minute * 60 + today.Second) / 10;

        }

 

        public int getMonthDay()

        {

            DateTime time = DateTime.Now;

            string timeString = string.Format("{0}{1}",time.Month,time.Day);

            return Convert.ToInt32(timeString);

        }

 

    }

So now ccnet will now use our labeling scheme, as long was we stick our new class in an assembly called ccnet.*.plugin.dll.  The config file bit looks like

  <labeller type="ourlabeller">
    <majorminor>2.0</majorminor>
  </labeller> 

We want the version of the assemblies to match the new generated label, so we need to read it in our NAnt buildfile.  CCNET stuffs the label in a property that gets passed to NAnt called ccnet.label, so we can read that in our NAnt build…

  <if propertyexists="ccnet.label">
   <script language="C#">
    <code><![CDATA[
    public static void ScriptMain(Project project) {
     //Shorten the project string (like 1.3.4.5, to 1.3.4)
     string projectVersion = project.Properties["ccnet.label"];
     project.Properties["project.version"] = projectVersion.Replace("_",".");
    }
   ]]></code>
   </script>
  </if>

Tuesday, 30 November 2004 16:22:47 (Pacific Standard Time, UTC-08:00)  #    Disclaimer  |  Comments [0]  | 
# Friday, 23 July 2004

Our continuous integration efforts came crashing down around our ears yesterday.  Our build started failing on Wednesday evening, after someone checked in a ton of changes all at once (not very continuous, I know).  That turned into a snowball of issues that took Scott and I all day yesterday to unravel.  Even after we'd fixed all the consequences of such large code changes (that was done maybe by noon) things continued not to work.  To make matters more dicey, at some point during the day our CVS server went TU, and had to be rebooted.  Builds continued to fail, mostly with weird timeout problems.  We were also seeing lots of locks being held open which was slowing down the build, contributing to the timeouts, etc. 

We ended up building manually just to get a build out, which was suboptimal but necessary. 

Thankfully, Scott was able to track down the problem last night.  Turns out that when the CVS box went down, our build server was in the middle of a CVS "tag" operation, a.k.a. labeling.  That left a bunch of crud on the CVS box that meant that subsequent tagging operations failed miserably, thus causing the timeouts, etc.  A few well placed file deletions on the CVS server cleaned things up, and we're (relatively) back to normal now. 

While I think that continuous integration is a fabulous idea, it's times like these that bring home just how many things have to go right at the same time for it to work.  What a tangled web we weave. :-)

Friday, 23 July 2004 10:54:00 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Monday, 12 April 2004

If you have any interest in builds or code maintenance (like I do) check out vil.  It's a code metrics tool that works directly on .NET assemblies.  The interesting thing (IMHO) about that approach is that when it gives metrics for lines of code, complexity, etc. that's all with regard to IL, and not the C# or VB.NET or whatever that you wrote in the first place. 

That makes sense when you consider all the stuff that get's compiled out, although particularly with things like lines of code it may be misleading.  Particularly in C#, where keywords can expand out to quite a bit of code.  Most of the other metrics still make sense with regard to IL, and of course even lines of code is useful when used for comparison rather than absolutist purposes. 

Monday, 12 April 2004 15:20:40 (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  |