I went to a more detailed (possibly too detailed) presentation on the ADO.NET entity model this morning.  This is pretty exciting stuff, although obviously people will have some reservations.  It's easy to dismiss EDM as just another in a long line of O/R mapping attempts, and while that's a valid criticism, I think there are some interesting and instructive differences.  Because EDM is built on top of support all the way down at the ADO.NET provider level, it's more than just an O/R map.  You can define entities (this is done using XML files for now) that span multiple tables, or otherwise hide complex structures in the underlying data store, as well as defining relationships between those entities that express a higher level relationship than a relational foreign key.  These relationships can be navigated at a much higher level, without having to deal with complex joins, etc.  Because that the Entity Model actually passes down to the data provider is an expression tree, the data provider is free to optomize into its native dialect as best it can.  So when you are taking advantage of complex relationships in your entity model, you aren't just fetching a bunch of objects and then making them relate to one another.  That relationship is handled at the provider level in ways that can be optimized, so that you aren't bringing back any more data then necessary. 

Plus, because of the deep support at the provider level, you can specify a query using the simplified entity syntax, but still return a forward only dataset for best performance, skipping the object translation step all together. 

Granted, they still have quite a bit of work ahead of them to improve the tooling, and make it as transparent as possible, but I think there's a lot of value to be had here. 

I'm looking forward to taking some time with the bits and trying out some different scenarios, so hopefully I'll have some good examples later.

I saw some great stuff yesterday afternoon on how to host WPF controls in WinForms and vice versa.  I've been wondering for a while if it would be possible to host WPF controls in MMC 3.0 for some management apps we need to write, and now I know it's at least possible, and possible with Whidbey, although the desinger experience will be better in Orcas for WPF on WinForms.

Jon Flanders did a very good presentation on real world tips and tricks for hosting WF in WCF services, without waiting for Silver (WF/WCF integration in Orcas).  Since our new app does exactly this, I learned some things I'll have to try out, specifically using the ManualSchedulerService.  I hadn't considered the implications of ASP.NET and WF contending for the same thread pool.  The manual scheduler might also simplify some things around identity management and impersonation that I've been puzzling over.  Jon is a very compelling speaker, and I got a lot out of his presentation.

To wrap up the day yesterday I got a preview of what's coming as part of the Acropolis project.  I was totally blown away by how much they've already accomplished, and how thoughtfully they have proceeded.  Acropolis is basically CAB for WPF, but they really took it in a positive direction.  While CAB is (IMHO) too technology focused, Acropolis is very much designed for building real applications quickly, and providing (again IMHO) exactly the right level of support.  I'm a big Smart Client fan, and I think they're really expanding the horizons for building smart client apps quickly and with a minimum of overhead.  If you are interested in build any kind of client app with WPF, be sure to follow their work.

This morning started off with a very good presentation on the new language features in C# 3.0 by Luke Hoban.  He did some very compelling demos to show off the new features, including the LINQ over Objects syntax.  Then, just to show there was no magic involved, he removed his reference to LINQ, and reimplemented the where and select operators using the new language features.  It really showed off the new features, and put them in a context that was very accessible.  Well done!

The rest of the day holds more on Acropolis, and more details of the LINQ over Entities project, which is pretty groovy stuff. 

Chris Anderson's session on using XAML as a modelling language for declaratively expressing programmer intent is easily the best thing I've seen so far.  I've been struggling to grok the beauty that is XAML for a while now, and while I'm not totally sure I've grokked it in its fullness, I'm a good deal closer now.  I think I'll have to play around with it for a while before it's fully cemented.  One thing he mentioned that I'm very interested in pursuing is using LINQ expression trees in non-data oriented ways.  There could be some huge opportunities for building declarative models using this technique.  Much to think on. 

I think this was actully the first time I'd seen Chris speak, and I was very impressed.  Not only by his style, but by his fabulous wardrobe.  Chris eschewed the MS-blue shirt for a fantastic brown and orange Aloha shirt with a huge (possibly embroidered) green and yellow palm tree across the front.  Excellent choice.

As an only vaguely related side note: how far back does the nerd fondness for Aloha shirts go?  Maybe it's just because we're in Florida, but I've seen some truly stunning examples of the genre.  Being a big fan (possibly even proponant) of the Aloha shirt myself, it's been fun checking out the good ones.  Better than birdwatching. 

To bring this back to something slightly more relavent, I think there are several developments underway, including the Software Factories initiative discussed a few Architecture Journals back, the rise of DSL Toolkit, etc. that make model driven development much more feasible, and it will be interesting to see what part XAML has to play there.  Very exciting times.

One more thing about TechEd today...

The food has been much better than last year.  The breakfasts have been pretty lackluster, but the lunches and after hours snacks have been first rate.  Lots of veggies, including some nice pickles and raw veggies at the partner expo last night.  The snacks have also been quite diverse.  I particularly appreciated the afternoon carrot sticks and dip, and not the usual ding-dongs and twinkies to go with your cheetos.  I must say that the Buffalo wings were better in Boston last year, but that's the exception that proves the rule so far. 

Hopefully tomorrow I'll have slept a bit, and have something less trivial to report (although it seems important just now).

More goodness today.  One of my favorites today was a presentation on the new features for building Smart Client applications with "Orcas".  Smart Clients hold a special place in my heart, so I'm glad to see them investing in better support.  Very cool stuff, including the new "SQL Compact Edition" which can run in-proc to your client app, and better still a new "sync provider" that allows you to write your smart client against the local "Compact Edition" database, then do a two way sync with a full SQL database, which provides a super-easy way to deal with offline use of your client app.  Also in Orcas, you'll be able to host WPF in WinForms and vice versa, which makes it easy to both extend and leverage existing applications.

Other highlights from the day include SQL 2008 support for "non-relational" data like files, spatial data, full text documents, etc.  They've really gone out of their way to make it easy to integrate all your data into SQL Server.

One big thing I'm taking way this year is Microsoft's continued commitment to responding to customer input.  In several arenas (SQL, Orcas, ADFS, etc) over and over I keep hearing "you said it was too hard, or not feature rich enough, so we fixed it".  It's really encouraging to see the extent to which real world customer input is having a direct impact on feature direction. 

Both of my BOF sessions went well.  The lunchtime session on NDepend was well attended, and sparked some interesting discussions.  This evening's session on ADAM and AzMan provoked some interesting responses.  One that surprised me was an assertion that MS must not intend to support AzMan because it doesn't get a managed interface in Longhorn Server.  I'm pretty sure I disagree with that statement, since there has been quite a bit of investment in the Longhorn (sorry Server 2008) version of AzMan, but I think it's illustrative of MS's lack of advertisement around AzMan.  It's a great feature, but it goes largely under-represented, so nobody takes it seriously.

Anyhow, busy day, once more into the breach tomorrow...

I've only got a few minutes between sessions, so I'm going to hit the highlights, more details to come later when I can focus a bit. ;)

The keynote was way better than last year's.  The focus this year seems to be much less on developer-related technology, and more on IT, and more specifically on what IT pros can do right now today, rather than any vision for the future.  Bob Muglia set the tone in the keynote by saying that they didn't want to give people any more MSBSVS, or Microsoft BullS**t Vision-Speak.  A very positive direction, and there's plenty to talk about.  I just came away from a session on the next version of the Identity Lifecycle Manger, and I think they're really nailed it.  Identity management in the enterprise is hard, and they've gone a long way towards fixing it.  If I was in IT, I'd be even more excited.

Yesterday I saw talks on the next version of ADFS (Active Directory Federation Services) and again they are headed in the right direction.  ADFS "2" will basically allow you to expose an STS on top of your AD store, with CardSpace support, etc.  Very cool. 

I also saw some stuff on the LINQ "Entity Data Model", which, in summary, is basically Rails-style active record support at the platform level, with some very advanced querying capabilities via LINQ.  This will be very useful in the future for getting rid of DAL code in the middle tier. 

Lastly, I saw an overview of SQL 2008 (Katmai) features.  Again, some very cool stuff, and new features that look like they've been developed in response to user feedback.  Always nice to see.  For all those groovy new mashups, there's a new LOCATION datatype.  Spacial data as first class citizen.  There's also some brilliant management-by-policy features that will make DBA's very happy.

Gotta run.  NDepend BOF at lunch, ADAM/AzMan tonight, so if you're reading this in a timely fashion, stop on by!

I'll be flying out for Orlando way too early on Sunday morning, but I should be recovered by Monday morning, so see you all there...

I finally got around to building some docs for our latest project, and decided to go with Sandcastle rather than NDoc for the time being.  I got the March CTP of Sandcastle and started trying to figure out how to get it working with our build. 

While the resulting CHM file came out looking really nice, I'd have to say that Sandcastle is WAY not ready for prime time.  There is little to no documentation, so figuring out how to customize things is largely an exercise for the user.  I was willing to exercise, but many aren't.   The build itself is very complicated, with 10-12 separate steps required to produce the CHM file, and lots of rough edges, like the fact that the MRefBuilder tool won't look for dependencies in the GAC, so I had to copy a bunch of core BCL assemblies out of the GAC and into my build directory to get the docs to build.  Lame. 

As it stands now, Sandcastle is useful only to serious build weenies, since it's too cumbersome for the average developer to deal with.  This is too bad, since it's being touted (at least in some circles) as the "right" solution for building docs going forward.  It smacks of something that the VS 2005 team built to create there docs, then decided to make public.  It's not quite there yet.

I've picked up another BOF session, BOF09: Exiting the Zone of Pain: Static Analysis with NDepend, scheduled for Tuesday, June 5, 2007 at 12:00 PM, in Room S331 A.  We'll be discussing how and why to use static analysis tools like NDepend.  I'm especially interested in hearing not just how people use tools like NDepend, but who uses them (in their orgainization) and how often.

As a reminder, I'll also be doing BOF51: ADAM and AzMan, later the same day at 6:30 PM, in Room N310 A.  Bring your stories about ADAM and AzMan, how you are using them, or why you aren't.  I think these tools are vastly under-appreciated, so I'm looking forward to spreading the love.

We've been estimating a large new project for the last week or so using the "planning poker" technique.  So far I think it's working out very well, although the biggest learning so far has been that you really need a solid task list before you enter this process.  On backlog items where we already had task lists defined, we didn't have any trouble coming to consensus about how long it would take.  This morning we entered into a backlog item without the task list already in place, and it led to an hour-long discussion that delved into design, etc. rather than getting any estimating done.  While I understand that part of the planning poker process is group learning and making sure everyone is on the same page, we would have made a lot more progress with even a tentative task list rather than none at all.

Now we know.

I was trying to come up with a set of guiding principles as we've been using them on my current project, with an eye toward educating new developers being assigned to the project, so I wrote up a PPT deck for the purpose.  Scott thought it would make a good, general purpose manifesto for any project, so here goes...

These are my opinions based on experience and preference, YMMV.  Much of this is common sense, some of it is arbitrary, but it's all stuff that I think is true.

• Our "guiding principles"
  • Test Driven Development
    • As much as possible, all of our code should be developed according to the TDD methodology.  If we stick with it, we will always have confidence in the current build (i.e. that it really works).
  • Continuous Integration
    • We want our build to produce something that looks as much as possible like a production deployment.  To that end, our build deploys a real live system and runs "integration" tests against it. 
• Unit tests
  • Two kinds of tests
    • Unit tests have no dependencies, or only depend on other assemblies owned by us, which can be mocked.  This means our unit tests should run on anyone's box without having to set up things like SQL, ADAM, etc.
    • Integration tests depend on code not owned by us, like SQL, ADAM, IIS, etc. 
  • Automation is equally possible for both sets of tests
    • NUnit/MbUnit for all of the tests
    • MbUnit allows for test "categories" like "unit" and "integration" and can run one at a time
    • Unit tests run on every build, integration tests run at night since they take much longer to run
  • All UI development should follow the MVP pattern for ease of testing
    • This includes Win/WebForms, MMC snapins, etc.
• Test coverage
  • 90%+ is the goal
    • This is largely achievable using proper mocking
  • NCover runs as part of the build, and reports are generated
• Buy, not build
  • Take full advantage of the platform, even if it only solves the 80% case
    • every line of code we don't have to write and maintain saves us time and money
  • Don't write a single line of code you don't have to
  • Take full advantage of .NET 3.0, SQL 2005, Windows 2003 Server, plan for- and test on Longhorn
  • Don't invent new solutions to solved problems
    • Even if you know how to write a better hashing algorithm than the ones in the BCL, it doesn't add value to your product, so it's not worth your time
• Limit compile time dependencies on code you don't own
  • Everything not owned by us should be behind an interface and a factory method
    • Easier to mock
    • Easier to replace
  • ILoggingService could use log4net today, EnterpriseLibrary tomorrow, but clients are insulated from the change at compile time
• Define your data contracts in C# (think "active record")
  • All persistent storage should be abstracted using logical interfaces
    • IUserProfileService takes a user profile data object, rather than methods that take parameter lists
    • IUserProfileService knows how to store and retrieve User Profile objects in ways important to your application
    • How a User Profile object gets persisted across databases/tables/XML/whatever is only interesting to the DBA, not consumers of the service
    • This means database implementations can change, or be optimized by DBAs without affecting the app
    • The application doesn't care about how database file groups or clustered indexes work, so define the contract, and leave that bit to a DBA
• Fewer assemblies is better
  • There should be a VERY good reason for creating a new assembly
  • The assembly is the smallest deployable unit, so it's only worth creating a new assembly if it means NOT shipping something else
  • Namespace != assembly name.  Roll up many namespaces into one physical assembly if they all must be deployed together.
• Only the public interface should be public
  • Only make classes and interfaces public if absolutely necessary
  • Test code should take advantage of InternalsVisibleTo attributes
  • VS 2005 defaults to creating internal, not public classes for a reason
  • If it's public, you have to support it for ever
• Windows authentication (good)
  • Just say no to connection strings
  • Windows authentication should be used to talk to SQL, ADAM, the file system, etc.
  • You can take advantage of impersonation without impersonating end users
    • Rather than impersonating end users all the way back to the DB, which is expensive and hard to manage, pick 3-4 windows accounts like "low privilege", "authenticated user", and "admin" and assign physical database access to tables/views/stored procs for those windows accounts
    • When users are authenticated, impersonate one of those accounts (low priv for anonymous users, authenticated user for same, and admin for customer service reps, for example)
• Tracing
  • Think long and hard about trace levels
    • Critical is for problems that mean you can't continue (app domain is going away)
    • Error means anything that broke a contract (more on that later)
    • Warning is for problems that don't break the contract
    • Info is for a notable event related to the application
    • Verbose is for "got here" debugging, not application issues
  • Use formatted resource strings everywhere for localization
  • For critical, error, or warning, your audience is not a developer
    • Make error messages actionable (they should tell the recipient what to do or check for to solve the problem)
• Error handling
  • Method names are verbs
    • the verb is a contract, e.g. Transfer() should cause a transfer to occur
  • If anything breaks the contract, throw an exception
    • if Transfer() can't transfer, for whatever reason, throw
    • Exceptions aren't just for "exceptional" conditions.  If they were really exceptional, we wouldn't be able to plan for them.
  • Never catch Exception
    • If it's not a problem you know about, delegate up the call stack, because you don't know what to do about it anyway
  • Wrap implementation specific exceptions in more meaningful Exceptions
    • e.g. if the config subsystem can't find the XML file it's looking for, it should throw ConfigurationInitializationException, not FileNotFoundException, because the caller doesn't care about the implementation.  If you swap out XML files for the database, the caller won't have to start catching DatabaseConnectionException instead of FileNotFoundException because of the wrapper.
    • Anytime you catch anything, log it
      • give details about the error, and make them actionable if appropriate
    • Rethrow if appropriate, or wrap and use InnerException, so you don't lose the call stack
• The definition of "done" (or, how do I know when a task is ready for QA?)
  • Any significant design decisions have been discussed and approved by the team
  • For each MyClass, there is a corresponding MyClassFixture in the corresponding test assembly
  • MyClassFixture exercises all of the functionality of MyClass (and nothing else)
  • Code coverage of MyClass is >=90%, excluding only lines you are confident are unreasonable to test
    • yes this is hard, but it's SOOOOO worth the effort
  • No compiler warnings are generated by the new code
    • we compile with warnings as errors.  if it's really not a valid or important warning, use #pragma warning disable/enable
  • Before committing anything to source control, update to get all recent changes, and make sure all unit and integration tests pass
  • FxCop should generate no new warnings for your new code
  • Compiling with warnings as errors will flush out any place you forgot documentation comments, which must be included for any new code
• There WAS a second shooter on the grassy knoll
  • possibly one on the railway overpass
  • (that's for anyone who bothered to read this far...)

Back in February, I mentioned that I was having problems testing our AzMan code on developer machines running Window XP instead of Windows 2003 server.  This was because the 2003 Server version of the AzMan interfaces weren't present on XP.  To deal with the issue, we actually put some code in our unit tests that reported success if running on XP when the error was encountered, so that everyone could have a successful build.  Drag.

Anyway, it turns out I wasn't the only one having this problem.  Helpful reader Jose M. Aragon says

We are trying to get AzMan into our authorization architecture, and were able to do it nicely, as soon as the development was done on Windows Server 2003. But, when we tried to use IAzClientContext2.GetAssignedScopesPage on a developers machine running XP sp2 it fails miserably with a type initialization. Searching on help, I found your entry. so we started to circumvent the problem by programming our own "GetAssignedScopes" method.

Meanwhile, on another XP workstation, the problem was not occurring. So, it turns out the difference resides in what was the Server 2003 admin pack package installed on every XP machine..

adminpak.exe sized 12.537KB on 24 Jan 2007 : Does NOT include IAzClientContext2 on azroles.dll

WindowsServer2003-KB304718-AdministrationToolsPack.exe sized 13.175KB on 13 March 2007 : allows using IAzClientContext2 on XP

It seems the two files where downloaded from microsoft.com/downloads, but the one on Server2K3 SP1 is the most recent AdminPak.

I uninstalled my original version of the Admin Pack on my XP-running laptop, installed the new one, and now all of our tests succeed without the hack!

Thanks Jose!