# Wednesday, July 30, 2003

When new data comes in from monitors, it needs to be processed and then stored in the DB.  (See Xml on Large Power Transformers and Industrial Batteries for some background.)  I’m currently pondering how to manage that, and what I’m thinking is that I’ll process the incoming data as XML, using XPath and some strongly typed methods for doing inserts.  The issue I’m wrestling with is how to get the XML data into the database.  We’re currently using SQLXML for some things, so I could style it into an updategram.  Initially I was thinking of just chucking the whole incoming XML doc into a stored proc, and then using SQL Server 2K’s OPENXML method to read the document inside the stored procedure and do all the inserts from there.  The advantage is that I can rip up the doc and do a bunch of inserts into normalized tables and keep it all in the context of one transaction in the sproc.  Also, it keeps me from having to write C# code to parse out all the stuff that’s in the doc and do the inserts from there (although that’s also an option).  Usually I’m opposed to putting much code into the DB, since it’s the hardest bit to scale, but in this case it wouldn’t be business logic, just data parsing, which seems like a pretty reasonable thing for a database application to do. 

With that all out of the way, my concern is that OPENXML is fairly complex, and would require some relatively involved TSQL (which I can do, but would rather not, and hate to maintain).  Also, I worry about it all being made irrelevant by Yukon, which is supposed to have some pretty wizzy new XML related data storage stuff.  

Another option would be to style the incoming data into a dataset and use ADO.NET for the dirty work.  Since I’m really only ever doing inserts, not updates, it should be pretty straightforward. 

Sigh.  Too many choices.

Wednesday, July 30, 2003 7:27:59 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [2]  | 
# Friday, July 18, 2003

It’s hard to tell how much of this is an unfinished documentation issue, but in the WSE 2 Tech Preview, there doesn’t seem to be any easy way to do encryption using symmetric keys.  The sample code in the documentation is that same as in WSE1, and it doesn’t compile any more, since the version of the EncryptedData constructor that takes an EncryptionKey is obsoleted.  Now you have to pass a SecurityToken, but there isn’t one for symmetric keys.  This seems like a pretty straightforward thing to want to do, so I don’t really want to write a custom binary token to do it.  There are some hints in the documentation that there is a plan for a symmetric key token (or maybe I just haven’t figured it out yet).   Right now this is a total pain, since I have a bunch of existing WSE1 code that uses symmetric keys.  If nothing better happens by the time WSE 2 is released, I may have to write a custom binary token anyway.  It’s nice that they’ve put in so much work to make Kerberos and X509 easy, but since my client is an embedded device, those aren’t really viable choices. 


Friday, July 18, 2003 2:03:25 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Tuesday, July 15, 2003

I know this is a bit after the fact, but I’ve had time to reflect.  Steve gave a very interesting talk on the feasibility of using SOAP/Web Services with embedded devices, which in his case was a printer.  I’m currently also working on an embedded Web Services project, so I was really interested in what Steve had to say.  His conclusion was that although there are some issues to be overcome, it is in fact feasible to use Web Services with an embedded device.  In the work I’ve done so far, I’d have to agree. 

The really interesting part for me what that the challenges Steve talked about with his printer were completely different ones from the challenges I’m facing talking to transformer monitors.   It sounded like the biggest hassle with using WS on the printer was dealing with large binary files as attachments (since it was actually a 3D printer, and 3D models can be very large).  There are several different ways of dealing with attachments to SOAP, and they all have their challenges. 

Our biggest challenge, on the other hand, is that since our monitors are meant to work over the WAN, we can’t assume connectivity to them all the time, which you probably can in a printer.  Also, it sounded like using Java on the printer wasn’t too big a problem, while we are working with straight C++ in a very constrained environment, for which there are many fewer tools available.

The short answer then to “is it feasible” is of course, “it depends”.  It worked for Steve.  So far it’s working for me.  I’ll keep my fingers crossed.

Tuesday, July 15, 2003 4:21:04 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [1]  | 
# Monday, July 14, 2003
A few hearty souls stuck it out until the bitter end to hear me ramble about XML on Large Power Transformers and Industrial Batteries. I got some good questions, and a couple of cool bits of feedback. I spent some time talking about the fact that one of the reasons we decided to go with XML Web Services in the embedded space was to take advantage of infrastructure implementations arising from GXA. For example, I don't have to wonder about how to deal with security of Web Services, because there's WS-Security, and better still in the future I'll be able to get a WS-Security appliance in the same way that I can get an SSL appliance today. Rich Salz, who gave the talk right before mine (Determining what Security your Web Services need) mentioned that his company makes such a thing today. Hooray! Very cool. With any luck, in the future I'll be able to get WS-Routing appliances also.
The other thing I talked about was the fact that our model is a little different from the usual Web Services models (Server to Server, or Smart Client) since we have to deal with the fact that our monitors aren't reachable (they have to call us) and that they can't maintain state about what data they've sent us. Given that fact that our monitors are meant to give utilities the information they need to keep their transformers from blowing up, Tim Ewald suggested that we call it the Exploding Client model. :)
Maybe it'll stick...
Monday, July 14, 2003 1:04:00 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [1]  | 
Aaron demonstrated some very cool techniques using XPath assertions to validate XML documents (specifically the contents of SOAP messages). He uses SOAP extensions and attributes that indicate which XPath assertions to apply to a given WebMethod. It apparently is much faster than doing full schema validation, and easier to set up, since most of the time you really only want to assert a few key requirements and don't need to validate the whole data structure exhaustively. I hadn't heard or Schematron before, but apparently is uses a similar technique using XPath assertions. There is a .NET implementation of Schematron here.
Monday, July 14, 2003 12:59:17 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Friday, July 11, 2003
I got a question yesterday about what was up with the formatting on my blog? How come it jumps around?
Well, with some help from Mike Amundsen at EraBlog.net where my blog is hosted, I'm now hosting that pages (but not the data) at my site. The permalinks on EraBlog now redirect to my site (http://www.cauldwell.net/patrick/blog) so if you use that link to get at my blog rather than /blogs/pcauldweblog the formatting will remain consistent.
For anyone interested in the implementation details, I'm using ASP.NET, using EraBlog's toolkit which makes SOAP requests back to EraBlog.net to get the data for my blog, then formatting it on my pages. Check this out for more details...
Friday, July 11, 2003 6:32:52 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
The sweet spot accoring to Tim:
Write schemas with global element declarations using anonymous types. To process them, wrap them in something that carries a DOM, exposes the Navigator interface, and give you type safe methods for modifying the documents.
This turns out to be very similar to some stuff I've been thinking about for processing/pipelining XML documents. More on this later...
Friday, July 11, 2003 5:48:39 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
Amazon gets Web Services. Jeff Barr is talking about how and why Amazon implements Web Services, and they really do it right. They have a real business model and Web Services (both SOAP and REST) that fit that business model and allow both Amazon and their associates to make money.
Their Web Service model is really just an extension of their earlier associate program, and basically just gives associates way more control over the look of their site and additional features they can support.
What I find really interesting is that there's a whole economy building up around this technology. There are even solutions available such as StorePerfect, which is a set of ASP.NET controls that wrap the Amazon web services, so all you have to do to set up a commerce site using Amazon is use these ASP.NET controls. So if you get a good domain name and do a little ASP.NET coding, you can set up a site that makes you money without having to handle any goods, process any payments, or ship anything. If that's not an appropriate use of Web Services, I don't know what is.
Friday, July 11, 2003 2:25:00 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
I wanted to follow up on the last question that was asked at the speakers panel this evening. To my interpretation, the question was basically "I'm trying to sell some end users on the idea of using XML, should they care that it's XML?" That may not have been the actually question, but I think that's what I heard.
Unfortunately, that's a pretty interesting question, and the answer got totally lost in XML-RPC vs. SOAP / how badly does WSDL suck bickering and I don't think we really answered the guy's question.
My answer (speaking only for myself here) is that absolutely they SHOULDN'T care that it's XML. We as developers care because we have to write implemenations. End users shouldn't care in the slightest if it's XML. If I give my CFO a some data in CSV format, I'm pretty sure he doesn't care that the format is CSV, or that it has delimiters, or any other thing about it. What he cares about is that when he double clicks on the file, it comes up in Excel looking like a spreadsheet.
The problem is twofold: 1) we're not there yet with XML. You can't just double click on an XML file and have it come up in Excel looking like a spreadsheet (unless you're running Office 2003 Beta, which does actually manage to do just that through an amazingly clever combination of shell extensions and processing instructions, but that's an exception) 2) XML received so much hype 2-3 years ago that everyone thought (including end users) that they should care, since it's such a great buzzword.
Those are the things that we as developers have to fix, so that end users will be as blissfully ignorant of XML as the should be (are?) of CSV.
Friday, July 11, 2003 2:15:14 AM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
I have to hand it to Keith: it's pretty impressive to see a guy interactively build a tool during the course of his talk that lets the audience heckle him interactively while he's giving the rest of the presentation.
For those of you who weren't there, Keith used WSE 2 to set up a non-IIS based SOAP server that accepted HeckleData structures and printed them out, then build an ASPX page with a form on it that used the WSE2 SoapClient to send HeckleData messages to the server. Not only that, but he gave out the URL to the form on his first slide, thereby exposing himself to heckling from the wirelessly connected audience.
All that while wearing someone else's belt, and weathering some pretty intense non-SOAP issued heckling.
I saw Keith's presentations on WSE 2 at TechEd also, and I can't wait to get my hands on it.
Friday, July 11, 2003 2:05:02 AM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
As Scott Hanselman mentions I've had some experience with SVG UI's in my current project at Serveron.
While I think SVG is a really cool idea, and amazingly useful for interactive graphics (if done succinctly) we had nothing but trouble trying to craft SVG UI components, which was largely the focus of Don's talk. There were just too many problems with different scripting/DOM models (you can access Adobe's SVG DOM from IE, but you can't get to IE from Adobe, etc.) to make it a long term workable solution. In retrospect I would have suggested (not having written the SVG personally) moving all the UI controls into ASPX and keeping the SVG strictly for graphics display.
While I thought Don's approach to some of the individual controls was cool (I really liked the analog gages), I don't think that SVG, or mor importantly the infrastructure build around SVG at present, is in good enough shape to be building UI's on it.
Scott's correct in that we've essentially scrapped the SVG UI for a WinForms based "Smart Client". We orignally banked on SVG to give us cross platform interactive graphics, but as always, it's not that simple. Adobe's SVG control, which seems to be pretty much the only game in town, doesn't work in Netscape 6, which pretty much kills the whole cross-platform thing, even if we could get the scripting to work.
Friday, July 11, 2003 1:58:03 AM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
# Thursday, July 10, 2003
Brian Jepson did some very cool demos using Perl to convert RSS into WML so he can read blogs while standing in line at the DMV (although not while driving, he was quick to point out). He also used the same technique to wrap Amazon's Web Services so that he can check on how his books are selling on Amazon. Pretty groovy stuff. He also (bless his heart) rewrote the same applications in C# (using Rotor on OS X) so that people can actually read them. :)
You can check out his sources here.
Better still, he demoed RSS.NET, which is a C#/.NET library for reading and writing RSS. Now I can go and fix up my Draco.NET RSS feed, since I unknowingly coded all the RSS by hand.
Thursday, July 10, 2003 5:15:40 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  | 
OK, I'm convinced. I need no further information on the ongoing debate about RSS.
Dave Winer just put forth a really great analogy on why RSS is the way to go. RSS is like a table (a physical one) in that we don't care how tables are made, we don't need a table standard, all we care about is that it holds up our coffee cup. So there it is in a nutshell: RSS holds up our coffee cups.
I must admit I have been from time to time in the strongly-typed, let's have a schema for everything camp but this and other arguments (such as loosely coupled SOAP services) are slowing winning me over to the idea that simple is better and there are good as useful reasons for not trying to specify the hell out of everything.
This makes me want to go back over some of my recent SOAP/Schema work and re-examine it in the light of this new maxim: Does it hold up my coffee cup?
Thursday, July 10, 2003 2:10:50 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [3]  | 
# Tuesday, July 08, 2003

This is a really interesting idea. It’s a .NET remoting channel that uses IIOP as its wire format.  This means that you can use it to interoperate with CORBA/J2EE/JavaRMI systems rather than using XML Web Services to do the same.   It looks like it’s not much more work then implementing Web Services in .NET, and the benefits you would get would be better performance (due to the binary serialization format) and the fact that your clients can take advantage of object references rather than essentially stateless methods exposed as Web Services.  The biggest drawback compared to XML Web Services is obviously that it only works for CORBA/J2EE/JavaRMI systems and not for all the other platforms for which there are Web Services implementations and not IIOP implementations.

Tuesday, July 08, 2003 1:44:24 PM (Pacific Daylight Time, UTC-07:00)  #    Disclaimer  |  Comments [0]  |