RaceReplay.net released

RaceReplay.net is 'live'... well, in beta form anyway.

Turns out Silverlight is a great medium for .NET developers to get on the RIA bandwagon. Flash always had a big learning curve - probably due to it's "designer-focused" background - but Silverlight and Xaml just seems to 'make sense' (to me, at least).

OT: Russ just pointed me to Flickrleech - waaay cool viewer for flickr photos.

EDIT: Silverlight vs. Flash: The Developer Story is an interesting point-of-view... although it was a loong time ago that I last developed anything in Flash, certainly they Silverlight platform makes a lot more "sense" to me.

More Silverlight

First, thought this platform comparison (Silverlight v Flash) was interesting - not sure how complete it is thought (and it's a bit early to claim Silverlight has a "Robust video publishing tools and third-party ecosystem" but Flash does not. Thought the SEO angle was interesting (wonder how many engines do follow links to "Xaml-behind", and not sure 'DRM' is a feature everyone likes.

Anyway, the saga of RaceReplay continues... after getting race splits (mostly) working I decided to give Silverlight a 'stress test' by loading all 400 competitors into the simulation (warning: the Xaml-behind is 7Mb, so loading may take a while).

The first thing that came to mind was "that Xaml is too big!". One way to solve this will be to send the animation data via JSON and pump Xaml from Javascript into Silverlight - pretty sure that will be a lot less verbose than repeated

While googling for other options, it turns out Silverlight also supports the ZIP format. The post suggests using it to package images, but ZIPping up Xaml (text/xml) would surely show much better compression gains - my 7Mb animation canvas compresses to 576kb, so now all I need to do is figure out if it's possible to extract and inject it into the Xaml DOM. Will see how that goes - the City2Surf has 60,000 runners (wonder how close Silverlight will get to animating those...)

Playing with Silverlight...

After mapping race courses, the next logical step seemed to be visualizing the races themselves, similar to BBC Tour coverage (link thanks to Russ) and at least one overseas marathon I've seen online (but can't remember which).

This 10k race demo shows how it could work, using Microsoft Silverlight beta 1.0 and Microsoft Expression Blend.

Got some other Silverlight ideas... stay tuned.

EDIT: Now calculates km markers automatically, and shows altitude as well. Now it really needs to support split times rather than race-average (so the 'runners' slow down on the hills!)

EDIT2: Although some 'matrix glitches' are present, km splits are now supported. Watch red catch green in the 2nd lap! The 'jumping' is due to a bug when a line segment is "split" to accomodate an intra-segement marker... kinda know where it is but too late (at night) to fix it now...

Linqaroo I

Playing with 101 LINQ samples and Searcharoo...

The simplest query (say, for the word "recent") over the existing 'index' looks like this

var x = from wf in catalog.WordFiles
        where wf.Text == "recent"
        select wf;

Digging deeper into the HashSet introduced in .NET 3.5, boolean search queries (which took ages to discuss & code in Searcharoo 2) can be easily described in Linq


so that a simple method call


will produce the these results (Console Apps - the easiest way to play with any new tech :)


Of course there's little indication of well it performs, or how it will work in the code itself (no query tree parser yet, so some work still to do)... maybe i4o - Indexed LINQ will come in handy, or at least provide some ideas for proper Searcharoo-backed Linq classes.

EDIT 30-Apr: It seems possible to play with the underlying structure of LINQ in .NET 2.0, although there's a bit more work to do exploring how robust the implementation is and how good the 3rd party (Hash)Set class is.

Searcharoo supports Medium Trust (and Office 2007)

The Searcharoo free ASP.NET search engine has a new version, which
supports searching on websites under Medium Trust hosting, and indexing Microsoft Office 2007 OpenXML files.

Slightly OT, the Searcharoo v4 article (the first version of the free ASP.NET search engine to support indexing Microsoft Office documents) narrowly lost March's favourite C# article on CodeProject.com 78 votes to 73...

Searcharoo.net gets a facelift...

The Searcharoo.net website has a new skin... much less 'fluro' than the old one (below :)



and a new, improved RecipeNow.net is coming soon...

Compact Framework... link drop

It's been a while since I've worked on CE/mobile, but getting back into it. All the stuff that's new/changed since my last 'real' mobile app makes for interesting reading...

What's New for Developers in Windows Mobile 6

Windows Communication Foundation (Compact Edition) is a great read.

OpenNETCF was an essential reference last time I checked...

Synchronization Strategies: email?!

SQL Server CE - latest name for SQL Everywhere; no it doesn't just run on Windows CE...

'Declarative' Google Maps - store locator sample

Just added another 'sample' app:
Store Locator: Help customers find you with Google Maps to CodeProject.

I started playing with GMapEZ declarative Google Maps API in Feb, originally just for mapping 'race' courses in Sydney for my running club. Later adapted it for a couple of 'proof of concept' samples related to mapping customer data and delivery truck routes (using Google's driving directions feature).

As well as the article, you can play with the NeedCoffeeNow sample online.

Open DOCX using C# to extract text for search engine

Parsing text-content out of different file formats for Searcharoo (or other search engines) can be accomplished a number of ways, including writing your own parser (eg. not too difficult for Html) or using an IFilter loader.

However there's always going to be new document types or formats where you want to build a custom parser... and today the new Word 2007 DOCX format is an example: I don't have Word 2007 installed on my PC so I doubt there's any IFilter implementations for it lying around here either.

A bit of background: the DOCX format is basically a ZIP file containing a directory-tree of Xml files, and from what I can gather the main body of a (Word 2007) DOCX file is located in word/document.xml within the main ZIP archive.

Using a .NET ZIP library based on System.IO.Compression it's relatively simple to open a DOCX file, extract the document.xml and read the InnerText, like this:

using System;
using System.IO;
using System.Xml;
using ionic.utils.zip;
... your code to populate the DOCX filename here ...
using (ZipFile zip = ZipFile.Read(filename))
{
    MemoryStream stream = new MemoryStream();
    zip.Extract(@"word/document.xml", stream);
    stream.Seek(0, SeekOrigin.Begin); // don't forget
    XmlDocument xmldoc = new XmlDocument();
    xmldoc.Load(stream);
    string PlainTextContent = xmldoc.DocumentElement.InnerText;
}

If you're using NET 3.0, the System.IO.Packaging.ZipPackage class is probably a better bet than the open source ZIP library for 2.0.

Now to do some reading on XLSX and PPTX formats...

Don't forget SQL Server 2005 'Compatibility Level' (ARITHABORT error on Xml query)

Working on a database recently that has been 'upgraded' from SQL Server 2000 to 2005. As part of the process, I added some 'Tools' stored procedures I commonly use (eg. ScriptDiagram).

For "some reason" that wasn't immediately obvious to me, I could execute these procs without trouble in SQL Server Management Studio BUT when I added some C# code that referenced a '2005-feature' stored procedure (such as using the new xml column.query syntax)...

SELECT failed because the following SET options have incorrect settings: 'ARITHABORT'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or query notifications and/or xml data type methods.

I started off trying to debug the C# code, thinking I'd done something wrong there... but then realised the Database Properties - Options - Compatibility Level must still be set to SQL Server 2000 (80). Changing that to SQL Server 2005 (90) got rid of the SELECT failed error... because I hadn't really 'upgraded' the database at all, just restored a 2000 backup into 2005.

Why the queries work in Studio still confuses me a little bit... but I'll worry about that some other time. Hope that helps someone - I didn't find anything useful on my first google for the error message.

Searcharoo C# search engine indexes Word, PDF and more...

The latest version of Searcharoo (4) is now available for download (and read about, of course) --
C# search engine: refactored to search Word, PDF and more.

The coolest feature - searching Microsoft Office docs (Word, Excel, PowerPoint) and Acrobat/PDF files - is based on someone else's article Using IFilter in C#.

Also put a bit of work into refactoring the code structure, layout, naming, etc. At least now it might pass a "code review" (if anyone cared to conduct one:)

The project now also has it's own website www.searcharoo.net - the latest article isn't quite there yet (requires some formatting) but over time I'll post more frequently to that site; it takes forever to write, format and upload decent, CodeProject-quality work...

Parsing DBP (Visual Studio Database Project) files

I was surprised how few people appeared to be trying to 'parse' Visual Studio Database Projects (*.dbp files). Why is it useful? If you want to construct a database from scripts in a project, you want to get exactly those scripts run - not any others that might be lying around the filesystem because Studio was too lazy to 'delete' them when they were removed from the Project (or perhaps they were retrieved from VSS).

Captain LoadTest had the same issue in 2005, and developed a
NAnt script to parse VS.Net database project files, which is a nice idea - but I really wanted to access the 'tree' of scripts from C#.

Thankfully, the open-source project AnkhSVN (which integrates the Subversion source control tool into Visual Studio) has exactly the code needed: ParsedSolution.cs and ParsedSolutionItem.cs.

I commented out some code ('context') and added this method, and that pretty much worked. Make sure you check the licence before using...
/// <summary>
/// Borrows code from the solution parser to JUST parse Database Project Files
/// </summary>
/// <param name="projectFileName">The full path to a *.DBP file</param>
public static ParsedSolutionItem OpenDatabaseProject(string projectFileName)
{
string projectFile = projectFileName; //GetProjectFile(projectFileName);

ParsedSolutionItem project = new ParsedSolutionItem();
project.Name = "Database"; //projectName;
project.FileName = projectFile;

if (System.IO.File.Exists(projectFile))
{
using (StreamReader reader = new StreamReader(projectFile))
{
string extension = Path.GetExtension(projectFile);
switch (extension)
{
case ".dbp":
ParseDatabaseProject(project, reader);
break;

default:
throw new ApplicationException("Trying to parse unknown project of type " + extension);
}
}
return project;
}
else
{
return null;
}
}


How to program it...

This brief 'essay' on developing a functional algorithm How to program it... is based on a 1945 book describing methods of problem solving.

The four basic steps are:

1. First, you have to understand the problem.


2. After understanding, then make a plan.


3. Carry out the plan.


4. Look back on your work. How could it be better?

I think it should be required reading for all developers - even pasted on your cubical wall!

Particularly when hiring, one of the "metrics" I try to measure in candidates is their sense of 'logic'. This is easier said than done, however; and actually what I am really interested in their natural ability or inclination to follow such a process.

It's easy to spot less 'logical' developers. They start at (and sometimes only ever do) Step #3! Sometimes, in the process of thrashing about in Step #3 they may accomplish some of Step #1. While fixing the inevitable bugs, they may also accidentally get some Step #4. But it's not an efficient process...

The best developers are built not bought

The best developers are built not bought

At the very bottom of Bloom's taxonomy of educational objectives for cognitive skills is knowledge. That is recall (or recognition) of data or information. Applied to development terms this means recognizing the C# syntax or recalling the correct syntax to execute a loop.

At the top of the cognitive taxonomy is synthesis and evaluation. Synthesis is to be able to bring together diverse elements to form a new whole solution. Evaluation is the ability to make judgments about the value of ideas, approaches, or materials. Developers need to consistently apply these higher levels of cognitive ability to effectively perform their work.

Most people find the idea expressed as thinking more natural than describing the difference as cognitive process. In general, the more thinking that is being done about higher level ideas, such as how to integrate pieces to form a solution, the better the developer.

Visual Studio 2005 Snippets - literal dollar sign $$$

Been using Visual Studio 2005 snippets since I installed it... double-[tab] is a common keystroke now. Of course, my first attempt to create a Snippet wasn't as simple as it should be thanks to the dollar sign escaping of variables. I wanted a 'standard' VSS region-wrapper as a snippet (yes, I realise I can also add it to our class file template).

#region Page History
/*
 * $History: $
 */
#endregion

but the editor interprets $History: $ as a variable and doesn't display it. Tried Google, but not easy to search for '$' and get useful results... after a few false starts trying to Xml entity encode it (ie. >) which also didn't work, I tried the double-dollar-sign $$ at random - and it worked!

Got Snippets is a great place to start when looking for useful snippets (or ideas/samples to build your own). Some of them are pretty complex.

Some other useful snippet links:

• Code Snippets in Visual Studio 2005 (DotNetJunkies)


• Code Snippet Editor for Visual Basic 2005 (MSDN)


• Snippy - Edit and Create VS 2005 Code Snippets (GotDotNet)

Generics, Serialization and NUnit

Latest article Generics, Serialization and NUnit posted on CodeProject.com. The googling I've done so far hasn't turned up much info on Unit Testing and Generics... seems to me that any time you write a class, you want to be able to test it, and Generic classes should be no exception; but there's not much support in NUnit for it at the moment...

NCover 'connection not established' error

I recently tried to set-up NCover 1.5 with a .NET 2.0 project on a new PC, but could not get past the error message Profiled process terminated. Profiler connection not established... the BAT file I was using looked like this (and was working fine on the build server)
@echo off
C:\DevTools2\NCover-1.5\NCover.Console.exe "C:\DevTools2\nunit-2.4.b1\bin\nunit-console.exe" "C:\Projects2\APPNAME\UnitTests\APPNAMEUnitTests.nunit" //a "Common.APPNAME.DataAccess;Common.APPNAME.Common; Common.APPNAME.RulesEngine;Common.APPNAME.Client" //ea "Common.CodeCoverageExcludedAttribute" //w "C:\DevTools2\nunit-2.4.b1\bin" //l "C:\Projects2\APPNAME\Build\NCover\NCover.log.txt" //x "C:\Projects2\APPNAME\Build\NCover\NCover.xml"

Of course, because NCover was working OK on other .NET 1.1 projects, I immediately assumed it was a 'framework' issue and spent fruitless time trying to 'figure out' why NCover couldn't "connect" to my application.

I gave up, only to be embarrassed a day or two later when this
NCover Error Message thread: Profiled process terminated. Profiler connection not established gave me the simple solution: I needed to register one of the NCover components (presumably so it can hook in to the CLR instrumentation or whatever)
regsvr32 CoverLib.dll

Doing an install of NCover from the .MSI would also have worked, but I've gotten so used to .NET applications being able to be 'xcopied' that I had just moved the NCover files from one PC to another and expected them to work.

atlas:UpdatePanel Cancel/Abort UpdateProgress

Thanks to this blog Trabalhando com o ASP.NET Atlas Framework - Parte III for finally giving me the hint on how to add a Cancel/Abort option to the atlas:UpdatePanel UpdateProgress template.

Just add an <input runat="server" type="button"> <asp:LinkButton id="abortButton" runat="server">

The trick is id=abortButton! You don't need to add any further client or server code/jscript/whatever. The presence of that control id is enough. When the UpdateProgress is displayed, you can click to cancel the Ajax XmlHttpRequest.

Why was this so difficult to figure out? I guess because if you use the Designer to create the ProgressTemplate it adds the button for you... but i was creating the controls in Source view, and it was NOT immediately obvious how the Abort/Cancel should be implemented.

EDIT 22-Feb-07 This post is out-of-date; the released version of ASP.NET AJAX doesn't work this way... just one of the breaking changes - check the Migration Guide: Converting Applications from “Atlas” CTP to ASP.NET AJAX RTM

,NET 3.0!

How did I not already know this... .NET Framework 3.0 will be released with Vista, and available for Windows XP and Windows Server 2003.

Interestingly (I think) it will just compile with the existing FX2.0 CLR, so I guess WinFX is just a mass of managed code? Still doesn't include LINQ or ObjectSpaces, so for database-focussed (ie. business) applications, what benefit does the renamed WinFX provide?

OT: great discussion-starter on Who needs value types anyway?, and this on .NET Framework 3.0 Naming...

RefreshPanel: precursor to Atlas

The Atlas UpdatePanel control seems to have it's genesis in the April 2005 RefreshPanel on gotdotnet.

The UpdatePanel has it's own 'book lite' on OReilly.com - USD9.99 doesn't seem to bad, although the fact they want to charge 'delivery' for overseas customers is very frustrating! The PDF itself is like an extended CodeProject article...