Compact Framework... link drop

It's been a while since I've worked on CE/mobile, but getting back into it. All the stuff that's new/changed since my last 'real' mobile app makes for interesting reading...

What's New for Developers in Windows Mobile 6

Windows Communication Foundation (Compact Edition) is a great read.

OpenNETCF was an essential reference last time I checked...

Synchronization Strategies: email?!

SQL Server CE - latest name for SQL Everywhere; no it doesn't just run on Windows CE...

'Declarative' Google Maps - store locator sample

Just added another 'sample' app:
Store Locator: Help customers find you with Google Maps to CodeProject.

I started playing with GMapEZ declarative Google Maps API in Feb, originally just for mapping 'race' courses in Sydney for my running club. Later adapted it for a couple of 'proof of concept' samples related to mapping customer data and delivery truck routes (using Google's driving directions feature).

As well as the article, you can play with the NeedCoffeeNow sample online.

Open DOCX using C# to extract text for search engine

Parsing text-content out of different file formats for Searcharoo (or other search engines) can be accomplished a number of ways, including writing your own parser (eg. not too difficult for Html) or using an IFilter loader.

However there's always going to be new document types or formats where you want to build a custom parser... and today the new Word 2007 DOCX format is an example: I don't have Word 2007 installed on my PC so I doubt there's any IFilter implementations for it lying around here either.

A bit of background: the DOCX format is basically a ZIP file containing a directory-tree of Xml files, and from what I can gather the main body of a (Word 2007) DOCX file is located in word/document.xml within the main ZIP archive.

Using a .NET ZIP library based on System.IO.Compression it's relatively simple to open a DOCX file, extract the document.xml and read the InnerText, like this:

using System;
using System.IO;
using System.Xml;
using ionic.utils.zip;
... your code to populate the DOCX filename here ...
using (ZipFile zip = ZipFile.Read(filename))
{
    MemoryStream stream = new MemoryStream();
    zip.Extract(@"word/document.xml", stream);
    stream.Seek(0, SeekOrigin.Begin); // don't forget
    XmlDocument xmldoc = new XmlDocument();
    xmldoc.Load(stream);
    string PlainTextContent = xmldoc.DocumentElement.InnerText;
}

If you're using NET 3.0, the System.IO.Packaging.ZipPackage class is probably a better bet than the open source ZIP library for 2.0.

Now to do some reading on XLSX and PPTX formats...

Don't forget SQL Server 2005 'Compatibility Level' (ARITHABORT error on Xml query)

Working on a database recently that has been 'upgraded' from SQL Server 2000 to 2005. As part of the process, I added some 'Tools' stored procedures I commonly use (eg. ScriptDiagram).

For "some reason" that wasn't immediately obvious to me, I could execute these procs without trouble in SQL Server Management Studio BUT when I added some C# code that referenced a '2005-feature' stored procedure (such as using the new xml column.query syntax)...

SELECT failed because the following SET options have incorrect settings: 'ARITHABORT'. Verify that SET options are correct for use with indexed views and/or indexes on computed columns and/or query notifications and/or xml data type methods.

I started off trying to debug the C# code, thinking I'd done something wrong there... but then realised the Database Properties - Options - Compatibility Level must still be set to SQL Server 2000 (80). Changing that to SQL Server 2005 (90) got rid of the SELECT failed error... because I hadn't really 'upgraded' the database at all, just restored a 2000 backup into 2005.

Why the queries work in Studio still confuses me a little bit... but I'll worry about that some other time. Hope that helps someone - I didn't find anything useful on my first google for the error message.

Searcharoo C# search engine indexes Word, PDF and more...

The latest version of Searcharoo (4) is now available for download (and read about, of course) --
C# search engine: refactored to search Word, PDF and more.

The coolest feature - searching Microsoft Office docs (Word, Excel, PowerPoint) and Acrobat/PDF files - is based on someone else's article Using IFilter in C#.

Also put a bit of work into refactoring the code structure, layout, naming, etc. At least now it might pass a "code review" (if anyone cared to conduct one:)

The project now also has it's own website www.searcharoo.net - the latest article isn't quite there yet (requires some formatting) but over time I'll post more frequently to that site; it takes forever to write, format and upload decent, CodeProject-quality work...

Parsing DBP (Visual Studio Database Project) files

I was surprised how few people appeared to be trying to 'parse' Visual Studio Database Projects (*.dbp files). Why is it useful? If you want to construct a database from scripts in a project, you want to get exactly those scripts run - not any others that might be lying around the filesystem because Studio was too lazy to 'delete' them when they were removed from the Project (or perhaps they were retrieved from VSS).

Captain LoadTest had the same issue in 2005, and developed a
NAnt script to parse VS.Net database project files, which is a nice idea - but I really wanted to access the 'tree' of scripts from C#.

Thankfully, the open-source project AnkhSVN (which integrates the Subversion source control tool into Visual Studio) has exactly the code needed: ParsedSolution.cs and ParsedSolutionItem.cs.

I commented out some code ('context') and added this method, and that pretty much worked. Make sure you check the licence before using...
/// <summary>
/// Borrows code from the solution parser to JUST parse Database Project Files
/// </summary>
/// <param name="projectFileName">The full path to a *.DBP file</param>
public static ParsedSolutionItem OpenDatabaseProject(string projectFileName)
{
string projectFile = projectFileName; //GetProjectFile(projectFileName);

ParsedSolutionItem project = new ParsedSolutionItem();
project.Name = "Database"; //projectName;
project.FileName = projectFile;

if (System.IO.File.Exists(projectFile))
{
using (StreamReader reader = new StreamReader(projectFile))
{
string extension = Path.GetExtension(projectFile);
switch (extension)
{
case ".dbp":
ParseDatabaseProject(project, reader);
break;

default:
throw new ApplicationException("Trying to parse unknown project of type " + extension);
}
}
return project;
}
else
{
return null;
}
}


How to program it...

This brief 'essay' on developing a functional algorithm How to program it... is based on a 1945 book describing methods of problem solving.

The four basic steps are:

1. First, you have to understand the problem.


2. After understanding, then make a plan.


3. Carry out the plan.


4. Look back on your work. How could it be better?

I think it should be required reading for all developers - even pasted on your cubical wall!

Particularly when hiring, one of the "metrics" I try to measure in candidates is their sense of 'logic'. This is easier said than done, however; and actually what I am really interested in their natural ability or inclination to follow such a process.

It's easy to spot less 'logical' developers. They start at (and sometimes only ever do) Step #3! Sometimes, in the process of thrashing about in Step #3 they may accomplish some of Step #1. While fixing the inevitable bugs, they may also accidentally get some Step #4. But it's not an efficient process...

The best developers are built not bought

The best developers are built not bought

At the very bottom of Bloom's taxonomy of educational objectives for cognitive skills is knowledge. That is recall (or recognition) of data or information. Applied to development terms this means recognizing the C# syntax or recalling the correct syntax to execute a loop.

At the top of the cognitive taxonomy is synthesis and evaluation. Synthesis is to be able to bring together diverse elements to form a new whole solution. Evaluation is the ability to make judgments about the value of ideas, approaches, or materials. Developers need to consistently apply these higher levels of cognitive ability to effectively perform their work.

Most people find the idea expressed as thinking more natural than describing the difference as cognitive process. In general, the more thinking that is being done about higher level ideas, such as how to integrate pieces to form a solution, the better the developer.

Visual Studio 2005 Snippets - literal dollar sign $$$

Been using Visual Studio 2005 snippets since I installed it... double-[tab] is a common keystroke now. Of course, my first attempt to create a Snippet wasn't as simple as it should be thanks to the dollar sign escaping of variables. I wanted a 'standard' VSS region-wrapper as a snippet (yes, I realise I can also add it to our class file template).

#region Page History
/*
 * $History: $
 */
#endregion

but the editor interprets $History: $ as a variable and doesn't display it. Tried Google, but not easy to search for '$' and get useful results... after a few false starts trying to Xml entity encode it (ie. >) which also didn't work, I tried the double-dollar-sign $$ at random - and it worked!

Got Snippets is a great place to start when looking for useful snippets (or ideas/samples to build your own). Some of them are pretty complex.

Some other useful snippet links:

• Code Snippets in Visual Studio 2005 (DotNetJunkies)


• Code Snippet Editor for Visual Basic 2005 (MSDN)


• Snippy - Edit and Create VS 2005 Code Snippets (GotDotNet)

Generics, Serialization and NUnit

Latest article Generics, Serialization and NUnit posted on CodeProject.com. The googling I've done so far hasn't turned up much info on Unit Testing and Generics... seems to me that any time you write a class, you want to be able to test it, and Generic classes should be no exception; but there's not much support in NUnit for it at the moment...

NCover 'connection not established' error

I recently tried to set-up NCover 1.5 with a .NET 2.0 project on a new PC, but could not get past the error message Profiled process terminated. Profiler connection not established... the BAT file I was using looked like this (and was working fine on the build server)
@echo off
C:\DevTools2\NCover-1.5\NCover.Console.exe "C:\DevTools2\nunit-2.4.b1\bin\nunit-console.exe" "C:\Projects2\APPNAME\UnitTests\APPNAMEUnitTests.nunit" //a "Common.APPNAME.DataAccess;Common.APPNAME.Common; Common.APPNAME.RulesEngine;Common.APPNAME.Client" //ea "Common.CodeCoverageExcludedAttribute" //w "C:\DevTools2\nunit-2.4.b1\bin" //l "C:\Projects2\APPNAME\Build\NCover\NCover.log.txt" //x "C:\Projects2\APPNAME\Build\NCover\NCover.xml"

Of course, because NCover was working OK on other .NET 1.1 projects, I immediately assumed it was a 'framework' issue and spent fruitless time trying to 'figure out' why NCover couldn't "connect" to my application.

I gave up, only to be embarrassed a day or two later when this
NCover Error Message thread: Profiled process terminated. Profiler connection not established gave me the simple solution: I needed to register one of the NCover components (presumably so it can hook in to the CLR instrumentation or whatever)
regsvr32 CoverLib.dll

Doing an install of NCover from the .MSI would also have worked, but I've gotten so used to .NET applications being able to be 'xcopied' that I had just moved the NCover files from one PC to another and expected them to work.

atlas:UpdatePanel Cancel/Abort UpdateProgress

Thanks to this blog Trabalhando com o ASP.NET Atlas Framework - Parte III for finally giving me the hint on how to add a Cancel/Abort option to the atlas:UpdatePanel UpdateProgress template.

Just add an <input runat="server" type="button"> <asp:LinkButton id="abortButton" runat="server">

The trick is id=abortButton! You don't need to add any further client or server code/jscript/whatever. The presence of that control id is enough. When the UpdateProgress is displayed, you can click to cancel the Ajax XmlHttpRequest.

Why was this so difficult to figure out? I guess because if you use the Designer to create the ProgressTemplate it adds the button for you... but i was creating the controls in Source view, and it was NOT immediately obvious how the Abort/Cancel should be implemented.

EDIT 22-Feb-07 This post is out-of-date; the released version of ASP.NET AJAX doesn't work this way... just one of the breaking changes - check the Migration Guide: Converting Applications from “Atlas” CTP to ASP.NET AJAX RTM

,NET 3.0!

How did I not already know this... .NET Framework 3.0 will be released with Vista, and available for Windows XP and Windows Server 2003.

Interestingly (I think) it will just compile with the existing FX2.0 CLR, so I guess WinFX is just a mass of managed code? Still doesn't include LINQ or ObjectSpaces, so for database-focussed (ie. business) applications, what benefit does the renamed WinFX provide?

OT: great discussion-starter on Who needs value types anyway?, and this on .NET Framework 3.0 Naming...

RefreshPanel: precursor to Atlas

The Atlas UpdatePanel control seems to have it's genesis in the April 2005 RefreshPanel on gotdotnet.

The UpdatePanel has it's own 'book lite' on OReilly.com - USD9.99 doesn't seem to bad, although the fact they want to charge 'delivery' for overseas customers is very frustrating! The PDF itself is like an extended CodeProject article...

Migrating to *ASP*.NET 2.0... from a Class Library project

In ASP.NET 1.1, Web Projects and the way they bind to Visual SourceSafe and a specific website make it *very* difficult to manage. I have always followed this advice to use
ASP.NET without web projects: the Project becomes a class library, and with a few minor VisualStudio configuration changes the source-control integration becomes a LOT easier.

But what happens when you want to migrate to ASP.NET 2.0? Do you:

1. keep using the 'class library hack', but maybe forgo VisualStudio 2005's cool new features for ASP.NET Designer Support


2. convert back to a Web Project in 1.1 then migrate to 2.0 (ie undo/reverse these instructions)


3. migrate the 'class library' project to 2.0 first, then convert it to the new (ie post Visual Studio 2005 released add-in) Web Application Project/li>
   
4. create a Web Site Project and copy all the existing pages into it


5. or something else...

If your website is part of a larger solution (ie other projects such as Business Objects), or you have more than one developer, or you are actually using source-control, let me save you some time and recommend number 3.

1. First open the Visual Studio 2003 Solution using 2005 and follow the steps to migrate to Framework 2.0.


2. Then make any code changes to get it to compile


3. Now for the tricky bit, open the <WebApplication>.csproj in NOTEPAD
   
   1. Add

          <ProjectTypeGuids>{349c5851-65df-11da-9384-00065b846f21};{fae04ec0-301f-11d3-bf4b-00c04f79efbc}</ProjectTypeGuids>

      near <ProjectGuid>
   
   
   2. Add

      <Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v8.0\WebApplications\Microsoft.WebApplication.targets" />

      near whatever other Import elements exist near the end of the file
   
   
   3. Add this to the end of the file (you may need to update some settings)

        <ProjectExtensions>
          <VisualStudio>
            <FlavorProperties GUID="{349c5851-65df-11da-9384-00065b846f21}">
              <WebProjectProperties>
                <UseIIS>False</UseIIS>
                <AutoAssignPort>True</AutoAssignPort>
                <DevelopmentServerPort>2555</DevelopmentServerPort>
                <DevelopmentServerVPath>/</DevelopmentServerVPath>
                <IISUrl>
                </IISUrl>
                <NTLMAuthentication>False</NTLMAuthentication>
              </WebProjectProperties>
            </FlavorProperties>
          </VisualStudio>
        </ProjectExtensions>
      </Project>

   
   
   
   


4. If you left the Solution open while editing, Visual Studio will detect the changes and offer to re-load. The first thing you should notice is the little 'world' in the Project icon (instead of the class-library square). The Project-Properties will also have a [Web] item, and all the benefits of ASP.NET Design-time, etc should be available.

Oh, and don't forget, if IIS is still pointing at the site's directory, you need to map the Framework 2.0 Handlers to it, using the Command Prompt...

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe -s W3SVC/1/ROOT

ILMerge is my 'tool of the week'

I've already been using ILMerge to merge output from a number of different projects in a solution into a single EXE or DLL for redistribution (or inclusion in other projects). It works fantastically, particularly via an ILMerge NAnt Task.

There are a few tricks, like remembering that Type info includes the assembly name! If you're directly referring to a Type in code, or a .config file, those references MUST CHANGE if that Type is subsequently merged into a 'new' assembly with ILMerge.

That's pretty basic stuff thought. Both
K. Scott Allen: Using MSBuild and ILMerge to Package User Controls For Reuse
and
David Ebbo : Turning an ascx user control into a redistributable custom control describe another interesting use : packaging up ASCX-type User Controls into redistributable DLLs. Great idea if you find building User Controls with a Design surface to be much easier than programmatically building Server Controls.

Seems like 2.0 has plenty of cool tricks to offer if you can think of them...

ASP.NET Atlas April CTP Installation Error

It was nice to know I wasn't the only one having problems installing Atlas April CTP due to cabinet file '_28942B4D9DE730210E7C27BD4FBAD377'...is corrupt.

I originally downloaded Atlas and did "Save As" (which is my habit, to install on multiple PCs). But as with quite a few others, it didn't work: "The cabinet file '_28942B4D9DE730210E7C27BD4FBAD377' required for this installation is corrupt and cannot be used. This could indicate a network error, an error reading from the CD-ROM, or a problem with this package."

I tried again, in both IE and FF, from my desktop and laptop (so four times) - clearing browser cache, etc (after the first time it broke, and I started googling for answers).

Then I noticed one of the comments on ASP.NET Atlas forum about the Publisher being 'Microsoft' versus 'Unknown', and it seemed like the versions I downloaded [Save As] and 'double-clicked' on were 'Unknown Publisher.
While not believing it would work , I tried again (with IE) BUT clicked [Run] instead... and the first dialog that appeared showed Publisher: Microsoft! The install worked fine from that point.

I don't know how/why the authenticode/whatever process would be different for a saved versus 'run straight away' install^, but after four failures across browsers/pcs and one success, I'd encourage others to at least give it a go -- click [Run] when you download!

^ actually, another post makes an interesting point about the effect of security/permissions in your IE 'downloads' folder versus your /My Documents/, /Desktop/ or wherever you might be placing the file when you [Save As]. I don't know enough about it to comment - but sounds feasible...?

Searcharoo v3 is LIVE

Ok, maybe not ALL CAPS exciting, but a new version of my 'basic' C# search engine and web spider Searcharoo is now 'ready'.

There's a pile of stuff I still wanted to add to it: creating interfaces and plug-ins for the Word and Number Normalization, a UK/Australian version of the stemmer, at least putting the 'language/culture' into the Stop/Go/Stemmer/Stopper interfaces so they could be language-specific and a lot more. They'll all have to wait for the next version...

'Support' and 'thanks' emails trickle in fairly regularly, hopefully that will continue.

I think the old articles are still worth reading...
Searcharoo: C# File system indexer and search (version 1)
Searcharoo: C# Search Engine and Web Spider (version 2)

"Cannot find the assembly" for Binary Deserialization

I have an ASP.NET search tool which stores a data structure in the Application cache. This 'problem' relates to the next version so stay tuned for the new code.

The structure is built up via a C# class which is defined in a .CS file. It takes a while to build (it crawls the web), so I don't want the build process to be running all the time.

The .CS file is compiled and executed 'on the fly' because it is only referenced in ASPX page @PAGE SRC=""

The assembly it is compiled into therefore has a random (and different) name every time the 'web application' restarts (eg. IIS is restarted, the computer is rebooted, or ever a change to web.config). The assembly name could be anything from "udlo.dll" to "9pd_.dll".

Rather than re-build the data structure every time the 'web application' restarts, I decided to serialize it to file, and look for the file to deserialize on restart, before rebuilding.

The XML serialized file is 77Mb+. Too big. The Binary serialized file is 4Mb. Better.

Imagine my frustration when - despite all the code looking right, the ASPX kept returning this error System.Runtime.Serialization.SerializationException: Cannot find the assembly ***
where *** was always a different, random dll name (because of the dynamic compilation of @PAGE SRC="" code files).

I already had an inkling that Binary-serialized objects could only be deserialized by the same assembly that serialized them, but I wasn't really sure why. Thankfully, goxman supplied some answers on CodeProject .NET XML and SOAP Serialization Samples, Tips.

Type information is also serialized while the class is serialized enabling the class to be deserialized using the type information. Type information consists of namespace, class name, assembly name, culture information, assembly version, and public key token. As long as your deserialized class and the class that is serialized reside in the same assembly it does not cause any problem. But if the serializer is in a separate assembly, .NET cannot find your class’ type hence cannot deserialize it...
In this case we can write our own class’s binder and tell the deserialization to use our binder while looking for our classes’ types.

As you can see - this is a real problem in my scenario - every time I need to deserialize my objects, it is because the application has restarted, the code is recompiled and by definition the assembly name is different!

Thankfully the CodeProject article goes on to demonstrate how to implement System.Runtime.Serialization.SerializationBinder and attach it to formatter.Binder... and everything worked!

Searcharoo gets some use!

It's cool when you randomly stumble on other people linking back to your code... like this blog linking to a Searcharoo user,
combining Searcharoo with Lucerne.

That, plus some positive comments on the CodeProject Searcharoo article has motivated me to get working on the code again!