From the past: Win32s was both awesome and awful

 WIN32S: a programmer's dream from the Windows 3.1 era    

Gather 'round, young 'uns, and let me give you some wisdom from the days of the Win32s compatibility library. When Windows was being made, it was a 16-bit system, meaning that pointers were short and it was painful to address much memory. I'll also share a key feature of technical decisions: how to make an early decision that you don't know the ramifications of.

Quick aside: the people who made the 8086 and compilers as well as the PC and the OS knew this, so pointers were "actually" 20 bits (1 meg) thanks to the selectors and corresponding weird compiler switches, plus there was weird back-compat with the "A20" shim, and of course there was additional weirdness for some bank-selection.

Enter the RS/1 statistical program, which I helped port from its VAX/VMS and Unix heritage to the PC running Windows 3.1 and NT. It was a "big" program that had been in active development by a team of capable developers for ten years or more, with a ton of features. It had been designed originally to work on constrained computers like the PDP-11 (also a 16 bit machine with its own pointer weirdness).

A key early decision was to use the Win32s library on Windows 3.1. This library let us use 32-bit pointers in our code, which was a big plus. A downside, though, is that we didn't have any way to know about any potential downsides. Marketing material then, as now, talks big about how great new technology is, and doesn't much mention possible issues.

But there was an incredibly tiny flaw in the Win32s implementation of "unlink" that was to have enormous implications. To explain the bug, you have to know two things:

Firstly, the unlink call is used to "delete" a file. It's called unlink because technically the file isn't "deleted": it's merely removed from a directory. If the file is only present in one directory and no program is using it, the file is also deleted. 

Secondly, RS/1 kept track of every "table" of data as a separate file, and it used tables for pretty much everything. This was actually awesome, and I think more programming environments would benefit from a table-first approach. Some tables were permanent, but others were "temporary" and might be just in memory or might have a file backing depending on the available memory. Remember that RS/1 was designed for low-memory environments, so this automatically shuffling of data between disk and memory was a critical part of the program.

The bug

The bug in unlink was that if you did an unlink on a file that didn't exist, it returned the wrong value. Specifically, it returned success. The Posix standard was to return failure. This was critical to the underpinnings of RS/1: it's how the program knew if a temporary file was just in memory or was actually backed by a file on disk. By returning the wrong values, some internal bookkeeping got confused, and would eventually cause a crash.

This was caught, BTW, by the incredibly good $systemtest() function that RS/1 shipped with. Any user, at any time, could run $systemtest() and the program would do a pretty solid job of verifying that it was all running correctly.

On a Windows 3.1 machine, the program crashing also meant that the whole computer crashed, which was not ideal for debugging. I finally tracked it down by running RS/1 in a debugger on two machines, one running Windows 3.1 (where it used the Win32s library and crashed) and one running Windows NT (which used the native implementation which worked perfectly). I then started doing a sort of binary search to trace what was different about the two systems as it did the $systemtest().

Key takeaways for making early technical decisions

Bet on the future, not the past. We could have just made a 16-bit app. But the future was clearly 32-bit for Windows

Everything that isn't mainstream has bugs. Your schedule should include time for them. A constant in my technical career is that libraries that don't get much use have bugs, and there isn't much management resolve to fix them. In our case, the mainstream was either 16-bit code on Windows or 32-bit code on VMS or Unix. Win32s was a weird little library (as was the "PharLap" DOS extender that an earlier PC version used). 

Workarounds are better than hoping for a bugfix. If your library has an issue, it's best to figure out a workaround. Waiting for a fix that might never come will delay shipping, possibly forever. 

Weird issues with WinUI3 and ALT key: so many beeps!

 Like any good developer, I want my WinUI3 app to have keyboard control over the menus. And once again, the terrible fit-and-finish of the WinUI3 framework is causing problems.

This time: every time the user presses the ALT key (like they are supposed to!) Windows decides to make the program BEEP!

This is reported as this bug 4379 and 9074. Some useless comments are here.

And somehow, the code I copy-pasted said that "SetWindowSubclass" et all were all in user32.dll. That's not true; it's in comctl32.dll. And did you know that if LINK.EXE isn't in your path, that DUMPBIN won't work?

Update: thanks to castorix's comment for  issue 4379, I have a workaround. See the full file at App.xaml.cs. All you have to do is weird stuff to get an HWND (ignoring the "CoreWindow" documentation since that's just wrong) and then subclass the window, catch the WM_MENUCHAR and then tell windows to close the menu. (seriously, return MNC_CLOSE, documented as "Informs the system that it should close the active menu". Telling windows to close the menu will in fact let the user open the menu with ALT keys just like it should.

Weird issues in WebView2 / WinUI3: handling the ALT key

Applies to: WinUI3 apps with a WebView2 and a menu

Problem: Normally users press the "Alt" ("Menu") key to use an app's menu. But after the user clicks on a WebView2 (as one does; it's a common practice), all Alt keys will be swallowed up by the WebView2.

Solution: capture the 'Alt' key and use that as a trigger to move focus. 

1. In the JavaScript for the one specific page I use, add a 'keydown' event listener. When the event.key is "Alt", send a special message to the enclosing C# code using the  window.chrome.webview.postMessage method.
2. In the C# code add a WebMessageReceived event callback. When the special message is received, set the app's focus to the parent object (cast the this.Parent to a FrameworkElement)    

The problem with this solution is that the first ALT is entirely swallowed up. The user has to press ALT twice to use the menus, but only when they have clicked on the WebView2.

Interesting discoveries: weirdly, when I wrote the C# code, Visual Studio 2022 hallucinated up a solution that involved constructing real objects like a KeyRoutedEventArgs and then calling made-up methods. And, FYI, the KeyRoutedEventArg has no public constructor. 

Links:

KeyRoutedEventArgs  Random difficult work-around and this one, too 

So many issues: 721 3352  984 5772 288 951 

Weird issues in WebView2 / WinUI3: using the ms-appx: scheme

 Update your WebView2 WinUI3 code to read ms-appx:/// URLS from your Assets folder

It was the simplest of APIs, it was the most irritating of APIs. The ms-appx:/// URL scheme is one of the unheralded awesome parts of the WinRT API set. With it, there's a simple and straightforward way for an app to combine a bunch of C# code and also some useful HTML code. I'm using it this month for my (upcoming) Simple GPS Explorer app so that I can display GPS traces on an OpenStreetMap map.

The ms-appx:/// URL scheme is also one of the irritations of moving from UWP apps over to WinUI3 and whatever they're calling the framework. That's because the PMs decided that a simple straightforward way to read local assets wasn't useful in programming.

This blog post shows how to use ms-appx:/// in WinUI3 and WebView2 in just a few steps. We're going to combine two different abilities:

1. We're going to replace the ms-appx:/// in all URLs with an HTTP://msappxreplacement/ . This switches both the scheme (ms-appx: to http:) and adds in a fake host name (msappxreplacement)

2. We'll use the WebView2 SetVirtualHostNameToFolderMapping method s that every HTTP URL that goes to hostname "msappxreplacement" will instead read a file from a folder relative to the app.

Just to make life more difficult, the WebView2 navigation flow doesn't actually let you do this easily :-(. You'll need two chunks of code that do three things:

1. Replace the URI with the ms-appx: scheme with an HTTP URL that points to the msappxreplacement hostname. This must be done in the NavigationStarting event handler and requires that the original navigation be canceled and a new navigation started.

2. Set the NavigationStarting event handler on your WebView2 object

3. Update your WebView2.CoreWebView2 with a virtual hostname redirector so that all HTTP URLs that use msappxreplacement as the hostname instead just read from a local directory.

Code snippets:

NavigationStarting event handler must cancel the original navigation and instead set the source to this other place. You can't just update the URI because the StartingEventArgs ".Uri" value doesn't let you. This handles the item #1 above.

        private void UiWebView_NavigationStarting(WebView2 sender, Microsoft.Web.WebView2.Core.CoreWebView2NavigationStartingEventArgs args)
        {
            // Input URL: ms-appx:///Assets/SimpleMapLeaflet/SimpleMapLeaflet.html
            // Updated URL: http://msappxreplacement/SimpleMapLeaflet.html
            var originalUri = args.Uri;
            if (originalUri.StartsWith("ms-appx:///"))
            {
                var replacementUri = originalUri.Replace("ms-appx:///", "http://msappxreplacement/");
                args.Cancel = true; // Cancel the original navigation.
                sender.Source = new Uri(replacementUri);
            }
        }

At initialization (like as part of your control Loaded event)

    uiWebView.NavigationStarting += UiWebView_NavigationStarting;
            await uiWebView.EnsureCoreWebView2Async();
            const String assetFolder = "Assets/HtmlSubdirectory";
            uiWebView.CoreWebView2.SetVirtualHostNameToFolderMapping
             ("msappxreplacement", assetFolder, 
               Microsoft.Web.WebView2.Core.CoreWebView2HostResourceAccessKind.Allow);

This code sets up the WebView2 to divert all URLs with an "msappxreplacement" hostname to the given directory. The directory will be in the app's installation directory (assuming you have a deployable app, of course). The "uiWebView" is the WebView2 that you put onto your XAML control. This handles the item #2 and #3 above.

Hints: the folder must exist for this to work! Otherwise the SetVirtualHostNameToFolderMapping will throw an exception.

Quick gripe: none of this should be needed. There's no reason why the WebView2 shouldn't come this way out of the box. And it's super painful that the ms-appx scheme is so far from handled by WebView2 that the only place to do the replacement is in NavigationStarting event. If the scheme isn't replaced there, WebView2 will simply fail and not even try to load the URL.

Helpful links:

See https://github.com/MicrosoftEdge/WebView2Feedback/issues/212

See also: https://github.com/microsoft/microsoft-ui-xaml/issues/1967

This restriction is not mentioned in https://learn.microsoft.com/en-us/microsoft-edge/webview2/get-started/winui

A lifetime of weird issues: VSCode and more

Upleveling my gripes + making better software

People who read my blogs know that I'll write mostly about things not working: painful Bluetooth protocols, awkward APIs, and inconvenient programs. That's good, but what advice would I give someone who's looking for a plan to improve, say, VSCode for C# developers.

Count up friction points, and reduce them

A classic PM thing is to make up a metric, measure it, do some work, and declare success. And hopefully measure again, so there's before and after numbers, but weirdly, that's not valued as much as you might think.

So what are the obvious friction points for VS Code? And for reference, here's the start-up screen for my VS Code. Look at all those projects!

The "Recent" list isn't actually my recent files. There's four things listed (blinky, ConstantHtmlTraffic, GenerateWiFiQrCode, and AstronomyOnThePersonalComputer. But in reality, the last files I edited was a set of Markdown files.

The Icons on the left are random, not planned. I have a longer gripe about not meeting the Windows design language, but the icons are particularly bad. Why is the "file" icon marked as "explorer"? Why is the universal "settings" gear marked as "options"? Why is there a "command palette" and yet there's also a "tasks" which isn't a set of my tasks; it's some other commands.

The POV of the app isn't very clear. The original POV was simple: it's an editor. Now it's morphed into a container of random stuff. 

Forward and back aren't consistent. For example, go to the welcome screen, and then run Github. The welcome screen is replaced by a big GitHub screen, but the "back" button doesn't take me back.

 Friction when figuring out a next step due to tone: The explorer in-line help says: "You haven't opened a folder" which feels accusatory instead of helpful. I've opened plenty of folders in my life including folders for VS Code. I just don't have one open right now. There's a fun old anecdote from the early days of computers. The IBM engineers had a hot new (and very expensive) computer with a light marked "idle". Watson had them change it to "ready". "Idle" means that a very expensive computer is wasting money; "Ready" means that an expensive computer is ready to get to work.

Friction because extra steps are demanded but not required: The button "Clone Repository" says "Clone a repository once the Git extension is activated". But in fact, the programmer need do nothing at this point.

Random UX is harder to puzzle out. The button "Clone Repository", when clicked, instantly jumps to a random-feeling part of the screen (it overlays on top of the search bar)

Friction in decoding what dialog boxes mean: The clone repo text box (sorry, there's no better name for it) says I can "pick a repository source". AFAICT, this is then tied to the second item in the fake dialog box, where I get clone from GitHub, but only if I log in. Which is weird, because I'm already logged into GitHub

Friction in cloning the repo. After cloning a repo, I'm asked if I want to open it. That's weird; it's hard to imagine a work flow where opening the repo I just cloned isn't the right choice. 

Friction in open folder. Clicking Open Folder after cloning a repo doesn't go to the folder I just opened. A common practice in Windows is for dialogs to remember previous uses.

Builds are flaky. I asked Chat how to build one of my projects. The resulting command was incorrect (it referenced a non-existing directory hierarchy for the csproj file). When the command was provided with a correct csproj file, it didn't build. The same project builds perfectly in Visual Studio.

Frenetic UX increased anxiety without providing value. When running a build command, the terminal frantically updates a clock every tenth of a second. As an example of the opposite way of life, I present my "Low Distraction Work Timer" that's designed to be entirely non-frenetic.

Added friction from the non-standard UX. VS Code doesn't follow any of the modern Windows UX principles here. It's not effortless or calm, or notably personal (in particular it loves to turn on dark mode even though I've never turned on dark mode on anything, ever). It's not familiar, it's not complete + coherent, or any of the other principles. 

 Solution! How to make it better!

Inspired by the Quick Settings Metric.

The best way to figure out what's wrong with VS Code (or any platform) is to design some simple work flows and use the Quick Settings Metric methodology: write down every single "decision" a user would have to make.

Some example workflows include:

1. Clone a C# project, find a line of code, set a breakpoint, and debug it
2. Create a WinUI3 C# project, add the latest WebView2, and make a one-page UX with a textblock title, a variable-size webview with a button to go to a single hardwired site with a custom User-Agent, and a scrolling textblock to track the webview navigation. Verify that the user-agent is correct. 
3. Create a Blazor web page ...
4. Create a command-line app ...

 Part of the Quick Settings Metric is that the harder the "decision" the user has to make, the more points it costs. Clicking an OK button is relatively cheap; selecting from a list is more, and making the user type something in is even more. For VS Code, telling the user to "log in" but they have to type in their MSA is more expensive than the better solution of allowing users to select from their already-in-use MSA.

Wierd issues in the WinUI3 / WebView2: Solving the User-Agent problem

 TL/DR: To verify that you are setting the right user-agent, you have to check the "web resource requested" event's copy of the request headers, not the "navigation starting" event's copy. And you need to call AddWebResourceRequestedFilter.

Background: why am I setting the user-agent, and why do I need to check it?

I'm writing a simple app to parse NMEA data from a small GPS tracker (because that's what I do for fun, that's why). I'm at the part of the project where I want to make a GPS trace, and the technology I've picked is the Open Street Maps project using the leaflet JavaScript library. It's actually really nice: it works smoothly, and the integration was mostly simple.

But ... one of the requirements Open Street Maps has is that if you make an app that works against their servers, you have to use a custom User-Agent header in the HTTP requests. This makes sense: they can filter requests based off of the user-agent if needed.

After a bit of work, it's not hard to set the user-agent. But how to verify that it's correct? The easiest way was to set an event handler on the WebView2 NavigationStarted event. That event provides a CoreWebView2NavigationStartingEventArgs object that includes the request headers. I just have to iterate through them (easy-peasy) and they check the User-Agent value.

And it's not set!

Side quest #1: timing

Web searches show that people do have trouble setting the User-Agent. In particular, it must be set before you get a navigation started event. But in my app, that's not a problem.

Side quest #2: threads and services

The documentation also helpfully points out that the user-agent is thoroughly screwed up if you have multiple WebView2 objects. Each object might well share some of the underlying bits, and so when you have two WebView2 objects, and the user-agent is set differently on them, you might well have the the wrong user-agent sent. This is just dumb, but I remember from previous work with Edge that it's a pretty complex beast.

Solution: check based on the WebResourceRequested event. This requires setting the event in a different place than normal (it must be set after calling the EnsureCoreWebView2Async method). It also requires setting a WebResourceRequestFilter, which hopefully won't cause a slowdown in my app. That callback also includes all of the request headers, and those headers show the correct User-Agent. 

Links: see my question at Setting a WebView2 UserAgent isn't updated in the NavigationStarting event - Microsoft Q&A. 

 

Weird issues in Visual Studio 2022: Solving the CS0534 error when solving the Json trimming problem

 TL/DR: Visual Studio 2022 is wrong, and your code is OK

Background: I'm having to switch from the awesome NewtonSoft JSON library over to some crappy System.Text.Json library (because VS thinks that breaking everyone to support Trimming is somehow ever OK). The sample code for that is wrong, but that's OK, it points to the right code.

Except that if you copy-paste the sample code, it fails:

The error, BTW, is CS5034: 'SourceGenerationContextErrorCS0534' does not implement inherited abstract member 'JsonSerializerContext.GetTypeInfo(Type)' and also '...does not implement inherited abstract member 'JsonSerializerContext.GeneratedSerializerOptions.get' and also 'CS7035: There is no argument given that corresponds to the required parameter 'options' of 'JsonSerializerContext(JsonSerializeroptions?)'.

Solution: just recompile. It turns out that every single thing about the error is wrong. The correct solution is to "recompile your code". That's because Visual Studio 2022 is focused on everything other than the day-to-day experience of programmers.