A quick guide to using Bluetooth serial ports from C#

Reading from classic Bluetooth serial ports isn't hard but does involve understanding 5 different classes and connecting them together. In this post, I'm show all the classes you need to make the code work, how to create or use them, and how they fit together. This walk-through will be in reverse order: I'll start with actually reading data and then work backwards through the classes. The recapitulation at the end will list the objects in forward order.

I use this code to connect to a small portable GPS device that sends National Maritime Electronics Industry (NMEA) formatted GPS information over a Bluetooth serial port device and translate and display the resulting messages in an easy-to-use app.

Starting with the DataReader class you need to read from your device, and ending up with the generic Device interfaces, here's the classes and objects you need to use your Bluetooth serial-port connected device.

Use a DataReader to read the data. 

Documentation DataReader:

Hints for using DataReader: call LoadAsync(count) to read from the serial port into the DataReader's internal memory and then ReadString() to get a string of the data. Be sure to set the InputStreamOptions to Partial. Partial means that whenever there's data the load will return. This is important because the serial port sends just a little data at a time.

Constructing the DataReader: construct the DataReader with the constructor that takes an IInputStream as a parameter

The IInputStream is from a connected StreamSocket. 

Documentation IInputStream:

An IInputStream is an interface for the concept of "getting stream of data (with no framing) from a specific source like a Bluetooth device or a network connection. You won't be calling any methods on it.

Getting the IInputStream: your IInputStream will be from a connected StreamSocket's InputStream property.

The StreamSocket is connected using data from an RfcommDeviceService objects

Documentation StreamSocket: 

Hints for the StreamSocket: StreamSocket is a single class that can handle multiple different type of connections. Common uses are for networking (where it's a classic TCP/IP network connection) and for Bluetooth classic. You'll just connect it and then use its InputStream property.

Constructing the StreamSocket: construct the StreamSocket with the default constructor with no parameters and then connect it using the RfcommDeviceService's ConnectionHostName and ConnectionServiceName. The hostname will look like "(00:19:01:48:4E:F5)" and the servicename will look like "Bluetooth#Bluetooth2c:0d:a7:c8:53:33-00:19:01:48:4e:f5#RFCOMM:00000000:{00001101-0000-1000-8000-00805f9b34fb}"

The RfcommDeviceService is from a BluetoothDevice object

Documentation RfcommDeviceService:

Hints for the RfcommDeviceService: The RfcommDeviceService is a little confusing because there's so much apparent overlap between a BluetoothDevice which supports an RfcommDeviceService. A good way to think of it is that my GPS Bluetooth serial device is the "BluetoothDevice" as seen by Windows. And the RfcommDeviceService is the serial port that could theoretically be connected to anything.

Getting the RfcommDeviceService: The RfcommDeviceService is gotten from the BluetoothDevice. There are two ways to do this: 

1. Call the GetRfcommServicesForIdAsync()  method to get just the specific serial service you want. This lets you ask for a serial port and not, for example, an ObexFileTransfer. I know ahead of time that I want the SerialPort service, so I use this method to get just the right matching services (although I  think there will always be just the one)
2. Call the GetRfcommServicesAsync() method get a list of all possible serial services and then pick your preferred service from it.

It's important to know that Device might support both the generic raw serial port as RfcommServiceId.SerialPort and might also support the more structured ObexFileTransfer. For my specific GPS device, it supports both the SerialPort and also the MFi/iAP(2) protocol. When I connect to the device, I see the NMEA messages, so it must be the right one.

BluetoothDevice object is initialized from a DeviceInformation Id

Documentation DeviceInformation:

The BluetoothDevice object that's used to get the RfcommDeviceService is used in the program to track device events like the connection being lost, and to ask for permission to access the device.

Constructing the BluetoothDevice: call the static BluetoothDevice.FromIdAync(deviceId) method, passing in the select DeviceInformation's Id property.

The DeviceInformation object is found with DeviceInformation.FindAllAsync(query)

The DeviceInformation object is used by the Windows device system to return information about devices in a Windows computer. A device can be an adapter, like a USB port or Bluetooth port, or a device like a video card or monitor, or an input device like a keyboard or mouse. 

Getting a DeviceInformation object. Devices are found using a query string in the Advanced Query Syntax (AQS) format. This format has no obvious documentation at learn.microsoft.com. The good news is that you can get pre-created strings for devices, so it's no great loss. Get a pre-created string from the static GetDeviceSelector() method on each device class. The methods often do not require any parameters; sometime they take in some kind of specialty parameter. For my program, I eventually need a BluetoothDevice, and I know it must be paired, so I use an AQS query string from BluetoothDevice.

Use the AQS string in the DeviceInformation.FindAllAsync() method. This returns a list of DeviceInformation objects. You'll have to look at each DeviceInformation object to decide which one to use. I often do a match using the Name property.

Hints for using the DeviceInformation: Most of code for reading and writing a Bluetooth serial port is very similar and can often be shared between your different Bluetooth projects. But matching a device is often unique to the device and will have to be changed for every program. It's best to not try to wrap the matching code into something "handier" or "easier to understand".

Recap: critical steps

Let's recapitulate the steps, but this time in the order you'll do it in your code. In these steps, names in UpperCase are classes and lowerCase are objects. 

The steps to reading from a Bluetooth serial device are:

1. Get a DeviceInformation object by using an AQS string from a static device GetDeviceSelector() method and passing that AQS string to the static DeviceInformation.FindAllAsync() method
2. Create a BluetoothDevice from the deviceInformation.Id property with the static BluetoothDevice.FromIdAsync() method
3. Find the correct RfcommDeviceService from the bluetoothDevice.GetRfcommServicesForIdAsync() method.
4. Construct a StreamSocket and then connect it with the rfcommDeviceService's ConnectedHostName and ConnectedHostService
5. Get an IInputStream from the streamSocket.InputStream property
6. Create a DataReader object with the DataReader constructor that takes an IInputStream as a parameter. Set the dataReader option to Partial
7. Call dataReader.LoadAsync() to get some of the serial data from the device, and call dataReader.ReadString() to get the data as a string.

Thanks for reading, and good luck!

Modbus: deciphering the CRC protocol

 The Modbus over Serial protocol doc has a clear, simple set of instructions on how to decode. You just have to know that two of the steps are in a reverse order, but it actually makes sense.

The Modbus protocol is used by the Daybetter LED light Bluetooth protocol. It's arguably a terrible fit for this: the protocol includes a bunch of stuff that isn't even slightly relevant with Bluetooth (like the CRC), but doesn't leverage any of the Bluetooth strengths (like the ability to split "color" from "on/off")

Modbus CRC Calculations explained

Here's the official explanation of the CRC calculations from page 14+15 of , modified into a numbered list. Absolutely no changes were made except to make it a list -- that's why there's still weird commas and periods. It's substantially the same as the steps in page 39 to 41 in section 6.2.2 AKA Appendix B.

During generation of the CRC,

1. each 8–bit character is exclusive ORed with the register contents. 
2. Then the result is shifted in the direction of the least significant bit (LSB), with a zero filled into the most significant bit (MSB) position.
3.  The LSB is extracted and examined. 
4.  If the LSB was a 1, the register is then exclusive ORed with a preset, fixed value. If the LSB was a 0, no exclusive OR takes place.  

Note that clear ordering: first you do a shift, then you look at the LSB and do the XOR. BTW, the "preset, fixed value" is  0xA001 (decimal 40961 or binary 1010 0000 0000 0001)

But when you look at the commonly-available Modbus CRC calculations from random Github repositories, the code always switches steps 2 and 3! The LSB is grabbed first! What the heck! why are the clear and unambiguous steps in the official docs not what everyone implements?

Everyone is right because of Microcontrollers!

When you look at high-level languages, a right-shift is just a right shift, possibly with the ability to decide what gets shifted into the MSB (either duplicating the old bit, so a negative number stays negative, or filling it with zeros). But that's not what microcontrollers programmed in assembly do!

A typical microcontroller will do the shift (often with lots more control over the bits) and will also set the carry flag. The next instruction you do can then be a conditional jump based on the carry bit and therefore based on the original LSB.

 And indeed, the flowchart diagram from the Modbus protocol doc, appendix B, page 40, labels the item 4 "if" statement as "Carry over" yes/no. They are expecting implementors to use the carry flag or the overflow flag depending on the microcontroller being programmed!

The Modbus flowchart is reproduced below.

Microcontroller assembly

Motorola (NXP) 6805 chip has arithmetic shift right and logical shift right as two different operations. In both cases the operand is shifted one bit to the right, with the LSB moving into the carry bit.

The 6502 is similar, as is the ARM chip where setting the flags is optional (LSRS in assembler as opposed to LSR). The Intel 8051 does this with a RRC A opcode () but you have to set the carry flag to zero before doing the instruction (if it's 1, then a 1 gets put into the MSB)

Helpful Links:

Modbus over Serial Line specification and implementation guide

EPUB format thoughts

 How hard can EPUB files be?

"EPUB is just HTML"! Hah!

I've got a fun EPUB ebook reader in the store; it's got two nifty features that, IMHO, all ebook readers should have (it can do an offline search of Project Gutenberg, and it's got a two-screen mode, so you can see both a critical image and the text that talks about the image in one spot).

Over the years, I've had to work around a lot of EPUB failures. Today's failure is thanks to the newer EPUB books that Project Gutenberg publishes.

Notably, each book seems to include a (pointless) <pre/> tag. The problem with that tag is that HTML does not support self-closing (void) elements. The Mozilla pages are super clear about that.

So my HTML renderer (which is just a WebView element) takes the <pre/> tag and reads it HTML style, like a <pre> tag that isn't closed. The entire rest of the book, which is most of it, is then displayed with pre-formatted lines. Because pre-formatted lines don't wrap (that's the point of the <pre> tag, the rest of the reading experience is mostly ruined.

SIGH. 

First thoughts on std::expected

 std::expected -- what can you do with it?

There's  a fancy new thing in C++: std::expected. There's a lot of pots about it's awesomeness, but what is it, really?

Answer: if you've got a std::expected, you can do three important things. Let's give a name to our std::expected: it's called result. And as a reminder: std::expected holds one of two values, which in this post I'll cleverly call "first" or "second".

1. Just result by itself is  a bool. When first is set, it returns true; when second is set, false.

2. *result is just first.

3. result.error() returns second.

You can also chain them together, assuming you have code where you want to do a bunch of stuff in order and will never have to refactor the code to be multi-threaded, or handle error conditions weirdly, or log something and then fail, or a lot of things. Oh, and using functions means lots of global variables, and the function must return a std::expected, not an int.

1. result.and_then(function) will call function with *result. The function can take exactly one variable, and the function will only be called if result was set to first.

2. result.or_else(function) is like the opposite of and_then. It's also a function that takes a single parameter (love them globals!) but take in the type of second, not first, and is only called if the second was set (the 'unexpected' branch).

Want to make a std::expected?

If you need to return a std::expected, and you're setting the first value, just return it and it's automatically cast into an object as needed. If you need to return the second value (unexpected), return a std::unexpected(). Be sure to not new it; just return std::unexpected("second value").

And some useless crap

What if you you've got a std::expected where first is an int, but you really need a double? Can you cast it? Nope, you can't. Casting is now called "transform".

There's a tranform_error(), too.

First thoughts: what about exceptions?

Lord knows. I guess exceptions aren't a thing any more?

What happens if I need to set a breakpoint? Answer: good luck with that. 

The source code the std::expected iat about 1500 lines of 

XAML Style Dictionaries

Step by step: adding consistent styles to a XAML app

XAML has a useful feature where you can set up a single file with a bunh of "style" configuration for your XAML (UWP) app, and then use those styles everywhere in your app. The plus side is having a centralized style configuration: all your titles can be one size, and all your technical text has the correct font, and all. The downside is that getting it set up is buried in a blizzard of docs. There's a lot of ways to set up your XAML styles, and Microsoft wants you to fully understand each of them in isolation.

In this simple blog post, I walk through the steps of one way to add centralized styles to your app and then be able to use them everywhere.

Create an AppDictionary.xaml file. In your project, Add > New and add a XAML ResourceDictionary. Call is AppDictionary.xaml.

Add your first style. I tend to prefix all my centralized styles with a small "s" like "sTitle".

    <Style x:Key="sTitle" TargetType="TextBlock">
        <Setter Property="FontWeight" Value="Bold" />
        <Setter Property="FontSize" Value="24" />
    </Style>    

Update the App.Xaml file to include this:

    <Application.Resources>

        <ResourceDictionary>

            <ResourceDictionary.MergedDictionaries>

                <XamlControlsResources xmlns="using:Microsoft.UI.Xaml.Controls" />

                <ResourceDictionary Source="AppDictionary.xaml" />

            </ResourceDictionary.MergedDictionaries>

        </ResourceDictionary>

    </Application.Resources>

The XamlControlsResources is seemingly a new thing for the Toolkit; it's essential to make the toolkit work. At least, that what is looks like to me.

Original, bad way: Update every page and control to include the AppDictionary

    <UserControl.Resources>

        <ResourceDictionary>

            <ResourceDictionary.MergedDictionaries>

                <ResourceDictionary Source="AppDictionary.xaml" />

            </ResourceDictionary.MergedDictionaries>

        </ResourceDictionary>

    </UserControl.Resources>

This isn't as good as just updating the App.Xaml because you have to update every page. By updating the App.Xaml file, you do it just once. At least, that's what I think will happen. If I was really an expert on this stuff, I wouldn't have struggled with it for a couple of hours last night :-)

Use the new style.  In a textblock, set the Style to be {StaticResource sTitle}.

        <TextBlock Style="{StaticResource sTitle}">Thingy Data</TextBlock>

Now you're styling with style, and it just took one short blog post to replaces multiple lengthy pages of explanation from the learn.microsoft.com site!

Weird Error to be aware of:

In my original App.Xaml file, I had the <Style> items right there in the file and then had the Resource Dictionary. But that could only be made working XAML by adding a "x:Key=..." to the ResourceDictionary. For a long time, this worked fine, but when I updated to the latest Toolkit, I got the following error. 

Windows.UI.Xaml.Markup.XamlParseException: 'The text associated with this error code could not be found.

Cannot find a Resource with the Name/Key ExpanderHeaderBackground [Line: 100 Position: 26]'

As far as I can tell, the "x:Key" is needed because when you have <Style> items in the <Application.Resources>, it's like it makes a default <ResourceDictionary>. So when you make a second <ResourceDictionary>, you have to give it a name.

But that's just a guess. What I know is that when I got the above error, I had an x:Key in my <ResourceDictionary> that I used for including the <XamlResourceControls> and when I moved everything around so I didn't need th x:Key, my app stopped crashing.

Overview: Installers for .NET Core

 Installer types for .NET Core applications

I've got a new application, and I want a nice installer for it. Visual Studio has a metric ton of different installer projects, variously called "publisher" and "setup" and "install" and "deploy" -- so many, many different adjectives.

TL/DR: Make a setup project

My goal is an installer that is a single file, probably .EXE but I'll do a .MSI or .MSIX or whatever. Additional points are awarded for easy instructions, instructions that are less than 5 years old, a simple build process, output that can be placed somewhere other than the "bin" directory, and, of course, an installer that actually works.

My project, FWIW, is a .NET "Core" (now it's just called ".NET") -- that's the new thing that replaces the "Framework" version of .NET. It's using some of the WinRT APIs, so the output type is "WinExe" and the target framework is "net7.0-windows10.0.22000.0".

Publish (Folder) from main project (23 MB)

Creates a folder with your real .EXE plus a .DLL plus some more DLLs plus a .JSON file plus more random files. They are pretty much all needed, and there's no actual "installer"; it's just random files that you can write an installer for.

Points deducted: not a single file, no installer

Publish (Click Once) from main project (24 MB)

Creates a folder with an MSI for installing, plus also a .EXE for installing, plus also a random set of "Application files" which is full of random files that have all been renamed. The output folder must be in the bin folder.

Points deducted: terrible multi-install experience, wizard is broken, very old docs

The multi-install experience happens when you do an install to test the installer, and then install from literally any other place. Instead of upgrading, or removing the old app, or installing side-by-side, the installer just fails with no suggested course of action.

The publish "wizard" is broken: you can select stuff to change in the installer. But when you select, for example, the "Sign Manifest" "tab", you don't get brought to that experience; the dialog box is unchanged. Instead you have to click the "next" button to get from one "tab" to the next. 

The documentation for "What's new" is from 2012. 

Setup Project from new project (7 MB)

Creates an .MSI for installing, plus a pointless SETUP.EXE that only works with the .MSI. This means I had to waste time figuring out which file I really needed. 

Points deducted for: bizarre and undocumented "editor". To use it, right-click the "Application Folder" and select "Add" and then "Project Output" and give the original project. 

The created installer is a clunky, old-fashioned installer, so the user has to click "next", then look at stuff they won't want to change, then click next again, click next on the UAC prompt, and then click close. And it gives a little warning to be sure to check with Windows Update for updates to .NET Framework, which this app, AFAICT, doesn't use at all (it's .NET Core). 

I really want to deduct points for the "Transitive" property, the description of which is "Determines whether the installer will reevaluate the Condition property for the selected item when reinstalling on a target computer". The words condition, property, select, and target computer really just raise new questions. 

Project: Govee E-Ink display

 The Govee H5074 E-Ink display project!

All complete! This project uses an Adafruit nrf52840 board running CircuitPython and their 2.13 inch e-ink display (the tri-color one) to pull in data via Bluetooth LE from a Govee H5074 temperate and humidity sensor.

Watch it on YouTube. Also, the code is up on Github; take a look!

Some of the challenging / fun parts: on reset, the device will look around for a Bluetooth "Current Time Service" so that the clock is set automatically (no need to every manually set it!). Reading in Bluetooth advertisements for the Govee H5074 was not as obvious as it should have been (and I've got a blog post about it), and the e-ink display was much more challenging than I though it would be.