Sunday, May 28, 2017

Your Bluetooth is bad (and you should feel bad)

Alternative title: your Bluetooth is great, and you should feel great!

Little Bluetooth devices are awesome!  What’s not to like about teeny little things with switches and lights, magnetic sensors and motors?  And over the years, I’ve learned what makes a device easier or harder to control.

But first, a quick TL/DR about Bluetooth for people who aren’t as familiar.  Some devices are like a serial port: you send bytes, and you get bytes, and the device maker has to invent a little protocol for what the bytes mean.

Picture break: some of the Bluetooth devices mentioned

Controller for MetaWearBest TI SensorTag BLEControl Program for BERO RobotsControl Program for TI BLE Lamp Development Kit

Left to right: MetaWear, SensorTag, BERO robot, TI Lamp kit

Other devices are BLE devices: each device has a set of “services”; each service has a set of “characteristics” with a name and value which can often be read, written, or can send a notification on change.  There are a bunch of standard services and characteristics.  For example, service 180f is the battery service; it includes characteristic 0x2a19, battery level which is a  single byte with a value 0 to 100.

Extra-wordy documentation!

I’m looking at you, otherwise awesome BBC micro:bit.  Here’s a short snippet from one characteristic:

Read                    Mandatory
Write                   Excluded
Write Without Response  Excluded
Signed Write            Excluded
Reliable Write          Excluded
Indicate                Excluded
Broadcast               Excluded
Writeable Auxiliaries   Excluded
Extended Properties     Excluded


That’s a big table with lots of rows, and it could all have been replaced with a single entry: “Supported operations: Read”.

Sloppy details!

Service 0x180a, device information, includes characteristic 0x2a29, Manufacturer name string.  It should be something like a company name, right?  Like “Texas Instruments” should be the manufacturer name on their delightful SensorTags.  In reality, the most common manufacturer names I see is a blank string, and literally the words, “Manufacturer name”. 

Similarly, each characteristic can be named.  TI is really good about adding clear names to their device characteristics; you can almost use just the names to guess how to control the device.  The micro : bit, less so. I had to update my Network Explorer program to convert the BBC GUIDs into more readable names.

More bluetooth devices


Left to right: BBC micro: bit, DOTTI, Hexiwear, Magic Lamp

Slow commands!

The Dotti is an 8x8 pixel light where each pixel can be set to any color using a Bluetooth command.  Too bad the write is a “write with response” which is extra-slow, so that actually filling up the screen is slow and painful instead of fast.  The BBC micro : bit, in contrast, has a single command that can fill its entire 5x5 pixel display in a single faster command.

Make the app to all the math!

The original 2541 SensorTag is a small device with a bunch of sensors – temperature, pressure, and more.  TI decided that “raw” access to the data was more important than “simple” access.  This is the temperature and pressure code that each app needs to write:

t_a = ((c0 * t_r / Math.Pow(2, 8) + c1 * Math.Pow(2, 6))) / Math.Pow(2, 16);
S = c2 + c3 * t_r / Math.Pow(2, 17) + ((c4 * t_r / Math.Pow(2, 15)) * t_r) / Math.Pow(2, 19);
O = c5 * Math.Pow(2, 14) + c6 * t_r / Math.Pow(2, 3) + ((c7 * t_r / Math.Pow(2, 15)) * t_r) / Math.Pow(2, 4);
p_a = (S * p_r + O) / Math.Pow(2, 14);
p_a = p_a / 100.0;

The newest SensorTag, the 1350, has values that can just be read directly. Except that they are the only 3-byte results I’ve ever seen.  Fun fact: there aren’t any helper libraries for reading in 3 byte values.

Ignore the math details!

Looking at you, Google Eddystone!  One of the values that an Eddystone beacon can produce is the temperature of the beacon; it’s documented to be a floating number in “8.8” format.

Specifically, here’s what the Eddystone spec says:

  • Beacon temperature is the temperature in degrees Celsius sensed by the beacon and expressed in a signed 8.8 fixed-point notation. If not supported the value should be set to 0x8000, -128 °C.

Now, that actually doesn’t look bad; there’s a (mediocre) example and a link to the exact details.  The link, however, goes to the main page for the current Cornell ECE 4760 course in Microcontrollers.  It’s not even to a specific lecture.  I eventually found lecture #11 to be useful.

(Weirdly, there’s a python library for the Ruuvi tag that says that the temperature data is in “8.8” format.  But they decode it using the RuuviTag way, where the second byte just has a value 0..99, and where the first byte actually isn’t in two’s complement.)

And some more little devices


Left to right: another MetaWear device, NOTTI, the 1350 SensorTag, and the AutoBike with a Bluetooth shifter

Pretend it’s serial!

The Magic Light BLE is a pretty standard Bluetooth-enabled, color-changing bulb.  It’s weirdness: that instead of having a simple characteristic for setting the color (like the TI beLight and pretty much all of the others!), they have a “serial protocol”.  There is a single characteristic to write data to, and a single one to read from, and you have to send in a set of bytes using some other protocol that they just made up.

For extra weirdness: they don’t think Bluetooth is reliable, so there’s a weird checksum on the command.

This is different from the puck.js device, where they do the same kind of thing, but it’s OK because the data you send is literally a stream of JavaScript commands (how cool is that?)

I also give a pass to my Autobike that has a computer-controlled continuously variable transmission with a stream of Bluetooth data.  It’s an older device from before all the BLE stuff was more standardized.  And I really like the bike.

And a word about my apps…

Some of my apps directly control devices: BERO Robots, Quirky Nimbus, Autobike, TI SensorTag 2541, Magic Light, TI BLE Lamp, MetaWear.  And my do-it-all Network Inspector gives a more raw approach to investigating Bluetooth devices.

And then there’s the Best Calculator, IOT edition.  It is programmable in BASIC (!), and the BASIC has a bunch of extensions to make all kinds of IOT and Bluetooth programming easy.  It’s got a free trial, and will save you time when you’re automating your life.  Give it a try!

Saturday, January 7, 2017

Measuring and Improving Variance in C# Task.Delay()

TL/DR: you can use both Task.Delay and SpinLock.SpinOnce() + StopWatch in order to delay by a precise amount while still allowing for a performant UI. 

Recently I tried to use the C# Task.Delay() in order to provide the delay in playing MIDI piano notes from old piano rolls that had been scanned in.  (Just playing the MIDI files didn’t meet my requirements for the project; I also wanted to animate the notes on a musical score and to highlight the notes on a virtual piano keyboard).  The results were disappointing: the resulting songs were musical, but with the note timings horribly off.



A quick investigation using the C# Diagnostics.StopWatch object showed why: the Task.Delay() was being asked to delay (in many cases) for 5 to 20 milliseconds.  MIDI are presented as a series of NOTE ON and NOTE OFF events; each includes a number of ticks to wait before performing the action.  In one typical MIDI file from a piano roll, there are 3778 notes with a median time to wait of 18.75 milliseconds; the Q1 point is 3.125 milliseconds and the Q3 at 62.5 milliseconds

Each call to Task.Delay() is not precise and often ends after the requested completion time.  This was called the “time overage” for the call.  The median time overage was 9.855 milliseconds with a Q1 of 5.3 milliseconds and Q3 of 12.34 milliseconds.  These overages were killing the musicality of the performance.

The first solution was to use a System.Threading.SpinWait lock for the correct number of milliseconds.  This worked as far as the music went, but the spin lock prevented the screen from being updated. 

The second solution was to do a Task.Delay() for each note except for the last 20 milliseconds of each note.  Each piano roll, as encoded by the MIDI files, includes plenty of longer requested time delays so that there are still plenty of Task.Delay() calls.  These calls enable the screen to stay refreshed.

The final data is quite pleasing.  The median time overage dropped from 18.75 milliseconds to a radically smaller 0.0188 milliseconds. 


Tuesday, April 19, 2016

The program that went “ding”!

I started with a simple requirement: build a timer program that, after 25 minutes, does a little “ding” sound, using the Universal Windows platform.

Making a sound is simple

The “make a sound” is simple enough; the fundamental API is the MediaElement, and it has a Play method that takes a stream.  Universal is a little awkward with streams; the final code:
  • Get the Windows.ApplicationModel.Package
  • from the InstalledLocation do a GetFileAsync passing in the normal  @“\Assets\Sounds\ding.wav” format.  Note my use of the @ in front of the quote.
  • given the file do an OpenAsync()
  • On the MediaElement do a SetSource() of the stream
  • Finally, you can Play().

Except when you don’t have focus

Unfortunately this runs into two problems, both discovered by an end user.  Firstly, it won’t ding when the app isn’t the foreground. Secondly, it doesn’t ding when the app is minimized.  You might think this is the same problem, but it’s not.
To work in the background you need to set the AudioCategory of the MediaElement to BackgroundCapableMedia.  This won’t work until you’ve also registered some callbacks to control the sound level; this is a requirement of the Windows OS.  Since all my sounds are short, I cheated, and the callbacks do nothing.
As a short aside, the other option is Communications; it’s the only other setting that works in the background.  However, my ding sounds are too short and don’t work properly.  Existing music would be faded out so users could hear the sounds, but that happed too slowly; by the time my sound was faded in, it was already over.  [Update: I'm also told by People At Microsoft that the Communications settings is wrong, wrong, wrong unless you are actually a communications app]

Convergence problems

Except that this is not fully converged.  For the phone you have to add callbacks to Windows.Media.SystemMediaTransportControls (getting the current one with GetForCurrentView()); the two callbacks you need are PropertyChanged and ButtonPressed.  On Desktop, you add callbacks to Windows.Media.MediaControl and you need a callback for about six different things.

App Manifest Woes

This still doesn’t work; you need to update the Package.appmanifest to declare that you do background audio.  Here the blogs are simply wrong, and present XAML that works until you try to compile for RELEASE mode.
The correct XAML is:
  <Extension Category="windows.backgroundTasks"
      <Task Type="audio" />

Note the EntryPoint item; the blog posts about this all mention a StartPage, which is wrong in this context.
But we’re still not done.

Hidden windows, quiet timers

So we have a great solution, and the app will ding when it’s minimized.  We can even prove it; run the app, wait for the ding, run minimized, and wait for the ding again.
That test turns out to be flawed.  IF you start the timer and minimize it before it has a change to ding the first time, it won’t ding in the background.  You have to make a foreground sound first.
Solution: add  a very short quiet ding right when the app starts. 
I also discovered that setting the ding to be looping would solve the problem, but it seemed like the first go-around would not play and then the second and subsequent ones would.

Oh, so you like listening to music?

There’s one problem I haven’t solved.  If you do all this on the phone, the “ding” will automatically stop your music from running.  On full (desktop/laptop/tablet) Windows, the music will simply fade out, let the ding happen, and then come back.  On the phone it stops dead and doesn’t resume.
This might not be bad, but the ding is actually optional; you can run in muted mode.  I normally run muted; otherwise it would distract by work mates.
  • Add a DingSilentlyOnce() that can do the silent ding
  • Call it at startup unless the user has the app muted
  • Call it when the user switches out of muted mode (either by pressing the mute button or by selecting a sound to play)
This still doesn’t solve the problem where the ding will stop my background music on the phone.


Making things go ding is only slightly painful until you need it to actually work, at which point four different changes are needed, none of which are properly documented.

Saturday, April 16, 2016

Giant List: The Art of English Murder: Writers on Project Gutenberg

The Art of the English Murder (“As Seen on the BBC as “A Very British Murder”) traces the current love of a good mystery back through time.  In addition to occasionally writing about programming, I also enjoy a good mystery.

Project Gutenberg is an awesome source of public-domain books that volunteers have scanned and proofread.  Naturally, it contains many of the authors listed in the book!  All Gutenberg book are free, but you should consider donating.

In this list, I’ve put all of the authors from the book whose works are available from Project Gutenberg.  The author is listed first with a link to Wikipedia, then the Gutenberg page for the author, and then links to specific mentioned works.

Often an author is mentioned in several chapters; I’ve happily duplicated all entries.  Sometimes an author is presented without any specific (traceable) work.  In these cases, only the page to the authors works are presented.

Not all chapters include references to traceable writers; these are simply skipped.  The later chapter deal with more modern writers whose works aren’t yet out of copyright are are therefore not in Project Gutenberg.


George Orwell: no listing

Thomas de Quincy: multiple entries including entries from the Lock and Key Library

Charles Dickens: multiple entries including  Bleak House

Dorothy L Sayers, creator of Lord Peter Whimsey : a single entry only; “Oxford Poetry” of which she’s the editor has has a single poem (and not a mystery)

Agatha Christie: two entries only

Margery Allingham: no entries

Ngaio Marsh: no entries

Graham Green: no entries

Chapter 1: A Connoisseur in Murder

Thomas de Quincy: multiple entries including Confessions of an English Opium Eater

Chapter 4: The Murder Circuit

Mary Shelley: multiple entries including Frankenstein, Frankenstein and  Frankenstein.

Elstree murder:

Thomas de Quincy: multiple entries

Thomas Carlyle:  multiple entries

Charles Dickens: multiple entries

Edward Bulwer-Lytton: multiple entries

Walter de la Mare: multiple entries

Sir Walter Scott: multiple entries

Chapter 6: True Crime

Shakespeare: multiple entries including Hamlet

Thomas de Quincy: multiple entries

Ann Radcliffe: multiple entries including The Mysteries of Udolpho

Jane Austen: multiple entries including Northanger Abbey

George Augustus Sala: entries for volumes 1 2 and 3 of The Strange Adventures of Captain Dangerous

Edward Bulwer-Lytton: multiple entries  including Pelham, or the Adventures of a Gentleman and Eugene Aram

Lytton Strachey: multiple entries including Eminent Victorians

Chapter 7: Charles Dickens

Charles Dickens: multiple entries  including Oliver Twist, exerpts from Household WordsBleak House

Newgate Calendar: entries for volume 1 and 2 of  The Newgate Calendar

Chapter 9: Stage Fright

Victoria and Albert Museum performance

Charles Dickens: multiple entries  including Bleak House

Chapter 11: Middle-Class Murderers and Medical Gentlemen

Thomas de Quincy: multiple entries including entries from the Lock and Key Library

Arthur Conan Doyle: multiple entries include Adventure of the Speckled Band

Chapter 13: Detective Fever

Wilkie Collins: multiple entries including Armadale and Moonstone

Charles Dickens: multiple entries 

Chapter 14: A New Sensation

Mary Elizabeth Braddon: multiple entries

Wilkie Collins: multiple entries including The Woman in White, Moonstone, Armadale

Dorothy L Sayers, creator of Lord Peter Whimsey : a single entry only; “Oxford Poetry” of which she’s the editor has has a single poem (and not a mystery) – and not The Gaudy Night, the book referenced in the chapter.

Sheridan le Fanu: multiple entries .  Sheridan is the research topic for detective Harriet Vane from Dorothy L. Sayer’s Gaudy Night.

Chapter 15: It is worse than a crime, Violet

Mary Elizabeth Braddon: multiple entries including Lady Audley’s Secret

Chapter 16: Monsters and Men

Robert Louis Stevenson: multiple entries including Doctor Jekyll and Mister Hyde 

Oscar Wilde: multiple entries including the Picture of Dorian Gray

Arthur Conan Doyle: multiple entries

Chapter 17: The Adventures of the Forensic Scientist

Arthur Conan Doyle: multiple entries including A Study in Scarlet and The Devil’s Foot

Eugène François Vidocq: multiple entries (all in French)

Completely missing from the book is my own favorite detective author, R. Austin Freemen: multiple entries including John Thorndkye’s cases

Chapter 18: Revelations of a Lady Detective

Andrew Forrester: a single entry which isn’t The Female Detective

Catherine Crowe: a single entry which isn’t The Adventures of Susan Hopley

Ann Radcliffe: multiple entries including The Mysteries of Udolpho 

Hans Christian Andersen: multiple entries none of which are really mysteries.  He’s mentioned as someone known by Catherine Crowe.

Charles Dickens: multiple entries but is just mentioned as a friend of Catherine Crowe

W. S. Hayward isn’t even part of Wikipedia or Project Gutenberg despite write Revelations of a Lady Detective

G. K. Chesterton: multiple entries including the Innocence of Father Brown

Chapter 19: the Women between the Wars

Agatha Christie: two entries only, neither of which is Murder is Easy or the Murder of Roger Ackroyd.

Edgar Wallace: multiple entries

Arthur Conan Doyle: multiple entries including Hound of the Baskervilles

John Buchan: multiple entries  including 39 steps and Greenmantle

E. W. Hornung: multiple entries  including several Raffles books.

Freeman Wills Crofts: one entry (the Pit Prop Syndicate)

G. K. Chesterton: multiple entries

Margery Allingham: no entries

Nicholas Blake: no entries

Chapter 20: The Duchess of Death

Agatha Christie: two entries only

Chapter 22: The Great Game

Dorothy L Sayers, creator of Lord Peter Whimsey : a single entry only; “Oxford Poetry” of which she’s the editor has has a single poem (and not a mystery)

Agatha Christie: two entries only

G. K. Chesterton: multiple entries

 Baroness Orczy: multiple entries

A. A. Milne: multiple entries including the Red House Mystery 

Ronald Knox: no entries

Arthur Conan Doyle: multiple entries

Charles Dickens: multiple entries

Wilkie Collins: multiple entries

Thomas de Quincy: multiple entries

Raymond Chandler: no entries

John Rhode (real name: Cecil Street): no entries

Ngaio Marsh: no entries

Chapter 23: Snobbery with Violence

Dorothy L Sayers, creator of Lord Peter Whimsey : a single entry only; “Oxford Poetry” of which she’s the editor has has a single poem (and not a mystery)

Colin Watson: no entries

Agatha Christie: two entries only

Val Gielgud: no entries

S. S. Van Dine: two entries under his real name as Willard Wright including Modern Painting: It’s Tendencies and Meanings but no mysteries.

Nicholas Blake: no entries

Ngaio Marsh: no entries

Edgar Wallace: multiple entries

Graham Green: no entries

Raymond Chandler: no entries

Chapter 24: The Dangerous Edge of Things

Dashiell Hammett: no entries

Raymond Chandler: no entries

Graham Green: no entries

Alfred Hitchcock: no entries

Postscript: the Decline of Murder

George Orwell: no entries

James Hadley Chase: no entries

Thomas de Quincy: multiple entries

Tuesday, January 26, 2016

XAML Binding a ComboBox to an Enum

Universal Apps: XAML Binding a ComboBox to an Enum

XAML sometimes has good news, and sometimes bad news for how easy we programmers can bind to types of lists.  Some "just work" and some are almost impossible.  The bad news is that most of the stackoverflow WPF solutions to binding a ComboBox to an enum don't work with Universal Apps.  The good news is that it's not actually hard to do.

The fundamentals are that
  1. You need to create a list of the enums.  A simple mechanism is something like Enum.GetValues(typeof(T)).Cast().ToList();
  2. XAML will happily bind a property of that list to a ComboBox ItemsSource
  3. Then bind the ComboBox SelectedItems to the an enum property

You will almost certainly want to display something a little nicer than just the enum values.  You can do this with a little IValueConverter.  It's possible to make a templated converter, so you can just pop in a standard class into your project.  The standard display XAML attribute isn't available for Universal, but it's trivial to make a small custom DisplayAttribute class. 

I've created a simple project with a nice helper classes; an entire sample project is now published on MSDN Samples pages along with complete instructions.

Saturday, October 3, 2015

Debugging your VisualState triggers

Windows 10 includes some the RelativePanel and new VisualState improvements to make it simpler to design “adaptive” displays.  But like all declarative languages, debugging is just simply painful.

To make the VisualState features work, you have to create a VisualStateManager in your grid (or similar); the VisualStateManager in turn as a VisualStateGroup; the VisualStateGroup has a set of VisualState items (you will always have two or more).  The VisualState in turn has a VisualState.StateTrigger (often an AdaptiveTrigger with a MinWindowsWidth) and VisualState.Setters to actually change your display.

Problem #1: if you mistype the name of your control, it’s just silently ignored.  It doesn’t even spit out a warning into the Output window.  If you are an inconsistent caser (like me), you’ll find this a rich productive bug farm.

Problem #2: the rules for the RelativePanel aren’t tricky, but you have to get used to them.  It’s tough to try to learn both the rules for RelativePanel while also learning how to make the AdoptiveTrigger work.

Problem #3: for a short while my C# project was set to “any CPU”.  It would “compile” and run with no warnings, and then run the last actual build.  All my “changes” would be completely ignored!

To fix your problems:

  1. Never set your project to “Any CPU”.  Yes, it used to be the awesome value, but now it’s the horrific value.  I don’t even know why they still include it as an option
  2. Never type your control names in the XAML designer.  Always do a cut-n-paste so that the name is guaranteed correct

Thursday, June 4, 2015

Data Binding -- good idea done badly, or bad idea implemented as well as can be hoped?

 The promise of data binding is that you can write your XAML UI code in one file, and then with a little bit of typing hook up the UI to your underlying data structures.  Which is great, until you have to do anything at all complex (e.g.,: I have a list of data, and I want a UI of all the data except for a subset of data that isn't right.

Worse, you get weird problems with data binding.  When you make a UserControl, you can just set up some public properties; the internals of your control bind to the properties, and everything is great.  Except you can't use a binding expression to set the values, and if you used the convenience this.DataContext = this in your constructor, you can't use binding to set common properties like 'Width' or 'Tags' any more.

Jerry Nixon has the answer in his Blog from 2013.

Short answer: use DependencyProperties and set (this.Content as FrameworkElement).DataContext = this; after you have this.InitializeComponent();'d.