OK sorry for the downtime. What happened is that I added a feature which added some new settings.
This is all fine and good.
Each user's settings are stored as pickled python objects in the DB. Not the best choice but also not really a problem. Weirdly when the updated settings were written back to the DB the data was corrupted by mysql!
"Invalid utf8mb4 character string: '800495'"
When the settings would get read back out moa would crash.
Fortunately I had a backup of the DB from before the update. The bad thing is that the workers had run after that DB dump was made so I couldn't be sure that the message IDs hadn't been updated so restoring ran the risk of duplicate messages.
I wrote some code to dump the pickled settings from the good db backup out to files, moved them to the server, and restored just the good settings.
It was a big mistake to store the settings as pickled objects so fixing that is my top priority.