Show more

Fortunately I had a backup of the DB from before the update. The bad thing is that the workers had run after that DB dump was made so I couldn't be sure that the message IDs hadn't been updated so restoring ran the risk of duplicate messages.

I wrote some code to dump the pickled settings from the good db backup out to files, moved them to the server, and restored just the good settings.

It was a big mistake to store the settings as pickled objects so fixing that is my top priority.

(2/2)

OK sorry for the downtime. What happened is that I added a feature which added some new settings.
This is all fine and good.

Each user's settings are stored as pickled python objects in the DB. Not the best choice but also not really a problem. Weirdly when the updated settings were written back to the DB the data was corrupted by mysql!

"Invalid utf8mb4 character string: '800495'"

When the settings would get read back out moa would crash.

(1/2)

@windd_imas

Terribly sorry but right after you hooked up Moa I had to rollback the database to address a serious bug. You'll have to set up the connection again.

Very sorry!

Unfortunately moa is down at the moment. I'm trying to squash a bug.

That was 9 days and now Moa is processing 405 bridges.

@limedaring The delay in cross-posting can be up to 5 minutes.

I just discovered that the Instagram API is hanging for *minutes* at a time. To work around this I'm randomzing the batches of accounts so that the sames ones don't always get stuck behind an Instagram job.

Last night I added a deferral function to cause a host to be skipped for 10 minutes if a worker gets a connection error. So far it's been working really well. The ongoing connection problems with mastodon.social aren't causing big delays like they normally would.

Welcome to all of the python devs that are signing up!

🐍🍍🐍🍍🐍🍍

I added 3 more workers to split up the queue and then mastodon.social stopped responding causing the workers to hit their time limits Β―\_(ツ)_/Β―

I'm going to add a feature to put a server in the penalty box if it responds with an error or timeout. All other accounts at that server will be skipped until a time limit is reached. Maybe 10 minutes?

Welcome to user #420 @adambrenecki@mastodon.social

πŸš¬πŸ‘½

If the per-bridge time doesn't change Moa should be able to scale up to ~1000 bridges and maintain a 5-minute delivery schedule.

It takes a worker about 90 seconds right now to process ~130 bridges. That's about 1.4 seconds to check mastodon, twitter, and possibly instagram for new messages.

I added queue splitting to keep the worker run times under control.

If there was some important change made in Moa how would you feel about receiving a DM about it?

Wish list:

β€’ remove the requirement for a twitter account so instagram->mastodon is possible without one.
β€’ Add support for Instagram->Pixelfed
β€’ More stats!

moa.party boosted

Thanks to @moa_party we're now tooting tweets and vice versa so expect to see a lot more here!

moa.party boosted

Thanks to @moa_party we're now tooting tweets and vice versa so expect to see a lot more here! tweeted by @LubuntuOfficial

Show more
pdx.social

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!