Mastodon on balena

Use this project to run Mastodon- the open-source microblogging platform- on balena

What

Mastodon is a popular open-source microblogging platform. It uses the Activity Pub specification to implement the federation. In practice, it means that one can subscribe and interact with users from external servers and see activity across the network.

Why

Setting up your own Mastodon server from scratch requires some effort. At a minimum, you’ll need your own domain, mail service account, and fairly deep technical knowledge to follow the installation steps. There are many hosting solutions out there that can furnish a server for a moderate subscription fee.

For those who want to have more control, balenaCloud provides an attractive alternative. If you have an old laptop or an inexpensive device like Intel NUC or Raspberry Pi, balena makes it a breeze to deploy Mastodon, with one noticeable benefit: you don’t even need your own domain!

How

Here are a few steps required to set up your own Mastodon server. There are some footnotes with details and alternatives you might find useful as well.

  1. Create an account on balena-cloud.com
  2. Click on “Getting Started” for a step-by-step walkthrough of onboarding your device with balena.
  3. Clone this git repo locally git clone https://github.com/dsevastianov/balena-mastodon. Now it’s time to update the configuration.
    • Open .gitignore and uncomment .balena. This is a directory with all the secrets, we want to make sure it’s not pushed to some git branch by mistake.
    • Open docker-compose.yml and update LOCAL_DOMAIN. This is your domain for federation, it cannot be changed once the server is up and running. Set it to your custom domain or balenaCloud address found in the balenaCloud dashboard (Public Device URL). It should look like abc.domain.com (no slashes).
    • (Optional) Open .balena/secrets/.env.production and update the SMTP section. By default it’s configured for Gmail, just update your credentials if this is your service of choice. Save and close the file. This step is optional to run the server, however, without SMTP credentials, new users will not be able to register on your server.
    • This step requires Docker installed. Run config.sh (or config.ps1 if you’re on Windows) in the shell. It will generate keys using the Mastodon image and update the config file accordingly.
  4. Finally, run balena push <your_fleet_name> from the root of the cloned repo. Once you see a unicorn (it may take a few minutes), you may want to follow it with balena logs <device_id> -t to monitor the deployment. If there’s an error in your configuration, please don’t worry – you can always run the push command again, and your data will survive the releases.

  1. Open the public device URL in a new browser window. It’s time to create your admin user.
    • Enter your username, email, and password, check the “I agree” box, and click Sign Up. Mastodon will send you an email with the link to confirm (if SMTP is appropriately configured). Feel free to ignore it, we will have to use Mastodon admin utility to set permissions anyway.

  • Head to the Balena-cloud dashboard for the device. Open terminal session for web service. This is where you can run all the Mastodon admin tasks, as described here. For now, we only need to give our new user admin rights.

  • Execute tootctl accounts modify <your_user_name> --role admin --confirm --approve

  • Click the “Account settings” link on the Mastodon confirmation page or just head straight to the home page.
  • If you are completely new to Mastodon and want to see how things work, open mastodon.social and find someone interesting.

  • Paste their username into the search box to follow them.

  • Over time, you will see the activity from this user on your timeline. More details on interacting with Mastodon instances can be found here. Enjoy!

Technical details

A bit about the journey.

Proxy

To figure out how to deploy Mastodon with minimum effort on balena, I started with the Dockerfile from the Mastodon repo. It quickly became apparent that a reverse proxy is needed to deal with SSL termination. My first choice was Traefik, as it offers many attractive features including dynamic configuration support and a built-in dashboard, and is actively developed. I was able to find some nice walkthroughs on the internet as well.

The first major problem was a redirect loop on the Traefik level. The trickiest thing to figure out was that the Mastodon server is never exposed to HTTPS traffic directly, as was pointed out here. So I had to explain to Traefik that yes, Mastodon is expecting HTTPS traffic and yes, it also has to be exposed to HTTP traffic for the redirect to work. Also, I had to explicitly set X-Forwarded-Proto and X-Forwarded-Port headers as they are stripped on SSL termination.

So far so good, but after running the service for some time I noticed that it freezes periodically, for no apparent reason. One of the obvious directions for improvement would be to enable caching for static files. It turned out that Traefik has limited support for this at the moment, and I was getting somewhat frustrated with the sheer size of its configuration.

The next experiment was to try Caddy, an open-source web server written in Go. I used this guide to start. Not only did those freezes go away completely, but I was also able to configure it to serve the static files directly, with noticeable improvement in the performance. The configuration is way more straightforward than that of Traefik as well.

Configuration

One of my goals was to make the deployment on balenaCloud as simple as possible. Mastodon installation requires the execution of multiple steps, including the generation of secrets, manual creation of the mastodon user and database structure in Postgresql, and so on.

DB

Mastodon is a Ruby On Rails service built around ORM model, hence all of the database manipulations are done from the Ruby code as opposed to through database scripts. I ended up adding standard postgres-client to the Mastodon image and scripting the database configuration as a part of the Mastodon server startup in docker-compose.yml. In theory, this solution has the disadvantage of slowing server startup times but I couldn’t notice any practical difference.

Secrets

This left me with the necessity of prompting a customer to add SMTP configuration and other sensitive data before the deployment of the service. Luckily, balena supports this scenario. Some of the secrets are very specific to Mastodon and pushing their handling to the customer felt wrong. The solution was to use the Mastodon Docker image to generate the secrets as prescribed and update the configuration file automatically.

Zero-configuration

I was thinking of making the whole process completely automated, so a customer could deploy a functioning service with zero configuration. It is possible to do, but it requires 1) abandoning support for custom domains and 2) ignoring SMTP configuration. It’s even possible to create an administrator account automatically, except that I would need to communicate the new password back to the user, in some secure manner. I decided against this approach because the custom domain feature seems too important to circumvent.

What’s next

The scope of this project is limited to a proof of concept. Here are a few possible directions to take from here:
* Scaling – the project was tested with only a few users/follows from other instances, so I don’t have a very good idea yet at which point the performance and space limitations will kick in. Mastodon documentation provides some guidance on tweaking the configuration accordingly. It also recommends weekly media cleanups to preserve the space, which can be baked in into the solution as well
* Mastodon supports external storage for user content, anything compatible with Amazon S3 or SWIFT should work
* ElasticSearch: current Mastodon implementation makes limited use of full-text search. It is relatively straightforward to enable
* Tor – Mastodon also supports “hidden services” to make it available on the darknet. Here’s the Caddy configuration to make it work

Conclusion

When reflecting on this journey, I have to admit that the most challenging part for me was to fully embrace the docker mindset. I spent a lot of time digging through Mastodon source code, until I realized that Docker gives enough flexibility to focus on deployment and configuration exclusively, without even knowing much about the implementation.

Hopefully, this project makes open-source software like Mastodon more accessible for non-programmers. The project can be a useful starting point for porting other Ruby on Rails services on the balena platform in the future.

Posted

in

Dmitry Sevastianov

Tags:

,