Prometheus is a great tool for monitoring all kinds of infrastructure or apps. There are “exporters” for many kinds of services, like Node exporter for Linux servers. When running a Django app, one may want to add custom metrics to monitor. Like number of new users, number of failed logins and other actions specific to your app. For those cases, the Prometheus Python Client may be a good fit. Despite the name, it offers all the tools needed to build your own exporter with custom metrics.

Borg is generally a very reliable backup tool. So it’s relatively rare to see broken repositories after minor issues, like network interruptions. More serious damage would be caused by issues with the underlying storage and file system. In such cases it can be necessary to run “extended” repair on the repo to save whatever can be saved and then continue using the repo by re-adding the missing data. I couldn’t find much documentation on this, except for an old Github issue.

One major drawback of offering your own IMAP/SMTP-based email service is that users need to enter a long list of details to set up their email client. Unfortunately there is no unified technique to make this step easier. You can still cover a good number of email clients by following the techniques below. This only covers the basic use case and you may want to look up further details if your use case is more specialized.

Spammers are using Google Forms to send their messages and Google hasn’t addressed the problem despite many discussions about it. If you are using Rspamd, Regex multimaps can help tackle the issue: First add a new rule for blacklisting based on content: # local.d/multimap.conf CONTENT_BLACKLISTED { type = "content"; filter = "oneline"; # can be headers, full, oneline, text, rawtext map = "${LOCAL_CONFDIR}/custom/bad_content.map"; symbols = ["IS_GOOGLE_FORM"]; regexp = true; } Then add some regex rules.

I’m a big fan of using hardware keys to secure important services, since they are even more secure that OTP tokens. If you are currently using a Yubikey (or similar) to secure services, you will be happy to hear that starting today, you can use your hardware key directly with OpenSSH in FIDO2/U2F mode. The relevant PR was just merged earlier today and it works as expected: Be sure to update or install the Homebrew openssh package: $ brew upgrade openssh Insert your hardware key Generate a new SSH key using: $ ssh-keygen -t ecdsa-sk -f ~/.

Today I finally got to upgrading our BorgBase.com Ceph cluster in the EU region from Mimic to Nautilus. The process is rather simple using Ceph-Ansible’s rolling-update.yml playbook, but I still want to document some gotchas and collect links to resources that may be useful for others.

I’m a big fan of Rspamd for filtering your emails for spam. One thing I wanted to achieve is to reject viruses from logged-in users. This would mainly happen if a user machine is compromised. Rspamd makes this quite easy using the user settings module. Using the snipped from that website, you can reject all emails with a score higher than e.g. 15. On the other hand, greylisting and adding a spam header is disabled for authenticated users.

Certain banking apps and e-wallets refuse to start on custom ROMs. Unrooting doesn’t seem to help much. The issue seems to arise from “insecure” settings in the ROM’s properties file. To reconfirm you can use the Rootbeer app. I suspect many apps use their library to check for signs of root access.

Update Feb 2020: You can now use a Yubikey directly via OpenSSH 8.2 in FIDO2/U2F mode. SSH is critical in most people’s devops process, be it remote server logins or Git commits. After reading about one too many stories about companies getting hacked that way, I decided to use Yubikeys to store my private SSH keys. You can either use the PIV- or OpenPGP module for this purpose. I decided to use the former because it’s better integrated and seems to be more reliable.

Update 2021: See here for another great collection of different email spoofs by Kaspersky. Recently I noticed a rise in spam emails with spoofed From-headers. This is unusual because almost all mail servers will require users to log in before sending emails. Below a typical example, which was already flagged as spam.As you can see, the sender- and recipient addresses are the same in Apple Mail. The content implies that the user’s mailbox was hacked.