Self-hosted Weekly #14

so much for sending the next newsletter from the beach/pool… :-D

Unfortunately our vacation was cut short due to the Corona situation escalating further and further and more and more countries restricting (air-)travel. In the end we had to pack our luggage again last tuesday, the alternative would have been to stay at least until the 16th of april.

Tools for keeping in contact

All around the world people are encourage to stay indoors and refrain from meeting in larger groups. All in all to limit potential exposure to the Corona virus and therefore “flatten the curve”. Luckily there are quite a lot of tools available to avoid falling into the trap of external systems when it comes to staying in contact with the outside world.

Run your own TURN service

coturn is a popular TURN and STUN service. A TURN server is needed you want tu use a WebRTC based video app (like Kopano Meet or Jitsi) and is in utilized to help establishing connections between peers in a session. With coturn-docker-image there is a small and regularly updated containerized variant of it.

Real time messaging

There are quite a lot of projects aiming to replicate and evolve the functionality of hosted services like Slack or Microsoft Teams. Based on my experience the most popular at the moment are Rocket Chat, Mattermost and Riot. But of these only Riot focusses on federation with other installations and services in an effort to not trap the admin on his own island. But sadly this also means that Riot with all its components is also not the easiest to setup.

In how blog Setting up Matrix and Riot with docker Melvyn shows to use the existing official containers images to spin up a local setup with docker-compose.

Online discussion board for less real time communication

Probably as old as the internet itself online message boards or forums have always been a place for open, non-realtime and continuos discussion. One of the serious pieces of open-source in this area is at the moment Discourse.

Written in Ruby it can scale much easier than former php applications were able to do. For when the vanilla functionality is not enough it can be extended through plugins to add e.g. LDAP login functionality.

Self-hosted social networks

Mastodon is already a well established alternative to hosted services such as Twitter, but sadly its a bit on the heavier side when you want to actually run it on your own. More lightweight alternative Pleroma recently reached its 2.0 milestone.

Tools for remote work

Kanban is a popular approach to visualize tasks and their progress and luckily there are two quite popular open-source tools to host ones own Kanban boards.

In the past (probably already four years) I have used Kanboard for quite a while, but sadly it was not really useable on mobile devices and sometimes felt a bit dated when it came to UI design. Another more modern looking solution is Wekan. If I wanted to set up a shared system today I’d probably give Wekan a try first.

Easy access to remote resources

With people encourage to work from home the usage of VPNs probably saw quite a rise in the last days. At my place of work we have locked down quite some resources to only allow access from within the office, even though these may be servers hosted externally limiting access from only certain locations improves security.

A tools that I have used quite extensively in the past is sshuttle instead of relying on tools like OpenVPN, sshuttle just needs an ssh endpoint to connect to and will afterwards relay traffic for a given ip segment. In my case I now utilise a reverse ssh tunnel that connects from a server in our office to a system running at my home. This proxied connection is my entry point for sshuttle.

In addition to this I am using a small collection of bash functions to tunnel to the desired systems without the need to mess with routes in my network stack. I cannot remember from where I originally got the functions, but I’ll add them below for anybody else interested:

# set default SSH server:
#   user@hostname or a host in ~/.ssh/config
TUNNEL_SERVER=office-tunnel

## tunnel all traffic including dns
alias tunnel='sshuttle --dns -vr $TUNNEL_SERVER 0.0.0.0/0'

# returns a list of IP addresses from given domain(s).
#
# Examples:
#  dns2ip google.com
#  dns2ip netflix.com movies.netflix.com
#
function dns2ip() {
        dig +short $* | sed "/[^0-9\.]/d" # use sed to remove non-IPv4 line e.g. alias
}

# tunnel specified domain(s) only.
#
# Examples:
#  tnlonly google.com
#  tnlonly netflix.com movies.netflix.com
#
function tunnel-only() {
        sshuttle -vr $TUNNEL_SERVER `dns2ip $*`;
}

# tunnel all traffic including DNS, except the specified domain(s).
#
# You can exclude certain TCP traffic using -x option.
#
# Examples:
#  tnlbut youku.com
#  tnlbut youku.com weibo.com
#
function tunnel-except() {
        sshuttle --dns -vr $TUNNEL_SERVER `dns2ip $* | sed "s/^/-x/"` 0/0; # use sed to append '-x' prefix
}

# vpn to a ssh server.
#
# -N flag tells sshuttle to figure out by itself the IP subnets to forward,
# and -H flag to scan for hostnames within remote subnets and store them temporarily in /etc/hosts.
#
# Examples:
#  vpnto my_office_server      # host in ~/.ssh/config
#  vpnto user@123.123.123.123
#
function tunnel-vpn() {
  sshuttle -HNvr $1;
}

#alias tunnel-root='sshuttle --dns -vr $TUNNEL_SERVER 192.168.101.0/0'

## run said tunnel in background
alias tunnel-bg='sshuttle --daemon --pidfile=/tmp/sshuttle.pid --remote=$TUNNEL_SERVER 0.0.0.0/0'
alias tunnel-kill='[[ -f /tmp/sshuttle.pid ]] && kill $(cat /tmp/sshuttle.pid) && echo "Disconnected."'

The above functions in my bashrc now allow me to simply call tunnel-only intranet.lan whenever I want to reach the systems available at this address.

Projects that provide easy shell access

Inspired through ShellHub, Teleport and ttyd I was looking for other applications that allow to easily expose terminal applications to the web.

Cloud Commander

Cloud Commander offers a classic two pane file manager and even bundles file viewers and editors as well as an online terminal.

TTY-share - Teamviewer for the terminal

tty-share enables Teamviewer like remote sessions for the Terminal. If remote assistance is needed the user just runs a command and shares the generated url, whoever has the url can then directly connect from a browser to an existing session.

Putting ssl certificates in Vault

In my local network I am using a central instance of Dehydrated to obtain certificates from Let’s Encrypt. These I am then copying to their final location with the hooks functionality of Dehydrated. One thing I wanted to look into during my holiday was instead utilizing Hashicorp Vault to securely store them and then simply fetching them on the required system. I already found a nice project that utilizes the hook system of Dehydrated to achieve just this.

The only missing pieces are then a Vault setup and a simple client to download the data from vault.

Polemarch - a web client for Ansible

Polemarch is an application to manage ones infrastructure through Ansible. Through its web based approach it makes Ansible easier to manage, control and share with other team members.

Monitoror - Easy wallboards

Monitoror is an application that can be used to provide a non-interactive, but still real time dashboard to visualize the status of e.g. open amount of tickets, current CI runs, etc.

Easily deploy Kubernetes clusters

k3sup can be used to easily deploy Kubernetes to any system reachable via SSH. In its scope its similar to kind which uses Docker to achieve easy local Kubernetes environments.

Dockerfiles - Another easy homeserver setup

Dockerfiles is a collection of ready made Dockerfiles and accompanying systemd Units to manage services encapsulated in containers. Its a bit barebones compared to the previously shared solution but also showcases that you don’t need a fancy new system to manage your applications, but can just use a bit of local code and systemd.

I am always looking for new projects to try out! Just send an email to selfhosted-newsletter@9wd.eu.

I hope you have enjoyed this issue. If this newsletter was useful to you please recommend it to colleges or tell them to subscribe to it via RSS.

Stay safe! Felix