Install a self-hosted matrix server (part 2)

How to install bridges in a self-hosted matrix

Install a self-hosted matrix server (part 2)

In part 1 we installed inside docker containers a fully-functional matrix server.

Here we'll setup bridges, in order to benefit full from what matrix brings to the game: connecting otherwise not-interoperable communication systems.

I thought I would do an explanation on how to setup bots, with discovering maubot and also setup a coturn server to be able to use audio / video, but I realise part 2 is long enough already so maybe in a 3rd part !

Bridges

Bridges are insane, these are the very reason you can have mutiple chat systems in one place.

IRC bridge on Freenode

The first one we're going to setup is the good old IRC.
Its config will live in the matrix-appservice-irc we create for this.

This one is pretty simple provided you have one or two things in mind.
The docker instructions are quite clear.

Here's a bare config that will be used for Freenode, it uses mostly the defaults you can find in the sample config:

homeserver:
  url: "https://matrix.YOURDOMAIN.COM"
  domain: "matrix.YOURDOMAIN.COM"
  enablePresence: true
ircService:
  servers:
    chat.freenode.net:
      name: "Freenode"
      port: 6697
      ssl: true
      sslselfsign: false
      sasl: false
      allowExpiredCerts: false
      sendConnectionMessages: true
      quitDebounce:
        enabled: false
        quitsPerSecond: 5
      modePowerMap:
        o: 50
        v: 1
      botConfig:
        enabled: true
        nick: "YOURNICK"
        username: "YOURIRCUSER"
        password: "YOURIRCPASS"
        joinChannelsIfNoUsers: true
      privateMessages:
        enabled: true
        federate: true
      dynamicChannels:
        enabled: true
        createAlias: true
        published: true
        joinRule: public
        groupId: +myircnetwork:localhost
        federate: true
      membershipLists:
        enabled: false
        floodDelayMs: 10000
        global:
          ircToMatrix:
            initial: false
            incremental: false
          matrixToIrc:
            initial: false
            incremental: false
        rooms:
          - room: "!fuasirouddJoxtwfge:localhost"
            matrixToIrc:
              initial: false
              incremental: false
        channels:
            ircToMatrix:
              initial: false
              incremental: false
        ignoreIdleUsersOnStartup:
          enabled: true
          idleForHours: 720
          exclude: "foobar"
      mappings:
          roomIds: ["!kieouiJuedJoxtVdaG:localhost"]
      matrixClients:
        userTemplate: "@irc_$NICK"
        displayName: "$NICK (IRC)"
        joinAttempts: -1
      ircClients:
        nickTemplate: "$DISPLAY[m]"
        allowNickChanges: true
        maxClients: 30
        ipv6:
          only: false
        idleTimeout: 10800
        reconnectIntervalMs: 5000
        concurrentReconnectLimit: 50
        lineLimit: 3
        realnameFormat: "mxid"
  bridgeInfoState:
    enabled: false
    initial: false
  ident:
    enabled: false
    port: 1113
    address: "::"
  logging:
    level: "debug"
    logfile: "debug.log"
    errfile: "errors.log"
    toConsole: true
    maxFiles: 5
  metrics:
    enabled: true
    port: 7001
    host: 127.0.0.1
    remoteUserAgeBuckets:
      - "1h"
      - "1d"
      - "1w"
  debugApi:
    enabled: false
    port: 11100
  provisioning:
    enabled: false
    requestTimeoutSeconds: 300
    ruleFile: "./provisioning.rules.yaml"
    enableReload: true
    roomLimit: 50
  passwordEncryptionKeyPath: "/data/passkey.pem"
  matrixHandler:
    eventCacheSize: 4096
  ircHandler:
  permissions:
   '*': admin
advanced:
  maxHttpSockets: 1000
  maxTxnSize: 10000000
sentry:
  enabled: false
  dsn: "https://<key>@sentry.io/<project>"
database:
  engine: "postgres"
  connectionString: "postgres://matrix_irc_user:CHANGEME@matrix-postgres-irc:5432/matrix_irc"

Just note the passwordEncryptionKeyPath: "/data/passkey.pem" entry: we will set that key inside the matrix-appservice-irc folder with the commandopenssl genpkey -out passkey.pem -outform PEM -algorithm RSA -pkeyopt rsa_keygen_bits:2048

Also note the database entry, here we set the postgres path to another db container which will be defined like this:

  matrix-postgres-irc:
    image: postgres:13
    restart: unless-stopped
    hostname: matrix-postgres-irc
    environment:
      - POSTGRES_PASSWORD=CHANGEME
      - POSTGRES_USER=matrix_irc_user
      - POSTGRES_DB=matrix_irc
    volumes:
      - ./.psqlrc:/root/.psqlrc:ro

Finally the bot has to be registered to Freenode, for that you can follow the wiki part related to that, it's well written.

Now this is where I had some troubles because I was certainly missing bits, you now need to register the bridge, for that you need a registration file, and you can generate it with the command from the docs

However you'll have to adapt it to the docker "spirit" ie thinking that localhost is the name of the host, so in the context of a doker-compose file the service name, here matrix-appservice-irc:9999.

And the node.js command runs in the container, meaning the config.yaml is in the mounted volume, ie in the container at /data/config.yaml, we need to specify the network flag otherwise it won't "see" the postgres container declared above:

docker run --rm --network=synapse_default -v $(pwd)/matrix-appservice-irc:/data matrixdotorg/matrix-appservice-irc node app.js -r -f appservice-registration-irc.yaml -u "http://matrix-appservice-irc:9999" -c /data/config.yaml -l irc_bot

The corresponding part in the compose file is the following:

  matrix-appservice-irc:
    image: matrixdotorg/matrix-appservice-irc@sha256:13f3c1a29394301690d8a36fe557693d40eeb2f8f17b4f413c7a987f0e409277
    restart: unless-stopped
    #entrypoint: ["sh", "-c", "sleep 2073600"]
    volumes:
      - ./matrix-appservice-irc:/data
    depends_on:
      - matrix-postgres-irc

You'll get this:

id: SHOULDBEGENERATED
hs_token: SHOULDBEGENERATED
as_token: SHOULDBEGENERATED
url: 'http://matrix-appservice-irc:9999'
sender_localpart: irc_bot
de.sorunome.msc2409.push_ephemeral: true
protocols:
  - irc
namespaces:
  rooms:
    - exclusive: false
      regex: '!kieouiJuedJoxtVdaG:localhost'
  aliases:
    - exclusive: true
      regex: '#irc_.*:matrix\.YOURDOMAIN\.COM'
  users:
    - exclusive: true
      regex: '@irc_.*:matrix\.YOURDOMAIN\.COM'
rate_limited: false

That was really the only "difficulty", and the final step is to use that appservice-registration-irc.yaml in the synapse container.
Now you get why, as we saw in part 1, this file is mounted in the synapse container and declared in the homeserver.yaml file:

app_service_config_files:
  - /mautrix-whatsapp/registration.yaml
  - /mautrix-signal/registration.yaml
  - /matrix-appservice-irc/appservice-registration-irc.yaml

You're set, run the matrix-appservice-irc container, and now you can send a PM to the Freenode NickServ ie @irc_NickServ:matrix.YOURDOMAIN.COM in order to register your matrix user, but this time in the riot app !

20210421_1617_874x705_1619014658

For the rest, joining channels etc you can read the docs, but the TLDR version is: you send a PM to the bridge bot you just set (his/her name is the one you send in the registration file, see sender_localpart: irc_bot ) and that's it !

I found the rest of the docs quite clear, just the beginning in the context of docker required some small tweaks.

Whatsapp bridge

The explanation will be shorter because it follows the same principles, and now we've seen that localhost is local in the context of the compose network created, there are no headaches anymore !

The relevant compose part looks like this:

  mautrix-whatsapp:
    image: dock.mau.dev/tulir/mautrix-whatsapp:v0.1.6
    restart: unless-stopped
    volumes:
      - ./mautrix-whatsapp:/data
    environment:
      - UID=1000
      - GID=1000
  mautrix-whatsapp-postgres:
    image: postgres:13
    restart: unless-stopped
    hostname: mautrix-whatsapp-postgres
    environment:
      - POSTGRES_PASSWORD=CHANGEME
      - POSTGRES_USER=mautrix_whatsapp_user
      - POSTGRES_DB=mautrix_whatsapp
    volumes:
      - ./.psqlrc:/root/.psqlrc:ro

Now the docs are very clear there so there is not much to add.
The process is simple and the same as before:

  1. run the container once to get a sample config
  2. edit it to your liking
  3. run another time the container and get the resistration file
  4. add the registration to the synapse config
  5. restart synapse container

The config.yaml will look like this, the important bits are as always:

1.the homeserver where you set you matrix url,
2.the appservice will also need to contain your container address if you changed it,
3.the database part is straightforward, except maybe the sslmode=disable which I found I can't remember where now
4. finally don't forget to setup the permissions part

The final yaml will look like this:

homeserver:
  address: https://matrix.YOURDOMAIN.COM
  domain: matrix.YOURDOMAIN.COM
  asmux: false
appservice:
  address: http://mautrix-whatsapp:29318
  hostname: 0.0.0.0
  port: 29318
  database:
    type: postgres
    uri: postgresql://mautrix_whatsapp_user:CHANGEME@mautrix-whatsapp-postgres:5432/mautrix_whatsapp?sslmode=disable
    max_open_conns: 20
    max_idle_conns: 2
  provisioning:
    prefix: /_matrix/provision/v1
    shared_secret: disable
  id: whatsapp
  bot:
    username: whatsappbot
    displayname: WhatsApp bridge bot
    avatar: mxc://maunium.net/NeXNQarUbrlYBiPCpprYsRqr
  as_token: GENERATED
  hs_token: GENERATED
metrics:
  enabled: false
  listen: 127.0.0.1:8001
whatsapp:
  os_name: Mautrix-WhatsApp bridge
  browser_name: mx-wa
bridge:
  username_template: whatsapp_{{.}}
  displayname_template: '{{if .Notify}}{{.Notify}}{{else}}{{.Jid}}{{end}} (WA)'
  community_template: ""
  connection_timeout: 20
  fetch_message_on_timeout: false
  delivery_receipts: false
  login_qr_regen_count: 2
  max_connection_attempts: 3
  connection_retry_delay: -1
  report_connection_retry: true
  aggressive_reconnect: false
  chat_list_wait: 30
  portal_sync_wait: 600
  user_message_buffer: 1024
  portal_message_buffer: 128
  call_notices:
    start: true
    end: true
  initial_chat_sync_count: 10
  initial_history_fill_count: 20
  initial_history_disable_notifications: false
  recovery_chat_sync_count: -1
  recovery_history_backfill: true
  chat_meta_sync: true
  user_avatar_sync: true
  bridge_matrix_leave: true
  sync_max_chat_age: 259200
  sync_with_custom_puppets: true
  sync_direct_chat_list: false
  default_bridge_receipts: true
  default_bridge_presence: true
  login_shared_secret: ""
  invite_own_puppet_for_backfilling: true
  private_chat_portal_meta: false
  bridge_notices: true
  resend_bridge_info: false
  whatsapp_thumbnail: false
  allow_user_invite: false
  command_prefix: '!wa'
  encryption:
    allow: false
    default: false
    key_sharing:
      allow: false
      require_cross_signing: false
      require_verification: true
  permissions:
    '*': 5
    '@admin:matrix.YOURDOMAIN.COM': 100
    matrix.YOURDOMAIN.COM: 10
  relaybot:
    enabled: false
    management: '!foo:matrix.YOURDOMAIN.COM'
    invites: []
    message_formats:
      m.audio: <b>{{ .Sender.Displayname }}</b> sent an audio file
      m.emote: '* <b>{{ .Sender.Displayname }}</b> {{ .Message }}'
      m.file: <b>{{ .Sender.Displayname }}</b> sent a file
      m.image: <b>{{ .Sender.Displayname }}</b> sent an image
      m.location: <b>{{ .Sender.Displayname }}</b> sent a location
      m.notice: '<b>{{ .Sender.Displayname }}</b>: {{ .Message }}'
      m.text: '<b>{{ .Sender.Displayname }}</b>: {{ .Message }}'
      m.video: <b>{{ .Sender.Displayname }}</b> sent a video
logging:
  directory: ./logs
  file_name_format: '{{.Date}}-{{.Index}}.log'
  file_date_format: "2006-01-02"
  file_mode: 384
  timestamp_format: Jan _2, 2006 15:04:05
  print_level: debug
  print_json: false
  file_json: false

Finally the registration file will be in the mautrix-whatsapp directory when generated but shared with the synapse container as we previously did for the irc bridge.