Self-Hosted Email Server

Prerequisites

• A VPS running Ubuntu 24.04 (recommended) or Debian
• A domain name with DNS access
• SSH access to the server

Secure SSH Access

After logging in, update your system:

sudo apt update && sudo apt upgrade -y

Edit the SSH configuration:

sudo vim /etc/ssh/sshd_config

Change the following settings (uncomment if needed):

Port 22222
PasswordAuthentication no
UsePAM no

Restart SSH:

sudo systemctl restart sshd

Setup Firewall

Mailcow uses Docker which requires iptables (not UFW). If UFW is installed, disable it:

sudo ufw disable
sudo apt remove ufw -y

Install iptables:

sudo apt install iptables iptables-persistent -y

Create a firewall script:

sudo vim /root/firewall.sh

Add the following:

#!/bin/bash

# Flush existing rules
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Set default policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT

# Allow loopback
iptables -A INPUT -i lo -j ACCEPT

# Allow established and related connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH
iptables -A INPUT -p tcp --dport 22222 -j ACCEPT

# Mailcow ports
iptables -A INPUT -p tcp --dport 25 -j ACCEPT    # SMTP
iptables -A INPUT -p tcp --dport 465 -j ACCEPT   # SMTPS
iptables -A INPUT -p tcp --dport 587 -j ACCEPT   # Submission
iptables -A INPUT -p tcp --dport 143 -j ACCEPT   # IMAP
iptables -A INPUT -p tcp --dport 993 -j ACCEPT   # IMAPS
iptables -A INPUT -p tcp --dport 110 -j ACCEPT   # POP3
iptables -A INPUT -p tcp --dport 995 -j ACCEPT   # POP3S
iptables -A INPUT -p tcp --dport 4190 -j ACCEPT  # Sieve
iptables -A INPUT -p tcp --dport 80 -j ACCEPT    # HTTP
iptables -A INPUT -p tcp --dport 443 -j ACCEPT   # HTTPS

# Save rules
iptables-save > /etc/iptables/rules.v4

echo "Firewall configured"

Make it executable and run it:

sudo chmod +x /root/firewall.sh
sudo /root/firewall.sh

Setup DNS Records

Configure your domain's DNS with the following records (replace example.com with your domain and YOUR_VPS_IP with your server's IP):

The MX record tells other mail servers to deliver emails for @example.com to mail.example.com.

Install Required Packages

sudo apt install -y git openssl curl gawk coreutils grep jq

Install Docker

Add Docker's official GPG key:

sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings

For Ubuntu:

sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

For Debian:

sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/debian
Suites: $(. /etc/os-release && echo "$VERSION_CODENAME")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

Install Docker:

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Install Mailcow

sudo umask 0022
cd /opt
sudo git clone https://github.com/mailcow/mailcow-dockerized
cd mailcow-dockerized
sudo ./generate_config.sh

When prompted:

• FQDN: Enter mail.example.com (not just example.com)
• Timezone: Press enter for default or enter your timezone
• Branch: Choose 1 for stable
• IPv6: Say Y if prompted to create IPv6 settings

Configure Mailcow

Edit the configuration:

sudo vim mailcow.conf

To save resources, disable ClamAV antivirus:

SKIP_CLAMD=y

Edit Docker Compose file:

sudo vim docker-compose.yml

In the redis-mailcow section, comment out the sysctls lines (fixes an issue on some VPS):

#sysctls:
#  - net.core.somaxconn=4096

Start Mailcow

Pull images and start:

sudo docker compose pull
sudo docker compose up -d

Wait about 5 minutes, then access the admin panel at:

https://mail.example.com

Default credentials: admin / moohoo

Secure the Admin Account

1. Go to System → Configuration
2. Under Access → Administrators, click Edit next to the admin user
3. Set a strong password
4. Enable Time-based OTP for 2FA

Add Your Domain

1. Go to E-Mail → Configuration
2. Click the Domains tab
3. Click Add Domain
4. Enter your domain (e.g., example.com)
5. Click Add domain and restart SOGo

Add DKIM and DMARC DNS Records

After adding your domain:

1. In the Domains list, click the DNS button next to your domain
2. Copy the DKIM key value

Add these DNS records:

You can verify your DNS configuration using: https://docs.mailcow.email/getstarted/prerequisite-dns/#testing

Create a Mailbox

1. Go to E-Mail → Configuration
2. Click the Mailboxes tab
3. Click Add Mailbox
4. Set username, full name, and password
5. Under Encryption policy, select:
   • Enforce TLS incoming
   • Enforce TLS outgoing
6. Uncheck Direct forwarding to SOGo (allows easier access to admin panel)
7. Save

Log out and log back in as the user (click "Login as user" instead of admin).

Set up 2FA for the user account, then click the blue button to access webmail.

Sending Emails (Relay Service)

Most VPS providers block outbound port 25 to prevent spam. You can receive emails, but sending requires a relay service.

How it works:

• Your server connects to the relay on port 587 (not blocked)
• The relay delivers using their servers on port 25
• Emails still appear from your domain

Popular relay services:

• SendGrid
• Mailgun
• Amazon SES

To configure a relay in Mailcow:

1. Go to System → Configuration
2. Under Routing, configure your relay host with the credentials from your relay service