Installing Sendmail/Postfix on Ubuntu and configuring your Ghost blog to use it

If you are using one of the "recommended" email providers such as Mailgun you should be fine following the official setup guide. Unfortunately, if you are hosting your email elsewhere (or, God forbid, running your own MTA), then Ghost can quickly let you down. For me, I have my mail hosted at Lunarpages - and Ghost refused to play nice despite all my efforts. Ghost uses Nodemailer for sending email. I think one of the main problems is that it's using an ancient version of Nodemailer (v0.7, when the current package is past v6). There is an issue on github on updating Nodemailer but nobody seems to care much. Makes one wonder if Ghost truly is dead. I hope not, because it is a good tool for what it does.

Gmail Config

The following worked for me when sending through gmail:

  "mail": {
    "from": "GMAIL_USERNAME@gmail.com",
    "transport": "SMTP",
    "options": {
      "host": "smtp.gmail.com",
      "port": 465,
      "secureConnection": true,
      "auth": {
        "user": "GMAIL_USERNAME@gmail.com",
        "pass": "PASSWORD"
      }
    }
  },

You will have to "allow less secure apps" for the above to work.

BUT this post is not about sending Ghost email through Gmail, but about sending it through a "regular" smtp server hosted for you at your ISP!

Quick Reference of Commands Used

# Edit email credentials, "cook" the file, postfix configuration, restart: 
$ sudo vi /etc/postfix/sasl_passwd
$ sudo postmap /etc/postfix/sasl_passwd
$ sudo vi /etc/postfix/main.cf
$ sudo service postfix restart
# Send a test email:
$ echo "This is a test email body." | mail -s "Subject" RECIPIENT@EMAIL.ADDRESS
# Restart postfix

Environment

Server: Ubuntu 18.10
Ghost: self-hosted, 2.25.2

Background

My specific setup is super simple. My host allows SMTP either with or without authentication. But however I configured Ghost, I would just run into this error:

Greeting never received

I tried various combinations for my config.production.json. I'll share it with you because some of this might work for you:

  mail: {
    transport: 'SMTP',
    options: {
      "host": "mail.???.com",
      "secure": false,
      "secureConnection": false, <!-- see https://www.saotn.org/send-email-ghost-using-smtp-authentication-tls-encryption/
	
      "port": 587,	<-- this is the NON-SSL port for me, but I also tried 465 
                        with "secure": true
      "ignoreTLS": true,	<-- force nodemailer not to use tls
      "auth": {
        "user": "???@???",
        "pass": "???"
      },
      "tls": {
        "rejectUnauthorized": false
      },
      "debug": true, <-- supposedly instructs nodemailer to write 
                         error messages to stdout
      "log": true	<-- supposedly adds logging to nodemailer, 
                        but couldn't find the actual logs
    }
  }

I tried watching the ghost log while using the email test button to see if there is any more info there:

tail content/logs/???_production.log

But the logs were of no help:


..."code":"ETIMEDOUT","name":"EmailError","statusCode":500,"level":"normal","message":"Failed to send email. Reason: Greeting never received.","help":"\"Please see https://docs.ghost.org/mail/ for instructions on configuring email.\"","stack":"EmailError: Failed to send email. Reason: Greeting never received.\n    at EmailError.GhostError (/var/www/???.com/en/versions/2.25.2/core/server/lib/common/errors.js:10:26)\n    at new EmailError (/var/www/???.com/en/versions/2.25.2/core/server/lib/common/errors.js:34:20)\n    at MailComposer.returnCallback (/var/www/???.com/en/versions/2.25.2/core/server/services/mail/GhostMailer.js:77:31)\n    at SMTPConnectionPool._onConnectionError (/var/www/???.com/en/versions/2.25.2/node_modules/simplesmtp/lib/pool.js:334:17)\n    at SMTPClient.emit (events.js:198:13)\n    at SMTPClient.EventEmitter.emit (domain.js:448:20)\n    at SMTPClient.<anonymous> (/var/www/???.com/en/versions/2.25.2/node_modules/simplesmtp/lib/client.js:293:18)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)\n\nError: Greeting never received\n    at SMTPClient.<anonymous> (/var/www/???.com/en/versions/2.25.2/node_modules/simplesmtp/lib/client.js:289:25)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)"},"msg":"Failed to send email. Reason: Greeting never received.","time":"2019-07-01T08:19:20.343Z","v":0}

After wasting way too much time trying to go the Ghost route, I decided to use a solid, Linux community supported mailer. After considering sendmail, as well as simpler alternatives such as ssmtp and msmtp and opensmtpd I ended up choosing postfix because it is widely used and has a lot of configuration options.

Towards a Solution

First, make sure sendmail exists on your platform: which sendmail

This should return something similar to /usr/sbin/sendmail

Set your FQDN

hostnamectl set-hostname new-hostname

vi /etc/hosts

Make sure local port 25 is open

For postfix to work, it needs to be able to receive incoming connections from localhost. Ensure that port 25 is open:

sudo apt install net-tools

netstat -tulpn | grep :25

Install Postfix

Install postfix and utilities, such as "mail":
$ sudo apt-get install mailutils
...alternatively, install postfix only:
$ sudo apt-get install postfix

On the second screen, set the actual FQDN of your server. This is not the same as hostname of the 3rd party SMTP server that you use.

You can reconfigure postfix at any time with sudo dpkg-reconfigure postfix

Set your Username/Password

This is a rundown of a detailed explanation:

$ sudo vi /etc/postfix/sasl_passwd
   Set the file to:
YOUR.ISP.SERVER:587 username:password
$ sudo vi /etc/postfix/main.cf
   Make sure you have: 
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
$ sudo chown root:root /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd
$ sudo chmod 0600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd
$ sudo postmap /etc/postfix/sasl_passwd
$ sudo service postfix reload

Configure Postfix

sudo vi /etc/postfix/main.cf

There are a plethora of options, you will probably need the following:

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = www.zensoft.hu
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = zen2, localhost.localdomain, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_protocols = all

# Enable auth
smtp_sasl_auth_enable = yes
smtp_tls_wrappermode = yes
# Set username and password
# A less secure method would be: smtp_sasl_password_maps = static:USERNAME@SERVER.com:YOUR_PASSWORD
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
# Turn on tls encryption
smtp_use_tls = yes
smtp_tls_security_level = encrypt
header_size_limit = 4096000
# Set external SMTP relay host here IP or hostname accepted along with a port number.
relayhost = mail.generalcomputing.com:465
# accept email from our web-server only
inet_interfaces = localhost

# https://serverfault.com/questions/147921/forcing-the-from-address-when-postfix-relays-over-smtp
sender_canonical_classes = envelope_sender, header_sender
sender_canonical_maps =  regexp:/etc/postfix/sender_canonical_maps
smtp_header_checks = regexp:/etc/postfix/header_check

A non-encrypted option would look like the above except:

#smtp_tls_wrappermode = yes
smtp_use_tls = no
#smtp_tls_security_level = encrypt

After updating run service postfix restart

Relay/Forward System Mail

If you want to forward all mail sent to root, then sudo vi /etc/aliases

# See man 5 aliases for format
postmaster:    root
root:          YOUR_EMAIL@ADDRESS.COM

And then run sudo newaliases

Relay www-data and other mail

You may run into a situation where mail to www-data is not relayed. For example, cron-job results are often emailed here.

/etc/aliases only allows you to forward mail that is headed for a local address, in other words, the recipient's domain must be one of the domains given in the mydestination = ... line of main.cf. Therefore, to make sure www-data is forwarded to whatever email you want, you must:

(1) Edit the file set in myorigin = /etc/mailname in main.cf, by default /etc/mailname and set it to one of the addresses listed in the mydestination = ... line. For example, add "localhost" to both as in this example:

/etc/postfix/main.cf

myhostname = xxx.domain.com
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = my-host-name, localhost

/etc/mailname:

localhost

/etc/aliases:

clamav: root
www-data: root
root:   RECIPIENT_EMAIL@SERVER.COM

Run newaliases and service postfix reload after editing these files.

Force the FROM Address

When testing postfix with echo "This is a test email body." | mail -s "Subject" RECIPIENT_EMAIL@SERVER.COM, the server will set the from address automatically to the user you are logged in as. Additionally, if any local app tries to send mail with sendmail, if it doesn't explicitly specify a from address then again the above will happen.

I have a dedicated email address for each of my servers, so to make things easy, I force the FROM address to always be that dedicated email address. There is a detailed answer to this on stackoverflow. Here's a quick rundown:

$ sudo vi /etc/postfix/main.cf
...add the following to the end:
sender_canonical_classes = envelope_sender, header_sender
sender_canonical_maps =  regexp:/etc/postfix/sender_canonical_maps
smtp_header_checks = regexp:/etc/postfix/header_check
$ sudo vi /etc/postfix/sender_canonical_maps
/.+/    newsender@address.com
$ sudo vi /etc/postfix/header_check
/From:.*/ REPLACE From: newsender@address.com

Allow Postfix through your Firewall

You would only need to take this step if you are using postfix to receive email!

sudo ufw allow postfix

Alternatively, you can allow only specific ports: sudo ufw allow 587

Test Postfix

echo "This is a test email body." | mail -s "Subject" -a "From: YOUR_EMAIL@SERVER.COM" RECIPIENT_EMAIL@SERVER.COM

Or, to send from the default address:

echo "This is a test email body." | mail -s "Subject" RECIPIENT_EMAIL@SERVER.COM

To watch for any errors, run this on a separater terminal while running the above:

sudo tail -f /var/log/mail.log
sudo tail -f /var/log/mail.err

Issues? Kill sendmail-mta

If you see something like this while starting postfix sudo systemctl start postfix and tailing mail.log sudo tail -f /var/log/mail.log:

fatal: bind 127.0.0.1 port 25: Address already in use

Then try sudo killall sendmail, if you see:

sendmail: no process found

Then you may need to sudo killall sendmail-mta

Check and Flush the mail queue

Check: sendmail -bp

Flush: postsuper -d ALL or sendmail -q -v

Explicitly tell ghost to use sendmail

To do so, set your ghost config email section to the following:

  mail: {
    "from": "Ghost <ghost@???.com>",
    "transport": "SMTP",
    "options": {
      "service": "sendmail",
    }
  },

Don't forget to service ghost restart