Relay Authentication Fails After a Password Rotation
mailgw runs Postfix 3.4 on Ubuntu 20.04 and relays all outbound mail through the ISP submission service at [smtp.isp.example]:587. Relevant settings are relayhost, smtp_use_tls = yes, smtp_tls_security_level = may, smtp_sasl_auth_enable = yes, smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd, and smtp_sasl_security_options = noanonymous.
The ISP rotated the mailbox password two days ago and an administrator edited /etc/postfix/sasl_passwd with the new value. Since then mailq grows and deferred messages show two different errors:
SASL authentication failed; server smtp.isp.example[192.0.2.25] said: 535 5.7.8 Authentication credentials invalid
SASL authentication failed; cannot authenticate to server smtp.isp.example[192.0.2.25]: no mechanism available
The ISP's status page states that AUTH is accepted only after STARTTLS and that the server offers LOGIN and CRAM-MD5. Disk space is fine, DNS resolves, and port 587 is reachable. Deferred messages are still within the default queue lifetime.
1. Distinguish the two error classes. Which one is caused by the password rotation and which by policy, and what do you check first for each?
Read the worked answer
They are independent faults and fixing only one leaves the queue stuck.
535 5.7.8 Authentication credentials invalid is the ISP rejecting the credential Postfix actually sent. The trap: Postfix never reads the text file - it reads /etc/postfix/sasl_passwd.db. The administrator edited the text file but evidently did not run postmap /etc/postfix/sasl_passwd, or ran it before the edit. Check with ls -l /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db: the .db must be newer. Then postmap -q "[smtp.isp.example]:587" hash:/etc/postfix/sasl_passwd prints the value that will actually be used, which catches a lookup key that does not match the relayhost exactly - the brackets and port are part of the key, and a bare smtp.isp.example entry will never be found. The file must also be root-owned and mode 0600.
no mechanism available is policy. With smtp_sasl_security_options = noanonymous, Postfix also excludes plaintext mechanisms while the session is not encrypted. Because smtp_tls_security_level = may permits a plaintext session, Postfix has no acceptable mechanism to offer in that state, and the server offers LOGIN and CRAM-MD5 only after STARTTLS. The fix is not to loosen SASL options but to require TLS, so AUTH happens inside an encrypted session.
Confirm both by reading maillog around one queue ID, and inspect the server's advertised mechanisms with openssl s_client -starttls smtp -connect smtp.isp.example:587.
2. Give the concrete parameter changes and the order of operations to clear the queue without losing mail.
Read the worked answer
Target configuration:
relayhost = [smtp.isp.example]:587
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
encrypt requires TLS but does not verify the certificate name; with a proper CA file, secure is better because it also validates the hostname against the relay. Port 587 uses STARTTLS, so do not set smtp_tls_wrappermode = yes - that is for implicit TLS on 465 and would cause a handshake failure here.
Order of operations:
- Write the correct password into /etc/postfix/sasl_passwd using the exact
[host]:portkey, thenpostmap /etc/postfix/sasl_passwdandchmod 600the text file. - Edit main.cf, then
postfix checkandsystemctl reload postfix. - Watch a single delivery attempt:
postqueue -fto force a retry andtail -f /var/log/mail.log. If it still fails, raise verbosity withpostconf -vor take one message and runsendmail -v -qstyle testing rather than flushing everything. - Only once one message is accepted should you flush the rest with
postqueue -f.
Do not use postsuper -d ALL; deferred mail is retried until maximal_queue_lifetime (5 days by default), so nothing is lost yet. Check qshape deferred to see whether the backlog is all outbound or contains bounces, and watch for default_destination_concurrency_limit if the ISP rate-limits after the flush.
3. How would you prove end to end that authentication and TLS now work, without sending test mail to a customer address?
Read the worked answer
Test the transport and the credential separately, then confirm the queue drains.
Transport and mechanisms: openssl s_client -starttls smtp -connect smtp.isp.example:587 -crlf shows the certificate chain and, after EHLO, the AUTH line the server advertises. That tells you whether LOGIN and CRAM-MD5 appear only after STARTTLS, and whether the certificate validates against your CA file.
Credential: authenticate outside Postfix with swaks --server smtp.isp.example:587 --tls --auth LOGIN --auth-user <user> --auth-password '<pass>' --to <your-own-test-address> --from [email protected]. A 235 response proves the password and mechanism; a 535 points straight back at the lookup table. Use an address you control, never a customer address, and use the same [host]:port semantics you configured.
Postfix path: send one message through the real queue with sendmail -v and read the transcript, or inspect a deferred message's recorded dialogue with postcat -q <QUEUEID>, which shows the exact server response from the last attempt. Then postqueue -p should be empty after a flush, and grep -i 'status=sent' /var/log/mail.log should show the relay accepting the message.
Finally confirm the effective configuration rather than the file: postconf -n | grep -E 'smtp_(tls|sasl)' must show encrypt, not may, because a parameter left in a later include can silently win. Record the swaks command and the postconf output in the ticket so the next rotation is a two-minute job.