Zimbra Unable to validate certificate chain

On installing SSL on the Zimbra mail server, i get the following error

zimbra@zim:~/boby$ /opt/zimbra/bin/zmcertmgr verifycrt comm /opt/zimbra/ssl/zimbra/commercial/commercial.key  /opt/zimbra/boby/zim_simplecloud_co_za.crt
** Verifying '/opt/zimbra/boby/zim_simplecloud_co_za.crt' against '/opt/zimbra/ssl/zimbra/commercial/commercial.key'
Certificate '/opt/zimbra/boby/zim_simplecloud_co_za.crt' and private key '/opt/zimbra/ssl/zimbra/commercial/commercial.key' match.
** Verifying '/opt/zimbra/boby/zim_simplecloud_co_za.crt' against '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt'
ERROR: Unable to validate certificate chain: /opt/zimbra/boby/zim_simplecloud_co_za.crt: C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
error 2 at 2 depth lookup:unable to get issuer certificate
zimbra@zim:~/boby$ 

This was due to SSL cert chain. The ca-bundle file they provided did not work with Zimbra. This is due to some issue with the order in which CA Certificate files are placed. Here is Zimba documentation related to this issue

https://wiki.zimbra.com/wiki/Fix_depth_lookup:unable_to_get_issuer_certificate

I checked with SSL provider, and they initially provided a combined SSL certificate, that has cert file + ca certificate. I tried to install it, but it did not work.

After showing SSL support the screenshot of the SSL install page, they provided me with 3 different files.

In the zimbra SSL install, you have option to add more intermediate CA by clicking “Add Intermediate CA” link.

The provided files are

root.ca
intermediate1.ca-bundle
intermediate2.ca-bundle

I tried to install it using UI, but it failed with some errors related to RemoteManager and port 22.

To install on Command line, first you need to login as user zimbra

su - zimbra

I copied all files provided by SSL provider to the server. Change to SSL folder

cd /opt/zimbra/ssl/zimbra/commercial/

Edited the file

vi commercial.crt

Pasted the SSL certificate content to this file. commercial.key file have the private key, this get auto generated during the CSR generation process.

Now i tried mixing those 3 files (CA certs) to create commerical_ca.crt, but it failed to work

zimbra@zim:~/ssl/zimbra/commercial$ cat  ~/boby/root.crt ~/boby/intermediate1.ca-bundle > /opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt
zimbra@zim:~/ssl/zimbra/commercial$ /opt/zimbra/bin/zmcertmgr verifycrt comm commercial.key commercial.crt
** Verifying 'commercial.crt' against 'commercial.key'
Certificate 'commercial.crt' and private key 'commercial.key' match.
** Verifying 'commercial.crt' against '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt'
ERROR: Unable to validate certificate chain: Error loading file /opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt
140015104063128:error:0906D066:PEM routines:PEM_read_bio:bad end line:pem_lib.c:815:
140015104063128:error:0B084009:x509 certificate routines:X509_load_cert_crl_file:PEM lib:by_file.c:259:
zimbra@zim:~/ssl/zimbra/commercial$

After few try, mixing ca certificate in the following order got it work.

zimbra@zim:~/ssl/zimbra/commercial$ cat   ~/boby/intermediate1.ca-bundle  ~/boby/intermediate2.ca-bundle ~/boby/root.crt > /opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt
zimbra@zim:~/ssl/zimbra/commercial$ /opt/zimbra/bin/zmcertmgr verifycrt comm commercial.key commercial.crt
** Verifying 'commercial.crt' against 'commercial.key'
Certificate 'commercial.crt' and private key 'commercial.key' match.
** Verifying 'commercial.crt' against '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt'
Valid certificate chain: commercial.crt: OK
zimbra@zim:~/ssl/zimbra/commercial$ 

Now installed SSL with

zimbra@zim:~/ssl/zimbra/commercial$ /opt/zimbra/bin/zmcertmgr deploycrt comm commercial.crt commercial_ca.crt
** Fixing newlines in 'commercial_ca.crt'
** Verifying 'commercial.crt' against '/opt/zimbra/ssl/zimbra/commercial/commercial.key'
Certificate 'commercial.crt' and private key '/opt/zimbra/ssl/zimbra/commercial/commercial.key' match.
** Verifying 'commercial.crt' against 'commercial_ca.crt'
Valid certificate chain: commercial.crt: OK
** Copying 'commercial.crt' to '/opt/zimbra/ssl/zimbra/commercial/commercial.crt'
'commercial.crt' and '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' are identical (not copied) at /opt/zimbra/bin/zmcertmgr line 1278.
** Copying 'commercial_ca.crt' to '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt'
'commercial_ca.crt' and '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt' are identical (not copied) at /opt/zimbra/bin/zmcertmgr line 1278.
** Appending ca chain 'commercial_ca.crt' to '/opt/zimbra/ssl/zimbra/commercial/commercial.crt'
** Importing cert '/opt/zimbra/ssl/zimbra/commercial/commercial_ca.crt' as 'zcs-user-commercial_ca' into cacerts '/opt/zimbra/common/lib/jvm/java/jre/lib/security/cacerts'
** NOTE: restart mailboxd to use the imported certificate.
** Saving config key 'zimbraSSLCertificate' via zmprov modifyServer zim.simplecloud.co.za...ok
** Saving config key 'zimbraSSLPrivateKey' via zmprov modifyServer zim.simplecloud.co.za...ok
** Installing imapd certificate '/opt/zimbra/conf/imapd.crt' and key '/opt/zimbra/conf/imapd.key'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' to '/opt/zimbra/conf/imapd.crt'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.key' to '/opt/zimbra/conf/imapd.key'
** Creating file '/opt/zimbra/ssl/zimbra/jetty.pkcs12'
** Creating keystore '/opt/zimbra/conf/imapd.keystore'
** Installing ldap certificate '/opt/zimbra/conf/slapd.crt' and key '/opt/zimbra/conf/slapd.key'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' to '/opt/zimbra/conf/slapd.crt'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.key' to '/opt/zimbra/conf/slapd.key'
** Creating file '/opt/zimbra/ssl/zimbra/jetty.pkcs12'
** Creating keystore '/opt/zimbra/mailboxd/etc/keystore'
** Installing mta certificate '/opt/zimbra/conf/smtpd.crt' and key '/opt/zimbra/conf/smtpd.key'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' to '/opt/zimbra/conf/smtpd.crt'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.key' to '/opt/zimbra/conf/smtpd.key'
** Installing proxy certificate '/opt/zimbra/conf/nginx.crt' and key '/opt/zimbra/conf/nginx.key'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.crt' to '/opt/zimbra/conf/nginx.crt'
** Copying '/opt/zimbra/ssl/zimbra/commercial/commercial.key' to '/opt/zimbra/conf/nginx.key'
** NOTE: restart services to use the new certificates.
** Cleaning up 3 files from '/opt/zimbra/conf/ca'
** Removing /opt/zimbra/conf/ca/ca.key
** Removing /opt/zimbra/conf/ca/ca.pem
** Removing /opt/zimbra/conf/ca/44fca4b1.0
** Copying CA to /opt/zimbra/conf/ca
** Copying '/opt/zimbra/ssl/zimbra/ca/ca.key' to '/opt/zimbra/conf/ca/ca.key'
** Copying '/opt/zimbra/ssl/zimbra/ca/ca.pem' to '/opt/zimbra/conf/ca/ca.pem'
** Creating CA hash symlink '44fca4b1.0' -> 'ca.pem'
** Creating /opt/zimbra/conf/ca/commercial_ca_1.crt
** Creating CA hash symlink '65ff7287.0' -> 'commercial_ca_1.crt'
** Creating /opt/zimbra/conf/ca/commercial_ca_2.crt
** Creating CA hash symlink 'fc5a8f99.0' -> 'commercial_ca_2.crt'
** Creating /opt/zimbra/conf/ca/commercial_ca_3.crt
** Creating CA hash symlink '157753a5.0' -> 'commercial_ca_3.crt'
zimbra@zim:~/ssl/zimbra/commercial$

Now rebooted the server, and after reboot SSL worked.