I am unable to get Sendmail to work with a TLS (STARTTLS) using version 22.04. I am able to get it work with the same config running on ArchLinux. Machines have near identical sendmail.cf & cert files with only difference is the FQDN's (each machine has a different name).
I am running: openssl s_client -starttls smtp -crlf -connect 127.0.0.1:587
to test the connection. I issue "AUTH PLAIN {coded user+PW}" and the response I get back is:40A7380BB97F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../ssl/record/ssl3_record.c:358
I suspect some issue with either the version of OPENSSL installed (3.0.2) or Sendmail (8.15.2-22ubuntu3) causing some sort of capability problem. ArchLinux has OpenSSL version 3.1.4m and sendmail 8.17.2. Sendmail on Arch also has "TLS_EC TLS_VRFY_PER_CTX" compiled in, which is missing from 22.04 LTS.
No issues with AUTH without using TLS as it will authenticate using plain, login, digest-md5, cram-md5.
line 358 in ssl_record.c is trigger from this statement:
if ((version >> 8) != SSL3_VERSION_MAJOR) {...
Perhaps its because Sendmail is pre-pending by sending "AUTH" back:
read from 0x55577e53fef0 [0x55577e545623] (5 bytes => 5 (0x5))0000 - **41 55 54 48 20** AUTHwrite to 0x55577e53fef0 [0x55577e549773] (24 bytes => 24 (0x18))0000 - **17 03 03 00 13**
On ArchLinux:
read from 0x56008b94e3b0 [0x56008ba2e3d3] (5 bytes => 5 (0x5))0000 - **17 03 03 00 2d** ....-read from 0x56008b94e3b0 [0x56008ba2e3d8] (45 bytes => 45 (0x2D))0000 - 91 de e1 c7
Note: 17 03 03 00, which is offset on 22.04 LTS since it sends "AUTH" before the version information
I am considering force updating OPENSSL & Sendmail to later versions to see if this fixes the issue. Let me know your thoughts.
Here the log from sendmail:
<-- AUTH PLAIN [HIDDEN] --- 235 2.0.0 OK Authenticated AUTH=server, relay=localhost [127.0.0.1], authid={userid}, mech=PLAIN, bits=0 disconnect level 1 in background, pid=813342 --- 421 4.4.1 FQDN Lost input channel from localhost [127.0.0.1]
Sendmail Server closes the TLS session after "OK Authenticated" & openssl returns the error "ssl3_get_record:wrong version number"
Thank You