Domino on Linux/Unix, Troubleshooting, Best Practices, Tips and more ...

 
alt

Daniel Nashed

 

How to find out which key to use to decrypt a S/MIME message

Daniel Nashed  23 October 2021 11:53:29


While troubleshooting a S/MIME issue for a customer I had to figure out why mails have not been decrypted.


Similar to Notes encryption, S/MIME uses a symmetric "session" key to encrypt the message.

This symmetric key is encrypted with the public key of each recipient.


But how does S/MIME know which key to use to decrypt the symmetric if the user has more than one key?


The solution is to find the serial number of the certificate used to encrypt the message.


An easy way on Windows would be to use the following command to get the serial number.


certutil.exe smime.p7m|findstr Serial

   Serial Number: 1e4a7e00d54d8237dd839a14afae3162


(the find string might differ in other languages)


This serial number is what you should look for in your e-mail certificate (which hopefully has a matching key).

You find the matching certificate for my example at the end of the post.


Now that we know what to search for, let me show the OpenSSL way.

It's a bit more complicated but using OpenSSL I can demonstrate encryption and analysis with an example.

And it also helps to understand the steps Microsoft
certutil.exe performs automatically for you.

-- Daniel




Encrypt some text via OpenSSL


First encrypted some text with an e-mail certificate I created with my CA.

The result of this encryption is in message format.


echo The yellow fox jumps .. > plain.txt

openssl smime -encrypt -aes-256-cbc -in plain.txt -out mail.msg daniel.pem



Get PKCS7 from message format


Getting the PKCS7 information from a message format does not need any extra format specification.


openssl smime -in mail.msg -pk7out -out msg.pk7



smime.p7m - Binary format needs the encoding specified


When exporting a smime.p7m attachment from Notes, you have to specify the binary format (DER).


openssl smime -in smime.p7m -inform DER -pk7out -out msg.pk7



To get the binary format of this test message, you can take the base64 part of the msg and convert it to binary format (I copied the base64 data into msg.b64).


openssl enc -d -base64 -in msg.b64 -out smime.p7m



Parse the PKCS7 information


Once you have the PKCS7 data, you can dump the results:


openssl asn1parse -in msg.pk7



Most of the details in the file are quite cryptic -- It's a dump of the ASN1 structure.

But after the CA certifier, you find the serial number, you can match with your certificate.


--------------------------------------------------------


Example output (email-addressed changed to acme.com):


  0:d=0  hl=4 l= 809 cons: SEQUENCE

   4:d=1  hl=2 l=   9 prim: OBJECT            :pkcs7-envelopedData

  15:d=1  hl=4 l= 794 cons: cont [ 0 ]

  19:d=2  hl=4 l= 790 cons: SEQUENCE

  23:d=3  hl=2 l=   1 prim: INTEGER           :00

  26:d=3  hl=4 l= 705 cons: SET

  30:d=4  hl=4 l= 701 cons: SEQUENCE

  34:d=5  hl=2 l=   1 prim: INTEGER           :00

  37:d=5  hl=3 l= 164 cons: SEQUENCE

  40:d=6  hl=3 l= 143 cons: SEQUENCE

  43:d=7  hl=2 l=  11 cons: SET

  45:d=8  hl=2 l=   9 cons: SEQUENCE

  47:d=9  hl=2 l=   3 prim: OBJECT            :countryName

  52:d=9  hl=2 l=   2 prim: PRINTABLESTRING   :DE

  56:d=7  hl=2 l=  12 cons: SET

  58:d=8  hl=2 l=  10 cons: SEQUENCE

  60:d=9  hl=2 l=   3 prim: OBJECT            :stateOrProvinceName

  65:d=9  hl=2 l=   3 prim: UTF8STRING        :NRW

  70:d=7  hl=2 l=  15 cons: SET

  72:d=8  hl=2 l=  13 cons: SEQUENCE

  74:d=9  hl=2 l=   3 prim: OBJECT            :localityName

  79:d=9  hl=2 l=   6 prim: UTF8STRING        :Hilden

  87:d=7  hl=2 l=  16 cons: SET

  89:d=8  hl=2 l=  14 cons: SEQUENCE

  91:d=9  hl=2 l=   3 prim: OBJECT            :organizationName

  96:d=9  hl=2 l=   7 prim: UTF8STRING        :NashCom

 105:d=7  hl=2 l=  11 cons: SET

 107:d=8  hl=2 l=   9 cons: SEQUENCE

 109:d=9  hl=2 l=   3 prim: OBJECT            :organizationalUnitName

 114:d=9  hl=2 l=   2 prim: UTF8STRING        :IT

 118:d=7  hl=2 l=  26 cons: SET

 120:d=8  hl=2 l=  24 cons: SEQUENCE

 122:d=9  hl=2 l=   3 prim: OBJECT            :commonName

 127:d=9  hl=2 l=  17 prim: UTF8STRING        :R2 NashCom MiniCA

 146:d=7  hl=2 l=  38 cons: SET

 148:d=8  hl=2 l=  36 cons: SEQUENCE

 150:d=9  hl=2 l=   9 prim: OBJECT            :emailAddress

 161:d=9  hl=2 l=  23 prim: IA5STRING         :certificates@acme.com

 186:d=6  hl=2 l=  16 prim: INTEGER           :1E4A7E00D54D8237DD839A14AFAE3162

 204:d=5  hl=2 l=  13 cons: SEQUENCE

 206:d=6  hl=2 l=   9 prim: OBJECT            :rsaEncryption

 217:d=6  hl=2 l=   0 prim: NULL

 219:d=5  hl=4 l= 512 prim: OCTET STRING      [HEX DUMP]:2724735F0707CC2B01C05CF7BBFD8C130DB96F5E90AA591AF3A63A0ED697E83541474911834154DBBA044E497D5470D31D98688F1F05E851F9E192E3A717CB10F1422C74F809E28C290B198162D093315F8F20AD30F7718D27E2AE9D0C07B8E2EBE9EEE0E884A934282F8A28EF1019C08B63A7C3913B7DE4215EA442E396C548F96E3BA3E4CF8AAE25A9DD16FF4F7A4CA289156595F14781B8856F7841A0DDD27FE006E4242A8B4EB2C10F3A61C17A1FAD5CAE78BAE745D6819B8DFC291CA0DF4E2BC2E44841A4A3B236B71A0B7D865E44B81B0EBF130A61333847D3C05A5CA52719AA3A12B4D3EA8D528D2B4EC7E7455499A9134E903705F4FE163832A21ED08176EA1C3DCA70000F4A8A3F71A797F45EF801995A8E7A3AED90C46A097C71DBB9B0AA6B65FF72D15996B4D6F30FD3EC1DF993118029202E785840267A59F118F4D36E078827CF72C5F415E12A9888E305179DFDADBD097E301178DB45181EDAE7E24FAEE3F2BCBB9572819B41E7A2F8C63B6D98AB2F1B8E0E763E33DD367AB674F5E882DBE0E05EC452AF94E6EACF35DF8A8E87FF1350B9A0AA6665042D5BE6ADFE0EFCB579128986490323019D6458160BB907C23DBD57E6EA238353931FDA62AD64AFC1C8979EEE471E52DDC16B8CDEDA7A632DA6C49D354C888B7B26431BA1FD476443D39FB56E95CCEDF1707B5B91FDFCA493B85E3A1053DDEB2011040B

 735:d=3  hl=2 l=  76 cons: SEQUENCE

 737:d=4  hl=2 l=   9 prim: OBJECT            :pkcs7-data

 748:d=4  hl=2 l=  29 cons: SEQUENCE

 750:d=5  hl=2 l=   9 prim: OBJECT            :aes-256-cbc

 761:d=5  hl=2 l=  16 prim: OCTET STRING      [HEX DUMP]:5E3D08C36CDC0848F6602059DBA444A3

 779:d=4  hl=2 l=  32 prim: cont [ 0 ]



--------------------------------------------------------


Certificate


#0

Subject    : DE/NRW/Hilden/NashCom/IT/Daniel Nashed/nsh@acme.com

SAN e-mail : nsh@acme.com

Iss e-mail : certificates@acme.com

Issuer     : DE/NRW/Hilden/NashCom/IT/R2 NashCom MiniCA/certificates@acme.com

Not Before : 2021.10.22 09:00:46

Not After  : 2022.10.23 09:00:46 (expires in 364.0 days)


Serial     : 1E4A7E00D54D8237DD839A14AFAE3162

Sign Alg   : sha256WithRSAEncryption

KeyUsage   : DigitalSignature

Extensions : KeyUsage, ExtKeyUsage

ExtKeyUsage: E-mail Protection

Key        : RSA 4096 bit



AuthKeyId  : 00:A2:6E:B2:B3:8D:0E:C7:0E:E3:7D:65:C8:0A:9E:B7:8E:04:E5:B1

SubjKeyId  : 9D:F1:7D:07:6C:91:26:1D:91:83:BC:00:7B:CF:B2:AE:7A:EC:1B:C2

MD5        : 96:19:0C:88:CE:A2:B9:00:22:E7:DC:49:B7:10:8E:03

SHA1       : 9F:C5:11:50:23:59:79:FB:A4:8A:C5:8C:DF:50:AC:2B:C5:FF:0E:04

SHA256     : 58:74:4B:A0:22:14:1B:87:76:26:DC:14:B8:E8:8A:0F:7F:E6:0A:2E:87:52:9F:3E:24:46:2C:05:6C:F1:87:7E



Links

    Archives


    • [IBM Lotus Domino]
    • [Domino on Linux]
    • [Nash!Com]
    • [Daniel Nashed]