Cannot properly configure OpenSSL on Windows

Question

I want to use KDSoap, which uses Qt, which uses OpenSSL, to use a web service that requires a secure connection, which is established using a key stored in USB HSM (SmartCard + Reader basically called Athena IDProtect Key 2.0).

I installed 64-bit OpenSSL from Shining Light Productions . Qt 5.8 can download it (I checked with QSslSocket ).

Problem

The problem is that even if it seems that the Qt network server should load the OpenSSL configuration file, it is not. In fact, I think OpenSSL is to blame here, not Qt because of the tests I did with the openssl tool provided.

First of all, the configuration is read because any syntax error in config makes the openssl tool complain about it the next time I run it. Secondly, with a clean configuration, this works:

 C:\Users\brokenthorn>openssl engine dynamic -t -pre SO_PATH:C:\OpenSSL-Win64\bin\capi.dll -pre LOAD -post list_certs (dynamic) Dynamic engine loading support [Success]: SO_PATH:C:\OpenSSL-Win64\bin\capi.dll [Success]: LOAD Loaded: (capi) CryptoAPI ENGINE [ available ] Certificate 0 <No Friendly Name> Subject: C = RO, L = Mun. Constanta, O = SC MINI-FARM SRL, CN = Valentina Calin, serialNumber = 2006051670CV93, name = Valentina Calin, GN = Valentina, SN = Calin Issuer: C = RO, O = certSIGN, OU = certSIGN Qualified CA Class 3 G2, CN = certSIGN Qualified CA Class 3 G2 Certificate 1 <No Friendly Name> Subject: CN = DESKTOP-IF670KN\\brokenthorn Issuer: CN = DESKTOP-IF670KN\\brokenthorn [Success]: list_certs 

But if I try to load the engine by identifier, after defining it in the configuration file, I get an error message stating that it cannot find the DLL file, except that it adds .dll.dll to the specified ID:

 OpenSSL> engine -t -pre ID:capi -pre LOAD (dynamic) Dynamic engine loading support [Success]: ID:capi [Failure]: LOAD 4684:error:25078067:DSO support routines:WIN32_LOAD:could not load the shared library:.\crypto\dso\dso_win32.c:179:filename(capi.dll.dll) 4684:error:25070067:DSO support routines:DSO_load:could not load the shared library:.\crypto\dso\dso_lib.c:233: 4684:error:260B6084:engine routines:DYNAMIC_LOAD:dso not found:.\crypto\engine\eng_dyn.c:467: [ unavailable ] 

Now I tried to create a copy of capi.dll and named it capi.dll.dll and the above command will start working:

 OpenSSL> engine -t -pre ID:capi -pre LOAD (dynamic) Dynamic engine loading support [Success]: ID:capi [Success]: LOAD Loaded: (capi) CryptoAPI ENGINE [ available ] 

My openssl.cfg is standard, with the exception of the last part, where I added kernel directives that should force OpenSSL to load the default CryptoAPI mechanism, which I need if I ever intend to open a secure socket using a key stored in HSM. I will put the full configuration here for full reference if something bad sticks out:

 # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME = . RANDFILE = $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids # To use this configuration file with the "-extfile" option of the # "openssl x509" utility, name here the section containing the # X.509v3 extensions to use: # extensions = # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca', 'req' and 'ts'. # Add a simple OID like this: # testoid1=1.2.3.4 # Or use config file substitution like this: # testoid2=${testoid1}.5.6 # Policies used by the TSA examples. tsa_policy1 = 1.2.3.4.1 tsa_policy2 = 1.2.3.4.5.6 tsa_policy3 = 1.2.3.4.5.7 #################################################################### [ ca ] default_ca = CA_default # The default ca section #################################################################### [ CA_default ] dir = ./demoCA # Where everything is kept certs = $dir/certs # Where the issued certs are kept crl_dir = $dir/crl # Where the issued crl are kept database = $dir/index.txt # database index file. #unique_subject = no # Set to 'no' to allow creation of # several ctificates with same subject. new_certs_dir = $dir/newcerts # default place for new certs. certificate = $dir/cacert.pem # The CA certificate serial = $dir/serial # The current serial number crlnumber = $dir/crlnumber # the current crl number # must be commented out to leave a V1 CRL crl = $dir/crl.pem # The current CRL private_key = $dir/private/cakey.pem# The private key RANDFILE = $dir/private/.rand # private random number file x509_extensions = usr_cert # The extentions to add to the cert # Comment out the following two lines for the "traditional" # (and highly broken) format. name_opt = ca_default # Subject Name options cert_opt = ca_default # Certificate field options # Extension copying option: use with caution. # copy_extensions = copy # Extensions to add to a CRL. Note: Netscape communicator chokes on V2 CRLs # so this is commented out by default to leave a V1 CRL. # crlnumber must also be commented out to leave a V1 CRL. # crl_extensions = crl_ext default_days = 365 # how long to certify for default_crl_days= 30 # how long before next CRL default_md = default # use public key default MD preserve = no # keep passed DN ordering # A few difference way of specifying how similar the request should look # For type CA, the listed attributes must be the same, and the optional # and supplied fields are just that :-) policy = policy_match # For the CA policy [ policy_match ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # For the 'anything' policy # At this point in time, you must list all acceptable 'object' # types. [ policy_anything ] countryName = optional stateOrProvinceName = optional localityName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional #################################################################### [ req ] default_bits = 2048 default_keyfile = privkey.pem distinguished_name = req_distinguished_name attributes = req_attributes x509_extensions = v3_ca # The extentions to add to the self signed cert # Passwords for private keys if not present they will be prompted for # input_password = secret # output_password = secret # This sets a mask for permitted string types. There are several options. # default: PrintableString, T61String, BMPString. # pkix : PrintableString, BMPString (PKIX recommendation before 2004) # utf8only: only UTF8Strings (PKIX recommendation after 2004). # nombstr : PrintableString, T61String (no BMPStrings or UTF8Strings). # MASK:XXXX a literal mask value. # WARNING: ancient versions of Netscape crash on BMPStrings or UTF8Strings. string_mask = utf8only # req_extensions = v3_req # The extensions to add to a certificate request [ req_distinguished_name ] countryName = Country Name (2 letter code) countryName_default = AU countryName_min = 2 countryName_max = 2 stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Some-State localityName = Locality Name (eg, city) 0.organizationName = Organization Name (eg, company) 0.organizationName_default = Internet Widgits Pty Ltd # we can do this but it is not needed normally :-) #1.organizationName = Second Organization Name (eg, company) #1.organizationName_default = World Wide Web Pty Ltd organizationalUnitName = Organizational Unit Name (eg, section) #organizationalUnitName_default = commonName = Common Name (eg server FQDN or YOUR name) commonName_max = 64 emailAddress = Email Address emailAddress_max = 64 # SET-ex3 = SET extension number 3 [ req_attributes ] challengePassword = A challenge password challengePassword_min = 4 challengePassword_max = 20 unstructuredName = An optional company name [ usr_cert ] # These extensions are added when 'ca' signs a request. # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This is required for TSA certificates. # extendedKeyUsage = critical,timeStamping [ v3_req ] # Extensions to add to a certificate request basicConstraints = CA:FALSE keyUsage = nonRepudiation, digitalSignature, keyEncipherment [ v3_ca ] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer # This is what PKIX recommends but some broken software chokes on critical # extensions. #basicConstraints = critical,CA:true # So we do this instead. basicConstraints = CA:true # Key usage: this is typical for a CA certificate. However since it will # prevent it being used as an test self-signed certificate it is best # left out by default. # keyUsage = cRLSign, keyCertSign # Some might want this also # nsCertType = sslCA, emailCA # Include email address in subject alt name: another PKIX recommendation # subjectAltName=email:copy # Copy issuer details # issuerAltName=issuer:copy # DER hex encoding of an extension: beware experts only! # obj=DER:02:03 # Where 'obj' is a standard or added object # You can even override a supported extension: # basicConstraints= critical, DER:30:03:01:01:FF [ crl_ext ] # CRL extensions. # Only issuerAltName and authorityKeyIdentifier make any sense in a CRL. # issuerAltName=issuer:copy authorityKeyIdentifier=keyid:always [ proxy_cert_ext ] # These extensions should be added when creating a proxy certificate # This goes against PKIX guidelines but some CAs do it and some software # requires this to avoid interpreting an end user certificate as a CA. basicConstraints=CA:FALSE # Here are some examples of the usage of nsCertType. If it is omitted # the certificate can be used for anything *except* object signing. # This is OK for an SSL server. # nsCertType = server # For an object signing certificate this would be used. # nsCertType = objsign # For normal client use this is typical # nsCertType = client, email # and for everything including object signing: # nsCertType = client, email, objsign # This is typical in keyUsage for a client certificate. # keyUsage = nonRepudiation, digitalSignature, keyEncipherment # This will be displayed in Netscape comment listbox. nsComment = "OpenSSL Generated Certificate" # PKIX recommendations harmless if included in all certificates. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer # This stuff is for subjectAltName and issuerAltname. # Import the email address. # subjectAltName=email:copy # An alternative to produce certificates that aren't # deprecated according to PKIX. # subjectAltName=email:move # Copy subject details # issuerAltName=issuer:copy #nsCaRevocationUrl = http://www.domain.dom/ca-crl.pem #nsBaseUrl #nsRevocationUrl #nsRenewalUrl #nsCaPolicyUrl #nsSslServerName # This really needs to be in place for it to be a proxy certificate. proxyCertInfo=critical,language:id-ppl-anyLanguage,pathlen:3,policy:foo #################################################################### [ tsa ] default_tsa = tsa_config1 # the default TSA section [ tsa_config1 ] # These are used by the TSA reply generation only. dir = ./demoCA # TSA root directory serial = $dir/tsaserial # The current serial number (mandatory) crypto_device = builtin # OpenSSL engine to use for signing signer_cert = $dir/tsacert.pem # The TSA signing certificate # (optional) certs = $dir/cacert.pem # Certificate chain to include in reply # (optional) signer_key = $dir/private/tsakey.pem # The TSA private key (optional) default_policy = tsa_policy1 # Policy if request did not specify it # (optional) other_policies = tsa_policy2, tsa_policy3 # acceptable policies (optional) digests = md5, sha1 # Acceptable message digests (mandatory) accuracy = secs:1, millisecs:500, microsecs:100 # (optional) clock_precision_digits = 0 # number of digits after dot. (optional) ordering = yes # Is ordering defined for timestamps? # (optional, default: no) tsa_name = yes # Must the TSA name be included in the reply? # (optional, default: no) ess_cert_id_chain = no # Must the ESS cert id chain be included? # (optional, default: no) # # Custom configuration for definition of CAPI engine # openssl_conf = openssl_init [openssl_init] engines = engine_section [engine_section] capi = capi_config [capi_config] ID = capi SO_PATH = C:\OpenSSL-Win64\bin\capi.dll LIST_ADD = 1 LOAD = 1 

Now, OpenSSL should by default load the capi engine to initialize the library, which happens in every application that is associated with this OpenSSL (and does not call special procedures that download or disable the download of the default configuration file), including the openssl command-line tool.

I don’t know yet, but ultimately I should also make it the default crypto provider.

In any case, adding the above configuration, and also before that, of course, I get the following:

 OpenSSL> engine -t (rdrand) Intel RDRAND engine [ available ] (dynamic) Dynamic engine loading support [ unavailable ] 

This should not be. The dynamic mechanism was supposed to turn into a capi engine ... as OpenSSL docs say.

CSP also works for HSM. I can list the loaded CSPs like this:

 C:\Users\brokenthorn>openssl engine dynamic -t -pre SO_PATH:C:\OpenSSL-Win64\bin\capi.dll -pre LOAD -post list_csps (dynamic) Dynamic engine loading support [Success]: SO_PATH:C:\OpenSSL-Win64\bin\capi.dll [Success]: LOAD Loaded: (capi) CryptoAPI ENGINE [ available ] Available CSPs: 0. Athena ASECard Crypto CSP, type 1 1. Microsoft Base Cryptographic Provider v1.0, type 1 2. Microsoft Base DSS and Diffie-Hellman Cryptographic Provider, type 13 3. Microsoft Base DSS Cryptographic Provider, type 3 4. Microsoft Base Smart Card Crypto Provider, type 1 5. Microsoft DH SChannel Cryptographic Provider, type 18 6. Microsoft Enhanced Cryptographic Provider v1.0, type 1 7. Microsoft Enhanced DSS and Diffie-Hellman Cryptographic Provider, type 13 8. Microsoft Enhanced RSA and AES Cryptographic Provider, type 24 9. Microsoft RSA SChannel Cryptographic Provider, type 12 10. Microsoft Strong Cryptographic Provider, type 1 11. OpenSC CSP, type 1 [Success]: list_csps 

Index 0 CSP is my HSP CSP. I also sent some simple commands using the openssl tool to HSM to confirm that it really works, and it does what it should (provider provider provider CSPI provider provider).

Any suggestions?