Skip to content

Resolve OpenSSL 4.0 build issues - #13476

Open
maskit wants to merge 12 commits into
apache:masterfrom
maskit:pr-13440-openssl4-build-fix
Open

Resolve OpenSSL 4.0 build issues#13476
maskit wants to merge 12 commits into
apache:masterfrom
maskit:pr-13440-openssl4-build-fix

Conversation

@maskit

@maskit maskit commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

Continues #13440 ("Resolve OpenSSL 4.0 build issues"), fixing the remaining CI failures on Ubuntu 20.04 and FreeBSD 13.1 (OpenSSL 1.1.1) flagged in review.

X509_NAME_get_index_by_NID() takes a non-const X509_NAME * on OpenSSL 1.1.1 but a const X509_NAME * on OpenSSL 3.0+ and post-constify OpenSSL/BoringSSL. The original PR widened several X509_NAME-typed variables to const to build against the newer, constified signature, which broke the older signature at three call sites:

  • src/tscore/X509HostnameValidator.ccvalidate_hostname()
  • src/iocore/net/SSLUtils.ccSSLMultiCertConfigLoader::load_certs_and_cross_reference_names()
  • plugins/experimental/txn_box/plugin/src/ts_util.ccssl_value_for()

Each call site now does const_cast<X509_NAME *>(...), matching the existing precedent in OCSPStapling.cc for the same kind of OpenSSL-version const mismatch.

Test plan

  • Built tscore, inknet, and txn_box targets locally against BoringSSL — all compile clean
  • Ran cmake --build build -t format — no additional changes
  • CI to confirm the Ubuntu 20.04 / FreeBSD 13.1 (OpenSSL 1.1.1) jobs that failed on Resolve OpenSSL 4.0 build issues #13440 now pass

@JosiahWI

JosiahWI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Did you consider the decltype workaround discussed in #13440? I would like to see an explicit justification for opting to use the less type safe const_cast approach.

@maskit

maskit commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

[approve ci]

@maskit

maskit commented Aug 3, 2026

Copy link
Copy Markdown
Member Author

@JosiahWI Thanks for the reminder. The use of auto made it a little cleaner. It didn't completely remove the const_casts, but I think this is ok for 10.2. After landing this, we should do cleanup that removes 1.1.1 support on master.

@JosiahWI

JosiahWI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

I think decltype will preserve the CV qualification on the type, unlike auto.

@JosiahWI

JosiahWI commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Oh, I see. The auto keyword also determines the correct constness in most cases. This looks good.

@maskit maskit added the TLS label Aug 3, 2026
@maskit maskit added this to the 11.0.0 milestone Aug 3, 2026
@maskit
maskit marked this pull request as ready for review August 3, 2026 19:46
Copilot AI review requested due to automatic review settings August 3, 2026 19:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR continues the effort to keep ATS building across OpenSSL versions (OpenSSL 1.1.1 through 4.0+/constified APIs and BoringSSL) by tightening const-correctness around X509/ASN1 access and updating code to use supported accessor functions instead of direct structure field access.

Changes:

  • Replace direct ASN1_STRING/ASN1_BIT_STRING field access (->data, ->length, ->type) with accessor APIs like ASN1_STRING_get0_data(), ASN1_STRING_length(), and ASN1_STRING_type().
  • Update several helper signatures and local variables to const where the OpenSSL APIs now expose const data (e.g., X509_NAME, X509_NAME_ENTRY, ASN1_STRING).
  • Adjust certificate subject-name manipulation in certifier to operate on a duplicated X509_NAME and set it back onto the certificate.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/tscore/X509HostnameValidator.cc Uses ASN1 accessor APIs for hostname checks and adjusts CN parsing types.
src/iocore/net/SSLUtils.cc Makes asn1_strdup() accept const ASN1_STRING* and propagates const correctness through subject CN extraction.
src/iocore/net/SSLNetVConnection.cc Updates debug helper to accept const X509_NAME*.
src/iocore/net/OCSPStapling.cc Replaces ASN1_BIT_STRING direct field access with ASN1 accessor APIs for digesting and const cleanup.
src/api/InkAPI.cc Uses const-correct X509_NAME_ENTRY / ASN1_STRING pointers when extracting CN during cert update.
plugins/experimental/txn_box/plugin/src/ts_util.cc Makes ssl_value_for() accept const X509_NAME* and uses a const_cast for OpenSSL 1.1.1 compatibility at one call site.
plugins/experimental/sslheaders/expand.cc Makes subject/issuer names const and replaces signature byte access with ASN1 accessor APIs.
plugins/experimental/cert_reporting_tool/cert_reporting_tool.cc Treats subject name as const X509_NAME*.
plugins/certifier/certifier.cc Duplicates and sets the subject name before adding CN to avoid mutating internal OpenSSL-owned structures directly.

Comment thread src/tscore/X509HostnameValidator.cc
Comment thread plugins/certifier/certifier.cc
@cmcfarlen

Copy link
Copy Markdown
Contributor

I had independently worked up a fix for this (#13482) before seeing this PR — standing that down in favor of yours. Handing over what I found in case it's useful, since some of it isn't covered here yet.

The diagnosis and the three X509_NAME_get_index_by_NID call sites match what I landed on exactly. The one thing worth flagging: neither CI nor a BoringSSL build can validate the OpenSSL 4 side of this. The Fedora CI job is Fedora 43 (OpenSSL 3.5), and per your own table in #13440 BoringSSL returns non-const from X509_get_subject_name — i.e. the same shape as 1.1.1. So a green CI run plus a BoringSSL build confirms the 1.1.1 regression is fixed, but can't exercise any of the const-return changes that OpenSSL 4 introduces.

I built this branch in a Fedora 45 / OpenSSL 4.0.1 container (BUILD_EXPERIMENTAL_PLUGINS=ON, ENABLE_CRIPTS=ON, BUILD_REGRESSION_TESTING=ON, ninja -k 0 to get past the first failure). Remaining errors:

File Errors
include/cripts/Certs.hpp 124
src/cripts/Certs.cc 4
plugins/lua/ts_lua_client_request.cc 4
plugins/lua/ts_lua_client_cert_helpers.h 3
src/iocore/net/unit_tests/test_SSLDHParams.cc 1

Specifically:

  • plugins/luaget_x509_name_string(X509_NAME *) needs a const X509_NAME * param (it only calls X509_NAME_print_ex, which is const on every version), and ts_lua_client_cert_helpers.h reads sig->length / sig->data directly, which breaks now that ASN1_STRING is opaque. Note this one builds by default — plugins/lua is gated on if(USE_LUAJIT), and Fedora 45 has luajit — so it's not an opt-in path.
  • src/criptsCerts.cc has the same opaque-ASN1_STRING issue in Signature::_load and _write_ip_address. ENABLE_CRIPTS defaults OFF so this is lower priority.
  • test_SSLDHParams.ccmake_cert_and_key mutates the subject name in place. Once the accessor returns const you have to dup, then call both X509_set_subject_name and X509_set_issuer_name, or the generated self-signed cert silently loses its CN.
  • Also broken on OpenSSL 4 but outside a default build: example/plugins/c-api/client_context_dump/client_context_dump.cc and tests/tools/plugins/ssl_client_verify_test.cc.

Two notes on approach, take or leave:

  1. const_cast handles the three call sites fine, but it can't handle Cripts. CertBase::X509Value::_load_name(X509_NAME *(*getter)(const X509 *)) takes a function pointer, and X509_get_subject_name's whole type changes in 4.0 — casting a function pointer and calling through it is UB. X509_getm_notBefore/notAfter also moved const the opposite direction (const X509 * in 1.1.1, X509 * in 4.0), so no single hardcoded spelling works there either. What worked for me was deducing the type from the accessor — auto * for locals, decltype(&X509_get_subject_name) for parameter and function-pointer types — which needs no casts and no version macros.
  2. The certifier.cc change (carried from Resolve OpenSSL 4.0 build issues #13440) leaks the duplicated name on the X509_NAME_add_entry_by_txt failure path — only the X509_set_subject_name path frees it — and doesn't null-check X509_NAME_dup.

Diffs for all of the above are on cmcfarlen:ossl4-compat (#13482) if you want to cherry-pick or copy from them; verified green on Fedora 45/OpenSSL 4.0.1, Fedora 43/OpenSSL 3.5.7, and Ubuntu 20.04/OpenSSL 1.1.1f + clang-12, with test_tscore and test_net passing on all three. Happy to just close mine and let you carry it however you prefer.

jeredfloyd and others added 5 commits August 3, 2026 16:21
OpenSSL 1.1.1 declares the first argument as non-const while newer
releases declare it const, so building against different versions
failed depending on which signature was in effect. Add a const_cast
at each call site to keep the const-qualified variables introduced by
this PR buildable against both, matching the existing precedent in
OCSPStapling.cc.
X509_get_subject_name and X509_NAME_get_index_by_NID have their
argument constness changed together across OpenSSL versions, so a
local variable declared with auto tracks whatever type is correct for
the OpenSSL version in use, without a cast. This applies only to the
two purely local variables; ts_util.cc ssl_value_for keeps its
const_cast since its parameter type is shared across multiple
callers.
X509HostnameValidator.cc cast ASN1_STRING_get0_data return value to
non-const before an ats_strndup call that only wants const char *;
drop the const instead of adding it back needlessly.

certifier.cc leaked the duplicated X509_NAME on the
X509_NAME_add_entry_by_txt failure path, and never checked
X509_NAME_dup for allocation failure.
@cmcfarlen
cmcfarlen requested a review from JosiahWI August 3, 2026 22:27
maskit added 7 commits August 3, 2026 16:44
CertBase::X509Value took hardcoded function pointer types for
X509_get_subject_name, X509_get_issuer_name, X509_getm_notBefore, and
X509_getm_notAfter, but those accessors change constness in different
directions across OpenSSL versions, so no single hardcoded signature
builds everywhere. Deduce the parameter type from the actual accessor
via decltype instead.

Signature::_load and _write_ip_address also read ASN1_STRING fields
directly, which breaks once the struct is opaque; switch to the
accessor functions used elsewhere in this codebase.
get_x509_name_string only reads through the name via
X509_NAME_print_ex, so accept a const X509_NAME * to match callers
that pass X509_get_subject_name/X509_get_issuer_name results
directly. get_x509_signature_string read the ASN1_STRING struct
fields directly, which breaks once the struct is opaque; use
ASN1_STRING_get0_data/ASN1_STRING_length instead.
make_cert_and_key mutated the X509_NAME returned by
X509_get_subject_name in place, which stops compiling once that
accessor can return const, and was already fragile since the
returned name is only a view into the certificate internal state.
Duplicate it, add the CN to the duplicate, and set it back as both
subject and issuer name since this is a self-signed certificate.
client_context_dump.cc, verify_cert.cc, and ssl_client_verify_test.cc
all held X509_get_subject_name/X509_NAME_get_entry/
X509_NAME_ENTRY_get_data results in hardcoded non-const locals or
parameters, which stops compiling once those accessors return const.
Switch to auto for the local variables and const for the
debug_certificate parameter, matching the read-only usage in each
case.
ssl_value_for is shared by four callers, each already deducing its
X509_NAME pointer type with auto from X509_get_subject_name or
X509_get_issuer_name, so unlike the other three call sites fixed
earlier in this series, a single hardcoded parameter type cannot
track the underlying accessor across OpenSSL versions. Templating
the parameter on the callers deduced type removes the cast entirely.
Bare decltype(&X509_get_subject_name) in a parameter list reads
poorly at each of the four call sites; name each getter type once
via using so the declarations and out-of-line definitions just say
what kind of accessor they take.
certifier.cc freed the duplicated X509_NAME manually on every path;
switch to a scoped_X509_NAME unique_ptr matching the file existing
scoped_X509/scoped_EVP_PKEY/scoped_SSL_CTX aliases so no path can
forget to free it.

ts_util.cc templated ssl_value_for just to defer the parameter type
to the caller; decltype(X509_get_subject_name(nullptr)) deduces the
same pointer type directly without turning it into a template.

Also drop the last C-style cast this series introduced in
X509HostnameValidator.cc in favor of reinterpret_cast.
Copilot AI review requested due to automatic review settings August 3, 2026 23:14
@maskit
maskit force-pushed the pr-13440-openssl4-build-fix branch from d31a219 to 01b9317 Compare August 3, 2026 23:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Suppressed comments (2)

plugins/certifier/certifier.cc:411

  • X509_NAME_add_entry_by_txt() expects a const unsigned char *, but this code uses a non-const cast (unsigned char *)commonName.c_str(). Avoid casting away constness here (and avoid C-style casts) to keep the code const-correct across OpenSSL variants.
  if (X509_NAME_add_entry_by_txt(n.get(), "CN", MBSTRING_ASC, (unsigned char *)commonName.c_str(), -1, -1, 0) != 1) {

plugins/experimental/txn_box/plugin/src/ts_util.cc:1119

  • The PR description says the OpenSSL 1.1.1 vs 3.x+ X509_NAME_get_index_by_NID() const-mismatch is handled by adding const_cast<X509_NAME*>(...) at the affected call sites, but this change instead introduces a decltype(X509_get_subject_name(nullptr)) alias and does not add any such const_cast (and none exist in-tree). Please update the PR description to match the implemented approach, or adjust the code to match the documented fix.
  using X509_NAME_ptr = decltype(X509_get_subject_name(nullptr));

  TextView
  ssl_value_for(X509_NAME_ptr name, int nid)
  {
    if (int loc = X509_NAME_get_index_by_NID(name, nid, -1); loc >= 0) {

unsigned char *astr;
str = X509_NAME_ENTRY_get_data(X509_NAME_get_entry(name, i));
// Convert to UTF-8
astrlen = ASN1_STRING_to_UTF8(&astr, str);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed real, but pre-existing on master since the function's original addition in 2015, unrelated to this PR's OpenSSL 4 changes. Filed as #13483.

Comment thread src/api/InkAPI.cc
Comment on lines +8319 to 8323
const int pos = X509_NAME_get_index_by_NID(X509_get_subject_name(cert.get()), NID_commonName, -1);
const X509_NAME_ENTRY *common_name = X509_NAME_get_entry(X509_get_subject_name(cert.get()), pos);
const ASN1_STRING *common_name_asn1 = X509_NAME_ENTRY_get_data(common_name);
char *common_name_str = reinterpret_cast<char *>(const_cast<unsigned char *>(ASN1_STRING_get0_data(common_name_asn1)));
if (ASN1_STRING_length(common_name_asn1) != static_cast<int>(strlen(common_name_str))) {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed real, but pre-existing on master since 2024, unrelated to this PR's OpenSSL 4 changes. Filed as #13484.

@maskit

maskit commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

[approve ci autest 2]

@maskit

maskit commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

I've updated the change and addressed all the comments. New code does not use const_cast at all now. I did see const_casts around the changes but those are just few of 500+. Those should be fixed separately.

@maskit

maskit commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

@jeredfloyd I added changes on top of your commits to cover the entire source tree. I'd appreciate it if you could test this change for Fedora 45.

@JosiahWI JosiahWI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great to me. Will wait for @jeredfloyd to check before approving. I have one question about the certifier logic.

Comment thread plugins/certifier/certifier.cc
Comment thread plugins/certifier/certifier.cc
Comment thread example/plugins/c-api/verify_cert/verify_cert.cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build work related to build configuration or environment TLS

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

5 participants