From 3ba8e529550b0c511ef5658a6955a567eebc5c37 Mon Sep 17 00:00:00 2001 From: sheetalarkadam <100380551+sheetalarkadam@users.noreply.github.com> Date: Tue, 7 Jul 2026 18:45:00 -0700 Subject: [PATCH] fix(cpp): honor SSL_CERT_FILE via CURLOPT_CAINFO for TLS on Android The bundled libcurl does not consult the SSL_CERT_FILE (or SSL_CERT_DIR) environment variable at request time: it was built with a compiled-in default CA path that does not exist on Android, and without the fallback that would otherwise read SSL_CERT_FILE. As a result the caller-provided trust store was never used and HTTPS requests to Azure failed verification ('self-signed certificate in certificate chain' for the catalog, 'unable to get local issuer certificate' for blob downloads). Every libcurl-based transport must therefore pass the bundle explicitly via CURLOPT_CAINFO, which libcurl always honors. Add a shared http::CaBundleFile() helper (reads SSL_CERT_FILE) and route all four transports through it: - http_client.cc (HttpRequestRaw) -> CurlTransportOptions.CAInfo - http_download.cc (HttpDownloadFile) -> CurlTransportOptions.CAInfo - blob_downloader.cc (ListBlobs + -> MakeBlobClientOptions() installs a DownloadBlob, via the Azure Storage CurlTransport(CAInfo) on the SDK's SDK's own transport) BlobClientOptions The WinHTTP (desktop Windows) path uses the OS trust store and is unaffected. Also corrects the AndroidBuildPlan.md SSL section, which previously claimed SSL_CERT_FILE was honored on its own. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- sdk_v2/cpp/docs/AndroidBuildPlan.md | 34 +++++++++++++++++----- sdk_v2/cpp/src/download/blob_downloader.cc | 32 ++++++++++++++++++-- sdk_v2/cpp/src/http/http_client.cc | 19 +++++++++++- sdk_v2/cpp/src/http/http_client.h | 8 +++++ sdk_v2/cpp/src/http/http_download.cc | 12 +++++++- 5 files changed, 93 insertions(+), 12 deletions(-) diff --git a/sdk_v2/cpp/docs/AndroidBuildPlan.md b/sdk_v2/cpp/docs/AndroidBuildPlan.md index c45c39dc..924f9771 100644 --- a/sdk_v2/cpp/docs/AndroidBuildPlan.md +++ b/sdk_v2/cpp/docs/AndroidBuildPlan.md @@ -192,11 +192,27 @@ When `--android_run_emulator --test` is specified, `build.py`: ### Problem -Statically-linked OpenSSL (built by vcpkg) **ignores the `SSL_CERT_DIR` environment variable** at -runtime. The `--openssldir` path is baked at build time to a non-existent location on Android. -`SSL_CERT_FILE` with a single concatenated PEM bundle *is* honored. +Statically-linked OpenSSL/libcurl (built by vcpkg) **ignores both the `SSL_CERT_DIR` and the +`SSL_CERT_FILE` environment variables** at request time. The default CA location (`--openssldir`) is +baked at build time to a path that does not exist on Android, and this libcurl build was not +compiled with the fallback that would otherwise consult `SSL_CERT_FILE`. As a result, setting either +environment variable alone has **no effect** — curl uses its (non-existent) compiled-in default and +verification fails with a misleading "self-signed certificate in certificate chain". -### Solution for Testing +> This was confirmed on-device: passing the device's own trust store via `SSL_CERT_FILE` still +> failed, while the *exact same* bundle passed explicitly as a CA file (`openssl s_client -CAfile`, +> equivalent to `CURLOPT_CAINFO`) verified `ai.azure.com` with `Verify return code: 0 (ok)`. Only the +> delivery mechanism differed. + +### Solution + +The Core reads the `SSL_CERT_FILE` environment variable and forwards it explicitly to libcurl via +`CurlTransportOptions.CAInfo` (`CURLOPT_CAINFO`), which libcurl **always** honors. See +`src/http/http_client.cc` and `src/http/http_download.cc`. Callers therefore still set +`SSL_CERT_FILE`, but its effect now comes from the Core forwarding it to `CAInfo` — not from OpenSSL +auto-reading the environment. + +### Building the bundle for testing `tools/android.py` builds a CA bundle at test time from the device's system certificates: @@ -207,7 +223,7 @@ runtime. The `--openssldir` path is baked at build time to a non-existent locati ``` cat /system/etc/security/cacerts/*.0 > /data/local/tmp/foundry_tests/cacert.pem ``` -3. Sets `SSL_CERT_FILE` pointing to the bundle +3. Sets `SSL_CERT_FILE` pointing to the bundle (the Core forwards it to `CAInfo`) This approach uses the device's actual trust store, so it works with any API level and includes all root CAs that the device trusts (including DigiCert Global Root G2 needed for Azure endpoints). @@ -220,9 +236,11 @@ the same pattern FL Core used. ### Key Insight -`SSL_CERT_DIR` does **not** work with statically-linked OpenSSL on Android. Always use `SSL_CERT_FILE` -with a concatenated PEM bundle. The error message when this fails is misleading: "self-signed -certificate in certificate chain" — it actually means OpenSSL cannot find **any** CA store. +Neither `SSL_CERT_DIR` nor `SSL_CERT_FILE` is auto-consulted by this statically-linked +OpenSSL/libcurl on Android. The CA bundle **must** be passed explicitly via `CURLOPT_CAINFO`; the +Core does this by forwarding `SSL_CERT_FILE`. The error message when the store is missing is +misleading: "self-signed certificate in certificate chain" — it actually means OpenSSL cannot find +**any** CA store. --- diff --git a/sdk_v2/cpp/src/download/blob_downloader.cc b/sdk_v2/cpp/src/download/blob_downloader.cc index c6a5e701..1fec2daf 100644 --- a/sdk_v2/cpp/src/download/blob_downloader.cc +++ b/sdk_v2/cpp/src/download/blob_downloader.cc @@ -4,6 +4,7 @@ #include "download/blob_download_state.h" #include "download/file_writer.h" #include "exception.h" +#include "http/http_client.h" #include "logger.h" #include "util/path_safety.h" #include "util/string_utils.h" @@ -18,11 +19,19 @@ #include #include #include +#include #include #include #include +// The Azure Storage SDK builds its own libcurl transport, which — like our other curl transports — +// does not honor SSL_CERT_FILE. On non-Windows builds we install a CurlTransport preconfigured with +// CAInfo (see MakeBlobClientOptions below). Desktop Windows uses the default WinHTTP transport. +#if !defined(FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT) +#include +#endif + namespace fl { namespace { @@ -32,6 +41,24 @@ namespace { /// memory at this many bytes regardless of chunk size. constexpr size_t kStreamingBufferBytes = 64 * 1024; +/// Builds BlobClientOptions with the CA bundle wired into the transport. The Azure Storage SDK +/// constructs its own libcurl transport internally, which does not consult SSL_CERT_FILE, so blob +/// downloads would otherwise fail TLS verification on Android with "unable to get local issuer +/// certificate". On non-Windows builds we install a CurlTransport preconfigured with CAInfo so +/// verification uses the caller-provided trust store; on desktop Windows the default WinHTTP +/// transport uses the OS store and is left untouched. +Azure::Storage::Blobs::BlobClientOptions MakeBlobClientOptions() { + Azure::Storage::Blobs::BlobClientOptions options; +#if !defined(FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT) + if (std::string ca_bundle = fl::http::CaBundleFile(); !ca_bundle.empty()) { + Azure::Core::Http::CurlTransportOptions curl_opts; + curl_opts.CAInfo = std::move(ca_bundle); + options.Transport.Transport = std::make_shared(curl_opts); + } +#endif + return options; +} + } // namespace // ======================================================================== @@ -55,7 +82,7 @@ AzureBlobDownloader::AzureBlobDownloader(ILogger& logger) : logger_(logger) {} std::vector AzureBlobDownloader::ListBlobs(const std::string& sas_uri) { try { - auto container_client = Azure::Storage::Blobs::BlobContainerClient(sas_uri); + auto container_client = Azure::Storage::Blobs::BlobContainerClient(sas_uri, MakeBlobClientOptions()); std::vector items; for (auto page = container_client.ListBlobs(); page.HasPage(); page.MoveToNextPage()) { @@ -137,7 +164,8 @@ void AzureBlobDownloader::DownloadBlob(const std::string& sas_uri, try { // Configure retry at the SDK level instead of a manual retry loop. // Exponential backoff: 2s initial delay, 30s cap, generous retry count. - Azure::Storage::Blobs::BlobClientOptions client_options; + // MakeBlobClientOptions wires the CA bundle into the transport (see its comment). + auto client_options = MakeBlobClientOptions(); client_options.Retry.MaxRetries = 10; client_options.Retry.RetryDelay = std::chrono::milliseconds{2000}; client_options.Retry.MaxRetryDelay = std::chrono::milliseconds{30000}; diff --git a/sdk_v2/cpp/src/http/http_client.cc b/sdk_v2/cpp/src/http/http_client.cc index ec8d0c52..f1fc9fa1 100644 --- a/sdk_v2/cpp/src/http/http_client.cc +++ b/sdk_v2/cpp/src/http/http_client.cc @@ -22,14 +22,24 @@ #endif #include +#include #include #include #include +#include #include namespace fl { namespace http { +std::string CaBundleFile() { + const char* cert_file = std::getenv("SSL_CERT_FILE"); + if (cert_file != nullptr && cert_file[0] != '\0') { + return cert_file; + } + return {}; +} + namespace { struct HttpRawResult { @@ -48,7 +58,14 @@ HttpRawResult HttpRequestRaw(const Azure::Core::Http::HttpMethod& method, #if defined(FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT) WinHttpTransport transport; #else - CurlTransport transport; + // The bundled libcurl does not honor the SSL_CERT_FILE environment variable (it was built with a + // compiled-in default CA path that does not exist on platforms like Android). Explicitly pass the + // CA bundle via CAInfo so the caller-provided trust store is actually used for TLS verification. + CurlTransportOptions curl_opts; + if (std::string ca_bundle = CaBundleFile(); !ca_bundle.empty()) { + curl_opts.CAInfo = std::move(ca_bundle); + } + CurlTransport transport(curl_opts); #endif // Build the request. For methods with a body (POST), attach a MemoryBodyStream. diff --git a/sdk_v2/cpp/src/http/http_client.h b/sdk_v2/cpp/src/http/http_client.h index 5bdcc64d..81c99edf 100644 --- a/sdk_v2/cpp/src/http/http_client.h +++ b/sdk_v2/cpp/src/http/http_client.h @@ -27,6 +27,14 @@ struct HttpRequestOptions { bool close_connection = false; }; +/// Returns the CA bundle file path from the `SSL_CERT_FILE` environment variable, or an empty +/// string when it is unset/empty. The bundled libcurl does not consult `SSL_CERT_FILE` +/// automatically (it was built with a compiled-in default CA path that does not exist on platforms +/// like Android), so every libcurl-based transport we construct — direct requests, file downloads, +/// and the Azure Storage blob client — must pass this explicitly as `CAInfo`. On desktop Windows the +/// WinHTTP transport uses the OS trust store and ignores this. +std::string CaBundleFile(); + /// Perform an HTTP POST and return status, headers, and body without throwing on non-2xx responses. /// Transport failures are returned as `status == 0` with the error message in `body`. HttpResponse HttpPostWithResponse(const std::string& url, diff --git a/sdk_v2/cpp/src/http/http_download.cc b/sdk_v2/cpp/src/http/http_download.cc index 7354235d..83e21d6b 100644 --- a/sdk_v2/cpp/src/http/http_download.cc +++ b/sdk_v2/cpp/src/http/http_download.cc @@ -2,6 +2,7 @@ // Licensed under the MIT License. #include "http/http_download.h" +#include "http/http_client.h" #include "logger.h" #include "util/string_utils.h" @@ -17,8 +18,10 @@ #include #endif +#include #include #include +#include namespace fl { @@ -40,7 +43,14 @@ bool HttpDownloadFile(const std::string& url, #if defined(FOUNDRY_LOCAL_USE_WINHTTP_TRANSPORT) WinHttpTransport transport; #else - CurlTransport transport; + // The bundled libcurl does not honor the SSL_CERT_FILE environment variable (it was built with a + // compiled-in default CA path that does not exist on platforms like Android). Explicitly pass the + // CA bundle via CAInfo so the caller-provided trust store is actually used for TLS verification. + CurlTransportOptions curl_opts; + if (std::string ca_bundle = http::CaBundleFile(); !ca_bundle.empty()) { + curl_opts.CAInfo = std::move(ca_bundle); + } + CurlTransport transport(curl_opts); #endif Request request(HttpMethod::Get, Url(url)); request.SetHeader("User-Agent", user_agent);