Skip to content

#1456: Cleaned up and refactored exaudfclient cc#642

Open
sgn4sangar wants to merge 21 commits into
masterfrom
refactoring/1456_cleanup_exaudfclient_cc
Open

#1456: Cleaned up and refactored exaudfclient cc#642
sgn4sangar wants to merge 21 commits into
masterfrom
refactoring/1456_cleanup_exaudfclient_cc

Conversation

@sgn4sangar

@sgn4sangar sgn4sangar commented May 25, 2026

Copy link
Copy Markdown
Contributor

relates to #1456

Comment thread exaudfclient/exa_vm_factory.h Outdated
Comment on lines +4 to +22
#ifdef ENABLE_JAVA_VM
#include "javacontainer/javacontainer_builder.h"
#endif //ENABLE_JAVA_VM

#ifdef ENABLE_PYTHON_VM
#include "python/pythoncontainer.h"
#endif //ENABLE_PYTHON_VM

#ifdef UDF_PLUGIN_CLIENT
#include "protegrityclient.h"
#endif //UDF_PLUGIN_CLIENT

#ifdef ENABLE_STREAMING_VM
#include "streaming_container/streamingcontainer.h"
#endif

#ifdef ENABLE_BENCHMARK_VM
#include "benchmark_container/benchmark_container.h"
#endif

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be moved into exaudfclient/exa_vm_factory.cc

Comment thread exaudfclient/exa_lib_loader.cc Outdated
if((error = dlerror()) != nullptr) {
std::stringstream sb;
sb << "Error when trying to load symbol '" << symbol_name << "': " << error;
fprintf(stderr, "dlsym error: %s loading symbol %s\n", error, sb.str().c_str());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already using streams for constructing sb, so we probably can also use cerr directly and don't need the string stream and the fprintf

Comment thread exaudfclient/exa_lib_loader.cc Outdated

void* handle = dlmopen(LM_ID_NEWLM, stdLibPath.c_str(), RTLD_NOW);
if (!handle) {
fprintf(stderr, "dlmopen error: %s; while loading %s\n", dlerror(), stdLibPath.c_str());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably could move this also to cerr

Comment thread exaudfclient/exa_set_env.cc Outdated

void setup_environment() {
if (::setenv("HOME", "/tmp", 1) == -1) {
fprintf(stderr, "Failed setting HOME env var: %s\n", std::strerror(errno));

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably move this also to cerr

Comment thread exaudfclient/exa_udfclient.cc Outdated
DBGVAR(std::cerr, libexaudflibPath);
void* handle = exa_load_libary(libexaudflibPath);
if (!handle) {
fprintf(stderr, "Failed to load library: %s\n", libexaudflibPath.c_str());

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already using cerr, so we probably can use it here, too

Comment thread exaudfclient/exa_udfclient.cc
Comment thread exaudfclient/exa_common/exa_lib_loader.cc
Comment thread exaudfclient/exa_common/exa_lib_loader.cc
Comment thread exaudfclient/exa_common/exa_set_env.cc
Comment thread exaudfclient/exa_udfclient.cc Outdated
exit(EXIT_FAILURE);
}
std::string libexaudflibPath;
#ifdef CUSTOM_LIBEXAUDFLIB_PATH

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should move the ifdef into its own function, this would make this function easier to read

Comment thread exaudfclient/exa_common/exa_vm_factory.cc
Comment thread exaudfclient/exa_udf_base.cpp Outdated

mb_useCtpgParser = false;
if(argc == 4) {
mb_useCtpgParser = is_use_ctpg_parser(argv[3]);

@tomuben tomuben May 27, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces a bug: If there is no fourth CLI parameter, the environment variable SCRIPT_OPTIONS_PARSER_VERSION is never eveluated!

Comment thread exaudfclient/exa_udf_base.cpp Outdated

// Parser option 2 means use ctpg parser.
// argv_parser_option is command line argument as it is
bool ExaUdfClientBase::is_use_ctpg_parser(const std::string& argv_parser_option) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_use_ctpg_parser is a strange name. Maybe just use_ctpg_parser?

Comment thread exaudfclient/exa_udf_clients.cpp Outdated
#endif //ENABLE_PYTHON_VM

void ExaUdfClientPython::usage(const std::string& programName) {
std::cerr << "Usage: " << programName << " <socket> lang=python <scriptOptionsParserVersion=1|2>"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scriptOptionsParserVersion is actually only relevant for Java

Comment thread exaudfclient/exa_common/exa_udf_base.h
Comment thread exaudfclient/exa_udf_base.cpp Outdated

setup_environment();
std::function<SWIGVMContainers::SWIGVM*()> vmMaker;
switch(m_lang) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Base class should be an abstract class and call here create_vm, however don't implement it. You mixed implementation of the client that supports all languages with the implementation of the abstract class. For the moment, we need the ExaudfclientBase as abstract class, ExaudfclientAllLangs, and we can add for each language its Exaudfclient{Lang} if we want to, however I would wait with later until we do the split into repos.

@tkilias tkilias left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidentally approved, please see my comments

Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_udf_base.cpp Outdated
Comment thread exaudfclient/exa_udf_base.cpp Outdated
Comment thread exaudfclient/exa_vm_factory.cc Outdated
Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_udf_clients.cpp Outdated
Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_udf_base.h Outdated
Comment thread exaudfclient/exa_common/exa_udf_client.cc
Comment thread exaudfclient/exa_common/exa_udf_client.h
Comment thread exaudfclient/exa_common/exa_udf_base.cc
Comment on lines +4 to +5
void* exa_load_libary(const std::string& stdLibPath);
void* exa_load_symbol(void *handle, const std::string& symbol_name); No newline at end of file

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be moved into a separate namespace?

@@ -0,0 +1,3 @@
#pragma once

void setup_environment(); No newline at end of file

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move into namespace?


void setup_environment() {
if (::setenv("HOME", "/tmp", 1) == -1) {
std::cerr << "Failed setting HOME env var: " << std::strerror(errno) << std::endl;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the old implementation we raised an exception.

std::cerr << "Failed setting HOME env var: " << std::strerror(errno) << std::endl;
}
if (::setlocale(LC_ALL, "en_US.utf8") == nullptr) {
std::cerr << "Failed setting locale: " << std::strerror(errno) << std::endl;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise exception?

Comment on lines +32 to +49
DBGMSG(std::cerr, "Load libexaudflib");
DBGVAR(std::cerr, libexaudflibPath);
void* exaudflib_handle = exa_load_libary(libexaudflibPath);
if (!exaudflib_handle) {
std::cerr << "Failed to load library: " << libexaudflibPath << std::endl;
exit(EXIT_FAILURE);
}
set_exaudflib_handle(exaudflib_handle);

MAIN_FUN exaudfclient_main = (MAIN_FUN)exa_load_symbol(exaudflib_handle, "exaudfclient_main");
SET_SWIGVM_PARAMS set_SWIGVM_params = (SET_SWIGVM_PARAMS)exa_load_symbol(exaudflib_handle, "set_SWIGVM_params");

setup_environment();
std::function<SWIGVMContainers::SWIGVM*()> vmMaker = create_vm();

SWIGVMContainers::SWIGVM_params = new SWIGVMContainers::SWIGVM_params_t(true);
set_SWIGVM_params(SWIGVMContainers::SWIGVM_params);
return exaudfclient_main(vmMaker, argc, argv);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix indention

void ExaUdfClient::parse_arguments(int argc, char** argv) {
// Now assumption is that all cmd line aguments are already validated.
m_languageArg = argv[2];

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add comment from old implementation

Suggested change
/*
* The given script-options-parser version set by the environment variable "SCRIPT_OPTIONS_PARSER_VERSION"
* must have priority over the CLI argument "scriptOptionsParserVersion=x".
* This allows clients to override the parser version in a specific UDF, if needed,
* via "%env SCRIPT_OPTIONS_PARSER_VERSION=x".
*/

Comment on lines +85 to +88
if executable.startswith("/opt/conda"):
expected_prefix = "/opt/conda"
else:
expected_prefix = "/usr"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't work.
For the udf-client with the bug (PATH variable without /opt/conda/bin) the executable was empty.

My suggestion: create two folders under ./test_container/tests/test/python3:
python_apt
python_conda
and then configure the flavor ci.config accordingly.

Also remember we should execute this test for the debug build.

For me it's ok to fix this later in another PR, we just need to create a ticket.

#include "exaudflib/vm/swig_vm.h"

void* handle;
static void* exaudflib_handle;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iniatilize with nullptr

exaudflib_handle = handle;
}

void* load_dynamic(const char* name) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check if exaudflib_handle != nullptr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants