Skip to content

[RUNTIME][VM] Lift LoadVMModule() interface to VM runtime#19851

Open
cbalint13 wants to merge 1 commit into
apache:mainfrom
cbalint13:tvm-vm-api
Open

[RUNTIME][VM] Lift LoadVMModule() interface to VM runtime#19851
cbalint13 wants to merge 1 commit into
apache:mainfrom
cbalint13:tvm-vm-api

Conversation

@cbalint13

@cbalint13 cbalint13 commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Lifts the LoadVMModule() API from disco submodule to vm submodule in the runtime.
Having it main in the VM runtime instead, not only disco benefits but becomes a usable public C++ API.

Related to #19849 which also close the gap to a functional c++ inference using pre-exported DSOs.


Issue

Using an exported DSO module from a simple C++ program:

// #include <runtime/disco/builtin.h> 
#include <runtime/vm/vm.h>

int main() {

    tvm::Device dev = {kDLCPU, 0};
    // auto mod = tvm::runtime::LoadVMModule("./lib.tar.so", dev);
    auto mod = tvm::runtime::vm::LoadVMModule("./lib.tar.so", dev);

    return 0;
}

Before

  • From initial disco submodule, tvm_runtime_extra was also needed, ending with a disco error:
$ c++ -I/usr/include/tvm dso-run-tvm.cpp -o dso-run-tvm -ltvm_ffi -ltvm_runtime -ltvm_runtime_extra
$ ./dso-run-tvm
terminate called after throwing an instance of 'tvm::ffi::Error'
  what():  Check failed: (ret) is false: The current thread is not a DiscoWorker thread
Aborted                    (core dumped) ./dso-run-tvm
$

After

  • Only tvm_runtime is needed, it works fine:
$ c++ -I/usr/include/tvm dso-run-tvm.cpp -o dso-run-tvm -ltvm_ffi -ltvm_runtime
$ ./dso-run-tvm
$

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors LoadVMModule and DSOLibraryCache by moving them from the Disco runtime module to the VM runtime module, updating the signature of LoadVMModule to accept a non-optional Device parameter, and wrapping its registration in a lambda to handle default devices. The review feedback highlights a missing <mutex> header in src/runtime/vm/vm.cc which could cause compilation issues, and suggests passing the path parameter by const std::string& instead of by value across several files to avoid unnecessary string copies. Additionally, it is recommended to remove the redundant TVM_RUNTIME_DLL specifier from the function definition.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/runtime/vm/vm.cc
Comment thread src/runtime/vm/vm.cc Outdated
Comment thread include/tvm/runtime/vm/vm.h Outdated
Comment thread src/runtime/extra/disco/builtin.cc
@cbalint13 cbalint13 marked this pull request as draft June 20, 2026 21:43
@cbalint13 cbalint13 marked this pull request as ready for review June 20, 2026 22:12
@tqchen

tqchen commented Jun 20, 2026

Copy link
Copy Markdown
Member

in this case, i think we should deliberately keep the function in disco because remote may need to load module this way. for local c++ applications, once can simply use the more detailed sequence of vm_load_executable/vm_initialization. The DSO cache is also very specific to disco i think to make sure no reload but may not be general applicable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants