What is Dynamic Library Loading?
Dynamic library loading, also known as dynamic loading or runtime dynamic linking, is a programming technique that allows a program to load external libraries (also known as dynamic link libraries or shared libraries) into memory and use their functionalities at runtime, rather than at compile time. These libraries contain compiled code that can be executed by the program, enabling modularization, code reuse, and flexibility.
Dynamic library loading is in contrast to static linking, where all the necessary code is included in the program's executable binary during the compilation process. With dynamic library loading, the program remains smaller in size, and libraries can be updated or replaced without requiring changes to the main program.
Dynamic Library Loading in Rust
Rust provides a safe and ergonomic interface for loading dynamic libraries at runtime. The libloading crate provides a cross-platform API for loading dynamic libraries and calling functions defined in those libraries. It also provides a safe wrapper around the dlopen
and dlsym
functions on Unix-like systems and the LoadLibrary
and GetProcAddress
functions on Windows.
The complete source code is available on GitHub.
Setup the workspace
Let's start by creating a new Rust project named lib-loading
:
cargo new lib-loading
# remove the src directory as we will be creating a workspace
Now, update the Cargo.toml
file to create a workspace and add two members to it:
[workspace]
members = [
"hello-world",
"executor",
]
Create a dynamic library
Let's now create a library named hello-world
with the following Cargo.toml
file:
cargo new hello-world --lib
[package]
name = "hello-world"
version = "0.1.0"
edition = "2021"
[lib]
crate-type = ["dylib"]
Note the crate-type
field in the [lib]
section. This tells the compiler to build a dynamic library instead of the default static library. This output type will create *.so
files on Linux, *.dylib
files on macOS, and *.dll
files on Windows.
You can read more about different crate-types in the Rust Reference.
Next, update the src/lib.rs
file to define a function named execute
that prints a message to the console:
#[no_mangle]
pub fn execute() {
println!("Hello, world!")
}
Name mangling is a compiler technique that encodes function names with additional information like the function's parameters and return type. However, this makes it difficult to call functions from other languages or dynamically loaded libraries.
The
no_mangle
attribute turns off Rust's name mangling, so that it has a well defined symbol to link to. This allows us to call the function from the dynamically loaded library.
Create a binary crate
Next, let's create a binary crate named executor
that will load the hello-world
library and call the execute
function defined in it.
cargo new executor
Add a dependency on the libloading
crate to the Cargo.toml
file:
[package]
name = "executor"
version = "0.1.0"
edition = "2021"
[dependencies]
libloading = "0.8"
Now, update the src/main.rs
file to load the hello-world
library and call the execute
function:
use libloading::{Library, library_filename, Symbol};
fn main() {
unsafe {
// Load the "hello_world" library
let lib = Library::new(library_filename("hello_world")).unwrap();
// Get the function pointer
let func: Symbol<fn()> = lib.get(b"execute").unwrap();
func() // Call the function
}
}
It's a good practice to use the
library_filename
function to get the platform-specific filename of the library. This function returnslibhello_world.so
on Linux,libhello_world.dylib
on macOS, andhello_world.dll
on Windows.
Build and run the project
First, let's build the project with cargo build
:
lib-loading on master [?] via 🦀 v1.69.0
➜ cargo build --release
Compiling cfg-if v1.0.0
Compiling hello-world v0.1.0 (/Users/gaurav.gahlot/workspace/playground/lib-loading/hello-world)
Compiling libloading v0.8.0
Compiling executor v0.1.0 (/Users/gaurav.gahlot/workspace/playground/lib-loading/executor)
Finished release [optimized] target(s) in 1.42s
Next, run the executor
using cargo run
:
lib-loading on master [?] via 🦀 v1.69.0 took 2s
➜ cargo run -p executor --release
Finished release [optimized] target(s) in 0.02s
Running `target/release/executor`
Hello, world!
Congratulations! You've successfully loaded a dynamic library and called a function defined in it.
When to use Dynamic Library Loading?
Dynamic library loading is useful in several scenarios where you want to achieve modularity, flexibility, and runtime extensibility in your applications. Here are some situations where dynamic library loading can be advantageous:
Plugin Systems: If you're building an application that supports plugins or extensions, dynamic library loading can allow you to load and unload these plugins at runtime without modifying or restarting the main application. This can be useful for applications like text editors, web browsers, or media players that support third-party plugins.
Modular Applications: When you're developing a large application, dynamic library loading can help you break down the functionality into smaller, manageable components. This can lead to faster development cycles, easier maintenance, and improved code organization.
Hot Reload: For certain types of applications like game development tools or graphical design software, dynamic library loading can enable hot reloading. This means you can modify parts of the application's code, compile them into a dynamic library, and load that library without restarting the entire application. This greatly speeds up development iterations.
Platform-Specific Implementations: If your application needs to interact with platform-specific features, dynamic library loading allows you to provide different implementations for different platforms. You can load the appropriate library based on the runtime environment, avoiding unnecessary code in the main application binary.
Versioning and Updates: When you want to provide updates or bug fixes to a specific part of your application, you can distribute and load only the updated dynamic library without needing users to update the entire application.
Resource Sharing: Dynamic libraries can be used to share resources like database connections, network sockets, or other system resources across multiple parts of your application. This can help optimize resource usage and improve performance.
Language Interoperability: If you need to interface with code written in other programming languages (e.g., C or C++), dynamic libraries provide a way to call functions written in those languages from your Rust code.
Security and Isolation: In some cases, you might want to isolate certain components of your application for security reasons. Dynamic libraries can help you encapsulate sensitive code, limiting its exposure to the main application.
Challenges of Dynamic Library Loading
Dynamic library loading also introduces complexity and potential challenges. Here are a few things to consider when using dynamic libraries:
Memory Management: You need to manage memory and resource allocation properly, ensuring that you release resources and unload libraries when they're no longer needed to avoid memory leaks.
Compatibility: Ensuring that dynamically loaded libraries are compatible with your application's version and other libraries can be challenging. Version mismatches can lead to crashes or unexpected behavior.
Unsafe Code: Working with dynamically loaded libraries often involves using the
unsafe
keyword due to the inherent risks associated with runtime operations that Rust's safety mechanisms cannot fully verify.Debugging: Debugging issues in dynamically loaded libraries can be more challenging compared to monolithic applications.
Performance Overhead: There can be a slight performance overhead when using dynamic libraries compared to statically linking code directly into your application binary.
Conclusion
In summary, dynamic library loading is most beneficial when you need to create modular, flexible, and extensible applications, but it requires careful design, proper resource management, and an understanding of the potential trade-offs. Rust's safety mechanisms can help you avoid many of the pitfalls associated with dynamic library loading, but you still need to be aware of the risks and challenges involved.
Top comments (3)
Really great article! It gave me a good introduction to dynamic libraries and inspired me. the simplicity and accuracy of Rust is always amazing. Further, I would like to know how static variables can be included and referenced externally, especially the &'static str type. Can you give me some advice? :D
thanks.
I have not tried it myself, so I don't really have an answer. Sorry.
It helps me a lot !! thanks