HAProxy's Reload Architecture
HAProxy uses a sophisticated socket transfer mechanism between old and new processes. This design choice leads to some interesting trade-offs and benefits.
The Master CLI Process
HAProxy's architecture includes a master CLI process that orchestrates the reload:
Example of HAProxy's master-worker socket handling */
static int proc_self_pipe[2];
void master_register_worker(struct worker *w) {
struct listener *listener;
list_for_each_entry(listener, &w->listeners, list) {
/* Transfer listener sockets to the new worker */
if (listener->state == LI_READY) {
listener_transfer_fd(listener, w);
}
}
}
Socket Transfer Magic
One of the most interesting aspects of HAProxy's implementation is how it handles socket transfers. The process involves several key steps:
- Socket Preparation
static int listener_transfer_fd(struct listener *l, struct worker *w) {
struct cmsg_fd_list fdlist;
struct msghdr msg;
struct iovec iov[1];
int ret;
/* Prepare socket data structure */
memset(&msg, 0, sizeof(msg));
msg.msg_iov = iov;
msg.msg_iovlen = 1;
/* Set up control message for FD passing */
msg.msg_control = &fdlist;
msg.msg_controllen = sizeof(fdlist);
}
- Graceful Connection Handover HAProxy ensures existing connections aren't disrupted during the reload:
void perform_soft_reload(void) {
/* 1. Keep accepting new connections in old process */
while (nb_running_tasks() > 0) {
/* 2. Process existing connections */
process_runnable_tasks();
/* 3. Check if we can stop */
if (should_exit()) {
break;
}
}
}
State Management During Reload
HAProxy's state management during reload is particularly clever. It handles several key aspects:
Connection State Preservation
struct connection {
unsigned int flags; /* Status flags */
enum obj_type *target; /* What this connection is about */
void *ctx; /* Application specific context */
/* ... other fields ... */
};
/* During transfer */
static void transfer_connection_state(struct connection *conn) {
/* Save essential connection data */
struct conn_state state = {
.flags = conn->flags,
.protocol_state = conn->ctx,
.ssl_state = conn->ssl_ctx
};
/* Transfer to new process */
send_state_to_new_process(&state);
}
SSL Session Handling
A particularly tricky part is managing SSL sessions during reload:
static int ssl_session_transfer(SSL *ssl, struct worker *new_worker) {
unsigned char *session_data;
unsigned int session_len;
/* Serialize SSL session */
session_data = ssl_serialize_session(ssl, &session_len);
if (!session_data)
return -1;
/* Transfer to new process */
return send_session_to_worker(new_worker, session_data, session_len);
}
Configuration Validation
Before any reload happens, HAProxy performs extensive configuration validation:
int check_config_validity(char *file) {
struct proxy *curproxy = NULL;
int cfgerr = 0;
/* Parse and validate configuration */
for (curproxy = proxy_list; curproxy; curproxy = curproxy->next) {
/* Check proxy settings */
cfgerr += proxy_cfg_check(curproxy);
/* Validate SSL configurations */
cfgerr += ssl_cfg_check(curproxy);
/* Check backend server configurations */
cfgerr += check_backend_cfg(curproxy);
}
return cfgerr;
}
Health Check Continuity
One often-overlooked aspect is maintaining health check states during reload:
struct check {
unsigned int status; /* health check status */
unsigned int result; /* test result */
int code; /* status code */
int duration; /* time it took to get the result */
};
void transfer_health_checks(void) {
struct server *srv;
struct check *check;
/* Iterate through all servers */
for (srv = servers; srv; srv = srv->next) {
check = &srv->check;
/* Save health check state */
if (check->status & CHK_ST_ENABLED) {
save_check_state(check);
}
}
}
Performance Considerations
HAProxy's reload mechanism is designed with performance in mind. Here are some key optimizations:
- Minimal Memory Overhead: Only essential state is transferred
- Efficient FD Passing: Uses kernel mechanisms for file descriptor transfer
- Progressive Transfer: Connections are handled gradually to avoid spikes
Top comments (0)