DEV Community

Cover image for HAProxy's Zero-Downtime Reload
Joshua Varghese
Joshua Varghese

Posted on

HAProxy's Zero-Downtime Reload

HAProxy's Reload Architecture

HAProxy uses a sophisticated socket transfer mechanism between old and new processes. This design choice leads to some interesting trade-offs and benefits.

The Master CLI Process

HAProxy's architecture includes a master CLI process that orchestrates the reload:

Example of HAProxy's master-worker socket handling */
static int proc_self_pipe[2];

void master_register_worker(struct worker *w) {
    struct listener *listener;

    list_for_each_entry(listener, &w->listeners, list) {
        /* Transfer listener sockets to the new worker */
        if (listener->state == LI_READY) {
            listener_transfer_fd(listener, w);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Socket Transfer Magic

One of the most interesting aspects of HAProxy's implementation is how it handles socket transfers. The process involves several key steps:

  1. Socket Preparation
static int listener_transfer_fd(struct listener *l, struct worker *w) {
    struct cmsg_fd_list fdlist;
    struct msghdr msg;
    struct iovec iov[1];
    int ret;

    /* Prepare socket data structure */
    memset(&msg, 0, sizeof(msg));
    msg.msg_iov = iov;
    msg.msg_iovlen = 1;

    /* Set up control message for FD passing */
    msg.msg_control = &fdlist;
    msg.msg_controllen = sizeof(fdlist);
}
Enter fullscreen mode Exit fullscreen mode
  1. Graceful Connection Handover HAProxy ensures existing connections aren't disrupted during the reload:
void perform_soft_reload(void) {
    /* 1. Keep accepting new connections in old process */
    while (nb_running_tasks() > 0) {
        /* 2. Process existing connections */
        process_runnable_tasks();

        /* 3. Check if we can stop */
        if (should_exit()) {
            break;
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

State Management During Reload

HAProxy's state management during reload is particularly clever. It handles several key aspects:

Connection State Preservation

struct connection {
    unsigned int flags;     /* Status flags */
    enum obj_type *target;  /* What this connection is about */
    void *ctx;             /* Application specific context */
    /* ... other fields ... */
};

/* During transfer */
static void transfer_connection_state(struct connection *conn) {
    /* Save essential connection data */
    struct conn_state state = {
        .flags = conn->flags,
        .protocol_state = conn->ctx,
        .ssl_state = conn->ssl_ctx
    };

    /* Transfer to new process */
    send_state_to_new_process(&state);
}
Enter fullscreen mode Exit fullscreen mode

SSL Session Handling

A particularly tricky part is managing SSL sessions during reload:

static int ssl_session_transfer(SSL *ssl, struct worker *new_worker) {
    unsigned char *session_data;
    unsigned int session_len;

    /* Serialize SSL session */
    session_data = ssl_serialize_session(ssl, &session_len);
    if (!session_data)
        return -1;

    /* Transfer to new process */
    return send_session_to_worker(new_worker, session_data, session_len);
}
Enter fullscreen mode Exit fullscreen mode

Configuration Validation

Before any reload happens, HAProxy performs extensive configuration validation:

int check_config_validity(char *file) {
    struct proxy *curproxy = NULL;
    int cfgerr = 0;

    /* Parse and validate configuration */
    for (curproxy = proxy_list; curproxy; curproxy = curproxy->next) {
        /* Check proxy settings */
        cfgerr += proxy_cfg_check(curproxy);

        /* Validate SSL configurations */
        cfgerr += ssl_cfg_check(curproxy);

        /* Check backend server configurations */
        cfgerr += check_backend_cfg(curproxy);
    }

    return cfgerr;
}
Enter fullscreen mode Exit fullscreen mode

Health Check Continuity

One often-overlooked aspect is maintaining health check states during reload:

struct check {
    unsigned int status;   /* health check status */
    unsigned int result;   /* test result */
    int code;             /* status code */
    int duration;         /* time it took to get the result */
};

void transfer_health_checks(void) {
    struct server *srv;
    struct check *check;

    /* Iterate through all servers */
    for (srv = servers; srv; srv = srv->next) {
        check = &srv->check;

        /* Save health check state */
        if (check->status & CHK_ST_ENABLED) {
            save_check_state(check);
        }
    }
}

Enter fullscreen mode Exit fullscreen mode

Performance Considerations

HAProxy's reload mechanism is designed with performance in mind. Here are some key optimizations:

  • Minimal Memory Overhead: Only essential state is transferred
  • Efficient FD Passing: Uses kernel mechanisms for file descriptor transfer
  • Progressive Transfer: Connections are handled gradually to avoid spikes

Top comments (0)