Graceful shutdown of a program involves
Sending SIGTERM(notify
program that its going to be killed) ,upon receiving this signal the program stops receiving further requests(could be web requests, database operations etc)Finalize ongoing/pending tasks(in-flight data)
Release resources (file locks, memory) and terminate(with exit status 0)
Process should be robust against sudden death(eg. power failure, hardware failures). Using robust message queue(beanstalkd) is recommended for such scenarios
Queue Concept-
- producer puts job in the Queue(eg. json, xml)
- consumer monitors that Queue and takes/reserve the available job
- consumer deletes the job from Queue
Read disposability
Graceful shutdown example for golang
https://pkg.go.dev/os/signal#Notify
package main
import (
"context"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
)
func main() {
// Hello world, the web server
helloHandler := func(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "You requessted %s - %s", req.URL, req.Method)
fmt.Println("Serving requests from hey...")
time.Sleep(time.Second * 2)
defer fmt.Println("remaining") // acts as remaining requests
}
api := &http.Server{
Addr: ":8080",
Handler: http.HandlerFunc(helloHandler),
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
serverErrors := make(chan error, 1)
shutdown := make(chan os.Signal, 1)
go func() {
log.Printf("main api listening on %s", api.Addr)
serverErrors <- api.ListenAndServe()
}()
signal.Notify(shutdown, os.Interrupt, syscall.SIGTERM)
//above is not blocking operation although waiting on shutdown channel
select {
case err := <-serverErrors:
log.Fatalf("Error while listening and starting http server: %v", err)
case <-shutdown:
log.Println("main: Starting shutdown")
const timeout = 5 * time.Second
// Context - it is used for Cancellation and propagation, the context.Background() gives empty context
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
err := api.Shutdown(ctx)
/*Shutdown gracefully shuts down the server without interrupting any active connections. Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down.If the provided context expires before the shutdown is complete, Shutdown returns the context's error, otherwise it returns any error returned from closing the Server's underlying Listener(s).*/
if err != nil {
log.Printf("main: Graceful shutdown didnot complete in %v:%v", timeout, err)
err = api.Close()
//Close() immediately closes all active net.Listeners and any connections in state StateNew, StateActive, or StateIdle. For a graceful shutdown, use Shutdown.
}
if err != nil {
log.Fatalf("main: could not stop server gracefully Error: %v", err)
}
}
}
additional reading: link1(for context timeouts)
Top comments (0)