The Importance of Request Timeouts

📣 This post originally appeared as The Importance of Request Timeouts on The Bearer Blog.

Networks are unreliable. Third-party dependencies, like APIs, are unreliable. This is why we build resiliency into our applications and services. Most prep work is focused around HTTP requests, but an aspect of them that is often overlooked is timeouts.

What are timeouts?

Timeouts set a max wait time on a request. We can think about timeouts in two contexts: the client and the server. Client timeouts set a limit on how long they are willing to wait for a response to come back—and in some cases be completed. Server timeouts set a limit on how long the connection should remain open. For example, if a client is taking an exceptionally long time to receive a response, the server may choose to close the connection. Be careful not to limit your thinking of client as browser and server as backend. The client is the application making the request.

You can also consider timeouts to be split into three types: a connect timeout, a write timeout, and a read timeout. Connect timeouts affect how long the client will wait for a connection to establish. Write timeouts affect how long the connection will wait while the client tries to send data, like a POST request. Read timeouts cover the amount of time it takes to actually receive the response back from the server. Some HTTP clients allow you to set each separately, and other's only allow you to set a total combined value.

Timeouts are all about "waiting." When your application makes a request, it has to wait for the response. Even in asynchronous code, eventually that action needs to be handled and processed. Without a timeout established, your code relies on the assumption that it will either receive a successful response or an error. In reality, this isn't always the case. As we said at the start, networks are unreliable. If the request is delayed along the way, and never receives an official error response, your application will hang indefinitely. This is why timeouts are important. They stop the indefinite hang scenario.

Libraries can't be trusted

We make assumptions about "battle-tested" libraries and packages. Their heavy use adds clout, but sometimes the "good default" for a large audience isn't always the best default for your needs. For a few examples:

• Requests, the leading Python library for making HTTP requests, does not have a default value for timeouts. They encourage you to set timeout values, but if you forget or assume a sensible default exists, your code will be susceptible to hanging connections.
• Go's net/http module also has no default timeout. You can, and should, set your own timeout value.
• The default browser implementations lack default timeouts, and in some cases lack easy ways to add timeouts. Both XHR and Fetch have issues, but most packages on NPM do incorporate ways to set timeouts. Axios, for example has a default of 0, which it interprets as infinite.
• Ruby's net/http has 60s defaults for all timeout types. This is pretty high for most use cases, but is at least a default value.

As we can see, even if you're using the language's built-in request module or one of the most popular packages available, you will likely hit problems with insufficient timeouts.

Choosing a timeout value

Determining your ideal timeout involves many variables. Connection timeouts should be kept short, while read and write timeouts are more dependant on the needs of the service. There are some considerations to make when determining the ideal timeout duration for each API call.

One way is to monitor the response time from the APIs you connect to using a tool like Bearer. This works well for helping estimate the average time a successful request takes, and how long a successful error takes. Another approach is to focus on the user experience. If a request is connected to the UI—directly or otherwise—your timeouts should be tied to how long you're willing to delay the interaction. For background-style tasks, higher timeouts are appropriate. For more immediate tasks, shorter timeouts are a better option. Combined with optimistic updates and a retry strategy, timeouts can directly improve your application's UX.

Another variable to consider is serverless functions. If your code is making a request from a serverless endpoint, make sure the timeout is no greater than the max invocation time of the function. The function's max invocation time is often a hard limit. Combined with a little buffer for the time it takes the request and response to move between your application and the remote server, these values can better inform your timeout value. For example: timeout = maxInvocationTime + averageResponseTime, where the average response time is the time it normally takes. This isn't a perfect formula, but it's a great place to start.

Finally, don't punish users for their connection speeds. This primarily applies to browsers, but a user on a 3G connection may require a significantly longer timeout when uploading an image than a user on a faster connection. Keep this in mind for any user-facing connections.

So what should you do?

Ideally, set specific timeouts for each API call your application makes. Use the needs of the interaction, or your services, to determine what the maximum timeout should be. If you're using an HTTP library or module that doesn't surface an easy configuration for this, consider switching to one that does.

Setting custom timeouts for every request can be tedious, especially when you don't have the data necessary to make an informed decision. One alternative approach is to use a solution like Bearer's active remediations to define max timeout values for specific types of requests. For example, you can set all third-party APIs that you know are further away to a higher timeout than those with local servers. This allows you to manage these limits separate from your codebase, which makes experimentation easier. It also makes adapting to unexpected network conditions easier.

Whatever you do, don't neglect timeouts. Don't assume the defaults are safe, and don't assume the network is reliable. Keep your applications resilient against problems that third-party APIs may experience and your users will never know when major issues happen. If you want to learn more about Bearer and our active remediation features, give our in-app agent's a try today.