An Attempt to return meaningful Problem Details responses for model binding errors in an ASP.NET Core Minimal API

Introduction

Model Binding is the process of binding parameters of the route handler to request parameters (to route segments, query string parameters, cookies, request headers or request body) or to services in the DI container.

The automatically added EndpointMiddleware, which is the last middleware in the request pipeline, performs model binding, then invokes the handler for the requested route with the parameter values extracted during model binding.

If an error occurs during model binding, then in Production environment, EndpointMiddleware returns a 400 or a 500 response with an empty response body whereas in Development environment it throws a BadHttpRequestException.

I wanted to return helpful and detailed Problem Details responses that would help the client fix an error that occurred during model binding if the error occurred due to a problem with the request (when EndpointMiddleware returns a 400 response with empty body in Production environment).

In order to do this I needed to understand exactly how EndpointMiddleware returns an error that it encountered during model binding.

How EndpointMiddleware returns Model Binding Errors

I have verified the following behaviour:

If a model binding error occurred but NOT due to an issue with contents of the HTTP request, then, in any ASP.NET Core environment, the EndpointMiddleware would return a 500 status code.

An example is when a service that was meant to be resolved from the DI container and passed in as an argument to the handler could not be resolved. No exception would be throw, only 500 would be set as the status code of HttpContext.Request

If the error occurred due to an issue with contents of the request, e.g. a required field or value is missing or a request parameter has an invalid/malformed value or a value of an incorrect data type or there is an issue with JSON request body, then

• In Development environment (and possibly in any non-Production environment), a BadHttpRequestException is thrown by the middleware. This has:

  • StatusCode property set to 400

  • a Message property that is informative such as:

    Failed to read parameter "CreateProductArgs createProductArgs" from the request body as JSON.

  • InnerException property which, if there was problem deserializing the JSON request body, would be System.Text.Json.JsonException.

    Again, this has an informative Message property:
    

    JSON deserialization for type 
    'problemdetailstestapi.CreateProductArgs' was 
    missing required properties, 
    including the following: price
    

• In Production Environment, the EndpointMiddleware sets status code 400 in the outgoing response with a blank response body.

  However we can enable throwing of BadHttpRequestException in Production environment by adding the following line in Program.cs before builder.Build() is called (from this issue in dotnet/aspnetcpore repo) to get our hands on this information.

  Now, even in Production, if a model binding error occurred due to an issue with contents of the request, then a BadHttpRequestException would be thrown by EndpointMiddleware. This is useful as the exception can be quite informative:
  

  builder.Services.Configure<RouteHandlerOptions>(
    options => { options.ThrowOnBadRequest = true; }
  );
  

Clearly, the messages above in BadHttpRequestException are useful but cannot be sent back to the client as doing so would reveal internal execution and implementation details.

I also do not want to parse them to extract information as for the same exception, the structure of the error message can be different in different sitautions. Also, exception messages may change in the future.

However, the messages are very useful for logging, at Information level or above, and that alone is a good reason to turn on throwing of BadHttpRequestException in Production environment (i.e. to catch and log the exception).

I have verified that an error does not gets logged (using .NET logging) from within EndpointMiddleware when there is a model binding exception binding some part of the request to handler parameters at level Information or above.

So we need log this exception ourselves. This can be done in one of at least two ways:

• via request logging at the reverse proxy (e.g. in a Web App in Azure App Service) to log all requests that resulted in a 400 response being returned.
• by writing a middleware that will catch and log the BadHttpRequestException exception thrown by EndpointMiddleware once throwing of this exception has been turned on in Production environment.

Returning model binding errors as Problem Details responses

Based on the above behaviour, I thought I could create and return informative Problem Details responses in the event of a model binding error by creating a middleware that would also log these exceptions using .NET logging, as follows:

• Enable throwing HttpBadRequestException in Production environment.

• Add a middleware just before RoutingMiddleware. This would catch catch a BadHttpRequestException.

• If a BadHttpRequestException exception is caught in the middleware that was thrown by the RoutingMiddleware rather than by an endpoint filter or by the invoked router handler then we have one of these two error situations:

  • If InnerException is System.Text.Json.JsonException then, since by default only a single request body is invalid in the specific sense that either a provided value is of an incorrect type, or a required value is missing. Report this with a ProblemDetails response.
  • Otherwise the issue with some other part of the request. Examples include the following: a requried route segment is missing or is of an incorrect value or the JSON body is missing. Report this with a ProblemDetails response.

• If a BadHttpRequestException was NOT caught in the middleware that was thrown by the EndpointMiddleware, i.e. no exception was caught, or an exception was caught but it was not BadHtpRequestException or a BadHttpRequestException was caught but it hadn’t been thrown by the EndpointMiddleware (i.e. had ben thrown by an endpoint filter or from somewhere in the invoked route handler), then we do not have the information to create a meaningful ProblemDetails response.

  So we just let it - the response or the exception - propagate up the request pipeline.

Was it worth it?

The trouble is, for all of this logic - which is not difficult to implement but is a bit convoluted - all I get is the ability to distinguish between two very broad errors in the request that between each other cover pretty much anything that could go wrong with the request (headers, route segments, query string, request body, cookies).

I implemented a middleware to do this (see below) and it was not pretty, (and it didn't include the check for the stack trace to see if the BadHttpRequestException had been thrown by the EndpointMiddleware or not; I give a sketch of how to do this below).

Essentially, the issue is that the BadHttpRequestException returned by EndpointMiddleware is not very machine readable.

Equally, we could say that model binding is a very opaque process that does not return precise or detailed errors when things go wrong.

Conclusion

I don’t see what value distinguishing between these two broad errors would add over just sending back a plain 400 with an empty response body.

The semantics of a 400 (Bad Request) status code are almost the same as these two errors (returned as Problem Details responses) taken together.

And in Production environment, a plain 400 is exactly what gets returned (as described above). So there's no need to do anything or change anything.

To quote my favourite part of the Problem Details specification RFC 9457:

truly generic problems -- i.e., conditions that might apply to any resource on the Web -- are usually better expressed as plain status codes. For example, a "write access disallowed" problem is probably unnecessary, since a 403 Forbidden status code in response to a PUT request is self-explanatory.

In order to get more specific errors out of the opaque model-binding process, I would need to basically reverse-engineer, at least partially, EndpointMiddleware's model-binding logic in a custom middleware that sits just before EndpointMiddleware.

I would be doing this just to pinpoint what, if anything, would go wrong in model binding so that I can send back useful and informative Problem Details responses.

I am not particularly inclined to do this because I cannot see how the value of this to the consumer of my current API project exceeds the investment in reverse-engineering, and keeping in sync, model-binding logic.

Isn't the whole point of model-binding in ASP.NET Core that it is free compared to something like Express/Node.js?

SO I WILL NOT IMPLEMENT THE SOLUTION SHOWN ABOVE.

To log requests that led to 400 being returned, I could just turn on request logging in my reverse proxy (currently Azure app Service).

The middleware was as follows:

using System.Text;
using System.Text.Json;

namespace problemdetailsmiddleware.Middleware;

public class ProblemDetailsForBadRequestMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<ProblemDetailsForBadRequestMiddleware> _logger;

    public ProblemDetailsForBadRequestMiddleware(RequestDelegate next, ILogger<ProblemDetailsForBadRequestMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (BadHttpRequestException ex)
        {
            context.Response.StatusCode = 200;
            await context.Response.WriteAsync(ex.StackTrace ?? "");
            return;

            context.Response.StatusCode = ex.StatusCode;
            _logger.LogError(ex, "BadRequestException occurred while processing HTTP request");
            if (ex.InnerException is JsonException)
            {
                //this would only happen if the 
                // var validationProblem = TypedResults.ValidationProblem(new Dictionary<string, string[]> {
                //     {"request body json", new string[] {@"An error occurred when parsing the provided request body. One of three things is likely to be wrong:

                //     1. The provided request body is not well-formed JSON
                //     2. A required key is missing in the request body JSON
                //     3. A value of an incorrect type is provided for a key in the request body JSON"}}
                // });

                var problem = TypedResults.Problem(
                    statusCode: StatusCodes.Status400BadRequest,
                    type: "http://example.com/problems/invalid-request-body-json",
                    title: "\"An error occurred while parsing request body JSON\","
                    detail: "Request body was provided but an error occurred when parsing it. One of three things is likely to be wrong: 1. The provided request body is not well-formed JSON 2. A required property is missing in the request body JSON 3. A value of an incorrect or incompatible type was provided for a property in the request body JSON"
                    );

                await problem.ExecuteAsync(context);
            }
            else
            {
                var problem = TypedResults.Problem(
                    statusCode: StatusCodes.Status400BadRequest,
                    type: "http://example.com/problems/missing-body-or-invalid-request-parameter-values",
                    title: "\"Request body is missing or a request parameter value is missing or invalid\","
                    detail: "Either the request body is required but missing, or the value of a request parameter - in request headers, query string, route segments or cookies - is either missing ( in case of a required parameter) or invalid (e.g. of an incorrect type). Check your request against the OpenAPI description of the operation."
                );
                await problem.ExecuteAsync(context);
            }
        }
    }

}


public static class ProblemDetailsForBadRequestMiddlewareExtensions
{
    public static IApplicationBuilder UseProblemDetailsForBadRequest(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<ProblemDetailsForBadRequestMiddleware>();
    }
}

Distinguishing between whether the BadHtpRequestException was thrown from EndpointMiddleware of from further down the request processing pipeline:

If every frame in the stack (each is a new line) begins with at Microsoft.AspNetCore.Http.RequestDelegateFactory until potentially a --- End of stack trace from previous location --- line is ensountered, then the exception was thrown from EndpointMiddleware (I believe RequestDelegateFactory create a RequestDelegate out of every middleware in the pipeline that is to be invoked). For example,

   at Microsoft.AspNetCore.Http.RequestDelegateFactory.Log.InvalidJsonRequestBody(HttpContext httpContext, String parameterTypeName, String parameterName, Exception exception, Boolean shouldThrow)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<HandleRequestBodyAndCompileRequestDelegateForJson>g__TryReadBodyAsync|102_0(HttpContext httpContext, Type bodyType, String parameterTypeName, String parameterName, Boolean allowEmptyRequestBody, Boolean throwOnBadRequest, JsonTypeInfo jsonTypeInfo)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass102_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at problemdetailsmiddleware.Middleware.ProblemDetailsForBadRequestMiddleware.InvokeAsync(HttpContext context) in C:\MyWork\problemdetails\problemdetailsmiddleware\Middleware\ProblemDetailsForBadRequestMiddleware.cs:line 20

If there are other frames, theses would come from an endpoint filter or from somewhere in the invoked route handler. In this case, not all frames, until the line --- End of stack trace from previous location --- is encountered, would start with at Microsoft.AspNetCore.Http.RequestDelegateFactory, as in this example from a Release build of a minimal API (for some reason the Release build also contained a .pdb, hence the topmost frame even tells youthat the error occurred at line 80):

   at Program.<>c.<<Main>$>b__0_3(IList`1 products, LinkGenerator linkGen, CreateProductArgs createProductArgs) in C:\MyWork\problemdetails\problemdetailsmiddleware\Program.cs:line 80
   at lambda_method4(Closure, EndpointFilterInvocationContext)
   at FluentValidation.AspNetCore.Http.FluentValidationEndpointFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass102_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at problemdetailsmiddleware.Middleware.ProblemDetailsForBadRequestMiddleware.InvokeAsync(HttpContext context) in C:\MyWork\problemdetails\problemdetailsmiddleware\Middleware\ProblemDetailsForBadRequestMiddleware.cs:line 20