ASP.NET Core Image Resizing Middleware

The .NET Framework people have it easy when it comes to image resizing. You can’t go past the excellent ImageResizer Package.

Unfortunately, at the time of writing this, I was unable to find a mature-enough cross-platform .net core equivalent.

Even forgetting about resizing and caching images on the fly from a URL, the options are still limited. This is because the Full .NET Framework uses the System.Drawing library, which wraps the Windows-only GDI library.

The options appear limited to one of:

• ImageSharp - looks superb, but it hasn’t officially shipped yet, and is still considered alpha stage by the authors.
• SkiaSharp - based on google’s Skia library, maintained by the Xamarin team. Installable from Nuget. They don’t ship Linux/Mac binaries but you can compile it from source pretty easily. It also seems to be a little faster than ImageSharp.
• ImageFlow - will have everything I need and should be a drop-in replacement once shipped. It’s incomplete at the time of writing and is presently in demo stage.

I went with SkiaSharp for this purely because it seems to be the most mature solution, though it has a few bugs of its own.

The Problem

I want to be able to, at a minimum:

• Resize images on the web server via a URL
• Optionally rotate them according to their EXIF headers (e.g. for phone camera uploads)
• Change the output format
• Change the quality
• Crop, pad, stretch to fit new size, or just set either the width or height and let it automatically determine the other dimension, or fit it within a bounding box.
• Preserve transparency if present
• Cache the output

The Solution

Middleware! I’ve talked about .net core’s middleware before, it’s pretty neat compared to the old way of doing things. This diagram from the docs illustrates the concept succintly:

All we need to do is write our own middleware. Our middleware should:

• Be injected into the pipeline ahead of the static file handler (to prevent the static file handler from handling image resize requests)
• Make a cursory inspection of the URL to determine if it’s an image, and if not, pass it on
• Make a further inspection to determine if it’s an image that actually exists on disk, with valid resize parameters, and if so, handle it, and prevent the rest of the pipeline from taking part in the request.
• Since resizing is an expensive operation, try to serve the request from a cache first, otherwise resize it and cache the output.

Doesn’t sound too complex… let’s get coding.

Dev Environment

• The dev environment, all tools & code will run on Mac, PC or Linux
• We are using Visual Studio Code and a command prompt / terminal window.
• Make sure you have the following Visual Studio Code Extensions (click “Extensions” in the sidebar): C#, C# Extensions

Solution Setup

In a new folder execute the following commands:

dotnet new sln
dotnet new classlib -o src/ImageResizer -f netstandard1.6
dotnet new mvc -o src/Web
dotnet sln add src/ImageResizer/ImageResizer.csproj
dotnet sln add src/Web/Web.csproj
dotnet add src/Web/Web.csproj reference src/ImageResizer/ImageResizer.csproj

code .

The above commands will create our ImageResizer library (to hold our middleware), an MVC web app with default scaffolding, add them both to a solution, add a reference to the ImageResizer library to our Web project, and then open the whole lot.

Once it’s open just accept all the prompts that VS code presents you with, which will restore missing packages etc.

The images in the default web app at the time of writing are SVG, which aren’t suitable for us. For testing purposes I’ve been using these images as they allow me to test the orientation headers.

Throw some images into your wwwroot/images folder.

The Middleware

The ImageResizer has a few dependencies, which can be added using a terminal window in the ImageResizer folder, with:

dotnet add package Microsoft.AspNetCore.Hosting.Abstractions
dotnet add package Microsoft.AspNetCore.Http
dotnet add package Microsoft.Extensions.Caching.Abstractions
dotnet add package Microsoft.Extensions.Caching.Memory
dotnet add package Microsoft.Extensions.Logging.Abstractions
dotnet add package SkiaSharp

Middleware

Create a new C# source file called ImageResizerMiddleware.cs and put our scaffolding in… let’s just make sure it’s handling resize requests first:

using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using SkiaSharp;
using System;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;

namespace ImageResizer
{
    public class ImageResizerMiddleware
    {
        struct ResizeParams
        {
            public bool hasParams;
            public int w;
            public int h;
            public bool autorotate;
            public int quality; // 0 - 100
            public string format; // png, jpg, jpeg
            public string mode; // pad, max, crop, stretch

            public static string[] modes = new string[] { "pad", "max", "crop", "stretch" };

            public override string ToString()
            {
                var sb = new StringBuilder();
                sb.Append($"w: {w}, ");
                sb.Append($"h: {h}, ");
                sb.Append($"autorotate: {autorotate}, ");
                sb.Append($"quality: {quality}, ");
                sb.Append($"format: {format}, ");
                sb.Append($"mode: {mode}");

                return sb.ToString();
            }
        }

        private readonly RequestDelegate _next;
        private readonly ILogger<ImageResizerMiddleware> _logger;
        private readonly IHostingEnvironment _env;
        private readonly IMemoryCache _memoryCache;

        private static readonly string[] suffixes = new string[] {
            ".png",
            ".jpg",
            ".jpeg"
        };

        public ImageResizerMiddleware(RequestDelegate next, IHostingEnvironment env, ILogger<ImageResizerMiddleware> logger, IMemoryCache memoryCache)
        {
            _next = next;
            _env = env;
            _logger = logger;
            _memoryCache = memoryCache;
        }

        public async Task Invoke(HttpContext context)
        {
            var path = context.Request.Path;

            // hand to next middleware if we are not dealing with an image
            if (context.Request.Query.Count == 0 || !IsImagePath(path))
            {
                await _next.Invoke(context);
                return;
            }

            // hand to next middleware if we are dealing with an image but it doesn't have any usable resize querystring params
            var resizeParams = GetResizeParams(path, context.Request.Query);
            if (!resizeParams.hasParams || (resizeParams.w == 0 && resizeParams.h == 0))
            {
                await _next.Invoke(context);
                return;
            }

            // if we got this far, resize it
            _logger.LogInformation($"Resizing {path.Value} with params {resizeParams}");

            await _next.Invoke(context);
        }

        private bool IsImagePath(PathString path)
        {
            if (path == null || !path.HasValue)
                return false;

            return suffixes.Any(x => x.EndsWith(x, StringComparison.OrdinalIgnoreCase));
        }

        private ResizeParams GetResizeParams(PathString path, IQueryCollection query)
        {
            ResizeParams resizeParams = new ResizeParams();

            // before we extract, do a quick check for resize params
            resizeParams.hasParams =
                resizeParams.GetType().GetTypeInfo()
                .GetFields().Where(f => f.Name != "hasParams")
                .Any(f => query.ContainsKey(f.Name));

            // if no params present, bug out
            if (!resizeParams.hasParams)
                return resizeParams;

            // extract resize params

            if (query.ContainsKey("format"))
                resizeParams.format = query["format"];
            else
                resizeParams.format = path.Value.Substring(path.Value.LastIndexOf('.') + 1);

            if (query.ContainsKey("autorotate"))
                bool.TryParse(query["autorotate"], out resizeParams.autorotate);

            int quality = 100;
            if (query.ContainsKey("quality"))
                int.TryParse(query["quality"], out quality);
            resizeParams.quality = quality;

            int w = 0;
            if (query.ContainsKey("w"))
                int.TryParse(query["w"], out w);
            resizeParams.w = w;

            int h = 0;
            if (query.ContainsKey("h"))
                int.TryParse(query["h"], out h);
            resizeParams.h = h;

            resizeParams.mode = "max";
            // only apply mode if it's a valid mode and both w and h are specified
            if (h != 0 && w != 0 && query.ContainsKey("mode") && ResizeParams.modes.Any(m => query["mode"] == m))
                resizeParams.mode = query["mode"];

            return resizeParams;
        }
    }
}

The above code checks if the request is for an image, and if it is, it extracts the query string into a struct, logs it, and then passes onto the next middleware in the chain. Builder Extensions So that we can easily add this middleware to our pipeline, we should write 2 very small extension methods to allow us to “AddImageResizer” and “UseImageResizer” in our MVC app. Let’s do that now. ImageResizerMiddlewareExtensions.cs

using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;

namespace ImageResizer
{
    public static class ImageResizerMiddlewareExtensions
    {
        public static IServiceCollection AddImageResizer(this IServiceCollection services)
        {
            return services.AddMemoryCache();
        }

        public static IApplicationBuilder UseImageResizer(this IApplicationBuilder builder)
        {
            return builder.UseMiddleware<ImageResizerMiddleware>();
        }
    }
}

We are adding MemoryCache here, since it will be a dependency for our Middleware… we can use the IServiceCollection to add other dependencies when we expand our Middleware in future. We’re simply wrapping the UseMiddleware statement into something a bit neater. Wire it up to MVC Easy to wire up. Just 3 lines of code. In the Web project’s Startup.cs file:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using ImageResizer;

namespace Web
{
    public class Startup
    {
        public Startup(IHostingEnvironment env)
        {
            var builder = new ConfigurationBuilder()
                .SetBasePath(env.ContentRootPath)
                .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
                .AddEnvironmentVariables();
            Configuration = builder.Build();
        }

        public IConfigurationRoot Configuration { get; }

        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            // Add framework services.
            services.AddMvc();
            services.AddImageResizer();
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            loggerFactory.AddConsole(Configuration.GetSection("Logging"));
            loggerFactory.AddDebug();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseBrowserLink();
            }
            else
            {
                app.UseExceptionHandler("/Home/Error");
            }

            app.UseImageResizer();
            app.UseStaticFiles();

            app.UseMvc(routes =>
            {
                routes.MapRoute(
                    name: "default",
                    template: "{controller=Home}/{action=Index}/{id?}");
            });
        }
    }
}

Test it

Before we test, we should make sure our hosting environment is set to “Development” so that we get the information logs in our terminal. On Windows (Vista and up)

setx ASPNETCORE_ENVIRONMENT "Development"

On Mac

nano ~/.bash_profile

then add this line to the file: export ASPNETCORE_ENVIRONMENT=development You will need to close and re-open your terminal window of choice for this change to take effect. From the Web folder, execute

dotnet restore
dotnet run

You should see it start up on port 5000 in “Development” environment. Now just navigate to a PNG or JPG image in your solution, with a query string for resizing, and you should see something like (without the actual resize):

Great it’s logging to console and letting our request through to the static file handler. Add some logic As at the time of writing, SkiaSharp seems to have a few issues. Notably, when loading an indexed 8-bit image (such as an optimized PNG), you encounter a few issues with resizing.

To overcome this, we need to write our own image loader that converts it to 32-bit. Here’s the image loader:

private SKBitmap LoadBitmap(Stream stream, out SKCodecOrigin origin)
{
    using (var s = new SKManagedStream(stream))
    {
        using (var codec = SKCodec.Create(s))
        {
            origin = codec.Origin;
            var info = codec.Info;
            var bitmap = new SKBitmap(info.Width, info.Height, SKImageInfo.PlatformColorType, info.IsOpaque ? SKAlphaType.Opaque : SKAlphaType.Premul);

            IntPtr length;
            var result = codec.GetPixels(bitmap.Info, bitmap.GetPixels(out length));
            if (result == SKCodecResult.Success || result == SKCodecResult.IncompleteInput)
            {
                return bitmap;
            }
            else
            {
                throw new ArgumentException("Unable to load bitmap from provided data");
            }
        }
    }
}

Cropping One of the resize parameters “mode”, can be used to crop to fit. We’ll need a helper method to crop our image.

private SKBitmap Crop(SKBitmap original, ResizeParams resizeParams)
{
    var cropSides = 0;
    var cropTopBottom = 0;

    // calculate amount of pixels to remove from sides and top/bottom
    if ((float)resizeParams.w / original.Width < resizeParams.h / original.Height) // crop sides
        cropSides = original.Width - (int)Math.Round((float)original.Height / resizeParams.h * resizeParams.w);
    else
        cropTopBottom = original.Height - (int)Math.Round((float)original.Width / resizeParams.w * resizeParams.h);

    // setup crop rect
    var cropRect = new SKRectI
    {
        Left = cropSides / 2,
        Top = cropTopBottom / 2,
        Right = original.Width - cropSides + cropSides / 2,
        Bottom = original.Height - cropTopBottom + cropTopBottom / 2
    };

    // crop
    SKBitmap bitmap = new SKBitmap(cropRect.Width, cropRect.Height);
    original.ExtractSubset(bitmap, cropRect);
    original.Dispose();

    return bitmap;
}

Padding Optionally, we’ll want to “pad” our image instead.

private SKBitmap Pad(SKBitmap original, int paddedWidth, int paddedHeight, bool isOpaque)
{
    // setup new bitmap and optionally clear
    var bitmap = new SKBitmap(paddedWidth, paddedHeight, isOpaque);
    var canvas = new SKCanvas(bitmap);
    if (isOpaque)
        canvas.Clear(new SKColor(255, 255, 255)); // we could make this color a resizeParam
    else
        canvas.Clear(SKColor.Empty);

    // find co-ords to draw original at
    var left = original.Width < paddedWidth ? (paddedWidth - original.Width) / 2 : 0;
    var top = original.Height < paddedHeight ? (paddedHeight - original.Height) / 2 : 0;

    var drawRect = new SKRectI
    {
        Left = left,
        Top = top,
        Right = original.Width + left,
        Bottom = original.Height + top
    };

    // draw original onto padded version
    canvas.DrawBitmap(original, drawRect);
    canvas.Flush();

    canvas.Dispose();
    original.Dispose();

    return bitmap;
}

Dealing with EXIF Rotation In SkiaSharp, the EXIF rotation comes through as an “Origin” value. You don’t generally need to deal with these for images that you produce, but you need to deal with it for images that others might be uploading because modern phones store their image rotations this way.

There are 8 different orientations, represented in SkiaSharp as an Origin value:

Not only can the image be rotated, but also flipped, or some combination of the 2.

SkiaSharp provides a canvas for us and 2D matrix transformations that would be ideally suited to the task of fixing the rotation, but after some experimenting with less than desirable outcomes, I decided simply to copy the pixels. The code for rotate and flip:

private SKBitmap RotateAndFlip(SKBitmap original, SKCodecOrigin origin)
{
    // these are the origins that represent a 90 degree turn in some fashion
    var differentOrientations = new SKCodecOrigin[]
    {
        SKCodecOrigin.LeftBottom,
        SKCodecOrigin.LeftTop,
        SKCodecOrigin.RightBottom,
        SKCodecOrigin.RightTop
    };

    // check if we need to turn the image
    bool isDifferentOrientation = differentOrientations.Any(o => o == origin);

    // define new width/height
    var width = isDifferentOrientation ? original.Height : original.Width;
    var height = isDifferentOrientation ? original.Width : original.Height;

    var bitmap = new SKBitmap(width, height, original.AlphaType == SKAlphaType.Opaque);

    // todo: the stuff in this switch statement should be rewritten to use pointers
    switch(origin)
    {
        case SKCodecOrigin.LeftBottom:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(y, original.Width - 1 - x, original.GetPixel(x, y));
            break;

        case SKCodecOrigin.RightTop:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(original.Height - 1 - y, x, original.GetPixel(x, y));
            break;

        case SKCodecOrigin.RightBottom:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(original.Height - 1 - y, original.Width - 1 - x, original.GetPixel(x, y));

            break;

        case SKCodecOrigin.LeftTop:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(y, x, original.GetPixel(x, y));
            break;

        case SKCodecOrigin.BottomLeft:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(x, original.Height - 1 - y, original.GetPixel(x, y));
            break;

        case SKCodecOrigin.BottomRight:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(original.Width - 1 - x, original.Height - 1 - y, original.GetPixel(x, y));
            break;

        case SKCodecOrigin.TopRight:

            for (var x = 0; x < original.Width; x++)
                for (var y = 0; y < original.Height; y++)
                    bitmap.SetPixel(original.Width - 1 - x, y, original.GetPixel(x, y));
            break;

    }

    original.Dispose();

    return bitmap;
}

That’s it for helper functions, almost. Load and resize our image Now we simply need to get our image data from disk (or cache), and apply our resize params to it. For this example, I’m loading from disk, and using an IMemoryCache, but I’d recommend abstracting the file loading and caching implementations to allow for a variety of sources (such as loading from S3, a remote URL, or caching in an IDistributedCache or on some file store.) Get image data

private SKData GetImageData(string imagePath, ResizeParams resizeParams, DateTime lastWriteTimeUtc)
{
    // check cache and return if cached
    long cacheKey;
    unchecked
    {
        cacheKey = imagePath.GetHashCode() + lastWriteTimeUtc.ToBinary() + resizeParams.ToString().GetHashCode();
    }

    SKData imageData;
    byte[] imageBytes;
    bool isCached = _memoryCache.TryGetValue<byte[]>(cacheKey, out imageBytes);
    if (isCached)
    {
        _logger.LogInformation("Serving from cache");
        return SKData.CreateCopy(imageBytes);
    }

    SKCodecOrigin origin; // this represents the EXIF orientation
    var bitmap = LoadBitmap(File.OpenRead(imagePath), out origin); // always load as 32bit (to overcome issues with indexed color)

    // if autorotate = true, and origin isn't correct for the rotation, rotate it
    if(resizeParams.autorotate && origin != SKCodecOrigin.TopLeft)
        bitmap = RotateAndFlip(bitmap, origin);

    // if either w or h is 0, set it based on ratio of original image
    if (resizeParams.h == 0)
        resizeParams.h = (int)Math.Round(bitmap.Height * (float)resizeParams.w / bitmap.Width);
    else if (resizeParams.w == 0)
        resizeParams.w = (int)Math.Round(bitmap.Width * (float)resizeParams.h / bitmap.Height);

    // if we need to crop, crop the original before resizing
    if (resizeParams.mode == "crop")
        bitmap = Crop(bitmap, resizeParams);

    // store padded height and width
    var paddedHeight = resizeParams.h;
    var paddedWidth = resizeParams.w;

    // if we need to pad, or max, set the height or width according to ratio
    if (resizeParams.mode == "pad" || resizeParams.mode == "max")
    {
        var bitmapRatio = (float)bitmap.Width / bitmap.Height;
        var resizeRatio = (float)resizeParams.w / resizeParams.h;

        if (bitmapRatio > resizeRatio) // original is more "landscape"
            resizeParams.h = (int)Math.Round(bitmap.Height * ((float)resizeParams.w / bitmap.Width));
        else
            resizeParams.w = (int)Math.Round(bitmap.Width * ((float)resizeParams.h / bitmap.Height));
    }

    // resize
    var resizedImageInfo = new SKImageInfo(resizeParams.w, resizeParams.h, SKImageInfo.PlatformColorType, bitmap.AlphaType);
    var resizedBitmap = bitmap.Resize(resizedImageInfo, SKBitmapResizeMethod.Lanczos3);

    // optionally pad
    if (resizeParams.mode == "pad")
        resizedBitmap = Pad(resizedBitmap, paddedWidth, paddedHeight, resizeParams.format != "png");

    // encode
    var resizedImage = SKImage.FromBitmap(resizedBitmap);
    var encodeFormat = resizeParams.format == "png" ? SKEncodedImageFormat.Png : SKEncodedImageFormat.Jpeg;
    imageData = resizedImage.Encode(encodeFormat, resizeParams.quality);

    // cache the result
    _memoryCache.Set<byte[]>(cacheKey, imageData.ToArray());

    // cleanup
    resizedImage.Dispose();
    bitmap.Dispose();
    resizedBitmap.Dispose();

    return imageData;
}

Should really have some exception handling, it could be a bit leaky.

For the cache key, I’m using a long based on the image last write time, the resize params, and the image path. The “unchecked” statement is to allow our cache key to overflow safely. Instead of throwing an exception, it just “rolls around” to the other extremity.

We just check the cache, return that, else, we load our bitmap, resize it, optionally rotate it using the EXIF header, crop, pad, or “max” (which is best fit), encode it in our format of choice, cache that, then return that data.

Wire it all up

Lastly, we need to call our GetImageData method from the middleware’s Invoke method. I’ve also added a “404” check. LastWriteTimeUtc returns midnight, January 1, 1601 if the file isn’t present. Inside Invoke:

public async Task Invoke(HttpContext context)
{
    var path = context.Request.Path;

    // hand to next middleware if we are not dealing with an image
    if (context.Request.Query.Count == 0 || !IsImagePath(path))
    {
        await _next.Invoke(context);
        return;
    }

    // hand to next middleware if we are dealing with an image but it doesn't have any usable resize querystring params
    var resizeParams = GetResizeParams(path, context.Request.Query);
    if (!resizeParams.hasParams || (resizeParams.w == 0 && resizeParams.h == 0))
    {
        await _next.Invoke(context);
        return;
    }

    // if we got this far, resize it
    _logger.LogInformation($"Resizing {path.Value} with params {resizeParams}");

    // get the image location on disk
    var imagePath = Path.Combine(
        _env.WebRootPath,
        path.Value.Replace('/', Path.DirectorySeparatorChar).TrimStart(Path.DirectorySeparatorChar));

    // check file lastwrite
    var lastWriteTimeUtc = File.GetLastWriteTimeUtc(imagePath);
    if(lastWriteTimeUtc.Year == 1601) // file doesn't exist, pass to next middleware
    {
        await _next.Invoke(context);
        return;
    }

    var imageData = GetImageData(imagePath, resizeParams, lastWriteTimeUtc);

    // write to stream
    context.Response.ContentType = resizeParams.format == "png" ? "image/png" : "image/jpeg";
    context.Response.ContentLength = imageData.Size;
    await context.Response.Body.WriteAsync(imageData.ToArray(), 0, (int)imageData.Size);

    // cleanup
    imageData.Dispose();

}

Test it

A quick test with padding, an EXIF rotation and a format change:

And another one, without the EXIF rotation, cropped to fit a square:

In Summary

Given that options on .net core at the time of writing are limited, I feel like this solution is as good as any, particularly once it’s got a bit more meat on it’s bones focused on performance and extensibility.

The source code is available on bitbucket to do with as you wish. Happy coding!