Integration Testing Set Up

At the very outset much of this was learned through trial and error and with the video help of people like Steve Smith, Nick Chapsas, Julie Lerman and others.  Useful links to some of their work is at the bottom of this post.

I'm writing this up because someone thought others could find it useful.  

There are going to be easier/better/smarter ways to do things, but this is eventually what worked for me.  I'm sure there will be plenty of groaning and "I can't believe he did that" utterings.  Your mileage will definitely vary.

My Integration Testing Approach

Environment:

  • .net Core 6
  • EF Core 6
  • Windows 11
  • Docker Desktop 10
  • MS SQL Server
  • CosmosDb
  • Azure B2C
  • xUnit
  • Visual Studio 2022

The app itself has lots of complex relationships and I wanted an actual integration testing scenario: 35 users, a dozen companies, even more locations, etc.  Setting them up individually for each test was a huge pain.  I needed a way to create a set of data and do it consistently.

xUnit will run all tests in a given class in sequence, but tests grouped in multiple classes will run simultaneously - a problem if you want the same data for each block of tests each time.  Steve Smith suggested separating "read" tests from "write" tests to make it easier and I wish I'd thought of that.

Instead, I decided to group my tests into test classes in the order that I was working on them: CRUD tests first, then validation tests, and so on.  So all User crud tests would go into a User_Crud test class and all validation tests would go into a User_Validation test class.

Each testing class would have its own data to work with.

LocalDb

For a while I ran them against localDb in Visual Studio 2022, but the clean up became tedious.  I decided I needed to create a complete SQL database at the start of each test class, but wanted to run multiple test classes at the same time.  Plus, I didn't want to be bothered to find all those databases in my localDb and get rid of them.  So, Docker.

TestContainers

I tried a package called TestContainers, which allows you, in C#, to configure and launch a container from within your test and then tear it down automatically when the test ends.  Sounded sexy, but when I finally did get it to work, it took longer to launch the container and create a database that it did to have a standing SQL container that ran all the time and just delete/recreate the database inside that server each time.  When I need to clean up everything, I just deleted the container and created a new one.  I did that with a menu entry on Windows Terminal.  

Like this:

cmd.exe /c docker run -e "ACCEPT_EULA=YES" -e "SA_PASSWORD=Please2022!" -p 1433:1433 --name IntegrationTestSQL -d mcr.microsoft.com/mssql/server:2017-CU28-ubuntu-16.04

There were also a few issues with the current version of Docker and which version of MS SQL Server would actually work.  2017 worked for me.  I eventually got it, but I'm a big fan of stable and simple, so nope.

Solution

Using the Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory, we need to create something that will take the app, removed the normally configured dbContexts and replace them with ones configured for the test.  

To test web applications, you would have a class that would inherit from WebApplicationFactory, like this:

public class UsersApiFactory : WebApplicationFactory<IApiAssemblyMarker>, IAsyncLifetime

And you would need to override:

  • ConfigureWebHost
  • InitializeAsync

as shown later.

Remember, at this stage we are modifying the server, not the test.  So when the test runs, nothing is different than if it were against a production server, except some differently configured dbContexts.  That's the theory, anyway.

What worked for me was:

  1. Use the standing Docker MS SQL Server container to host SQL Server (you can get a connection string from there and it'll be the same for all your test classes).
  2. Create a database named after the name of the test class - this will be unique for each test class, so no conflicts with other tests running at the same time.
  3. Delete the database by that name and re-seed it - now we have a standard set of complex data for each test, whether we need all of it or not.
  4. Clear out the old dbContext configurations and put in the ones for the tests.
  5. Run all the tests in that class.

Code

In the WebApplicationFactory you inherit from, there are two interesting methods that I found (probably more - I'm lazy and have ADD):

  1. InitializeAsync which runs first - allowing you to do stuff like create/recreate and populate a database
  2. ConfigureWebHost which runs at the start of the test, where you can reconfigure the web host that you're testing.

But here's the catch: you can't change the signature on either one of these methods, so you can't use dependency injection for ScopeFactories and other handy things.  This is a little limiting, but what can you do?

Here's my initialization code:

  public async Task InitializeAsync()
  {
    try
    {
      await using var dbContext = ApiFactoryHelpers.CreateSeedDataDbContext(nameof(InvitationApiFactory));
      var seedFactory = new SeedDataFactory(dbContext);
      _usersDictionary = await seedFactory.CreateAllUsers();
      _accountsDictionary = await seedFactory.CreateEntries();
      await dbContext.DisposeAsync();
      await ApiFactoryHelpers.DeleteDocumentDatabase(nameof(InvitationApiFactory));
    }
    catch (Exception e)
    {
      Log.Error(e.Message);
    }

I'm using a helper method to get a dbContext that I can use to insert my data.  I also delete the document database(s) to clean those up.

  public static AuthorizationDbContext CreateSeedDataDbContext(string databaseName, bool useLocalHost = false)
  {
    var connection = SetDatabaseNameInConnectionString(databaseName, useLocalHost);

    var builder = new DbContextOptionsBuilder<AuthorizationDbContext>();

    builder.UseSqlServer(connection);

    var context = new AuthorizationDbContext(builder.Options, null);
    context.Database.EnsureDeleted();
    context.Database.EnsureCreated();
    context.Database.Migrate();
    return context;
  }

And here's the configuration code:

  protected override void ConfigureWebHost(IWebHostBuilder builder)
  {
    base.ConfigureWebHost(builder);
    ApiFactoryHelpers.ResetDbContexts(builder, nameof(InvitationApiFactory));
    ApiFactoryHelpers.ResetDocumentDb<Invitation>(builder, nameof(InvitationApiFactory), nameof(Invitation), "/accountId");
  }

And here are the internals to those helper methods - starting with resetting the EF Core dbContext:

  public static void ResetDbContexts(IWebHostBuilder builder, string databaseName, bool useLocalHost = false)
  {
    builder.ConfigureTestServices(services =>
    {
      var cs = GetConnectionString(useLocalHost);

      var connection = SetDatabaseNameInConnectionString(databaseName, useLocalHost);
      var documentDatabaseName = Constants.TestDbName;
      var documentConnection = Constants.TestDocumentDbConnectionString;

      services.RemoveAll(typeof(DbContext));

      services.RemoveAll(typeof(TenantDbContext));
      services.RemoveAll(typeof(AuthorizationDbContext));

      services.RemoveAll(typeof(DbContextOptions<TenantDbContext>));
      services.RemoveAll(typeof(DbContextOptions<AuthorizationDbContext>));

      services.AddDbContext<AuthorizationDbContext>(options =>
      {
        options.UseSqlServer(connection);
      });

      services.AddDbContext<TenantDbContext>(options =>
      {
        options.UseSqlServer(connection);
      });
    });
  }

And then resetting the CosmosDb:

  public static void ResetDocumentDb<T>(IWebHostBuilder builder, string databaseName, string container, string partitionKey) where T : class, IItem
  {
    builder.ConfigureTestServices(services =>
    {
      services.RemoveAll(typeof(CosmosRepository<>));
      services.RemoveAll(typeof(IRepository<>));

      services.AddCosmosRepository(options =>
      {
        options.CosmosConnectionString = Constants.TestDocumentDbConnectionString;
        options.DatabaseId = databaseName;
        options.ContainerPerItemType = true;
        options.AllowBulkExecution = true;
        options.SerializationOptions = new RepositorySerializationOptions
        {
          Indented = true,
          PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase,
          IgnoreNullValues = false,
        };
        options.OptimizeBandwidth = false;
        options.ContainerBuilder.Configure<T>(options =>
        {
          options.WithContainer(container);
          options.WithPartitionKey(partitionKey);
        });
      });
    });
  }

A couple of small details:

  1. I created a wrapper around the CosmosRepository package because I don't like exceptions coming back from my data stores.  I trap all the errors in that layer and then bubble up a result that's more meaningful.  Yes, this means checking for success each time, but it also means I maintain flow control, pass multiple error messages and content to the spot I need them, etc.
  2. These test runs are slower than some people like.  But at the end of the tests, each database is still standing and it gives me a chance to see what, exactly, I managed to screw up.  It's been super helpful.
  3. Those SeedFactory methods sprinkled around are a bunch of static methods to create standardized data - nothing special there.  I'm using Guid values for Ids, so it's consistent.

Tests

In the code below, you can see I expose some things from the ApiFactory that are helpful in my tests:

private readonly InvitationApiFactory _apiFactory;
private readonly HttpClient _httpClient;
private readonly Account _account;
private ISeedAccountBase _defaultAccount = new GingerGaucho();
private readonly ITestOutputHelper _output;

  public InvitationsCrud(InvitationApiFactory apiFactory, ITestOutputHelper output)
  {
    _apiFactory = apiFactory;
    _output = output;
    _httpClient = _apiFactory.AuthenticatedClient(_defaultAccount.Owner.Email);
    _account = _apiFactory.GetAccount(_defaultAccount.GetType().Name);
  }

All of my calls are authenticated against Azure B2C so I create a list of users with a Resource Owner Password Credentials login which I use to get an actual live, B2C access token.

I jam that access token into the _httpClient and all my calls are authenticated.  In the event I need another authenticated user for some test, the _apiFactory contains a dictionary of the accounts and users and I can always spin up another httpClient with a different auth token in it.

I hope that helps someone.

DevBetter a great site for coaching and other help.  

DDD course - A great Pluralsight course making sense of many architectural aspects

EF Core 6 Course - Julie's videos on EF Core are simply the best

Nick Chapsas Testing Video - Test Containers explained - in fact, subscribe to everything this guy does.  I learn something every time, without fail.

TestContainers - See above.

CosmosRepository - A wrapper for accessing CosmosDb that really has a lot going for it.

Julie Lerman - Her website links to countless articles on EF Core, etc.

Steve Smith - Steve Smith is one of the best.  Practical, useful stuff.  Get his newsletter, pay attention.