Unit Testing

MongoDb C# Driver 2 - A Nice(ish) Abstraction by James Heywood

I've been doing a bit of work in the evenings of late to put something together that better demonstrates some of my back end development as I realised there was no example data access code in my bitbucket repos. 

So I've resurrected part of an old project, the project was an on-line car owners portal for Porsche, in fact some previous posts on this blog relate to this, I had to remove the name Porsche from those articles (and screen grabs) under threat from a previous employer, but I feel enough time has passed now that it's ok to mention their name, if not well we'll see what happens!

You can see my efforts in the following repository; https://github.com/jdheywood/4um - a rough estimate of the time I've spent on this is somewhere between 10 and 15 hours over the course of a week.


Anyway the part I have chosen to rewrite to show what I can do is a mini forum, where users can ask questions and other users can reply. There is a search function and the ability to favourite questions and answers so users can revisit these as and when needed. 

When we built this for the client it was a single .NET MVC application using forms authentication, knockout.js to enhance the user interface and MongoDb for the data storage. I've decided to rewrite this as an API and an SPA so that I can better separate the data from the interface, my plan is to recreate the knockout interface once the API is ready and then after that have a go at writing interfaces using a variety of JS frameworks to contrast and compare these approaches.

The user interface (when ready) will consist of two forms, one for users and another for admins (or maybe one form that adapts based on the user's role?), with a JS framework using ajax to call the API for data operations. A stretch goal would be to add OAuth support to ease registration and access, but that's way down the line.



For now I've been concentrating on getting the back end up and running, I thought this would be relatively quick to do but Mongo have rewritten their C# driver completely since I last looked at it so I've ended up re-writing this part completely. Also when I built this a couple of years ago we had very little test coverage so I've tried to address this shortcoming.

So what I have so far is a nice(ish) abstraction over the Mongo C# drivers (if I do say so myself!) and the beginnings of an API that uses the repositories I've created to access the MongoDb collections.

Yes I know repositories are so 2000s and in the 20 teens it's all about CQRS, but hey I can always refactor these at a later date, I wanted to concentrate on getting the mongo abstraction in place first before I changed too much.

I have written unit tests for the various factories that abstract the MongoDb drivers and integration tests to demonstrate and prove the repositories functionality. 

I've also created a simple console application to load sample data to mongo, as the integration tests set up and tear down the data they use when they finish you have no data, hence the need for loading sample data to demonstrate the API.

If you have access to a Windows machine and Visual Studio 2013 you can pull down the repo, build the solution and see this running. You'll need to install MongoDb first though, instructions/links can be found in the readme of my github repo.

If you run the integration tests you can see the repositories in action, similarly if you run the Forum.Tools.Samples console application then fire up the API and hit one of the endpoints; /api/questions, /api/answers, /api/searchterms you can see the API returning data from MongoDb.


Good, Bad, Better

Some things I like about this
- The abstraction over MongoDb via MongoContext, MongoClientFactory, MongoDatabaseFactory and Generic MongoCollectionFactory
- The use of SimpleInjector for IOC/Dependency Injection, especially the ability to verify the container
- The use of MongoContext in the API Startup as a single entry point to set up the data access
- The integration tests covering the repositories

Some things I would change given time;
- Refactor the repositories into Commands and Queries to allow easier testing
- Introduction of API data transfer objects and mapping between domain objects and these DTOs
- Swap out the use of an installed MongoDb for an embedded or hosted version
- Use AutoFixture customisations to build test and sample data instead of semi-hand rolling these objects

The next things for me to do to get on with this project;
- Define the API routes and build out the API layer
- Change the API to return JSON instead of XML
- Unit and Integration tests of the API
- Install swagger to self-document the API; https://github.com/swagger-api/swagger-ui 
- Set up a simple web application
- Markup and styling for the web app
- Introduction of knockout to provide UI functionality

So quite a few things to do before this is a functioning application, however the data access layer is well defined, covered by tests and helps to demonstrate the kinds of applications I've built, I hope you like it.



One final point to note is that I had trouble unit testing my repositories due to issues substituting the MongoCollection objects that the repository methods use, I have used integration test to add test coverage because of this issue. Even though we need integration tests anyway to actually test data storage and access we should be able to write unit tests for our repositories which run quickly and are less expensive in terms of maintenance, deployment and execution, so how do we do this?

The issue stems from the collection objects and the fluent API that the MongoDb C# driver provides, when trying to set the .Returns(object) for the chained fluent API methods off a collection NSubstitute throws errors as it doesn't know which method you are trying to set the .Returns() on.

After some googling on this and how best to resolve what I've determined is that this is a problem with my code and abstraction, rather than NSubstitute (way to state the bleeding obvious eh?). The way to resolve this is to introduce another layer to the abstraction, something along the lines of an IForumMongoCollection, and have this implement methods of IMongoCollection, this way my MongoCollectionFactory can return my ForumMongoCollections and I can mock/substitute these in my unit tests.

So a simple idea when you step back and think it through, it alluded me at the time I was writing this code though, but that's what you get for coding late into the night. So that's now top of my to do list, once that's in place I can start building out the application properly.

Anyway enough blogging for now folks, hope you like this article and the repository, as ever throw comments and questions at me below if you are so inclined.



NCrunch Project Build Issues by James Heywood

Hello all, it's been a while since the last post, mainly because I couldn't think of what to post about. I have a few bits and pieces on the go but nothing complete enough to warrant a post, so instead I did nothing! However I had a thought the other day, the next time I find a fix to a problem I'm having with something technical at work I can post about that, as it's a good way of documenting a fix for others who may be having the same issue and it also means I have something to say!

So today I have been setting up NCrunch, to help monitor test coverage of the project I'm currently working on. If you're not aware of NCrunch head on over to their site as it's a great piece of software to assist your TDD. It takes away the pain of compilation/build and let's you focus on the more productive parts of TDD, like writing tests and meeting them in the code/system under test, i.e. the bit we want to be doing as developers!

Anyway, I had fun setting up the projects initially as NCrunch just wouldn't load one of my unit test projects, the error message received was the old classic "could not load file or assembly", specifically referring to Newtonsoft.Json.dll in my case. After a spot of reading of the NCrunch documentation site I came across this page suggesting that the project build issue I was experiencing was due to implicit file/build dependencies.

Briefly, NCrunch works asynchronously testing your code as you write it, which is great, in order to do this though it has to copy the related assemblies and resources needed by the code you are testing to a working folder for it's own compilation and execution and it is this that was causing my issue.

There are several suggested fixes, however the least intrusive in my opinion was to use the Additional files to include configuration setting, this seemed a safer option than hacking the .proj file of the test assembly.

To do this I went to the NCrunch menu in Visual Studio, selected Configuration, then selected my troublesome project. In the General section of the project configuration I then clicked the small ellipsis button in the Additional files to include setting and selected the offending DLL (and related XML file) for Newtonsoft.Json and then clicked OK.

NCrunch then displayed a warning; "NCrunch detected unusual behaviour around the resolution of referenced assemblies required by this project in its test environment...." and so on, which I hastily ignored. You can view this and any other warnings through the NCrunch Test window at any time, so don't be worried about skipping over a warning like this if you are following this yourself.

After this all I then needed to do was to disable and re-enable NCrunch from the NCrunch menu in Visual Studio et voila I am now crunching tests, sweet!

So there you have it, hopefully that is of some use to someone out there, if so let me know.