Knockout.js, .NET MVC & MongoDb / by James Heywood

So this is a little overview of a project I worked on a while ago utilising these three technologies. The idea of this post is to show how a request/response cycle is handled from the initial request of a page by the browser, through the server code, to the data store and back again.

The stack is an ASP.NET MVC 3 web application, using the Razor view engine, Knockout.js for client side UI functionality and MongoDb for data storage. The web application makes use of a Windsor container to provide IOC, server side logic is separated using a service and repository pattern and constructor dependency injection is used throughout.

The application is hosted on a Windows 2008 R2 Server provided by Rackspace, and we have a staging server of the same spec which we can bring up and take down as and when needed to save costs, we do this by snapshotting an image of the staging box and backing this up to storage, again provided by Rackspace.

I have discussed the application in a previous post here but from the point of view of MongoDb schema design, in summary it is a responsive web application for Porsche owners and the area of the application that we will discuss in this post is the Knowledge Base. 

As a note when I first published this article my previous employer rang me up to complain about my use of the name Porsche (as if somehow mentioning this was a sin?!) so I amended the article at their request, with hindsight I feel that I should have just stuck to my guns, after all I did the work and have a right to discuss it, so long as I don't give away trade secrets, anyway rant over!

Knowledge Base

This is an area where owners can access useful information about their cars as well as ask, search and bookmark answers to questions asked either by themselves or other owners. There is an administrative interface to the knowledge base so that Car staff can respond to and manage questions, however we are only concerned with the user or owner side of the knowledge base for now, and in particular the landing page for this area that shows recently answered questions as well as controls to search for answers and ask a question.

Initial Request

To enter the knowledge base area from the site navigation the user requests the page /knowledgebase/findanswers this request is routed to the FindAnswers method of the KnowledgeBaseController by the "Default" routing rule shown below, this will be familiar to you if you have worked with ASP.NET MVC as it is the out-of-the-box routing rule set up for you in the project template.

public class RouteConfig
{
    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.MapRoute(
            name: "Default",
            url: "{controller}/{action}/{id}",
            defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional },
            namespaces: new string[] { "CarPlus.Web.Controllers" }
        );

    }
}

As you can see we have added our own defaults and namespaces properties to the route to catch requests for controllers and/or actions that we do not support, in this instance the request would be routed to HomeController.Index() and handled there appropriately.

The Controller

Back to our example, if we take a quick look at a snippet of our KnowledgeBaseController we can see that the Index() method simply redirects to the FindAnswers() method.

public class KnowledgeBaseController : JsonController
{
    private readonly IKnowledgeBaseService _knowledgeBaseService;

    public KnowledgeBaseController(IOwnerService ownerService,
                                                          IKnowledgeBaseService knowledgeBaseService)
        : base(ownerService)
    {
        _knowledgeBaseService = knowledgeBaseService;
    }

    public ActionResult Index()
    {
        return RedirectToAction("findanswers");
    }

    public ActionResult FindAnswers()
    {
        return View();
    }
}

To digress briefly you can see that our KnowledgeBaseController inherits from JsonController, this is the parent controller for all of the owner accessible areas of the site and it provides a method to return Json encoded data, which is used by the various AJAX requests that are made by our Knockout.js ViewModels, but more on that later. Our JsonController is shown below for reference. 

[NoCache]
public abstract class JsonController : AuthController
{
    public JsonController(IOwnerService ownerService)
        : base(ownerService)
    { }

    protected override JsonResult Json(object data, 
                                                              string contentType, 
                                                              System.Text.Encoding contentEncoding, 
                                                              JsonRequestBehavior behavior)
    {
        return new JsonNetResult
        {
            Data = data,
            ContentType = contentType,
            ContentEncoding = contentEncoding,
            JsonRequestBehavior = behavior
        };
    }
}

As you can see this in turn inherits from AuthController and passes a reference to the OwnerService to this so that this can manage user and session data, but I'll detail more on that in another post some time. 

Getting back to the point, as we can see in the KnowledgeBaseController code above, the FindAnswers() method simply returns the relevant view, by convention this is the view in our project /Views/KnowledgeBase/FindAnswers.cshtml

The View

The key parts of the view FindAnswers.cshtml are shown below. I have edited this code for clarity to just show the parts relevant to our use of knockout, for example I have removed any reference to classes and some of the non-semantic markup needed for layout and styling purposes. What is left should just be what I need to explain to you how the knockout bindings work.

FYI the views are written using the Razor view engine that is part of ASP.NET MVC, again if you have used ASP.NET MVC you should be familiar with this, although having said that there is precious little we are using from this engine in the sample code I show below, bonus points if you spot the Razor syntax!

<div id="knowledge-base-wrapper">

    <section id="searchResults">

        <div data-bind="visible: showFindAnswers">
            
            <h1>Knowledge Base</h1>
            
            <div id="search-results">

                <h2>Recent questions</h2>
                <p data-bind="visible: recentAnswers().length === 0">

                There are currently no recently answered questions.</p>

                <ul data-bind="foreach: recentAnswers">
                    <li data-bind="visible: answers().length > 0">
                        <h3 data-bind="text: questionText"></h3>
                        <ul>
                            <li>
                                <div data-bind="foreach: answers">
                                <span data-bind="text: dateTime"></span>
                                <a href="#" data-bind="click: $root.createBookmark.bind

                                ($root, $data), css: { selected:  isBookmarked}">

                                Bookmark this answer

                                </a>
                                <h4>Answer</h4>
                                <p data-bind="text: answerTextSummary"></p>
                                <button data-bind="click: $root.selectAnswer.bind

                                ($root, $parent, $data)">

                                Read More

                                </button>
                            </li>
                        </ul>
                    </li>
                </ul>
            </div>
        
        </div>
        
    </section>
    
</div>


@section scripts {
    <script type="text/javascript">
        $(document).ready(function () {
            ko.applyBindingsWithValidation

            (new window.CarPlus.ViewModels.KnowledgeBaseViewModel

                     ("@ViewBag.URLSiteRoot", 'findAnswers'),

                     document.getElementById("knowledge-base-wrapper")

            );
        });
    </script>
}

There is more in this view, for example we have a search function as well as the ability to submit a new question to the knowledge base, however for our purposes we are going to focus on how the recently answered questions are loaded into the DOM by knockout post page load.

As you can see at the bottom of the view code is the scripts section (Razor syntax, plus one if you spotted it). I have removed the references to our scripts for clarity so we can concentrate on the important part which is where we bind our knockout view model to the DOM. As you can see the ko.applyBindingsWithValidation call does this, and contains another bit of Razor syntax which is to pass in the site root or domain to the ViewModel constructor from a ViewBag property, (plus two if you spotted both). 

The ViewModel

Below is a snippet of our ViewModel, with just the relevant parts to this article shown to keep this easier to follow;

window.CarPlus.ViewModels.KnowledgeBaseViewModel = function (siteRoot, dataContext) {
    var self = this;
    self.siteRoot = siteRoot;

    // Observables

    self.recentAnswers = ko.observableArray([]);
    self.showFindAnswers = ko.observable(true);
    self.currentSection("knowledge-base");

    self.search = ko.observable().extend({ required: { message: 'Please enter a search term.'} });

    // Load view model data.
    if (dataContext == 'findAnswers') {

        self.getRecentAnswers();
    }
    if (dataContext == 'getMyQuestions') {
        self.getMyQuestions(true);
    }
    if (dataContext == 'getMyBookmarks') {
        self.getMyBookmarks();
    }
    // Set the ko validation group if necessary.
    if (ko.validation) {
        self.Errors = ko.validation.group(self);
    }
};

As you can see the constructor of the VieWModel takes two parameters, the first is the siteRoot which is the environment specific URL of the site domain, this allows us to alter the domain for the various AJAX calls that the client side code makes depending on whether we are in development, staging or live environments. The second parameter is the dataContext, the context in which we are using the ViewModel.

This ViewModel is used on three different pages of the application, this parameter allows the ViewModel to load data and model behaviour relevant to the page the user is viewing. For our purposes we are concentrating on the FindAnswers page so our dataContext is 'findAnswers'. 

Just a quick thought on this, it may be tidier to split the ViewModel into three, one for each page, rather than have a single ViewModel tailor its data and behaviour for multiple pages. This would be an ideal candidate for refactoring if we had the time to revisit this code. As it stands we have a large and complex ViewModel in use across multiple pages, which although it keeps code in one place, makes it more difficult to manage in the long term as changes can affect multiple pages so require greater testing.

Namespacing

Back to our explanation, as our dataContext is 'findAnswers' we can see that the first thing our ViewModel does after initialising various observables/observableArrays is to call 'getRecentAnswers'. This function is declared against a namespace that we have added to the window object, in the same way that the ViewModel itself is namespaced. Our namespaces are managed in a javascript file imaginatively called CreateNamespaces.js which ensures that the various namespaces we use exist, a brief snippet of this is shown below relating to our case study here; 

window.CarPlus = window.CarPlus || {};
window.CarPlus.Utilities = window.CarPlus.Utilities || {};
window.CarPlus.ViewModels = window.CarPlus.ViewModels || {};
window.CarPlus.KnowledgeBase = window.CarPlus.KnowledgeBase || {};
window.CarPlus.KnowledgeBase.Models = window.CarPlus.KnowledgeBase.Models || {};
window.CarPlus.KnowledgeBase.Constants = window.CarPlus.KnowledgeBase.Constants || {};
window.CarPlus.KnowledgeBase.DataServices = window.CarPlus.KnowledgeBase.DataServices || {};

Function and Promise

The ViewModel defines the function getRecentAnswers, which can be seen below; 

window.CarPlus.ViewModels.KnowledgeBaseViewModel.prototype.getRecentAnswers = function () {
    var self = this;
    self.recentAnswers([]);
    var promise = window.CarPlus.KnowledgeBase.DataServices.getRecentAnswers

    (this.siteRoot, self.recentAnswers, self.answerTextSummaryLength);
    promise.done(function (data) {
    });
    promise.fail(function () {
        toastr.error('Sorry, there was a problem getting recent answers.');
    });
    promise.always(function () {
    });
};

This uses jQuery promise() to call into the knowledge base data service, which requests the most recently answered questions from the server. The observable array self.recentAnswers is passed into this function and is populated by the function when the promise is resolved. As this observable array is bound to the DOM once it is populated the DOM element(s) bound to the contents of this array are updated and the recently answered questions are displayed for the user of the application.

If there is a failure, that is if the promise is not resolved by the data service, we use a javascript library called toastr to inform the user. I highly recommend toastr, its a tiny library, easy to use and a nice user experience, also the demo uses a quote from The Princess Bride so the author has great taste in movies. We planned to use it only in development but the client really liked the feedback toasts so it ended up in the finished application.

DataService and Deferred

The data service getRecentAnswers function is shown below for reference. The data service functions are responsible for resolving (or not if something goes wrong) the promises made by the view model functions, so we can see the use of jquery Deferred() here to keep or break the promises.

This also makes use of our constants, which are defined in a separate javascript file, again imaginatively called 'Constants.js'. This basically holds the controller names for the various parts of the application so as to allow the data services to build up the urls they need to call to get and post data.

getRecentAnswers = function (url, recentAnswersObseravble) {
    var deferred = $.Deferred();
    window.CarPlus.DataServices.AjaxHelperService.ajaxRequest

    ("get", url + window.CarPlus.KnowledgeBase.Constants.url + 'getRecentAnswers')
        .done(getSucceeded)
        .fail(getFailed);

    function getSucceeded(data) {
        $.map(data.recentQuestions,
            function (item) {
                if (item.Answers) {
                    var question = new window.CarPlus.KnowledgeBase.Models.Question

                    (item.Id, item.Text, item.DateTime, item.NiceDateTime);
                    recentAnswersObseravble.push(question);
                    if (item.Answers && item.Answers.length > 0) {
                        var answers = $.map(item.Answers, function (answer) {
                            return new window.CarPlus.KnowledgeBase.Models.Answer

                            (answer.Id, answer.Text, question.id, question.text, answer.Public, answer.IsBookmarked);
                        });
                        answers.forEach(function (answer) {
                            question.answers.push(answer);
                        });
                    }
                    return question;
                }
            });
        deferred.resolve(recentAnswersObseravble);
    }

    function getFailed() {
        deferred.fail();
    }

    return deferred.promise();
}

For reference the part of Constants.js we are concerned with looks like below; 

window.CarPlus.KnowledgeBase.Constants = function() {
    return {
        url: '/knowledgebase/',
        iamgeUploadLimit: 2000,
        imageUploadTypes: ['png', 'jpeg', 'bmp', 'tiff', 'jpg']
    };
}();

Server Side

So what does the request for recent answers look like when we hit the server? If you have been following carefully (or if I have explained this well, the jury is out on this latter part so let me know in the comments if any of this makes sense please!) the data service will be calling the following url for its recent answer data; <domain>/knowledgebase/getRecentAnswers.

This is routed to the GetRecentAnswers() method of our KnowledgeBaseController via the ASP.NET MVC routing rule we have in place and this action looks like this; 

[HttpGet]
public JsonResult GetRecentAnswers()
{
    var recentQuestions = _knowledgeBaseService.GetNRecentlyAnswered

                                        (_numberOfRecentlyAnsweredQuestionsToFetch, true);

    return Json(new { recentQuestions = recentQuestions });
}

The private variable _numberOfRecentlyAnsweredQuestionsToFetch is defined in the controller and is set by a configuration setting so it can be easily changed without having to rebuild the server side .NET code. At present this is set to 2.

The KnowledgeBaseService is injected into the controller through our use of the Windsor IOC container and constructor dependency injection. The service implements the interface IKnowledgeBaseService and the concrete class for this is declared in our container, which itself is initialised on Application_Start() in Global.asax.cs. 

Service

The service call looks something like this; 

public List<Question> GetNRecentlyAnswered(int maxResults, 
                                                                          bool hideRemoved)
{
    // get top N answers ordered by datetime
    var answers = _answerRepository.GetNRecent(maxResults);

    // use these answers to get questions (by question id against answer)
    var questionIds = new string[answers.Count];
    for (int i = 0; i < answers.Count; i++)
    {
        questionIds[i] = answers[i].QuestionId;                
    }
    var questions = _questionRepository.GetByIdArray(questionIds, hideRemoved);

    // sort the questions retrieved by the answers associated with them (descending).
    var sortedQuestions = questions
                            .OrderByDescending(q => q.Answers
                                .Where(a => a.Public == true)
                                .Max(a => a.DateTime))
                            .ToList();

    return sortedQuestions;
}

As you can see the service makes use of repositories, as we have a nice separation of concerns in our application. These repositories are declared against their interfaces in the service constructor and as with the controller our container is responsible for tying these interfaces to their implementation on application start, when the service is constructed the dependant repositories are injected and available for use by the service.

Repository

Just to follow this through then, the repository calls used by the service above are shown below. Please note the nice syntax of the MongoDb C# driver, I can't praise these drivers enough, they make working with Mongo in C# a breeze.

public List<Answer> GetNRecent(int maxResults)
{
    var query = Query.And(Query.EQ("Public", true), Query.EQ("Removed", false));
    var result = collection.FindAs<Answer>(query)
                                .SetSortOrder(SortBy.Descending("DateTime"))
                                .SetLimit(maxResults);

    return result.ToList();
}

public List<Question> GetByIdArray(string[] questionIds, bool hideRemoved)
{
    if (!hideRemoved)
    {
        var query = Query.In("_id", new BsonArray(questionIds));
        var result = collection.FindAs<Question>(query);
        return result.ToList();
    }
    else
    {
        var query = Query.And(
                        Query.In("_id", new BsonArray(questionIds)), 
                        Query.EQ("Removed", false));
        var result = collection.FindAs<Question>(query);
        return result.ToList();
    }
}

One final point to note about our repositories, regarding tying these to the Mongo database and collection. If we look at the constructor of our AnswerRepository we can see that we declare and initialise private variables for our Mongo database (or document store, although the driver refers to this as a database, perhaps to make it easier for developers new to NoSQL?) and collection and then set these via configuration settings. There has to be a better way to tie these together yet maintain some configuration over this, perhaps I could hook something up to the container, any suggestions or thoughts on this are most welcome.

public class AnswerRepository : IAnswerRepository
    {
        private MongoDatabase database;
        private MongoCollection collection;

        public AnswerRepository()
        {
            string connectionString = ConfigurationSettings.AppSettings["mongo.connstr"] as string;
            var server = MongoServer.Create(connectionString);
            
            string dbname = ConfigurationSettings.AppSettings["mongo.dbname"] as string;
            database = server.GetDatabase(dbname);

            string collectionName = ConfigurationSettings.AppSettings["mongo.collection.answers"] as string;
            collection = database.GetCollection<Answer>(collectionName);
        }
           
    ...
}

POCO

As you will have seen in the repository and service code above we deserialise the Mongo JSON (or BSON) data into .NET objects. We refer to these as POCO, which I believe stands for Plain Old CLR Objects, as that is what they are. They're not tied to any ORM as such (other than our implicit Mongo code which can be thought of as an ORM of sorts, albeit a very basic one), they are simply objects we use to work with data in our application.

We have two main objects, Question and Answer. One question can have many answers, so we have a nested document/object structure. This relates nicely back to my previous blog post on this application and issues around schema design for MongoDb, have a read here if you're interested.

Below are the classes for our Question and Answer POCOs for reference. As you will see the use of POCO may seem odd as we have the data in JSON and we ultimately pass it to the ViewModel in JSON but it does allow us to add a few nice things such as custom constructors so we can create new objects from posted data (or from stored data, for example if we want to duplicate or clone an object) as well as helper functions, so in our Question object we have a NiceDateTime() method to provide us with formatted date strings for use in the JSON we pass to the client.

namespace CarPlus.Models.POCO
{
    public class Question : Entity
    {
        public Question()
        { }

        public Question(string jsonData)
        {
            ...
        }

        public Question(BsonValue bsonValue)
        {
            ...
        }

        public string Id { get; set; }
        public int OwnerIdAsked { get; set; }
        public int OwnerIdAnswered { get; set; }
        public DateTime? DateTime { get; set; }
        public string Text { get; set; }
        public string Model { get; set; }
        public string Area { get; set; }
        public int Views { get; set; }
        public bool Approved { get; set; }
        public bool Referred { get; set; }
        public bool Archived { get; set; }
        public bool Removed { get; set; }
        public bool Replied { get; set; }

        public Answer[] Answers { get; set; }
        
        #region Helpers

        public string NiceDateTime
        {
            get
            {
                if (DateTime == null)
                {
                    return "";
                }
                else
                {
                    return DateTime.Value.ToString("HH:mm dd-MM-yyyy");
                }
            }
        }

        #endregion
    }

    public class Answer : Entity
    {
        public Answer()
        { }

        public Answer(string jsonData)
        {
            ...
        }

        public string Id { get; set; }
        public string QuestionId { get; set; }
        public int OwnerId { get; set; }
        public DateTime? DateTime { get; set; }
        public string Text { get; set; }
        public string[] Tags { get; set; }
        public int Views { get; set; }
        public bool Public { get; set; }

        public bool IsBookmarked { get; set; }

    }
}

An Odd Marriage

Once we have the data we need in POCO form from Mongo we then serialise this back to JSON (via the Json() Method of the JsonController) and return it to the ViewModel on the client who then consumes this data and binds it to the DOM. So we have an odd marriage of technology here in that we store data as JSON in Mongo and work with data as JSON in Knockout but in the middle we have to deserialise to objects to work with the code in C# and then re-serialise to JSON for the ViewModels. 

We decided to use .NET as it played to our strengths as a development team and allowed us to get started quickly, however if I had my time again I would look at using another platform for the web application, in particular I would use a dynamically typed and interpreted language, such as python or maybe node rather than a statically typed and compiled language like C# as this fits better with the data we store and the way we manipulate this in the UI.

I doubt whether this will ever be re-written, and its no real problem as the application works well and can be supported by the development team as is. Further to this the use of ASP.NET MVC and C# was fun and as stated above the C# drivers for Mongo are great, certainly much better than the python drivers I have used briefly on other projects. Another benefit is that we can add value to the data and application via the POCO in the form of helpers and constructors, so its not a bad marriage, just perhaps a little odd but with nice benefits.

ViewModel Models

So we serialise the POCO back to JSON and return this to the Data Service, which then satisfies the promise made by the ViewModel. The data returned from the Data Service is a representation of the POCO but in a form that can be easily bound to the DOM by the ViewModel, to demonstrate this the Question and Answer models we work with client side are shown below;

window.CarPlus.KnowledgeBase.Models.Question = function (id, text, dateCreated, niceDateTime) {
    var self = this;
    self.id = id;
    self.questionText = text;
    self.dateCreated = moment(dateCreated).startOf('day').fromNow();
    self.niceDateCreated = niceDateTime;
    self.answers = ko.observableArray([]);
};

window.CarPlus.KnowledgeBase.Models.Answer = function (id, answerText, questionId, questionText, isPublic, isBookmarked) {
    var self = this;
    self.id = id;
    self.questionId = questionId;
    self.questionText = questionText;
    self.answerText = answerText;
    self.isBookmarked = isBookmarked;
    self.public = ko.observable(isPublic);
};

As you can see one Question can have many Answers, in our Question model above this is represented by the observabe array 'answers'. Also if you compare this to our Question POCO you can see we only map the properties that we want to use in the DOM and our ViewModel operations, so some properties of the POCO may be ignored when we return data via the Data Service. One quick point to note is that on our Question model above we make use of a library called moment.js which is a great way to display blog like time stamps (1 day ago, 10 minutes ago, etc), we also make use of our POCO helper method value NiceDateTime so that we have both dates available in useable formats should we wish to use these to display to the user.

The DOM

If we now return to our DOM we can see that we have a <ul> bound to the observable array 'recentAnswers' as well as a <p> which is only visible if we have some recent answers to show. The 'recentAnswers' observable array is an array of window.CarPlus.KnowledgeBase.Models.Question objects, and each of these has a property called 'answers' which is itself an array of window.CarPlus.KnowledgeBase.Models.Answer objects, so this allows us to show a list of questions and for each question, the related answers, as shown below;

<div id="search-results">
    <h2>Recent questions</h2>
    <p data-bind="visible: recentAnswers().length === 0">

      There are currently no recently answered questions.</p>

    <ul data-bind="foreach: recentAnswers">
        <li data-bind="visible: answers().length > 0">
            <h3 data-bind="text: questionText"></h3>
            <ul>
                <li>
                    <div data-bind="foreach: answers">
                    <span data-bind="text: dateTime"></span>
                    <a href="#" data-bind="click: $root.createBookmark.bind($root, $data),

                                                         css: { selected:  isBookmarked}">Bookmark this answer</a>

                    <h4>Answer</h4>
                    <p data-bind="text: answerTextSummary"></p>
                    <button data-bind="click: $root.selectAnswer.bind($root, $parent, $data)">Read More</button>
                </li>
            </ul>
        </li>
    </ul>
</div>

For each question (recentAnswer) we display the question text and then we iterate through each answer to this question and display these answers within a nested <ul>.

You may have noticed that we have some extra functionality available to the user here, for each answer the user can create a bookmark, or select the answer (to show a detailed view), these are bound to the ViewModel functions createBookmark and selectAnswer respectively. These are worth a quick mention as they help to demonstrate binding context.

Root and Parent

As the elements that the createBookmark and selectAnswer functions are bound to are nested within a foreach binding the context of these functions needs specifying. If we simply had $createBookmark.bind() we would be trying to bind to the createBookmark function of an Answer, which does not exist so would throw an error.

What we need to do is ensure that we bind at the right level, hence the use of the $root prefix to these function bindings. We could use $parent here, however this would also fail, as we are nested within two foreach bindings we need to use $root which places us at the root of the ViewModel, rather than simply the $parent of the Answer, which in our case would be the Question and again would fail as Question has no function called createBookmark.

The knockout binding context is discussed in more detail here and is pretty self explanatory, however it has caught me out a few times during development so its well worth reading and remembering.

Summary

To summarise then, we have seen how we can structure an application using ASP.NET MVC, Mongo and Knockout and we have followed a use case (to show recently answered questions in our knowledge base) from initial request to response, through the ViewModel initialisation, AJAX data requests and responses and finally data binding to the DOM and display of the results.

Along the way I have thrown in a few random thoughts and plenty of snippets of code to demonstrate how it all hangs together, I hope this has been interesting and also maybe of some use to someone out there working on a similar application. As always your comments and thoughts are most welcome and finally thanks for reading this somewhat long winded post.

Oh and as I wrote this post I learnt a couple of things, a) I may need to split my rambling posts into smaller chunks and b) I need to figure out how to better format code snippets, perhaps some custom CSS will help. Anyway enough for now.

Cheers,
James