Testing Neo4j.Driver (4.1.1) Part 2 – Session Config

Fact

In my previous post we showed how we could test a few different areas of the Neo4j.Driver nuget package.

One area we didn’t touch was the SessionConfig – as part of the IAsyncSession. Now, for the most part – things are done in interfaces, which makes our lives easier – as we can Mock them easily.

SessionConfig is a different beast.

Let’s look at some example code:

var session = driver.AsyncSession(builder => builder
    .WithDatabase("movies")
    .WithDefaultAccessMode(AccessMode.Read));

Here we get an IAsyncSession that is configured for Read access, and against the movies database.

Let’s imagine we have decided to store our data in different databases, Movies in the movies database, Actors in the actors database.

We can change our existing code (in the GitHub) repo to be like the code above, and all our tests would still work.

Well. Two changes:

1 – change the Mock for the IDriver.AsyncSession method to be:

driverMock
    .Setup(x => x.AsyncSession(It.IsAny<Action<SessionConfigBuilder>>()))
    .Returns(sessionMock.Object);

2 – Change Test2_UsesTheAsyncSession_ToGetTheMovie() to Verify:

driverMock.Verify(x => x.AsyncSession(It.IsAny<Action<SessionConfigBuilder>>()), Times.Once);

Nothing is checking that the AsyncSession being created is actually pointing at the correct database.


Mocking The SessionConfig / SessionConfigBuilder

WARNING The code we’re going to use here can break if Neo4j change the Driver code.

Approach wise – we’re going to take the same approach we took with the Mock of the Transaction Functions where we inject our own ‘Func’.

To do that, we’ll want code that looks like this:

mockDriver
    .Setup(x => x.AsyncSession(It.IsAny<Action<SessionConfigBuilder>>()))
    .Returns((Action<SessionConfigBuilder> action) =>
    {
        action( /* SOMETHING !!! */ );
        return mockSession.Object;
    });

What we need to work out, is how to create the ‘SOMETHING’ as we can’t Mock it. So, let’s look at SessionConfigBuilder a bit more.

First – it’s sealed – and has no default constructor, the only one that is there is the internal Constructor that takes a SessionConfig.

Looking a bit more, we can see that the WithDatabase method sets the Database property on the SessionConfig, and a bit further down there is a method called Build() (again internal) which returns the internal Config.

So, let’s have a look at SessionConfigpublic – so we can see it, sealed – so we can’t extend it, and it only has 1 constructor which as above is internal.

Hmmm. OK. Positives – the SessionConfig class is basically a holder for data, so if we could get one of those into a SessionConfigBuilder – we could get it constructed, then, we should be able to call Build() on our builder and get the config to check the settings.

We have to assume Neo4j have tested that setting the database does indeed use the correct database. That is not in the scope of these tests, nor should it be.

Get you a SessionConfig

Luckily, SessionConfig has the default constructor, so we can make an instance in one line:

var sc = FormatterServices.GetUninitializedObject(typeof(SessionConfig)) as SessionConfig;

This gives us an uninitialized version of the SessionConfig object, which was easy!

Get you a SessionConfigBuilder

With sc in hand, how do we construct our builder?

With the only constructor being an internal one with a parameter, we’re going to have to go full reflection. So. Let’s get the constructor:

var ctr = typeof(SessionConfigBuilder)
    .GetConstructors(BindingFlags.NonPublic | BindingFlags.Instance)
    .Single(c => c.GetParameters().Length == 1 && c.GetParameters().Single().ParameterType == typeof(SessionConfig));

We could probably just get away with:

var ctr = typeof(SessionConfigBuilder)
    .GetConstructors(BindingFlags.NonPublic | BindingFlags.Instance)
    .Single();

But, the extra checks mean that we will hopefully be slightly more future proof, as I can see the addition of extra constructors being relatively common.

Anyhews, once we have that, we need to call our constructor:

ctr.Invoke(new object[] {sc}) as SessionConfigBuilder;

Which we can wrap into a method:

private static SessionConfigBuilder GenerateSessionConfigBuilder()
{
    var sc = FormatterServices.GetUninitializedObject(typeof(SessionConfig)) as SessionConfig;

    var ctr = typeof(SessionConfigBuilder)
        .GetConstructors(BindingFlags.NonPublic | BindingFlags.Instance)
        .Single(c => c.GetParameters().Length == 1 && c.GetParameters().Single().ParameterType == typeof(SessionConfig));

    return ctr.Invoke(new object[] {sc}) as SessionConfigBuilder;
}

Which means our GetMocks method can now become:

private static void GetMocks(
    out Mock<IDriver> driver, 
    out Mock<IAsyncSession> session, 
    out Mock<IAsyncTransaction> transaction, 
    out Mock<IResultCursor> cursor,
    //RETURN OUT THE BUILDER
    out SessionConfigBuilder sessionConfigBuilder)
{
    var transactionMock = new Mock<IAsyncTransaction>();
    var sessionMock = new Mock<IAsyncSession>();
    sessionMock
        .Setup(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task<List<Movie>>>>()))
        .Returns((Func<IAsyncTransaction, Task<List<Movie>>> func) => { return func(transactionMock.Object); });

    var cursorMock = new Mock<IResultCursor>();
    transactionMock
        .Setup(x => x.RunAsync(It.IsAny<string>(), It.IsAny<object>()))
        .Returns(Task.FromResult(cursorMock.Object));

    // GENERATE OUR BUILDER
    var builder = GenerateSessionConfigBuilder();
    var driverMock = new Mock<IDriver>();
    driverMock
        .Setup(x => x.AsyncSession(It.IsAny<Action<SessionConfigBuilder>>()))
        .Returns((Action<SessionConfigBuilder> action) =>
        {
            action(builder); //CALL THE BUILDER
            return sessionMock.Object;
        });

    driver = driverMock;
    session = sessionMock;
    transaction = transactionMock;
    cursor = cursorMock;
    sessionConfigBuilder = builder;
}

Get you a Build() method

To test our code, we need one last thing. To get the SessionConfig from the builder, to be able to see we’ve set the correct properties with the correct values. internally they use .Build() – so let’s write our own one.

I like extension methods for this sort of thing, so first let’s work out how to do it, then make it all look nice.

How is not really any different to the constructor stuff we looked at earlier – that’s right – reflection – which has the same problems (they might change it!).

var buildMethod = typeof(SessionConfigBuilder).GetMethod("Build", BindingFlags.NonPublic | BindingFlags.Instance);

It’s internal – so we need the NonPublic flags, and the name "Build". Once we have that – we need to Invoke that method on the SessionConfigBuilder instance we have:

var config = buildMethod.Invoke(builder, null) as SessionConfig;

Let’s wrap that up in a nice extension method:

public static class SessionConfigBuilderExtensions
{
    public static SessionConfig Build(this SessionConfigBuilder scb)
    {
        var buildMethod = typeof(SessionConfigBuilder).GetMethod("Build", BindingFlags.NonPublic | BindingFlags.Instance);
        return buildMethod?.Invoke(scb, null) as SessionConfig;
    }
}

OK, and let’s now finish our test:

[Fact]
public async Task Part2_Test1_UsesTheCorrectDatabase()
{
    const string expectedDb = "movies";
    GetMocks(out var mockDriver, out _, out _, out _, out var builder);
    var movieStore = new MovieStore(mockDriver.Object);
    await movieStore.GetMovie_Part2("Valid");

    var config = builder.Build();
    config.Database.Should().Be(expectedDb);
}

Now we can test that the database we expect is what we actually pass in. This is extra useful when we’re doing this against a method where we take in the database, as it means we can make sure our code is actually passing that parameter onto the session properly.

Using Neo4j.Driver? Now you can EXTEND it!

Some Code

Hot on the heels of Neo4jClient 4.0.0, I was doing some work with the Neo4j.Driver (the official client for Neo4j), and in doing so, I realised I was writing a lot of boiler plate code.

So I started adding extension methods to help me, and as my extension methods became more involved, I moved them to another project, and then… well… decided to release them!

TL;DR; you can get the Neo4j.Driver.Extensions package on Nuget, and the GitHub page is here.


The Problem

Let’s first look at the problem. Neo4j.Driver is quite verbose, you end up having lots of ‘magic strings’ throughout the codebase, which can lead to problems in runtime, and one of the reasons we’re using .NET is to try to avoid runtime errors when we can get compilation errors.

Let’s take a look at a ‘standard’ read query. Here we’re executing the following Cypher to get a movie with a given title.

MATCH (m:Movie)
WHERE m.title = $title
RETURN m

We MATCH a Movie based on it’s title and return it, easy. Code wise we have this:

public async Task<Movie> GetMovieByTitle(string title)
{
    var session = _driver.AsyncSession();
    var results = await session.ReadTransactionAsync(async tx =>
    {
        var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
        var cursor = await tx.RunAsync(query, new {title});
        var fetched = await cursor.FetchAsync();

        while (fetched)
        {
            var node = cursor.Current["m"].As<INode>();
            var movie = new Movie
            {
                Title = node.Properties["title"].As<string>(),
                Tagline = node.Properties["tagline"].As<string>(),
                Released = node.Properties["released"].As<int?>()
            };
            return movie;
        }

        return null;
    });

    return results;
}

We’re using transactional functions to give us future proofing should we decide to connect to a Cluster, as the function will retry the query if the current cluster member we’re connected to takes the unfortunate decision to ‘move on’.

Let’s take a closer look at the creation of the Movie object:

var node = cursor.Current["m"].As<INode>();
var movie = new Movie
{
    Title = node.Properties["title"].As<string>(),
    Tagline = node.Properties["tagline"].As<string>(),
    Released = node.Properties["released"].As<int?>()
};
return movie;

In our first line, we pull the Current IRecord from the IResultCursor, by an identifier, and attempt to get it as an INode. This is ok, I know in my query that’s what I’ve asked for (RETURN m). Then I proceed to go through the properties in my Movie – assigning the properties from the node into the right place.

For the string properties (title and tagline) it’s just an As<string>() call – which works fine, as if the property isn’t there, we just get null anyway. The released property is more complex, to make the code simpler – I’ve used .As<int?>() (nullable int) as I happen to know the default movies database does have some nodes without the released property – as we’re schema-free here.

What I could have done would be:

var released = node.Properties["released"].As<int?>();
if(released.HasValue)
    Released = released.Value;
else { /* ?!?!? */ }

Which would make my Movie class slightly tighter – but I guess, if my data can have nodes without the property – then my models should too… 🙂

GetValue

Aaaaanyways. The first step in the extension methods was to create a ‘GetValue’ method:

var node = cursor.Current["m"].As<INode>();
var movie = new Movie
{
    Title = node.GetValue<string>("title"),
    Tagline = node.GetValue<string>("tagline"),
    Released = node.GetValue<int?>("released")
};
return movie;

This method will return default if the Properties property doesn’t contain a given key, otherwise it will try to call As<T> with the associated Exceptions (FormatException and InvalidCastException) that can take place.

I mean. If that was it, you’d be right to say that this was a waste of time. But, it does simplify the call a bit… onwards!

GetContent (NetStandard 2.1 only)

I’d like it to be a bit simpler, so let’s try to sort out the while loop. Remember we have this:

var cursor = await tx.RunAsync(query, new {title});
var fetched = await cursor.FetchAsync();

while (fetched)
{
    var node = cursor.Current["m"].As<INode>();
    /* object creation code here */
}

We FetchAsync from the cursor, then while that returns true we parse our Current into INode and then create our obj.

Instead of the Fetch/While loop, we can do this instead:

var cursor = await tx.RunAsync(query, new {title});
await foreach(var node in cursor.GetContent<INode>("m")) 
{
    /* object creation code here */
}

This removes the cursor.Current["m"].As<INode>() line, and simplifies the while into a pleasing foreach.

NB. This is NetStandard 2.1 as it uses the IAsyncEnumerable interface which isn’t in NetStandard 2.0

ToObject

OK, so we’re starting to look a bit better, some of the boiler plate is going, is there anything else we can do?

OF COURSE!

Instead of all the object creation, requiring you to go through the properties (and what if you add one later in development and forget to update this code!?!) – we can use ToObject

This works on an INode (and we’ll see later other things), and allows you to pass in a Type as a generic parameter, which will be parsed, and returned to you filled if possible:

var cursor = await tx.RunAsync(query, new {title});

await foreach (var node in cursor.GetContent<INode>("m")) 
    return node.ToObject<Movie>();

Neo4jProperty

We should probably pause here to talk about Neo4jPropertyAttribute so far, we’ve had the properties from Neo4j all Lowercase, but the observant of you will have noticed that the Movie class seems to have Upper camel case naming conventions as .NET typically does.

When we’re doing the GetValue approach – not such an issue – as we define the identifier ourselves (GetValue('title')). I think it’s probably pretty obvious that I’m going to be using Reflection here to work out what property to put where but OH NOES my properties are all Upper camel case, but the data is all Lower camel case. WHAT TO DO?

This is how a property is normally defined:

public string Title {get;set;}

Basic stuff. But we can add the Neo4jProperty attribute:

[Neo4jProperty(Name = "title")]
public string Title {get;set;}

And lo and behold, the properties will be reflected properly!
As an added extra – you can also tell the serialization process to Ignore a property if you want:

[Neo4jProperty(Ignore = true)]
public string Title {get;set;}

So the full Movie class looks like:

public class Movie
{
    [Neo4jProperty(Name = "title")]
    public string Title { get; set; }

    [Neo4jProperty(Name = "released")]
    public int? Released { get; set; }

    [Neo4jProperty(Name = "tagline")]
    public string Tagline { get; set; }
}

GetRecords with ToObject (NetStandard 2.1 only)

So – we’ve seen it works on INode, but, what if we want to return a different thing than just a node, what about, the properties? So change our Query to:

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m.title AS title, m.tagline AS tagline, m.released AS released"

Well. No worries! We’ve still got a working prospect:

await foreach (var record in cursor.GetRecords())
    return record.ToObject<Movie>();

Here, I’m using the GetRecords extension method to get each IRecord and attempt to cast it to a Movie. This works as the properties of Movie match the names of the aliases in the Cypher.

RunReadTransactionForObjects

In the previous examples, for simplification, I’ve not shown the code around the outside, but – if we take the most recent one (GetRecords example), it actually looks like this:

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
var session = _driver.AsyncSession();
var results = await session.ReadTransactionAsync(async x =>
{
    var cursor = await x.RunAsync(query, new {title});

    await foreach (var node in cursor.GetContent<INode>("m")) 
        return node.ToObject<Movie>();

    return null;
});

return results;

And we’ll do that for almost all the queries we’re going to run, so let’s look at how we can reduce that code as well…

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
var session = _driver.AsyncSession();

var movie = 
    (await session.RunReadTransactionForObjects<Movie>(query, new {title}, "m"))
    .Single();
return movie;

I’ve added some newlines to make it a bit more readable, but now, we take the session and call RunReadTransactionForObjects<T> on it, to return the results of the query as T (in this case Movie).

The RunReadTransactionForObjects<T> method is returning an IEnumerable<T> hence I can use .Single() (or indeed any of the LINQ methods).

Put it all together

There are other extension methods in there, and some which are no doubt missing (PR away!) but I’m quite pleased that I can get from this code:

public async Task<Movie> GetMovieByTitle(string title)
{
    var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
    var session = _driver.AsyncSession();
    var results = await session.ReadTransactionAsync(async tx =>
    {        
        var cursor = await tx.RunAsync(query, new {title});
        var fetched = await cursor.FetchAsync();

        while (fetched)
        {
            var node = cursor.Current["m"].As<INode>();
            var movie = new Movie
            {
                Title = node.Properties["title"].As<string>(),
                Tagline = node.Properties["tagline"].As<string>(),
                Released = node.Properties["released"].As<int?>()
            };
            return movie;
        }

        return null;
    });

    return results;
}

To:

public async Task<Movie> GetMovieByTitle(string title)
{
    var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
    var session = _driver.AsyncSession();

    var results = 
        (await session.RunReadTransactionForObjects<Movie>(query, new {title}, "m"))
        .Single();
    return results;
}

PHEW!

Long post eh?

If you got this far – well done! Please, try it, log bug reports (on GitHub– comments on here are easy to miss).