Testing Neo4j.Driver (4.1.1) Part 1

There are a few challenges when dealing with the official Neo4j.Driver when it comes to testing, over a period of time, I’ve hit a few of them, and thought it would be good to share them with you.

TL;DR; This is all be available on GitHub

So, let’s write a method, in which we pass a title in, and get the Movie that the title relates to. A few assumptions:

  1. Movie exists as a class, with a Title (string), Tagline (string) and Released (int) property
  2. The method will be in a MovieStore class, which we’re not testing (we assume it’s already been tested)

Nuget wise – we’re going to be using:

I’m not going to go full TDD – I think we can skip a bit ahead, and start with a base stub that we can test:

public class MovieStore
{
    private readonly IDriver _driver;

    public MovieStore(IDriver driver)
    {
        _driver = driver;
    }

    public async Task<Movie> GetMovie(string title)
    {
        return null;
    }
}

Test 1

If I call with an invalid title, I get null back

[Fact]
public async Task Test1_ReturnsNull_WhenInvalidTitleGiven()
{
    var movieStore = new MovieStore( /* IDriver?? */ );
    var actual = await movieStore.GetMovie("invalid");

    actual.Should().BeNull();
}

Our first problem, we need to mock an IDriver instance for the MovieStore to be constructed, this is why we have Moq:

[Fact]
public async Task Test1_ReturnsNull_WhenInvalidTitleGiven()
{
    var driverMock = new Mock<IDriver>();
    var movieStore = new MovieStore( driverMock.Object );
    var actual = await movieStore.GetMovie("invalid");

    actual.Should().BeNull();
}

Running our test – and it passes, for the obvious reason that – well, we only return null.

Test 2

Jumping ahead – we might go with ‘passing in a valid title gets a Movie’. But code-wise to get to this stage there’s actually quite a lot we need to consider.

  1. We need a Session
  2. With that Session we need to run a Transaction Function (we’ll come to why a bit later on)
  3. In that function, we need to execute some Cypher
  4. We need to parse the results of that Cypher
  5. Each result from above we’ll need to parse into a Movie (in this case there really should only be 1 result)
  6. We need to return that Movie out of the function

So, I guess we ought to test that we get a Session to work with:

[Fact]
public async Task Test2_UsesTheAsyncSession_ToGetTheMovie()
{
    var driverMock = new Mock<IDriver>();
    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("valid");

    driverMock.Verify(x => x.AsyncSession(), Times.Once);
}

A failing test!
To fix it – let’s add our code into our method:

public async Task<Movie> GetMovie(string title)
{
    _driver.AsyncSession();
    return null;
}

As a ‘be the best person’ thing – we should be disposing of that Session as well, which is where we meet our next Mock target. Closing the Session is a method of the IAsyncSession object, not the IDriver. So we need our Mock<IDriver> to supply an IAsyncSession:

[Fact]
public async Task Test2a_ClosesTheAsyncSession_ToGetTheMovie()
{
    //Our new mock
    var sessionMock = new Mock<IAsyncSession>();

    var driverMock = new Mock<IDriver>();

    //We setup the driverMock to Return the sessionMock object when anything asks for the AsyncSession
    driverMock
        .Setup(x => x.AsyncSession())
        .Returns(sessionMock.Object);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("valid");

    //Now we can check the session is closed.
    sessionMock.Verify(x => x.CloseAsync(), Times.Once);
}

Our new code looks like:

public async Task<Movie> GetMovie(string title)
{
    var session = _driver.AsyncSession();
    await session.CloseAsync();
    return null;
}

Good – now run our tests, OH NOES!! the previous 2 tests have just broken, throwing NullReferenceExceptions due to the fact that the AsyncSession isn’t mocked for them.

I would heartily recommend NCrunch or Resharper to automatically run your tests as soon as possible to get feedback directly when writing tests.


Decision time!

There are a few options here, the first is to ignore the error – and for the Test2_UsesTheAsyncSession_ToGetTheMovie() this is an option. With this test we could catch the exception, which arguably is the right thing to do as we don’t actually care if it succeeds – only that we attempt to open the session.

The second option is to copy the setup code to the other 2 methods, for Test1 this is the right choice, as the result should be null if it succeeds, but there is nothing to return.

I’m no purist, and I don’t particularly want my tests covered with the same boiler plate code, so I’m going to extract my setup code into another method, and use that in place:

//We're returning 'Mock' versions so we can do verification later if we want to
private static void GetMocks(out Mock<IDriver> driver, out Mock<IAsyncSession> session)
{
    var sessionMock = new Mock<IAsyncSession>();

    var driverMock = new Mock<IDriver>();
    driverMock
        .Setup(x => x.AsyncSession())
        .Returns(sessionMock.Object);

    driver = driverMock;
    session = sessionMock;
}

This method will be changed over time I imagine, adding more mocks, maybe some more default options etc to use this in my Test2a test we do:

[Fact]
public async Task Test2a_ClosesTheAsyncSession_ToGetTheMovie()
{
    //Getting the mocks here.
    GetMocks(out var driverMock, out var sessionMock);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("valid");

    sessionMock.Verify(x => x.CloseAsync(), Times.Once);
}

For the other tests (and I’ll just show Test1 here), I can use the _ to ignore the out parameters I’m not interested in

[Fact]
public async Task Test1_ReturnsNull_WhenInvalidTitleGiven()
{
    //See the use of _ here for the session mock, as I don't need it
    GetMocks(out var driverMock, out _);
    var movieStore = new MovieStore(driverMock.Object);
    var actual = await movieStore.GetMovie("invalid");

    actual.Should().BeNull();
}

Test 3

Way up above – step 2 was:

2. With that Session we need to run a Transaction Function (we’ll come to why a bit later on)

The simplest way for us to run the Cypher we want to run is just running RunAsync on the Session. The problem with this approach is when we decide to move to a Cluster, it is recommended we use Transaction Functions, as they automatically retry connections for us, and due to the way Causal Clusters work, that takes the effort away from us – the developers – which can only be a good thing!

So, the test:

[Fact]
public async Task Test3_OpensAReadTransaction()
{
    GetMocks(out var driverMock, out var sessionMock);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("valid");

    sessionMock.Verify(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task>>()), Times.Once);
}

Hmm what’s this It.IsAny stuff that’s suddenly appeared? Well, we only care that we use a ReadTransaction – at this stage we don’t care what is actually going on in the transaction function, merely that we’re using one.

Our method now looks like:

public async Task<Movie> GetMovie(string title)
{
    var session = _driver.AsyncSession();
    await session.ReadTransactionAsync(tx => { return null; });

    await session.CloseAsync();
    return null;
}

Test 4

It’s executing cypher time!

So, quick recap. We have our code opening a Session, and in that Session we’re opening a ReadTransaction function. All good – now, we need to do something in our transaction.

Let’s first work out our Cypher. We want a Movie by title so:

MATCH (m:Movie) WHERE m.title = $title RETURN m

I’m using a parameter $title for the title, as that makes Neo4j run better for subsequent queries.
So, we want to call that in our transaction, how do we do that?

public async Task<Movie> GetMovie(string title)
{
    //I usually put this at the top, to make it obvious.
    //For big queries - I'll use '@' to multiline it to make it more readble.
    const string query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";

    var session = _driver.AsyncSession();
    //RunAsync is async, so we pass in the 'tx' as 'async'
    await session.ReadTransactionAsync(async tx =>
    {
        //We await a call to 'RunAsync'
        await tx.RunAsync(query, new {title});
    });

    await session.CloseAsync();
    return null;
}

At this point, all I’m doing is executing some Cypher, I’m not dealing with the results, all I want to check is that the cypher is correct.

This is a contrived example, given it’s a const at the top of the method, would I really bother testing the Cypher is correct? Maybe not, but for a query which is composed, I might want to. It’s also useful to be able to test that the title is being passed in as a parameter rather than a string. Which is the kind of thing an overly optimistic developer can do by accident. Also – this is a way to show how you would test this.

In a break from tradition, I’ve shown you the code before the test, and that’s because it’s easier to show how to test it if you can see the code.

We’ll need to Mock the tx part of this, to be able to check what is called. tx in this case is an IAsyncTransaction – so let’s Mock that:

var transactionMock = new Mock<IAsyncTransaction>();

Now the slightly tricky bit – we need to get this mock into the actual call that is being made. This is so we can verify what is called.

We have a mock Session – so let’s Setup the call:

sessionMock
    .Setup(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task>>()))
    .Returns((Func<IAsyncTransaction, Task> func) =>
    {
        func(transactionMock.Object);
        return Task.CompletedTask;
    });

Let’s take it apart a bit. First – we .Setup the call – we don’t care what the actual call is, only that it matches the pattern (Func<IAsyncTransaction, Task>), and at the moment it does.

Next we .Returns a Task.CompletedTask – this maps to the Task part of the Funcbut we also have this (Func<IAsyncTransaction, Task> func) => bit – which is the most important bit. Basically we’re taking the parameter from the Setup and injecting our own IAsyncTransaction: func(transactionMock.Object);.

Because we use our Mock – we can now Verify that the code is called correctly:

[Fact]
public async Task Test4_ExecutesTheRightCypher()
{
    const string expectedCypher = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
    GetMocks(out var driverMock, out _, out var transactionMock);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("valid");

    transactionMock.Verify(x => x.RunAsync(expectedCypher, It.IsAny<object>()), Times.Once);
}

We’ve modified our GetMocks method to return the transactionMock as an out parameter. We’re also not testing the parameter itself. Merely that the Cypher is correct. So. Let’s test that parameter:

[Fact]
public async Task Test4a_ExecutesUsingTheRightParameter()
{
    const string expectedParameter = "valid";
    GetMocks(out var driverMock, out _, out var transactionMock);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie(expectedParameter);

    transactionMock.Verify(x => x.RunAsync(It.IsAny<string>(), expectedParameter), Times.Once);
}

Sorted. Oh wait. Hmmm. Doesn’t work – Ahhh, RunAsync takes an object (or the overload we’re using does), and we actually pass it in as new { title } – Anonymous type time!

We need to do some custom comparison here, for that we’ll use It.Is:

transactionMock.Verify(x => x.RunAsync(It.IsAny<string>(), It.Is<object>(o => <COMPAREHERE>)), Times.Once);

The tricky bit here is the <COMPAREHERE> bit. What we have is an anonymous type, so we can’t just do: o == expectedParameter, we need to get the value out. My first attempt was to try:

((dynamic)o).title == expectedParameter

But that didn’t work as expression trees can’t contain dynamic objects – so I learnt something 🙂 Generally, when I’ve got to this point, it’s a method we’re after. We have need 3 parameters, the first is the actual object o itself. Next, we need the expectedValue – and because we’re pulling from an AnonymousObject we’ll need to know the name of the property we’re looking for. This last is important, as we could pass in more than one parameter (new {title, title1 = title, title2 = title});

Luckily, in another project I have one of these for just such an occassion – who’d have thought?!

private static bool CompareParameters<T>(object o, T expectedValue, string propertyName) 
{
    var actualValue = (T) o.GetType().GetProperty(propertyName)?.GetValue(o);
    actualValue.Should().Be(expectedValue);
    return true;
}

We use reflection to get the value from the o and then cast (T) it to T – then call actualValue.Should().Be(expectedValue) – which will throw an exception if it’s not. Finally, we return true; and we do this as It.Is needs a bool response, and if no exceptions have been thrown, then we’re all good.

Putting that into the codebase, our call now looks like:

[Fact]
public async Task Test4a_ExecutesUsingTheRightParameter()
{
    const string expectedParameter = "valid";
    GetMocks(out var driverMock, out _, out var transactionMock);

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie(expectedParameter);

    transactionMock.Verify(x => x.RunAsync(It.IsAny<string>(), It.Is<object>(o => CompareParameters(o, expectedParameter, "title"))), Times.Once);
}

Test 5

OK, where have we got to? Step 4, parsing our results. This makes sense, at the moment, all we’re doing is running Cypher.

So the trick here is to Mock the result from the query we pass in, we do this and don’t connect to a Neo4j instance for a couple of reasons, the first is that I don’t want to test that Neo4j works. I think at this point, it’s safe to assume that executing a query will work. I don’t know that the query is right – and that would require integration testing – but at this stage, I want to know that assuming the connection is ok, and the query has run – and we get back valid data that I can parse that. The other reason is that I don’t have to write a load of setup code, and probably some powershell scripts to bring down an instance of Neo4j, start it, etc. At least not in this post!

So what does the data from Neo4j look like?

When I’m in this sort of situation, particularly if I’m investigating what the outcome of a query will look like, I go to my trusted friend, LinqPad, running the following query:

var driver = GraphDatabase.Driver("neo4j://localhost:7687", AuthTokens.Basic("neo4j","neo"));
var session = driver.AsyncSession();

var cursor = await session.RunAsync("MATCH (m:Movie) WHERE m.title = 'The Matrix' RETURN m");
await cursor.FetchAsync();
cursor.Dump();

Gets me:

So we get an IResultCursor with a property call Current, and inside that an IRecord, which has Keys and Values properties.

The Keys property contains my m that I’m returning from the query (RETURN m) – and I can see that, that is a Node – which for us, is an INode.

The likelihood is that the INode contains the actual data we want, so if we add:

cursor.Current["m"].As<INode>().Dump();

We get:

Bingo! All the datas. So to test this, we need to Mock the following:

  • IResultCursor – with setups for the Current["m"] property indexer, and the FetchAsync method – as we need to call that to succeed – so we ought to ensure we do call it.
  • INode – with the Properties property.

Why are we not Mocking the IRecord? Well – this is because we can bypass it and just return the INode from the Current["m"] call. If we were getting the IRecord from the Current property directly, and then accessing it, we would need to Mock it.

Now, we’re reach an interesting point. Would it be better for us to make a Stub instead of a Mock for any of these?

We could implement a TestNode : INode pretty easily, but we’d end up implementing a lot more than we need to test our code base.

I would argue for this – we should approach it from a Mock point of view, and if at somepoint it makes sense to Stub then go for that at that point. For now… Mocking…

So, the IResultCursor comes from the IAsyncTransaction, and we already have that mocked, so we can just access it and add a new Setup:

var cursorMock = new Mock<IResultCursor>();
transactionMock
    .Setup(x => x.RunAsync(It.IsAny<string>(), It.IsAny<object>()))
    .Returns(Task.FromResult(cursorMock.Object));

We don’t need to setup anything on cursorMock at the moment, as the default for a Moq mock is Loose which means it won’t throw an exception when a call is made on a method that isn’t setup.

[Fact]
public async Task Test5_CallsFetchAsyncToGetTheNextRecord()
{
    GetMocks(out var driverMock, out _, out var transactionMock);

    var cursorMock = new Mock<IResultCursor>();
    transactionMock
        .Setup(x => x.RunAsync(It.IsAny<string>(), It.IsAny<object>()))
        .Returns(Task.FromResult(cursorMock.Object));

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("Valid");

    cursorMock.Verify(x => x.FetchAsync(), Times.Once);
}

Which leads to our method being:

public async Task<Movie> GetMovie(string title)
{
    const string query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";

    var session = _driver.AsyncSession();
    await session.ReadTransactionAsync(async tx =>
    {
        var cursor = await tx.RunAsync(query, new {title});
        await cursor.FetchAsync();
    });

    await session.CloseAsync();
    return null;
}

So, next step, let’s make the IResultCursor return valid data, begin with an INode:

const string expectedTitle = "Title";
const string expectedTagline = "Tagline";
const int expectedReleaseDate = 2000;

var nodeMock = new Mock<INode>();
nodeMock.Setup(x => x.Properties["title"]).Returns(expectedTitle);
nodeMock.Setup(x => x.Properties["tagline"]).Returns(expectedTagline);
nodeMock.Setup(x => x.Properties["released"]).Returns(expectedReleaseDate);

I suspect we’ll be extracting that out into a method, in fact, let’s just do that:

private static Mock<INode> GetMockNode(string title = "Title", string tagline = "Tagline", int? released = 2000)
{
    var nodeMock = new Mock<INode>();
    nodeMock.Setup(x => x.Properties["title"]).Returns(title);
    nodeMock.Setup(x => x.Properties["tagline"]).Returns(tagline);
    nodeMock.Setup(x => x.Properties["released"]).Returns(released);
    return nodeMock;
}

I’ve also removed the consts as in this case, I’m not actually checking that it’s doing it right, merely that it is attempting to get the values.

We need to return our node mock now:

var nodeMock = GetMockNode();
var cursorMock = new Mock<IResultCursor>();
cursorMock.Setup(x => x.Current["m"]).Returns(nodeMock.Object);

OK, and lastly, verify that the node is called on:

nodeMock.Verify(x => x.Properties["title"], Times.Once);
nodeMock.Verify(x => x.Properties["tagline"], Times.Once);
nodeMock.Verify(x => x.Properties["released"], Times.Once);

Now, the above code is the reason I have mocked each indexer as opposed to just making Properties return a new Dictionary<string, object> initialized with the values. Using a Dictionary will work but it restricts our verifiability to only checking if the Properties property was hit, not what specifically was hit. So all I could do was:

nodeMock.Verify(x => x.Properties, Times.Exactly(3));

But, calling node.Properties["title"] 3 times, would still pass, and that’s not right. That’s wrong.

Test 6

  1. Each result from above we’ll need to parse into a Movie (in this case there really should only be 1 result)

Hmmm, 2 things here, first, convert to a Movie – I actually did that in the last test (tbh – I know how this is going to come out and I’m cheating a bit as I suspect this is already quite long!). The other thing is the words ‘Each result’.

Each result

There could be more than 1.

This is bad news. Our method only returns Movie not IEnumerable<Movie> – and we’re certainly not parsing more results. Now we have two choices. Change the method to return IEnumerable, or leave it, and assume our data is clean and normalized, and there can be only one movie with a title (given that the movie industry appears to work primarily with remakes at the moment, this seems unlikely.)

We’re going to change the signature, only so we can show how to mock the multiple items scenario. Because, 90% of the time, you’ll probably end up doing that, and well, we may as well cover it. If you’re not doing that – then – you can still learn.

So. Signature change. Task<Movie> –> Task<IEnumerable<Movie>> – which due to the way we’ve currently written tests (i.e. not checking results etc) means we’re actually all good with the change.

So, let’s setup our mock of the IResultCursor to return 1 movie with the first FetchAsync and false for the second FetchAsync.

[Fact]
public async Task Test6_CallsFetchAsyncUntilFalseReturned()
{
    GetMocks(out var driverMock, out _, out var transactionMock);

    var nodeMock = GetMockNode();
    var cursorMock = new Mock<IResultCursor>();
    cursorMock.Setup(x => x.Current["m"])
        .Returns(nodeMock.Object);

    cursorMock
        .SetupSequence(x => x.FetchAsync())
        .Returns(Task.FromResult(true))
        .Returns(Task.FromResult(false));

    transactionMock
        .Setup(x => x.RunAsync(It.IsAny<string>(), It.IsAny<object>()))
        .Returns(Task.FromResult(cursorMock.Object));

    var movieStore = new MovieStore(driverMock.Object);
    await movieStore.GetMovie("Valid");

    cursorMock.Verify(x => x.FetchAsync(), Times.Exactly(2));
}

Moq has a SetupSequence method allowing us to configure the responses, in this case true then false. We then check we call FetchAsync() twice.

Now. When we change the code to reflect this, we’re probably going to break one of our other tests as we don’t have the FetchAsync setup to return anything.

OK, it was only Test5a_AttemptsToGetTheData() we needed to fix, due to the fact that the default response from FetchAsync will be false so – we never go into our loop to get data:

var cursor = await tx.RunAsync(query, new {title});
//Assign the result to a variable
var fetched = await cursor.FetchAsync();

//While that's 'true'
while (fetched)
{
    /* All the node reading code here */

    //Then see if we have another one to get
    fetched = await cursor.FetchAsync();
}

Right, finally, we’re going to get the actual Movie instances…

Test 7

Eh?! What about the rest of the previous step? As we need to return the Movie to be able to test it, that fits into our:

  1. We need to return that Movie out of the function

And this is in no way because this post has gotten longer than I imagined.

Unfortunately, the changes we need to do here will break a lot of our tests, so far, we’ve been returning nothing from our ReadTransactionAsync method, which means the signature has been: Func<IAsyncTransaction, Task>, but now we’re changing it to Func<IAsyncTransaction Task<IEnumerable<Movie>>> and that means our Mocks for the IAsyncSession need to be updated.

Fortunately, as we’d extracted that mock setup to one method, we can change it there and fix all the broken tests in one go.

Or so I thought

Uh oh! The mock setup did this:

sessionMock
    .Setup(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task>>()))
    .Returns((Func<IAsyncTransaction, Task> func) =>
    {
        func(transactionMock.Object);
        return Task.CompletedTask;
    });

To change it to actually return the Func we call, we need to go with:

sessionMock
    .Setup(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task<List<Movie>>>>()))
    .Returns((Func<IAsyncTransaction, Task<List<Movie>>> func) =>
    {
        return func(transactionMock.Object);
    });

At which point we discover, that the first tests start to fail as they call on things which are not there (IResultCursor) – so we should add those into our default mock setups.

We add:

var cursorMock = new Mock<IResultCursor>();
transactionMock
    .Setup(x => x.RunAsync(It.IsAny<string>(), It.IsAny<object>()))
    .Returns(Task.FromResult(cursorMock.Object));

to the GetMocks method – this allows us to get most of the tests working, the other broken ones are where (as in Test3_OpensAReadTransaction) we’re testing that sessionMock.Verify(x => x.ReadTransactionAsync(It.IsAny<Func<IAsyncTransaction, Task>>()), Times.Once); is called, but this should be Func<IAsyncTransaction, Task<List<Movie>>>.

Why List<Movie> and not IEnumerable<Movie>? Well, because we put our results into a List<Movie> and return that, and yes, I could call AsEnumerable but, meh, I can live with a proper List.

[Fact]
public async Task Test7_ReturnsTheMovie()
{
    GetMocks(out var driverMock, out _, out var transactionMock, out _);

    const string expectedTitle = "Foo";
    const string expectedTagline = "Bar";
    const int expectedReleased = 1900;
    var nodeMock = GetMockNode(expectedTitle, expectedTagline, expectedReleased);

    /* Same setup as for Test6 - removed to make the post a bit smaller */

    var movies = (await movieStore.GetMovie("Valid")).ToList();

    movies.Should().HaveCount(1);
    var movie = movies.First();
    movie.Title.Should().Be(expectedTitle);
    movie.Tagline.Should().Be(expectedTagline);
    movie.Released.Should().Be(expectedReleased);
}

First off, we use the same basic setup as for Test6_CallsFetchAsyncUntilFalseReturned, and this is because we’re doing roughly the same thing.

I’m using the consts to allow me to confirm the values are what I’m saying they should be.

Here we come to a slightly odd question – if the answers I got back were wrong, say, for example, Title came back as "Title" – is it the code I’m testing that is wrong, or is it the Mock? Both places could give the error.

Who tests the tests?

Anyhews, our Test7 above will fail at the moment as we don’t return the actual values, we’ve changed our ReadTransactionAsync code to be like:

await session.ReadTransactionAsync(async tx =>
{
    var cursor = await tx.RunAsync(query, new {title});
    var fetched = await cursor.FetchAsync();

    //What we're outputting
    var output = new List<Movie>();
    while (fetched)
    {
        /* Movie extraction code */

        //Add it to the output
        output.Add(movie);
        fetched = await cursor.FetchAsync();
    }

    //Return that output!
    return output;
});

But the containing method doesn’t return it:

public async Task<IEnumerable<Movie>> GetMovie(string title)
{
    const string query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";

    var session = _driver.AsyncSession();
    await session.ReadTransactionAsync(async tx =>
    {
        /* ReadTransactionCode */
    });

    await session.CloseAsync();
    return null;
}

All we need to do is capture the output, and return it:

public async Task<IEnumerable<Movie>> GetMovie(string title)
{
    const string query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";

    var session = _driver.AsyncSession();

    //Capture the output
    var results = await session.ReadTransactionAsync(async tx =>
    {
        /* ReadTransactionCode */
    });

    await session.CloseAsync();

    //Return the output
    return results;
}

Changing the code to do this, fixes that test, but breaks our very first one – that we return null when there are no movies that match the title.

Presently – it returns an Empty collection.

Is this correct? Should it be null or Empty? As we’re now returning IEnumerable – I prefer to return an Empty collection – this is because I might well end up doing something like:

foreach(var movie in await store.GetMovies("title")) { /* CODE */ }

I don’t want to have to check for null.

So, let’s change that test from:

[Fact]
public async Task Test1_ReturnsNull_WhenInvalidTitleGiven()
{
    GetMocks(out var driverMock, out _, out _, out _);
    var movieStore = new MovieStore(driverMock.Object);
    var actual = await movieStore.GetMovie("invalid");

    actual.Should().BeNull();
}

to:

[Fact]
public async Task Test1_ReturnsEmptyCollection_WhenInvalidTitleGiven()
{
    GetMocks(out var driverMock, out _, out _, out _);
    var movieStore = new MovieStore(driverMock.Object);
    var actual = await movieStore.GetMovie("invalid");

    actual.Should().BeEmpty();
}

Now all our tests pass.


I’m going to stop here for the moment as I think it’s probably long enough, and we cover quite a lot of the Driver.

I’ll do another (shorter) post going into more detail about the IAsyncSession mocking techniques, as there are some complications with the SessionConfigBuilder that we need to address.

Depending on when you read this – there may already be the code in the repo. 🙂

Using Neo4j.Driver? Now you can EXTEND it!

Some Code

Hot on the heels of Neo4jClient 4.0.0, I was doing some work with the Neo4j.Driver (the official client for Neo4j), and in doing so, I realised I was writing a lot of boiler plate code.

So I started adding extension methods to help me, and as my extension methods became more involved, I moved them to another project, and then… well… decided to release them!

TL;DR; you can get the Neo4j.Driver.Extensions package on Nuget, and the GitHub page is here.


The Problem

Let’s first look at the problem. Neo4j.Driver is quite verbose, you end up having lots of ‘magic strings’ throughout the codebase, which can lead to problems in runtime, and one of the reasons we’re using .NET is to try to avoid runtime errors when we can get compilation errors.

Let’s take a look at a ‘standard’ read query. Here we’re executing the following Cypher to get a movie with a given title.

MATCH (m:Movie)
WHERE m.title = $title
RETURN m

We MATCH a Movie based on it’s title and return it, easy. Code wise we have this:

public async Task<Movie> GetMovieByTitle(string title)
{
    var session = _driver.AsyncSession();
    var results = await session.ReadTransactionAsync(async tx =>
    {
        var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
        var cursor = await tx.RunAsync(query, new {title});
        var fetched = await cursor.FetchAsync();

        while (fetched)
        {
            var node = cursor.Current["m"].As<INode>();
            var movie = new Movie
            {
                Title = node.Properties["title"].As<string>(),
                Tagline = node.Properties["tagline"].As<string>(),
                Released = node.Properties["released"].As<int?>()
            };
            return movie;
        }

        return null;
    });

    return results;
}

We’re using transactional functions to give us future proofing should we decide to connect to a Cluster, as the function will retry the query if the current cluster member we’re connected to takes the unfortunate decision to ‘move on’.

Let’s take a closer look at the creation of the Movie object:

var node = cursor.Current["m"].As<INode>();
var movie = new Movie
{
    Title = node.Properties["title"].As<string>(),
    Tagline = node.Properties["tagline"].As<string>(),
    Released = node.Properties["released"].As<int?>()
};
return movie;

In our first line, we pull the Current IRecord from the IResultCursor, by an identifier, and attempt to get it as an INode. This is ok, I know in my query that’s what I’ve asked for (RETURN m). Then I proceed to go through the properties in my Movie – assigning the properties from the node into the right place.

For the string properties (title and tagline) it’s just an As<string>() call – which works fine, as if the property isn’t there, we just get null anyway. The released property is more complex, to make the code simpler – I’ve used .As<int?>() (nullable int) as I happen to know the default movies database does have some nodes without the released property – as we’re schema-free here.

What I could have done would be:

var released = node.Properties["released"].As<int?>();
if(released.HasValue)
    Released = released.Value;
else { /* ?!?!? */ }

Which would make my Movie class slightly tighter – but I guess, if my data can have nodes without the property – then my models should too… 🙂

GetValue

Aaaaanyways. The first step in the extension methods was to create a ‘GetValue’ method:

var node = cursor.Current["m"].As<INode>();
var movie = new Movie
{
    Title = node.GetValue<string>("title"),
    Tagline = node.GetValue<string>("tagline"),
    Released = node.GetValue<int?>("released")
};
return movie;

This method will return default if the Properties property doesn’t contain a given key, otherwise it will try to call As<T> with the associated Exceptions (FormatException and InvalidCastException) that can take place.

I mean. If that was it, you’d be right to say that this was a waste of time. But, it does simplify the call a bit… onwards!

GetContent (NetStandard 2.1 only)

I’d like it to be a bit simpler, so let’s try to sort out the while loop. Remember we have this:

var cursor = await tx.RunAsync(query, new {title});
var fetched = await cursor.FetchAsync();

while (fetched)
{
    var node = cursor.Current["m"].As<INode>();
    /* object creation code here */
}

We FetchAsync from the cursor, then while that returns true we parse our Current into INode and then create our obj.

Instead of the Fetch/While loop, we can do this instead:

var cursor = await tx.RunAsync(query, new {title});
await foreach(var node in cursor.GetContent<INode>("m")) 
{
    /* object creation code here */
}

This removes the cursor.Current["m"].As<INode>() line, and simplifies the while into a pleasing foreach.

NB. This is NetStandard 2.1 as it uses the IAsyncEnumerable interface which isn’t in NetStandard 2.0

ToObject

OK, so we’re starting to look a bit better, some of the boiler plate is going, is there anything else we can do?

OF COURSE!

Instead of all the object creation, requiring you to go through the properties (and what if you add one later in development and forget to update this code!?!) – we can use ToObject

This works on an INode (and we’ll see later other things), and allows you to pass in a Type as a generic parameter, which will be parsed, and returned to you filled if possible:

var cursor = await tx.RunAsync(query, new {title});

await foreach (var node in cursor.GetContent<INode>("m")) 
    return node.ToObject<Movie>();

Neo4jProperty

We should probably pause here to talk about Neo4jPropertyAttribute so far, we’ve had the properties from Neo4j all Lowercase, but the observant of you will have noticed that the Movie class seems to have Upper camel case naming conventions as .NET typically does.

When we’re doing the GetValue approach – not such an issue – as we define the identifier ourselves (GetValue('title')). I think it’s probably pretty obvious that I’m going to be using Reflection here to work out what property to put where but OH NOES my properties are all Upper camel case, but the data is all Lower camel case. WHAT TO DO?

This is how a property is normally defined:

public string Title {get;set;}

Basic stuff. But we can add the Neo4jProperty attribute:

[Neo4jProperty(Name = "title")]
public string Title {get;set;}

And lo and behold, the properties will be reflected properly!
As an added extra – you can also tell the serialization process to Ignore a property if you want:

[Neo4jProperty(Ignore = true)]
public string Title {get;set;}

So the full Movie class looks like:

public class Movie
{
    [Neo4jProperty(Name = "title")]
    public string Title { get; set; }

    [Neo4jProperty(Name = "released")]
    public int? Released { get; set; }

    [Neo4jProperty(Name = "tagline")]
    public string Tagline { get; set; }
}

GetRecords with ToObject (NetStandard 2.1 only)

So – we’ve seen it works on INode, but, what if we want to return a different thing than just a node, what about, the properties? So change our Query to:

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m.title AS title, m.tagline AS tagline, m.released AS released"

Well. No worries! We’ve still got a working prospect:

await foreach (var record in cursor.GetRecords())
    return record.ToObject<Movie>();

Here, I’m using the GetRecords extension method to get each IRecord and attempt to cast it to a Movie. This works as the properties of Movie match the names of the aliases in the Cypher.

RunReadTransactionForObjects

In the previous examples, for simplification, I’ve not shown the code around the outside, but – if we take the most recent one (GetRecords example), it actually looks like this:

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
var session = _driver.AsyncSession();
var results = await session.ReadTransactionAsync(async x =>
{
    var cursor = await x.RunAsync(query, new {title});

    await foreach (var node in cursor.GetContent<INode>("m")) 
        return node.ToObject<Movie>();

    return null;
});

return results;

And we’ll do that for almost all the queries we’re going to run, so let’s look at how we can reduce that code as well…

var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
var session = _driver.AsyncSession();

var movie = 
    (await session.RunReadTransactionForObjects<Movie>(query, new {title}, "m"))
    .Single();
return movie;

I’ve added some newlines to make it a bit more readable, but now, we take the session and call RunReadTransactionForObjects<T> on it, to return the results of the query as T (in this case Movie).

The RunReadTransactionForObjects<T> method is returning an IEnumerable<T> hence I can use .Single() (or indeed any of the LINQ methods).

Put it all together

There are other extension methods in there, and some which are no doubt missing (PR away!) but I’m quite pleased that I can get from this code:

public async Task<Movie> GetMovieByTitle(string title)
{
    var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
    var session = _driver.AsyncSession();
    var results = await session.ReadTransactionAsync(async tx =>
    {        
        var cursor = await tx.RunAsync(query, new {title});
        var fetched = await cursor.FetchAsync();

        while (fetched)
        {
            var node = cursor.Current["m"].As<INode>();
            var movie = new Movie
            {
                Title = node.Properties["title"].As<string>(),
                Tagline = node.Properties["tagline"].As<string>(),
                Released = node.Properties["released"].As<int?>()
            };
            return movie;
        }

        return null;
    });

    return results;
}

To:

public async Task<Movie> GetMovieByTitle(string title)
{
    var query = "MATCH (m:Movie) WHERE m.title = $title RETURN m";
    var session = _driver.AsyncSession();

    var results = 
        (await session.RunReadTransactionForObjects<Movie>(query, new {title}, "m"))
        .Single();
    return results;
}

PHEW!

Long post eh?

If you got this far – well done! Please, try it, log bug reports (on GitHub– comments on here are easy to miss).