Pagination testing with Guzzle Mocks

Having a lot of microservices means that writing a service client become a daily routine for me.

Writing a client for a REST API service is usually nothing too difficult. I strongly favor this over just ad-hoc calling some endpoints via Curl or Guzzle. The client can handle all error handling and retry logic and possibly other things (like API throttling). And mainly, it is fully tested. I usually write a few functional tests and then use mock tests to achieve 100% coverage. While there are many discussions about the benefits of 100% test coverage (I find this article). In case of clients I find it extremely useful to have 100% coverage and it is actually easy to achieve.

Pagination is one thing that definitely deserves code coverage, because it can really bite. So it’s something like the following piece of code for a simple offset-limit pagination.

public function listJobs(JobListOptions $listOptions): array
{
    $jobs = [];
    $i = 1;
    do {
        $request = new Request('GET', 'jobs?' . 
           implode('&', $listOptions->getQueryParameters()));
        $chunk = $this->sendRequest($request);
        $jobs = array_merge($jobs, $chunk);
        $listOptions->setOffset($i * $listOptions->getLimit());
        $i++;
    } while (count($chunk) === $listOptions->getLimit());
    return $jobs;
}

So, I write a couple of tests for that, one of them being the basic assumption that the pagination actually paginates, that would be the following code using the Guzzle Mock Handler.

public function testClientGetJobsPaging(): void
{
    $queue[] = new Response(
        200,
        ['Content-Type' => 'application/json'],
        (string) json_encode(['foo' => 'bar'])
    );
    $mock = new MockHandler($queue);
    $requestHistory = [];
    $stack = HandlerStack::create($mock);
    $stack->push($Middleware::history($requestHistory));
    $client = $this->getClient(['handler' => $stack]);
    $jobs = $client->listJobs((new JobListOptions()));
    $request = $mock->getLastRequest();
    self::assertEquals(
        'offset=1000&limit=100',
        $request->getUri()->getQuery()
    );
}

Now, the question is how to fill the response queue for the mock handler. So my first idea would be to create as many Response objects as I need pages (10 in this case), each containing as many objects as the page size is (100 in this case). So I have the two nested calls of array_fill() which fill the mock queue:

$queue = array_fill(
    0, 
    10,
    new Response(
        200,
        ['Content-Type' => 'application/json'],
        (string) json_encode(array_fill(
            0,
            100,
            ['foo' => 'bar']
        ))
    )
);

That’s cool. All the pages are the same, but I don’t care about that at all because I’m writing the client and not the server. So that’s cool.

Except that it doesn’t work at all.

If you actually make this work, then you’ll realize that every time you run it, the second page will be completely empty.

The culprit is that a new Response creates a new stream inside. If you’re very observant, you could have spotted that I have actually created only a single instance of the Response object. Therefore I have actually created only a a single stream and once that stream is used up to the end, there is no one to rewind it. So the “trick” is to actually create a new response for every page, e.g using an inline function:

$queue = array_fill(0, 10, function () use ($jobData): Response {
    return new Response(
        200,
        ['Content-Type' => 'application/json'],
        (string) json_encode(array_fill(
            0,
            100,
            $jobData
        ))
    );
});

And that’s all folks.