Httpclient Retry

[arunnabraham]

Hello team I need help/advice regarding HTTP retry mechanism in the HTTP client, I tried in a way to create a retry, but the results coming are not stable. (I am quite new to working on async) . I would like to see a simpler implementation of this. Please let me know.

[SimonFrings]

Hey @arunnabraham, thanks for bringing this up 👍

You can achieve a retry mechanism using a similar example as this one:

<?php

require __DIR__ . '/vendor/autoload.php';

$client = new React\Http\Browser();

$promise = $client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
    // promise fullfilled
    var_dump($response->getHeaders(), (string)$response->getBody());
}, function (Exception $e) {
    // promise rejected
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

This examples shows a simple promise API. We try to send a request to "google.com/" and then define what should happen if it was successful (promise fulfilled) and what happens if something goes wrong (promise rejected). In our successful case we would just var_dump the response headers and body. If an error occurs, we would just show the error message. You can read more about promises in our documentation: https://github.com/reactphp/http#promises

We can now come back to your retry mechanism, which means we want to try again if something goes wrong, so let's change this example a bit:

<?php

require __DIR__ . '/vendor/autoload.php';

$client = new React\Http\Browser();

$promise = $client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
}, function () {
    // retry logic
});

This example just uses the rejected part for the retry logic. There are still some things you have to take into consideration e.g. how often do you retry. Be aware that this $client->get() method doesn't return the response itself, but a promise containing the response. You can use the await() function from our async component to return whatever the promise resolved to.

$result = React\Async\await($promise);

We're also currently working on v3 for ReactPHP with the goal to make everything easier, so there will be some simpler implementations in the future. I hope this helps 👍

[arunnabraham]

Thanks @SimonFrings I never tried retry logic with reject function, one of them I think I may never able to catch the response object (please correct me if I am wrong). I tried with option with $client->withRejectErrorResponse(false), where I could implement the retry and easly filtering the status code to enable the retry function.

Now my issue was I was unable to get the response 200 result even if I tried.

But I would expect maybe as an improvement like you said like below implementation :)

https://laravel.com/docs/9.x/http-client#retries

[SimonFrings]

I had something like this in mind if you want to retry at least one time:

$client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
}, function () use ($client) {
    $client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
        var_dump($response->getHeaders(), (string)$response->getBody());
    }, function () {
        // next retry or error
    });
});

[arunnabraham]

Thanks, @SimonFrings. I will test this and let you know. BTW I will say my precise requirement, for now, is to handle 500, and later may focus on delay and retry for 429. And currently, I am experimenting on this to add 3 retries with maybe a recursion or loop.

I will look into this idea and may come up with a solution :). BTW just to know, these things are nowadays much needed especially when dealing with 500 or 429s. and I am also eagerly waiting for a simpler abstraction your team might implement on this.

And as of now I may need help even on delay, especially in the case of 429s. My project includes 3 to 4 APIs to handle millions of data working together to have a final merge result. Anyways thanks for your advice once again.

[arunnabraham]

Hi @SimonFrings I was testing today with an implementation of retry with delay. Please let me know your thoughts. It is just a prototype. And it works for the logic I was looking for.

https://github.com/arunnabraham/reactphp-httpclient-retry-and-delay/tree/dev

[SimonFrings]

@arunnabraham Nice, first make it work is always my motto 👍

Only thing I saw is that you're never using the $response argument in your retry() function, but as you said it's still a prototype. It's hard to say if something's wrong by just looking at the code, seems fine at a first glance. I'd recommend to write tests to assure even edge cases work as expected, but that's up to you.