Using Python decorators to retry request

Using Python decorators to retry request

if you do not mind installing a library you could use the tenacity (github.com/jd/tenacity) module. one of their examples:

import random
from tenacity import retry, stop_after_attempt

# @retry  # retry forever
@retry(stop=stop_after_attempt(3))
def do_something_unreliable():
    if random.randint(0, 10) > 1:
        raise IOError(Broken sauce, everything is hosed!!!111one)
    else:
        return Awesome sauce!

print(do_something_unreliable())

this also allows you to specify the number of tries or seconds you want to keep retrying.

for your case this might look something like this (not tested!):

@retry(stop=stop_after_attempt(3))
def retry_get():
    result = requests.get(
            url, auth=HTTPBasicAuth(COMMON_USERNAME, COMMON_PASSWORD)).json()
    if error not in result:
        raise RequestException(result)

You can use a decorator like this and handle your own exception.

def retry(times, exceptions):
    
    Retry Decorator
    Retries the wrapped function/method `times` times if the exceptions listed
    in ``exceptions`` are thrown
    :param times: The number of times to repeat the wrapped function/method
    :type times: Int
    :param Exceptions: Lists of exceptions that trigger a retry attempt
    :type Exceptions: Tuple of Exceptions
    
    def decorator(func):
        def newfn(*args, **kwargs):
            attempt = 0
            while attempt < times:
                try:
                    return func(*args, **kwargs)
                except exceptions:
                    print(
                        Exception thrown when attempting to run %s, attempt 
                        %d of %d % (func, attempt, times)
                    )
                    attempt += 1
            return func(*args, **kwargs)
        return newfn
    return decorator

@retry(times=3, exceptions=(ValueError, TypeError))
def foo1():
    print(Some code here ....)
    print(Oh no, we have exception)
    raise ValueError(Some error)

foo1()

Using Python decorators to retry request

Instead of using decorators, the probably better solution is to move the request to its own function, arriving at a structure similar to this:

no_of_retries = 3

def make_request(url):
    for i in range(0,no_of_retries):
        try:
            result = requests.get(url, auth=HTTPBasicAuth(COMMON_USERNAME, COMMON_PASSWORD)).json()
            if error not in result:
                return result
            else:
                continue
        except Exception as e:
            continue
    return result

def check_status():
    result = make_request(http://something/status)

def load_file():
    result = make_request(http://something/file)

This way, you avoid duplicate code while encapsulating the request. If you were to use a decorator, you would need to wrap the whole load_file() method which would prevent you from further processing the requests result within this function.