Quick way to check if image exists on remote url in python

Question

I am using the python-requests library to fulfill my requests.

On the website’s homepage, I get a bunch of images and show them to the user. Sometimes these images are deleted and I get a broken image url.

So, I want to check if images exist.

Here is what I did:

items = Item.objects.filter(shop__is_hidden=False, is_hidden=False).order_by("?")[:16]

existing_items = []

for item in items:
    response = requests.head(item.item_low_url)
    if response.status_code == 200:
        existing_items.append(item)

But it will take a little longer than I want.

Is there a faster way?

Answer 1

Your requests are blocked and synchronous, so it takes a little time. In simple words, this means that the second request does not start until the first one finishes.

, .

; , ( , , -, , ).

, , :

• , .
• , ( , ).
• , .

# 1, ( , ).

, , # ​​2 - , , .

, :

• URL- .
• ( ).
• .

, ; , grequests:

import grequests

# Create a map between url and the item
url_to_item = {item.item_low_url: item for item in items}

# Create a request queue, but don't send them
rq = (grequests.head(url) for url in url_to_item.keys())

# Send requests simultaneously, and collect the results,
# and filter those that are valid

# Each item returned in the Response object, which has a request
# property that is the original request to which this is a response;
# we use that to filter out the item objects

results = [url_to_item[i.request.url]
           for i in filter(lambda x: x.status_code == 200,
                           grequests.map(rq)))]