Raccoon Or Raccoon Dog?

For the search engine we'll use DuckDuckGo via the duckduckgo-search package (from pypi) and its ddg_images function..

# python
from functools import partial
from pathlib import Path
from time import sleep

import os, warnings

# pypi
from duckduckgo_search import ddg_images
from dotenv import load_dotenv

import torch

# fastai
from fastai.data.all import (
    CategoryBlock,
    DataBlock,
    parent_label,
    RandomSplitter,
)
from fastai.vision.all import (
    download_images,
    get_image_files,
    ImageBlock,
    Resize,
    resnet18,
    resize_images,
    verify_images,
    vision_learner,
    error_rate,
    PILImage,
)

from fastcore.net import urlsave

# monkey shines
from graeae import Timer

TIMER = Timer()
load_dotenv()

DATA_PATH = Path(os.environ["FASTAI_DATA"])/"raccoon-vs-tanuki"
assert torch.cuda.is_available()

Note: The DATA_PATH is where we're going to store the images we download. We are going to use a function (parent_label) that uses the folders within this directory to label the images within the folders (so images in a folder named "herbert" will be labeled "herbert"). This means that it should only have the folders that we are going to use to build the model. I originally set it to the fastai root root data path which then made the data loader think that all the other data folders were labels as well, so I created a sub-folder named "raccoon-vs-tanuki" to isolate the images I need to train the model.

We'll start by checking that our searcher is working using the keywords "tanuki" and "racoon". First, what does ddg_images return when we search for tanuki?

o = ddg_images("tanuki", type_image="photo", max_results=1)
print(o)

[{'title': 'Tanuki | Animal Jam Fanon Wiki | Fandom', 'image': 'https://vignette.wikia.nocookie.net/ajfanideas/images/6/6a/God_damnit.png/revision/latest?cb=20190222141158', 'thumbnail': 'https://tse3.mm.bing.net/th?id=OIP.74LPltCuN75QxFq2RHLhywHaFj&pid=Api', 'url': 'https://ajfanideas.fandom.com/wiki/Tanuki', 'height': 1200, 'width': 1600, 'source': 'Bing'}]

So it looks like it returns a list of json/dict objects. I'll print it out in a table to maybe make it easier to see. First, the title contains 'pipes' that break the table so I'll replace them with dashes.

o[0]["title"] = o[0]["title"].replace("|", ",")

Now the table.

print("|Key | Value|")
print("|-+-|")
for key, value in o[0].items():
    print(f"|{key}| {value}|")

Key	Value
title	Tanuki , Animal Jam Fanon Wiki , Fandom
image
thumbnail	https://tse3.mm.bing.net/th?id=OIP.74LPltCuN75QxFq2RHLhywHaFj&pid=Api
url	https://ajfanideas.fandom.com/wiki/Tanuki
height	1200
width	1600
source	Bing

Looking at the image URL you might not guess that it was an image of a tanuki (is the tanuki named "God damnit"?), but the title suggests that it is. Interestingly, if you follow the URL to the page where the image comes from you'll see that it's a wiki dedicated to a game called "Animal Jam" but the author of the page says that they couldn't find an image of the tanuki from the game so it is, indeed, a photo of a real tanuki, not a game character.

That's the output of ddg_images but we created get_image_urls to make it a little simpler to get just the URLs so let's search for "tanuki" images again but this time I'm going to download and show the image so I'll specify that I want to pull the image from the Public Domain.

TANUKI_URLS = get_image_urls("tanuki", max_images=1, license_image="Public")
print(TANUKI_URLS[0])

https://c.pxhere.com/photos/04/ff/animal_marten_raccoon_dog_tanuki_enok_obstfuchs_omnivore_fur-705793.jpg!d

The images are usually pretty big so let's download a thumbnail of the image and take a look at it to make sure we're getting the image we expect. The original fastai notebook uses the fastai download_url function (which appears to come from another fastai project called fastdownload (github, documentation)) but, it looks like all this function is doing is starting a progress bar (which I can't use here) and then calling urlsave from another fastai library called fastcore (documentation) so I'll use urlsave instead.

THUMBS = get_image_urls("tanuki", max_images=1, license_image="Public", key="thumbnail")
TANUKI_OUTPUT = "/tmp/tanuki_thumb.jpg"
urlsave(url=THUMBS[0], dest=TANUKI_OUTPUT)

And Here's the thumbnail we downloaded.

Seems to work. One disadvantage to using the get_image_urls to alias the ddg_images function is that we end up throwing away the other information, so to get the source URL to see the page where the image comes from we have to make another function call.

print(get_image_urls("tanuki", max_images=1, license_image="Public", key="url")[0])

https://pxhere.com/en/photo/705793

This image comes from pxhere.com which appears to be a public domain image hosting site.

Now for the raccoon.

RACCOON_OUTPUT = "/tmp/raccoon_thumb.jpg"
RACCOON_URLS = get_image_urls("raccoon",
                             max_images=1,
                             license_image="Public", key="thumbnail")

urlsave(url=RACCOON_URLS[0],
        dest=RACCOON_OUTPUT)

print(get_image_urls("raccoon",
                     max_images=1,
                     license_image="Public", key="url")[0])

http://www.publicdomainpictures.net/view-image.php?image=33712&picture=raccoon-4&large=1

Now that we've done a little check of what our function does we can move on to creating our dataset using it. When you download an archived dataset from fastai it saves it to the ~/.fastai/ directory, so I'll put this dataset there too. I'll use fastai's download_images function to do the actual downloading.

print(download_images.__doc__)

Download images listed in text file `url_file` to path `dest`, at most `max_pics`

These are the arguments it takes.

We'll add two extra keywords - "sun" and "shade" to the search to hopefully get images that match those conditions and between each search query I'll put in a sleep so that we aren't hitting the server too hard. We'll also use fastai's resize_images to make sure that none of the images are too big. The argument max_size gives the maximum number of pixels either dimension (height or width) can have.

def download_and_resize(destination: Path, search_terms: str, max_size: int=400) -> None:
    """Download images and resize them

    Args:
     destination: path to parent folder
     search_terms: keywords to use to search for images
     max_size: maximum size for height and width of images
    """
    download_images(
        dest=destination,
        urls=get_image_urls(SEARCH_TERMS)
    )

    resize_images(path=destination,
                  max_size=max_size,
                  dest=destination)
    return

The path argument is the source of the images and the dest is where you want to put the resized images. Normally I don't suppose you'd want to remove the original images, but in this case I do so they're set to the same folder.

ANIMALS = ("tanuki", "raccoon")

PAUSE = 10
PAUSE_BETWEEN_SEARCHES = partial(sleep, PAUSE)
CONDITIONS = tuple(("", "sun ", "shade "))

print(f"Estimated Run Time: {len(CONDITIONS) * len(ANIMALS) * PAUSE + 15} seconds")

with TIMER:
    print("Searching for:")
    for animal in ANIMALS:
        destination = DATA_PATH/animal
        destination.mkdir(exist_ok=True, parents=True)

        for condition in CONDITIONS:
            SEARCH_TERMS = f"{animal} {condition}"
            print(f" - '{SEARCH_TERMS}'")
            download_and_resize(destination, SEARCH_TERMS)
            PAUSE_BETWEEN_SEARCHES()

Estimated Run Time: 75 seconds
Started: 2022-12-08 18:24:00.371278
Searching for:
 - 'tanuki '
 - 'tanuki sun '
 - 'tanuki shade '
 - 'raccoon '
 - 'raccoon sun '
 - 'raccoon shade '
Ended: 2022-12-08 18:27:07.360951
Elapsed: 0:03:06.989673

Some of the images might be invalid for whatever reason, we'll use a fastai builtin function (verify_images) to check them and Path.unlnk to delete the files that were deemed invalid. verify_images works by trying to open each file as an image. It adds some parallelism to speed it up but isn't doing anything fancy and, depending on how many files you have and their size, might take a little while to cmoplete.

with TIMER:
    failed = verify_images(get_image_files(DATA_PATH))
    failed.map(Path.unlink)
print(f"{len(failed)} images were deemed failures.")

Started: 2022-12-08 18:28:17.277347
Ended: 2022-12-08 18:28:22.806488
Elapsed: 0:00:05.529141
18 images were deemed failures.

Let's look at the output of the learner.predict method when we pass the model the picture of a raccoon dog that we looked at when we were looking at the duckduckgo search example.

TANUKI_PATH = "/tmp/tanuki_image.jpg"
urlsave(url=TANUKI_URLS[0], dest=TANUKI_PATH)
prediction, probability_index, probabilities = predict(TANUKI_PATH)

This is a raccoon.
Probability it's a raccoon: 0.99

RACCOON_PATH = "/tmp/raccoon_image.jpg"
urlsave(url=RACCOON_URLS[0], dest=RACCOON_PATH)
prediction, probability_index, probabilities = predict(RACCOON_PATH)

This is a raccoon.
Probability it's a raccoon: 1.00

It's really sure this is a raccoon.