A simple way to get images for your dataset

Whenever we begin a machine learning project, the first thing that we need is a dataset. Dataset will be the pillar of your training model. You can build the dataset either automatically or manually. Here I am going to share about the manual process.

What is the dataset?

Dataset is the collection of specific data for your ML project needs. The type of data depends on the kind of AI you need to train. Basically, you have two datasets:

Training
Testing

Composition at 90% and 10% respectively

Whenever you are training a custom model the important thing is images. Yes, of course, the images play a main role in deep learning. The accuracy of your model will be based on the training images. So, before you train a custom model, you need to plan how to get images? Here, I’m going to share my ideas on the easy way to get images for a dataset.

Get images from google

Yes, we can get images from Google. Using the Download All Images browser extension we can easily get images in a few minutes. You can check out here for more details about this extension! It is available on major browsers. Unfortunately, this extension is not available on the Safari browser.

Once you download images using this extension, you will see the downloaded images in a folder with random filenames. We can rename the files or remove the .png file using the below Python script.

Remove PNGs from the downloaded image folder.

import glob
import pathlib

for file in glob.glob('frames/*.png'):
  path = pathlib.Path(file)
  path.unlink()

Rename your image files

import os
os.getcwd()
collection = "images/cat"
for i, filename in enumerate(os.listdir(collection)):
  print(filename)
  os.rename(collection + "/" + filename, collection + "/cat" + str(i) + ".jpg")

Get images from videos

Here we have another way to prepare images for the Dataset. We can easily extract images from video files. Detecto gives a simple solution to get images from the video. Refer Detecto for more information.

pip3 install detecto

Using the following code we can extract images from video files.

from detecto.utils import split_video 
#split_video('video file path', 'image save path', frame size) 
split_video('images/cat.mp4', 'images/cat/', step_size=10)

Using your set of images

You can take pictures of objects which you will use to train your model. The important note is to make sure your images are not beyond 800×600. This will help your dataset train much quicker.

from PIL import Image
import os
import argparse

def rescale_images(directory, size):
  for img in os.listdir(directory):
    im = Image.open(directory+img)
    im_resized = im.resize(size, Image.ANTIALIAS)
    im_resized.save(directory+img)
 

if __name__ == '__main__':
  parser = argparse.ArgumentParser(description="Rescale images")
  parser.add_argument('-d', '--directory', type=str, required=True, help='Directory containing the images')
  parser.add_argument('-s', '--size', type=int, nargs=2, required=True, metavar=('width', 'height'), help='Image size')
  args = parser.parse_args()
  rescale_images(args.directory, args.size)
##run this command
##python convert_image_resolution.py -d images/ -s 800 600

I have prepared a video and explained about the above process. Please check out the below video blog.

Conclusion

As an ML noob, I need to figure out the best way to prepare the dataset for training a model. I hope this will be useful. My ultimate idea is to create a Python package for this process. 🙂

See you in my next article 🙂