Geospatial Digital Informatics Lab

This tutorial series provides a comprehensive introduction to image processing and machine learning through three interconnected parts. In Part 1, students utilize Python and the PIL library to merge multiple images into a single row, establishing foundational image manipulation skills that can be applied to NASA GLOBE Observer’s four directional landscape images to create panoramic views for enhanced environmental analysis. Part 2 guides students through training an image classification model using Google’s Teachable Machine platform, demonstrating how to create custom AI models without extensive coding knowledge. Part 3 advances the workflow by teaching students how to adapt their Teachable Machine models for use in Google Colab, including techniques for processing multiple images and saving classification results for further analysis.

Lab Part 1: Merging Images with Python and PIL (Python Imaging Library) 

 

Objective: By the end of this lab, you will be able to use Python and the Python Imaging Library (PIL) to merge four images into a single row.

 

Step 1: Import Required Libraries 

Firstly, we will need to import the PIL library and Google Colab’s drive module. The PIL library will be used to work with images, and the drive module will allow us to access files in our Google Drive.

from PIL import Image

from google.colab import drive

 

Step 2: Mount Google Drive 

 

Before we can load the images from Google Drive, we need to mount it into the Colab environment. Running this code will prompt you to authorize Colab to access your Google Drive. Follow the link, authorize access, copy the provided code, paste it into the input box that appears in your Colab notebook, and finally hit enter to complete the mounting process.

 

drive.mount(‘/content/drive’)

Step 3: Load the Images 

Next, we will load the images that we wish to merge. Please ensure that all the images are of the same size for best results. You will need to replace ‘/content/drive/My Drive/image1.jpg’, etc., with the paths of your actual images.

image1 = Image.open(‘/content/drive/My Drive/image1.jpg’)

image2 = Image.open(‘/content/drive/My Drive/image2.jpg’)

image3 = Image.open(‘/content/drive/My Drive/image3.jpg’)

image4 = Image.open(‘/content/drive/My Drive/image4.jpg’)

Step 4: Determine the Dimensions for the Merged Image 

Our objective is to merge the images into a 1×4 grid (a single row with four columns). Therefore, we need to create a new image with dimensions that can hold all four images. Its width will be four times the width of an individual image, and its height will be equal to the height of an individual image.

# Get the dimensions of the first image

width, height = image1.size

 

# Create a new, empty image with appropriate size

combined_image = Image.new(‘RGB’, (4 * width, height))

Step 5: Paste the Images into the Merged Image 

Now that we have our new image, we will paste each image into the appropriate location.

# Paste the images into the combined image

combined_image.paste(image1, (0, 0))

combined_image.paste(image2, (width, 0))

combined_image.paste(image3, (2 * width, 0))

combined_image.paste(image4, (3 * width, 0))

Step 6: Save the Merged Image 

Finally, we will save our newly created image. You can specify the name and the format of the image (here, we are saving it as a JPEG).

# Save the new image

combined_image.save(‘/content/drive/My Drive/combined_image.jpg’)

Conclusion 

You have successfully created a new image by merging four images into a single row. This is a powerful technique that can be used in various applications, such as creating collages or data augmentation for deep learning. Keep exploring the PIL library, as it provides a wide range of other functionalities that can help you manipulate and process images.

Full Code of Part 1: https://colab.research.google.com/drive/1Ff9Q3mXn_Adrz9xMo_30-upiD5SBo5zV?usp=sharing 

 

Lab Part 2: Training the AI Model with Teachable Machine

Materials Needed:

1. Computer with internet access
2. Google Account
3. Organized image data

Instructions:

Begin by launching your preferred web browser. Navigate to the Teachable Machine website, which can be found at the following URL: https://teachablemachine.withgoogle.com/.

Upon reaching the homepage of Teachable Machine, locate and click on the “Get Started” button. This will lead you to a new page where you are to select the “Image Project” button. This action initiates a new project specifically tailored for image classification.

The workspace you are now presented with is divided into classes. Each class represents a unique category that the AI model will be trained to recognize. By default, you are provided with two classes, aptly named Class 1 and Class 2. You are encouraged to rename these classes to reflect the land cover types you are working with (e.g., “Forest”, “Water”, “Urban”, “Agriculture”). To upload the corresponding images for each class, click on the “Upload” button located in each class box. This step is to be repeated for each land cover type.

Once you have successfully uploaded the images, proceed by clicking on the “Train Model” button, conveniently located at the top of the page. Teachable Machine will now utilize the images you have provided to train the AI model. Please note that the duration of this process is dependent on the volume of data and the complexity of the model, and may take a few minutes.

Example screenshot:

Upon completion of the training process, you are now able to test the model. This can be achieved by using the webcam or by uploading a new image. The model will then provide a prediction of the land cover type, based on the training it has undergone.

 If the performance of the model is not up to your satisfaction, you have the option to tune it. This can be achieved by adjusting the sliders located under the “Advanced” section in the “Train Model” tab. Here, you can modify parameters such as the learning rate, epochs, and batch size. After making the necessary adjustments, click “Train Model” again to apply these changes.

When you are satisfied with the performance of your model, you can proceed to export it for use in other applications. To do this, click on the “Export Model” button at the top of the page and follow the instructions to either download or host your model.

 

 

Full Code for Part 2: https://colab.research.google.com/drive/1bSH4sEzCPFmMmtNKrnmTfxbnZVTPfBFf?usp=sharing 

 

Lab Tutorial 3: Adapting a Teachable Machine Model for Google Colab

 

In this lab, we’ll dive into the exciting world of machine learning, using a model trained with Teachable Machine. Teachable Machine is an intuitive, web-based tool by Google that allows users to create machine learning models without any prior coding experience. But what if we want to take these models and apply them outside of the Teachable Machine environment? Well, that’s what we’re going to explore today.

 

Step 1: Import Libraries

 

Firstly, let’s import some necessary libraries. We’ll need Keras for handling our model, PIL (Python Imaging Library) for image manipulation, and numpy for numerical operations:

from keras.models import load_model  

from PIL import Image, ImageOps  

import numpy as np  

Next, for clarity’s sake, we’ll disable numpy’s scientific notation. This will ensure that when we print our results, they will be in a format that is easier to interpret:

np.set_printoptions(suppress=True)

Step 2: Google Drive Integration

Here’s where we take a departure from the original script. The original script was written assuming that the model and associated files were present in the same directory as the script. However, when working in a Google Colab environment, we generally store our files in Google Drive.

So, let’s mount our Google Drive to the Colab notebook so we can access those files:

from google.colab import drive

drive.mount(‘/content/drive’)

This code will prompt you to authorize Google Colab to access your Google Drive. You’ll be given a URL to visit where you can allow this access, and then you’ll be given a code to paste back into the Colab notebook.

 

Step 3: Load Model and Labels

 

Now, we need to load our model and the associated labels. Since we’re working in Google Colab and have mounted our Google Drive, we adjust the path to point to where the files are stored in the Drive:

model = load_model(“/content/drive/My Drive/converted_keras/keras_model.h5”, compile=False)

class_names = open(“/content/drive/My Drive/converted_keras/labels.txt”, “r”).readlines()

 

Note: Be sure to replace the paths with the actual paths to your model and labels in your Google Drive.

Step 4: Image Preprocessing

 

Before we can feed an image into our model for prediction, it needs to be preprocessed.

Just like in the original code, we’ll first create an empty numpy array of the right shape:

 

data = np.ndarray(shape=(1, 224, 224, 3), dtype=np.float32)

And then we open an image using PIL and convert it to RGB:

image = Image.open(“/content/drive/My Drive/wetransfer_land-cover-activity-images_2022-05-13_2246/test-land-cover-activity-images/test/test-20.jpg”).convert(“RGB”)

This is another area where we’ve deviated from the original script. Instead of specifying the image path directly, we’ve altered the path to pull the image from a location in our Google Drive.

 

Step 5: Make a Prediction

 

The steps for making a prediction remain the same as in the original script. However, bear in mind that, depending on the number of classes you have in your model, the class names will be stored in the class_names list that was loaded from labels.txt earlier.

Also, it’s important to note that the class names are loaded as string with leading and trailing whitespaces, so when printing the predicted class, we slice it as class_name[2:] to remove the leading whitespace and the newline character.

# Predicts the model

prediction = model.predict(data)

index = np.argmax(prediction)

class_name = class_names[index]

confidence_score = prediction[0][index]

 

# Print prediction and confidence score

print(“Class:”, class_name[2:], end=””)

print(“Confidence Score:”, confidence_score)

That’s it! Now you know how to adapt a Teachable Machine model for use in Google Colab. This powerful combination allows you to take advantage of the simplicity of Teachable Machine for training models, with the flexibility of Google Colab for applying those models. Happy coding!

 

Step 6: Processing Multiple Images and Saving Results

 

Our machine-learning model is currently only capable of processing one image at a time. However, for large-scale projects, we’ll need our model to process entire directories of images. In this step, we’ll upgrade our code to process all images in a directory and save the results to a table for further analysis.

 

6.1: Define the Image Processing Function

 

First, we’ll define a function to handle the image preprocessing and prediction steps that we previously walked through. This function, process_image, takes an image path as input, processes the image, feeds it into our model, and returns the predicted class and confidence score:

 

def process_image(image_path):

    # Create the array of the right shape to feed into the keras model

    data = np.ndarray(shape=(1, 224, 224, 3), dtype=np.float32)

 

    # Load and format the image

    image = Image.open(image_path).convert(“RGB”)

    size = (224, 224)

    image = ImageOps.fit(image, size, 1)

 

    # Turn the image into a numpy array and normalize it

    image_array = np.asarray(image)

    normalized_image_array = (image_array.astype(np.float32) / 127.5) – 1

 

    # Load the image into the array and make a prediction

    data[0] = normalized_image_array

    prediction = model.predict(data)

    index = np.argmax(prediction)

    class_name = class_names[index]

    confidence_score = prediction[0][index]

 

    return class_name, confidence_score

 

6.2: Initialize a Result Dataframe

 

To store our results, we’ll use a Pandas dataframe, which can be thought of as a table where each row represents a different image, and each column represents a different attribute of that image (i.e., the image name, the predicted class, and the confidence score):

 

results = pd.DataFrame(columns=[“Image”, “Class”, “Confidence Score”])

 

6.3: Process All Images in the Directory

Next, we’ll iterate over each image file in our directory, process it using our process_image function, and add the results to our dataframe:

 

dir_path = “/content/drive/My Drive/wetransfer_land-cover-activity-images_2022-05-13_2246/test-land-cover-activity-images/test”

for file in os.listdir(dir_path):

    if file.endswith(“.jpg”):  # Only process .jpg files

        image_path = os.path.join(dir_path, file)

        class_name, confidence_score = process_image(image_path)

        results = results.append({“Image”: file, “Class”: class_name, “Confidence Score”: confidence_score}, ignore_index=True)

Remember to replace dir_path with the actual path to your image directory.

 

6.4: Save Results to a CSV File

 

Finally, after processing all the images, we’ll write the results to a CSV file, which is a widely-used format for storing tabular data:

 

results.to_csv(‘/content/drive/My Drive/wetransfer_land-cover-activity-images_2022-05-13_2246/Model_Results.csv’, index=False)

 

Remember to replace the path with the actual path where you want to save your results.

That’s it! You’ve now successfully upgraded your script to process an entire directory of images and output the results to a CSV file. This will be invaluable for handling larger datasets and projects. Happy coding!