Math 248 Final Project (2023); Building a predictive machine learning model to estimate image locations

Why? Big picture:

1. Use images to track down criminals
2. Disaster response/affected area identification
3. Travel/tourism; identifying landmarks, monuments, or historical sites from images
4. Historical research/archaeological identification
5. Military strategy

Analytical approach to location identification?

Unable to obtain an exact solution through analytical methods as there are many variables involved.

But, general digital image processing can be done through Singular Value Decomposition, Fourier analysis, or other analytical methods which can extract/classify important features from images.

These can then be used to inform a model of some of the subtle but complex features of an image that can help to make an accurate prediction possible.

Numerical approach to location identification?

Machine learning model trained on large datasets of geotagged images. More complex models could even take advantage of feature-based matching and begin to classify things like cars, trees, plants, architecture, etc. and then give us a precise area in which the picture could have been taken.However, this is also a mixed approach as it requires multiple analytical functions that allow the model to interpret input data and return a meaningful output

Requires extremely high computing capabilities, memory, and massive datasets along with very complex, finely tuned ML models.

Combination of Analytical, Numerical, and Programmatic methods in the current project:

Data collection

In order to collect a large set of geotagged images I used an online image-based location guessing game called Geoguessr.

Then, using pyautogui and a script that captures the coordinates of each location, data collection was able to be automated through a program.

Each round was screenshotted and saved along with the corresponding coordinates and then the program would move on to the next round.

import time
import os
import pyautogui
import pyperclip
import webbrowser
import numpy as np
import cv2


ROUNDS_PER_GAME = 5
NUM_GAMES = 50


OUTPUT_DIR = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\ScrapScreenshots"
GEOGUESSR_URL = "https://www.geoguessr.com/maps/59a1514f17631e74145b6f47/play"


LOAD_TIME = 3
PROCESS_TIME = 2
WAIT_TIME = 2

webbrowser.open_new_tab(GEOGUESSR_URL)


time.sleep(LOAD_TIME)
time.sleep(1)


rounds_played = 0

games_played = 0


while games_played < NUM_GAMES:
    while rounds_played < ROUNDS_PER_GAME:
        time.sleep(LOAD_TIME)
        screenshot = pyautogui.screenshot()

        output_path = os.path.join(OUTPUT_DIR, f"game_{games_played}_round_{rounds_played}.png")
        screenshot.save(output_path)

        pyautogui.hotkey("ctrl", "r")
        time.sleep(LOAD_TIME)

        pyautogui.hotkey("shift", "alt", "g")

        time.sleep(1) #e
        
        pyautogui.hotkey("ctrl", "l")
        
        time.sleep(.3) #e
    
        pyautogui.hotkey("ctrl", "c")
        
        time.sleep(.3) #e
        
        pyautogui.hotkey("ctrl", "w")
        
        
        url = pyperclip.paste()

        coordinates = url.split("/")[-1]

        latitude, longitude = coordinates.split(",")

        #print(f"Latitude: {latitude}, Longitude: {longitude}")

        # Open the file in write mode
        with open(r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Coordinates\ScrapCoordinates.txt", "a") as f:
            f.write(f"Latitude: {latitude}, Longitude: {longitude}\n")
        
        
 

        # Make a random guess
        time.sleep(1)
        pyautogui.click(2000,1500)
        time.sleep(1)
        #pyautogui.click(1800,900)
        
        import random
        import pyautogui

        coords_list = [(1400,800), (1420,900), (1590,900), (1579,777), (1620,790), 
                      (1785,860), (1800,900), (1890,988), (1876,823), (1637,970), 
                      (1440,985), (1609,734), (1778,772)]

        # Randomly select one of the coordinates from the list
        x, y = random.choice(coords_list)

        # Click on the selected coordinate
        pyautogui.click(x, y)


      


        time.sleep(1)
        pyautogui.click(1600,1550)
    
        

        time.sleep(2)
        pyautogui.press('space')

        time.sleep(PROCESS_TIME)

        pyautogui.keyDown("space")
        pyautogui.keyUp("space")

        rounds_played += 1

        time.sleep(WAIT_TIME)

    pyautogui.keyDown("space")
    pyautogui.keyUp("space")

    rounds_played = 0

    games_played += 1

    time.sleep(WAIT_TIME)

Feature collection

Analytical approach: Images could have their singular value decomposition matrix computed in order to reduce the size of images while retaining much of the information.

import matplotlib.pyplot as plt
# Let's read the image file. It will be saved as 3 dimensional array, one layer for each channel.
picture = plt.imread("geoguessr.png")
# Let's see how python saves this image file
print('Type of the image : ' , type(picture))
print(f'Shape of the image : {picture.shape}')
print(f'Image Height {picture.shape[0]}')
print(f'Image Width {picture.shape[1]}')
print(f'Dimension of Image {picture.ndim}')
# Let's display the image
plt.imshow(picture)

Type of the image :  <class 'numpy.ndarray'>
Shape of the image : (1440, 2412, 4)
Image Height 1440
Image Width 2412
Dimension of Image 3

<matplotlib.image.AxesImage at 0x2aa00c1e8e0>

import matplotlib.cm as cm
R=picture[:,:,0] # red channel

# Choose a channel and perform singular value decomposition. Do not print the matrices
# since they are very big, just show their image
# Find the singular value decomposition of R
U,sigma,Vt=np.linalg.svd(R)
# store sigma in a diagonal matrix that has the same shape as R
Sigma=np.zeros(R.shape) # Sigma has the same shape as R
m=np.amin(R.shape) # pick the smaller number between the number of rows and columns 
Sigma[0:m,0:m]=np.diag(sigma) # place the singular values on the diagnal of Sigma
# Let's visualize the above product
# split the figure into 4 subplots
fig, subs=plt.subplots(nrows = 1, ncols=4, figsize=(10,5))
subs[0].imshow(R,cmap=cm.Greys_r)
subs[0].set_title('R')
subs[1].imshow(U,cmap=cm.Greys_r)
subs[1].set_title('U')
subs[2].imshow(Sigma,cmap=cm.Greys_r)
subs[2].set_title('Sigma')
subs[3].imshow(Vt,cmap=cm.Greys_r)
subs[3].set_title('Vt')

Text(0.5, 1.0, 'Vt')

Then we could save each of these compressed matrix representation for each image and have them as input into specific nodes, however, this would have greatly increased the computational complexity required in training the model. Adding any more layers reliably caused training to take too long or it just drained memory and killed the kernel.

Also computing the SVD matrix for each image in the dataset wasn't very appealing.

So instead just the raw pixel values were fed into the models.

Model training

Picking the best model architecture is largely a process of trial-and-error along with taking into account what features would work best for that specific project and ones computing capabilities.

Architecture decisions

Activation function - Allows the model to learn complex patterns of input and connect these with certain outputs when some predefined function threshold is satisfied. Many of the models attempted used a sigmoid activation function.

A sigmoid activation function gives an output between 0 and 1 for all x and is defined by: $$S(x) = \frac{1}{1+e^{-x}}$$

Loss function - How model performance gets defined, quantifies the difference between actual output and predicted output. Mean squared error was the main loss function used: $$MSE = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y_i})^2$$

Optimizer - Which inputs to consider and how to iteratively minimize loss over the models training. The Adam optimization algorithm was used in each model.

adam = lambda x: (x ** 3)-(3 *(x ** 2))+7
x = np.linspace(-1,3,500)

plt.plot(x,adam(x))
plt.show()

import sympy as sp
function = lambda x: (x**3) - (3 *(x ** 2))+7


#def adam_optimizer(x_new, x_prev, precision, l_r, beta1, beta2, epsilon):
def adam_optimizer(x_new=0.5, x_prev=0, precision=0.001, l_r=0.6, beta1=0.9,beta2= 0.99,epsilon= 10**-8):
    
    x_list, y_list = [x_new], [function(x_new)]
    
    vd_x = 0
    sd_x = 0
    
    t=1
    
    while abs(x_new - x_prev) > precision:
        
        x_prev = x_new
        
        d_x = -sp.diff(x_prev)
        
        vd_x = beta1 * vd_x + ((1-beta1) * d_x)
        
        sd_x = beta2 * sd_x + ((1-beta2) * np.square(d_x))
        
        vd_x = vd_x / (1-(beta1)**t)
        sd_x = sd_x / (1-(beta2)**t)
        
        nd_x = vd_x / np.sqrt(sd_x + epsilon)
        
        x_new = x_prev + (l_r * nd_x)
        
        x_list.append(x_new)
        
        y_list.append(function(x_new))
        
        t+=1
        
adam_optimizer()
print("Local minimum occurs at: "+ str(x_new))
print("Number of steps: " + str(len(x_list)))

plt.subplot(1,2,2)
plt.scatter(x_list, y_list, c="g")
plt.plot(x_list,y_list, c="g")
plt.plot(x,function(x), c="r")
plt.title("Adam Optimizer")
plt.show()

plt.subplot(1,2,1)
plt.scatter(x_list,y_list,c="g")
plt.plot(x_list,y_list,c="g")
plt.plot(x,function(x),c="r")
plt.xlim([1.0,2.1])
plt.title("Zoomed in Adam to Key Area")
plt.show()

Takes an initial x value and iteratively updates it based on the Adam optimization algorithm until the minimum x value that satisfies the function is reached.

Model 1

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D

reducedheight=624
reducedwidth=416
reduced2height=312
reduced2width=208

input_layer = tf.keras.layers.Input(shape=(reduced2height, reduced2width, 3))
x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(input_layer)
x = tf.keras.layers.MaxPooling2D((2, 2))(x)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(2, activation='sigmoid')(x)
output_layer = x

model = tf.keras.models.Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='mean_squared_error')

# Paths
images_folder = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\2ReducedScreenshots4"
coordinates_file = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Coordinates\Coordinates4.txt"

# Lists for images & coordinates
X_train = []
y_train = []

# Loop through coordinates file and load images and coordinates
with open(coordinates_file, 'r') as f:
    for i, line in enumerate(f):
        lat, lon = map(float, line.split(','))
        y_train.append([lat, lon])
        img_path = f"{images_folder}/{i+1}.png"
        img = tf.keras.preprocessing.image.load_img(img_path, target_size=(reduced2height, reduced2width))
        img_arr = tf.keras.preprocessing.image.img_to_array(img)
        X_train.append(img_arr)
        
y_train = np.array(y_train)
X_train = np.array(X_train) / 255.0

# Training
model.fit(X_train, y_train, epochs=150, batch_size=106)

# Save 
model.save('model1.h5')

Epoch 1/150
1/1 [==============================] - 4s 4s/step - loss: 3816.4590
Epoch 2/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 3/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 4/150
1/1 [==============================] - 2s 2s/step - loss: 3799.2859
Epoch 5/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 6/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 7/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2866
Epoch 8/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 9/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 10/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 11/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2866
Epoch 12/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2866
Epoch 13/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2866
Epoch 14/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 15/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 16/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 17/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 18/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 19/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 20/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 21/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 22/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 23/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 24/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 25/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 26/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 27/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 28/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 29/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 30/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 31/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 32/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 33/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 34/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 35/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 36/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 37/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 38/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 39/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 40/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 41/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 42/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 43/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2859
Epoch 44/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 45/150
1/1 [==============================] - 1s 1s/step - loss: 3799.2864
Epoch 46/150

Model 2

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D

reducedheight=624
reducedwidth=416

input_layer = tf.keras.layers.Input(shape=(reducedheight, reducedwidth, 3))
x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(input_layer)
x = tf.keras.layers.MaxPooling2D((2, 2))(x)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(2, activation='linear')(x)
output_layer = x

model = tf.keras.models.Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='mean_absolute_error')

# set the path to the folder containing the images
images_folder = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\ReducedScreenshots4"

# set the path to the coordinates file
coordinates_file = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Coordinates\Coordinates4.txt"

# initialize lists to store the image data and the corresponding coordinates
X_train = []
y_train = []

# loop through the coordinates file and load the images and coordinates
with open(coordinates_file, 'r') as f:
    for i, line in enumerate(f):
        lat, lon = map(float, line.split(','))
        y_train.append([lat, lon])
        img_path = f"{images_folder}/{i+1}.png"
        img = tf.keras.preprocessing.image.load_img(img_path, target_size=(reducedheight, reducedwidth))
        img_arr = tf.keras.preprocessing.image.img_to_array(img)
        X_train.append(img_arr)
        
y_train = np.array(y_train)
X_train = np.array(X_train) / 255.0

# Train your model
model.fit(X_train, y_train, epochs=10, batch_size=106)

# Save your trained model
model.save('secondmodel.h5')

Epoch 1/10
1/1 [==============================] - 91s 91s/step - loss: 47.8657
Epoch 2/10
1/1 [==============================] - 59s 59s/step - loss: 206.9561
Epoch 3/10
1/1 [==============================] - 65s 65s/step - loss: 96.9743
Epoch 4/10
1/1 [==============================] - 66s 66s/step - loss: 61.9084
Epoch 5/10
1/1 [==============================] - 67s 67s/step - loss: 102.4971
Epoch 6/10
1/1 [==============================] - 66s 66s/step - loss: 88.0211
Epoch 7/10
1/1 [==============================] - 67s 67s/step - loss: 55.5881
Epoch 8/10
1/1 [==============================] - 66s 66s/step - loss: 44.4580
Epoch 9/10
1/1 [==============================] - 67s 67s/step - loss: 62.6375
Epoch 10/10
1/1 [==============================] - 67s 67s/step - loss: 66.0519

Model ?

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D

reducedheight=624
reducedwidth=416
reduced2height=312
reduced2width=208

input_layer = tf.keras.layers.Input(shape=(reduced2height, reduced2width, 3))
x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(input_layer)
x = tf.keras.layers.MaxPooling2D((2, 2))(x)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dense(2, activation='linear')(x)
output_layer = x

model = tf.keras.models.Model(inputs=input_layer, outputs=output_layer)
model.compile(optimizer='adam', loss='mean_absolute_error')


images_folder = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\2ReducedScreenshots4"
coordinates_file = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Coordinates\Coordinates4.txt"


X_train = []
y_train = []


with open(coordinates_file, 'r') as f:
    for i, line in enumerate(f):
        lat, lon = map(float, line.split(','))
        y_train.append([lat, lon])
        img_path = f"{images_folder}/{i+1}.png"
        img = tf.keras.preprocessing.image.load_img(img_path, target_size=(reduced2height, reduced2width))
        img_arr = tf.keras.preprocessing.image.img_to_array(img)
        X_train.append(img_arr)
        
y_train = np.array(y_train)
X_train = np.array(X_train) / 255.0


model.fit(X_train, y_train, epochs=250, batch_size=106)


model.save('9bmodel.h5')

Epoch 1/250
1/1 [==============================] - 2s 2s/step - loss: 47.8796
Epoch 2/250
1/1 [==============================] - 1s 1s/step - loss: 50.7596
Epoch 3/250
1/1 [==============================] - 1s 1s/step - loss: 44.8550
Epoch 4/250
1/1 [==============================] - 1s 1s/step - loss: 44.0209
Epoch 5/250
1/1 [==============================] - 1s 1s/step - loss: 45.6251
Epoch 6/250
1/1 [==============================] - 1s 1s/step - loss: 45.6070
Epoch 7/250
1/1 [==============================] - 1s 1s/step - loss: 44.1454
Epoch 8/250
1/1 [==============================] - 1s 1s/step - loss: 43.1977
Epoch 9/250
1/1 [==============================] - 1s 1s/step - loss: 43.0630
Epoch 10/250
1/1 [==============================] - 1s 1s/step - loss: 43.2931
Epoch 11/250
1/1 [==============================] - 1s 1s/step - loss: 43.4820
Epoch 12/250
1/1 [==============================] - 1s 1s/step - loss: 42.8589
Epoch 13/250
1/1 [==============================] - 1s 1s/step - loss: 42.1419
Epoch 14/250
1/1 [==============================] - 1s 1s/step - loss: 41.5516
Epoch 15/250
1/1 [==============================] - 1s 1s/step - loss: 41.2088
Epoch 16/250
1/1 [==============================] - 1s 1s/step - loss: 40.9894
Epoch 17/250
1/1 [==============================] - 1s 1s/step - loss: 40.6004
Epoch 18/250
1/1 [==============================] - 1s 1s/step - loss: 39.9710
Epoch 19/250
1/1 [==============================] - 1s 1s/step - loss: 39.8112
Epoch 20/250
1/1 [==============================] - 1s 1s/step - loss: 39.3099
Epoch 21/250
1/1 [==============================] - 1s 1s/step - loss: 38.8843
Epoch 22/250
1/1 [==============================] - 1s 1s/step - loss: 38.6357
Epoch 23/250
1/1 [==============================] - 1s 1s/step - loss: 38.0674
Epoch 24/250
1/1 [==============================] - 1s 1s/step - loss: 37.8486
Epoch 25/250
1/1 [==============================] - 1s 1s/step - loss: 37.3307
Epoch 26/250
1/1 [==============================] - 1s 1s/step - loss: 37.0721
Epoch 27/250
1/1 [==============================] - 1s 1s/step - loss: 36.6593
Epoch 28/250
1/1 [==============================] - 1s 1s/step - loss: 36.2210
Epoch 29/250
1/1 [==============================] - 1s 1s/step - loss: 36.1351
Epoch 30/250
1/1 [==============================] - 1s 1s/step - loss: 36.0248
Epoch 31/250
1/1 [==============================] - 1s 1s/step - loss: 35.5917
Epoch 32/250
1/1 [==============================] - 1s 1s/step - loss: 35.3932
Epoch 33/250
1/1 [==============================] - 1s 1s/step - loss: 34.7562
Epoch 34/250
1/1 [==============================] - 1s 1s/step - loss: 34.8373
Epoch 35/250
1/1 [==============================] - 1s 1s/step - loss: 34.1313
Epoch 36/250
1/1 [==============================] - 1s 1s/step - loss: 34.0866
Epoch 37/250
1/1 [==============================] - 1s 1s/step - loss: 33.6807
Epoch 38/250
1/1 [==============================] - 1s 1s/step - loss: 33.2089
Epoch 39/250
1/1 [==============================] - 1s 1s/step - loss: 33.0222
Epoch 40/250
1/1 [==============================] - 1s 1s/step - loss: 32.4434
Epoch 41/250
1/1 [==============================] - 1s 1s/step - loss: 32.4833
Epoch 42/250
1/1 [==============================] - 1s 1s/step - loss: 31.8353
Epoch 43/250
1/1 [==============================] - 1s 1s/step - loss: 32.4152
Epoch 44/250
1/1 [==============================] - 1s 1s/step - loss: 32.4978
Epoch 45/250
1/1 [==============================] - 1s 1s/step - loss: 31.8730
Epoch 46/250
1/1 [==============================] - 1s 1s/step - loss: 33.1622
Epoch 47/250
1/1 [==============================] - 1s 1s/step - loss: 31.8217
Epoch 48/250
1/1 [==============================] - 1s 1s/step - loss: 31.7596
Epoch 49/250
1/1 [==============================] - 1s 1s/step - loss: 31.4781
Epoch 50/250
1/1 [==============================] - 2s 2s/step - loss: 30.1977
Epoch 51/250
1/1 [==============================] - 1s 1s/step - loss: 31.3086
Epoch 52/250
1/1 [==============================] - 1s 1s/step - loss: 30.6575
Epoch 53/250
1/1 [==============================] - 1s 1s/step - loss: 30.2680
Epoch 54/250
1/1 [==============================] - 1s 1s/step - loss: 29.3512
Epoch 55/250
1/1 [==============================] - 1s 1s/step - loss: 30.4134
Epoch 56/250
1/1 [==============================] - 1s 1s/step - loss: 30.1490
Epoch 57/250
1/1 [==============================] - 1s 1s/step - loss: 28.4511
Epoch 58/250
1/1 [==============================] - 1s 1s/step - loss: 29.0918
Epoch 59/250
1/1 [==============================] - 1s 1s/step - loss: 28.4195
Epoch 60/250
1/1 [==============================] - 1s 1s/step - loss: 27.9689
Epoch 61/250
1/1 [==============================] - 1s 1s/step - loss: 28.3027
Epoch 62/250
1/1 [==============================] - 1s 1s/step - loss: 28.1919
Epoch 63/250
1/1 [==============================] - 1s 1s/step - loss: 27.7883
Epoch 64/250
1/1 [==============================] - 1s 1s/step - loss: 27.0944
Epoch 65/250
1/1 [==============================] - 1s 1s/step - loss: 27.9872
Epoch 66/250
1/1 [==============================] - 1s 1s/step - loss: 26.4471
Epoch 67/250
1/1 [==============================] - 1s 1s/step - loss: 27.2331
Epoch 68/250
1/1 [==============================] - 1s 1s/step - loss: 27.4740
Epoch 69/250
1/1 [==============================] - 1s 1s/step - loss: 25.9115
Epoch 70/250
1/1 [==============================] - 1s 1s/step - loss: 27.0197
Epoch 71/250
1/1 [==============================] - 1s 1s/step - loss: 26.3466
Epoch 72/250
1/1 [==============================] - 1s 1s/step - loss: 25.7100
Epoch 73/250
1/1 [==============================] - 1s 1s/step - loss: 27.0924
Epoch 74/250
1/1 [==============================] - 1s 1s/step - loss: 25.8658
Epoch 75/250
1/1 [==============================] - 1s 1s/step - loss: 25.5012
Epoch 76/250
1/1 [==============================] - 1s 1s/step - loss: 24.9663
Epoch 77/250
1/1 [==============================] - 1s 1s/step - loss: 24.2341
Epoch 78/250
1/1 [==============================] - 1s 1s/step - loss: 25.3283
Epoch 79/250
1/1 [==============================] - 1s 1s/step - loss: 25.0195
Epoch 80/250
1/1 [==============================] - 1s 1s/step - loss: 24.4320
Epoch 81/250
1/1 [==============================] - 1s 1s/step - loss: 23.4963
Epoch 82/250
1/1 [==============================] - 1s 1s/step - loss: 24.0489
Epoch 83/250
1/1 [==============================] - 1s 1s/step - loss: 25.3733
Epoch 84/250
1/1 [==============================] - 1s 1s/step - loss: 22.7593
Epoch 85/250
1/1 [==============================] - 1s 1s/step - loss: 24.4719
Epoch 86/250
1/1 [==============================] - 1s 1s/step - loss: 23.1589
Epoch 87/250
1/1 [==============================] - 1s 1s/step - loss: 24.3848
Epoch 88/250
1/1 [==============================] - 1s 1s/step - loss: 23.2592
Epoch 89/250
1/1 [==============================] - 1s 1s/step - loss: 22.4096
Epoch 90/250
1/1 [==============================] - 1s 1s/step - loss: 23.5773
Epoch 91/250
1/1 [==============================] - 1s 1s/step - loss: 23.1194
Epoch 92/250
1/1 [==============================] - 1s 1s/step - loss: 21.9507
Epoch 93/250
1/1 [==============================] - 1s 1s/step - loss: 24.0753
Epoch 94/250
1/1 [==============================] - 1s 1s/step - loss: 21.5673
Epoch 95/250
1/1 [==============================] - 1s 1s/step - loss: 21.0657
Epoch 96/250
1/1 [==============================] - 1s 1s/step - loss: 22.5837
Epoch 97/250
1/1 [==============================] - 1s 1s/step - loss: 21.5843
Epoch 98/250
1/1 [==============================] - 1s 1s/step - loss: 20.6949
Epoch 99/250
1/1 [==============================] - 1s 1s/step - loss: 22.9338
Epoch 100/250
1/1 [==============================] - 1s 1s/step - loss: 20.6623
Epoch 101/250
1/1 [==============================] - 1s 1s/step - loss: 20.4262
Epoch 102/250
1/1 [==============================] - 1s 1s/step - loss: 20.1143
Epoch 103/250
1/1 [==============================] - 1s 1s/step - loss: 21.8979
Epoch 104/250
1/1 [==============================] - 1s 1s/step - loss: 19.6504
Epoch 105/250
1/1 [==============================] - 1s 1s/step - loss: 20.0143
Epoch 106/250
1/1 [==============================] - 1s 1s/step - loss: 20.9091
Epoch 107/250
1/1 [==============================] - 1s 1s/step - loss: 20.0024
Epoch 108/250
1/1 [==============================] - 1s 1s/step - loss: 20.8162
Epoch 109/250
1/1 [==============================] - 1s 1s/step - loss: 19.0240
Epoch 110/250
1/1 [==============================] - 1s 1s/step - loss: 20.9618
Epoch 111/250
1/1 [==============================] - 1s 1s/step - loss: 19.5081
Epoch 112/250
1/1 [==============================] - 1s 1s/step - loss: 20.3560
Epoch 113/250
1/1 [==============================] - 1s 1s/step - loss: 18.2747
Epoch 114/250
1/1 [==============================] - 1s 1s/step - loss: 18.9250
Epoch 115/250
1/1 [==============================] - 1s 1s/step - loss: 20.3445
Epoch 116/250
1/1 [==============================] - 1s 1s/step - loss: 19.6638
Epoch 117/250
1/1 [==============================] - 1s 1s/step - loss: 18.4023
Epoch 118/250
1/1 [==============================] - 1s 1s/step - loss: 21.6483
Epoch 119/250
1/1 [==============================] - 1s 1s/step - loss: 19.5740
Epoch 120/250
1/1 [==============================] - 1s 1s/step - loss: 23.0128
Epoch 121/250
1/1 [==============================] - 1s 1s/step - loss: 22.7240
Epoch 122/250
1/1 [==============================] - 1s 1s/step - loss: 20.8968
Epoch 123/250
1/1 [==============================] - 1s 1s/step - loss: 24.2487
Epoch 124/250
1/1 [==============================] - 1s 1s/step - loss: 26.8437
Epoch 125/250
1/1 [==============================] - 1s 1s/step - loss: 22.5640
Epoch 126/250
1/1 [==============================] - 1s 1s/step - loss: 23.9679
Epoch 127/250
1/1 [==============================] - 1s 1s/step - loss: 26.3539
Epoch 128/250
1/1 [==============================] - 1s 1s/step - loss: 25.1717
Epoch 129/250
1/1 [==============================] - 1s 1s/step - loss: 23.2745
Epoch 130/250
1/1 [==============================] - 1s 1s/step - loss: 22.8669
Epoch 131/250
1/1 [==============================] - 1s 1s/step - loss: 22.2165
Epoch 132/250
1/1 [==============================] - 1s 1s/step - loss: 23.3095
Epoch 133/250
1/1 [==============================] - 1s 1s/step - loss: 19.6610
Epoch 134/250
1/1 [==============================] - 1s 1s/step - loss: 20.8925
Epoch 135/250
1/1 [==============================] - 1s 1s/step - loss: 23.1063
Epoch 136/250
1/1 [==============================] - 1s 1s/step - loss: 18.1124
Epoch 137/250
1/1 [==============================] - 1s 1s/step - loss: 21.1634
Epoch 138/250
1/1 [==============================] - 1s 1s/step - loss: 22.2659
Epoch 139/250
1/1 [==============================] - 1s 1s/step - loss: 19.8411
Epoch 140/250
1/1 [==============================] - 1s 1s/step - loss: 18.4647
Epoch 141/250
1/1 [==============================] - 1s 1s/step - loss: 18.6412
Epoch 142/250
1/1 [==============================] - 1s 1s/step - loss: 18.2165
Epoch 143/250
1/1 [==============================] - 1s 1s/step - loss: 16.9282
Epoch 144/250
1/1 [==============================] - 1s 1s/step - loss: 17.7990
Epoch 145/250
1/1 [==============================] - 1s 1s/step - loss: 16.3455
Epoch 146/250
1/1 [==============================] - 1s 1s/step - loss: 17.3285
Epoch 147/250
1/1 [==============================] - 1s 1s/step - loss: 16.7343
Epoch 148/250
1/1 [==============================] - 1s 1s/step - loss: 16.7389
Epoch 149/250
1/1 [==============================] - 1s 1s/step - loss: 17.4412
Epoch 150/250
1/1 [==============================] - 1s 1s/step - loss: 15.4228
Epoch 151/250
1/1 [==============================] - 1s 1s/step - loss: 17.2138
Epoch 152/250
1/1 [==============================] - 1s 1s/step - loss: 16.6765
Epoch 153/250
1/1 [==============================] - 1s 1s/step - loss: 16.5052
Epoch 154/250
1/1 [==============================] - 1s 1s/step - loss: 16.9876
Epoch 155/250
1/1 [==============================] - 1s 1s/step - loss: 15.3521
Epoch 156/250
1/1 [==============================] - 1s 1s/step - loss: 17.7502
Epoch 157/250
1/1 [==============================] - 1s 1s/step - loss: 16.6037
Epoch 158/250
1/1 [==============================] - 1s 1s/step - loss: 16.2105
Epoch 159/250
1/1 [==============================] - 1s 1s/step - loss: 16.2873
Epoch 160/250
1/1 [==============================] - 1s 1s/step - loss: 14.5265
Epoch 161/250
1/1 [==============================] - 1s 1s/step - loss: 15.7861
Epoch 162/250
1/1 [==============================] - 1s 1s/step - loss: 15.4989
Epoch 163/250
1/1 [==============================] - 1s 1s/step - loss: 15.7612
Epoch 164/250
1/1 [==============================] - 1s 1s/step - loss: 15.4552
Epoch 165/250
1/1 [==============================] - 1s 1s/step - loss: 14.5955
Epoch 166/250
1/1 [==============================] - 1s 1s/step - loss: 15.8952
Epoch 167/250
1/1 [==============================] - 1s 1s/step - loss: 14.4517
Epoch 168/250
1/1 [==============================] - 1s 1s/step - loss: 14.9291
Epoch 169/250
1/1 [==============================] - 1s 1s/step - loss: 14.1119
Epoch 170/250
1/1 [==============================] - 1s 1s/step - loss: 14.7515
Epoch 171/250
1/1 [==============================] - 1s 1s/step - loss: 14.5801
Epoch 172/250
1/1 [==============================] - 1s 1s/step - loss: 13.9979
Epoch 173/250
1/1 [==============================] - 1s 1s/step - loss: 14.1730
Epoch 174/250
1/1 [==============================] - 1s 1s/step - loss: 14.1033
Epoch 175/250
1/1 [==============================] - 1s 1s/step - loss: 14.5011
Epoch 176/250
1/1 [==============================] - 1s 1s/step - loss: 13.4942
Epoch 177/250
1/1 [==============================] - 1s 1s/step - loss: 13.8387
Epoch 178/250
1/1 [==============================] - 1s 1s/step - loss: 13.5401
Epoch 179/250
1/1 [==============================] - 1s 1s/step - loss: 14.1932
Epoch 180/250
1/1 [==============================] - 1s 1s/step - loss: 13.2535
Epoch 181/250
1/1 [==============================] - 1s 1s/step - loss: 12.7654
Epoch 182/250
1/1 [==============================] - 1s 1s/step - loss: 13.6487
Epoch 183/250
1/1 [==============================] - 1s 1s/step - loss: 13.7149
Epoch 184/250
1/1 [==============================] - 1s 1s/step - loss: 12.7656
Epoch 185/250
1/1 [==============================] - 1s 1s/step - loss: 13.2535
Epoch 186/250
1/1 [==============================] - 1s 1s/step - loss: 13.6148
Epoch 187/250
1/1 [==============================] - 1s 1s/step - loss: 12.9784
Epoch 188/250
1/1 [==============================] - 1s 1s/step - loss: 15.8797
Epoch 189/250
1/1 [==============================] - 1s 1s/step - loss: 14.0448
Epoch 190/250
1/1 [==============================] - 1s 1s/step - loss: 14.6393
Epoch 191/250
1/1 [==============================] - 1s 1s/step - loss: 15.7126
Epoch 192/250
1/1 [==============================] - 1s 1s/step - loss: 13.3680
Epoch 193/250
1/1 [==============================] - 1s 1s/step - loss: 16.6963
Epoch 194/250
1/1 [==============================] - 1s 1s/step - loss: 17.5479
Epoch 195/250
1/1 [==============================] - 1s 1s/step - loss: 11.9767
Epoch 196/250
1/1 [==============================] - 1s 1s/step - loss: 16.2771
Epoch 197/250
1/1 [==============================] - 1s 1s/step - loss: 16.3960
Epoch 198/250
1/1 [==============================] - 1s 1s/step - loss: 12.0919
Epoch 199/250
1/1 [==============================] - 1s 1s/step - loss: 18.1310
Epoch 200/250
1/1 [==============================] - 1s 1s/step - loss: 20.1208
Epoch 201/250
1/1 [==============================] - 1s 1s/step - loss: 14.8812
Epoch 202/250
1/1 [==============================] - 1s 1s/step - loss: 16.7356
Epoch 203/250
1/1 [==============================] - 1s 1s/step - loss: 18.8710
Epoch 204/250
1/1 [==============================] - 1s 1s/step - loss: 17.4078
Epoch 205/250
1/1 [==============================] - 1s 1s/step - loss: 12.6090
Epoch 206/250

1/1 [==============================] - 1s 1s/step - loss: 17.8885
Epoch 207/250
1/1 [==============================] - 1s 1s/step - loss: 17.8507
Epoch 208/250
1/1 [==============================] - 1s 1s/step - loss: 13.5359
Epoch 209/250
1/1 [==============================] - 1s 1s/step - loss: 17.2445
Epoch 210/250
1/1 [==============================] - 1s 1s/step - loss: 18.9892
Epoch 211/250
1/1 [==============================] - 1s 1s/step - loss: 18.0703
Epoch 212/250
1/1 [==============================] - 1s 1s/step - loss: 12.3952
Epoch 213/250
1/1 [==============================] - 1s 1s/step - loss: 17.9644
Epoch 214/250
1/1 [==============================] - 1s 1s/step - loss: 19.5275
Epoch 215/250
1/1 [==============================] - 1s 1s/step - loss: 18.6053
Epoch 216/250

Testing how the model does

import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense, Flatten, Conv2D, MaxPooling2D

# Load the trained model
model = tf.keras.models.load_model('9bmodel.h5')
norm=120
images_folder = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Model9ScreenshotsValidation"
X_test = []
for i in range(1, 6):
    img_path = f"{images_folder}/{i}.png"
    img = tf.keras.preprocessing.image.load_img(img_path, target_size=(312,208))
    img_arr = tf.keras.preprocessing.image.img_to_array(img)
    X_test.append(img_arr)
X_test = np.array(X_test)

# Predictions on the new data
y_pred = model.predict(X_test)/norm

# Predicted coordinates
predictions=print(y_pred[0:5])
predictions

1/1 [==============================] - 0s 213ms/step
[[ 48.1873    83.85427 ]
 [ 15.641254 -92.640236]
 [ 10.250204  29.397371]
 [ 71.2296    42.37426 ]
 [ -9.368075  61.64105 ]]

import folium

def plot_coordinates_on_map(file_location, additional_coordinates=None):
    # Load coordinates from file
    coordinates = []
    with open(file_location, 'r') as f:
        for line in f:
            lat, lon = map(float, line.split(','))
            coordinates.append((lat, lon))

    # Create map centered on the first coordinate
    map_center = coordinates[0]
    my_map = folium.Map(location=map_center, zoom_start=6)

    # Add markers for each coordinate
    for i, coord in enumerate(coordinates):
        folium.Marker(location=coord, icon=None).add_to(my_map)
        folium.Marker(
            location=coord,
            icon=None,
            # Display the point number as a text label
            popup=str(i),
            # Add a text label offset to make it visible
            tooltip=f'Point {i}',
            ).add_to(my_map)
        
        # Add a dotted line between the corresponding coordinates from both lists
        if additional_coordinates and i < len(additional_coordinates):
            folium.PolyLine([coord, additional_coordinates[i]], color="red", dash_array='5').add_to(my_map)

    # Add markers for guess
    if additional_coordinates:
        for i, coord in enumerate(additional_coordinates):
            folium.Marker(location=coord, icon=folium.Icon(color='red')).add_to(my_map)
            folium.Marker(
                location=coord,
                icon=None).add_to(my_map)
    
    return my_map

file_location = r"C:\Users\ttdog\OneDrive\Desktop\Training Data\Coordinates\CoordinatesValidation.txt"
additional_coordinates = [[48.1873, 83.85427], [15.641254, -92.640236], [10.250204, 29.397371], [71.2296, 42.37426], [-9.368075, 61.64105]]
my_map = plot_coordinates_on_map(file_location, additional_coordinates)
my_map

Make this Notebook Trusted to load map: File -> Trust Notebook

What limited these models from accurate predictions?

Lack of compute power

1. Data collection involving frequent crashes--> Extremely small dataset(106 image/coordinate pairs)
2. Simple architecture/low number of nodes in the model--> Not effectively capturing important features

Lack of understanding of complexities of network architecture--> Slow improvement, underfitting, overfitting, etc.

Could this actually work?

One approach:

1. Locations/images classified by country instead of coordinates

Over 50,000 training images

Performs better than the average person- able to guess the correct country 50% of the time