# Using a Logistic regression along with Neural Networks for Cat vs Non-Cat Image Classification

In this blog, we will be covering up the concepts of using the logistic regression along with neural networks, applying forward and backward propagation, and then applying them to the practice in order to build your image recognition system i.e a cat classifier in this case. This cat classifier takes an image as an input and then it predicts whether the image contains a cat or not with 70% accuracy and the tools used will be Jupyter Notebook and the code is written in python.

Let’s see a bit of theory about Neural networks and Logistic regression.

# How do computers interpret Images

For a computer, an image is just an array of values. Typically it’s a 3-dimensional (RGB) matrix of pixel values.

For example, a 6x 6 RGB abstract image representation would look like this.

Where each pixel has a specific value of red, green, and blue that represents the color of a given pixel. Neural networks process images using matrixes of weights called filters (features) that detect specific attributes such as vertical edges, horizontal edges, etc. Moreover, as the image progresses through each layer, the filters are able to recognize more complex attributes.

# General Architecture

Mathematical Expressions to be used:

Here L represents the Loss function

Computing cost function using the formula:

The main steps for building a Neural Network are:

1. Define the model structure (such as number of input features)
2. Initialize the model’s parameters
3. Loop:
• Calculate current loss (forward propagation)
• Calculate current gradient (backward propagation)

# Step 1: Creating a new Notebook

import numpy as np
import matplotlib.pyplot as plt
import h5py
import scipy
from PIL import Image
from scipy import ndimage

%matplotlib inline

My Github has the entire code written below: https://github.com/aditimukerjee/Cat-and-Non-cat-for-logistic-regression

`train_set_x_orig` is a NumPy-array of shape (m_train, num_px, num_px, 3)

Each image is of shape (num_px, num_px, 3) where 3 is for the 3 channels (RGB) and num_px is the height equal to the width of a training image. Thus, each image is square (height = num_px) and (width = num_px).

# Step 3: Analyzing the dataset

print (“Number of training examples: m_train = “ + str(m_train))
print (“Number of testing examples: m_test = “ + str(m_test))
print (“Height/Width of each image: num_px = “ + str(num_px))
print (“Each image is of size: (“ + str(num_px) + “, “ + str(num_px) + “, 3)”)
print (“train_set_x shape: “ + str(train_set_x_orig.shape))
print (“train_set_y shape: “ + str(train_set_y.shape))
print (“test_set_x shape: “ + str(test_set_x_orig.shape))
print (“test_set_y shape: “ + str(test_set_y.shape))

`Number of training examples: m_train = 209Number of testing examples: m_test = 50Height/Width of each image: num_px = 64Each image is of size: (64, 64, 3)train_set_x shape: (209, 64, 64, 3)train_set_y shape: (1, 209)test_set_x shape: (50, 64, 64, 3)test_set_y shape: (1, 50)`

# Step 3: Reshaping the dataset

Flattening array means converting a multidimensional array into a 1D array.

A trick when you want to flatten a matrix X of shape (a,b,c,d) to a matrix X_flatten of shape (b ∗ c ∗ d, a) is to use:

X_flatten = X.reshape(X.shape, -1).T

Removing transpose it will be
X_flatten = X.reshape(-1, X.shape) where -1 is the unspecified element of new the shape of x

Since x is originally of shape (m_train, num_px, num_px, 3) since already used x.shape aka m_train , the remaining elements in the new shape should be num_px * num_px * 3 to make sure x can be reshaped so instead of specifying num_px * num_px * 3, we specify it as -1

train_set_x_flatten = train_set_x_orig.reshape(train_set_x_orig.shape, -1)
test_set_x_flatten = test_set_x_orig.reshape(test_set_x_orig.shape, -1).T

print (“train_set_x_flatten shape: “ + str(train_set_x_flatten.shape))
print (“train_set_y shape: “ + str(train_set_y.shape))
print (“test_set_x_flatten shape: “ + str(test_set_x_flatten.shape))
print (“test_set_y shape: “ + str(test_set_y.shape))
print (“sanity check after reshaping: “ + str(train_set_x_flatten[0:5,0]))

`train_set_x_flatten shape: (12288, 209)train_set_y shape: (1, 209)test_set_x_flatten shape: (12288, 50)test_set_y shape: (1, 50)sanity check after reshaping: [17 31 56 22 33]`

To represent color images, the red, green, and blue channels (RGB) must be specified for each pixel, and so the pixel value is actually a vector of three numbers ranging from 0 to 255.

train_set_x = train_set_x_flatten/255
test_set_x = test_set_x_flatten/255

# Step 4: Sigmoid function

def sigmoid(z):
s = 1/(1+np.exp(-z))
return s

`print (“sigmoid([0, 2]) = “ + str(sigmoid(np.array([0,2]))))Output : sigmoid([0, 2])[ 0.5 0.88079708]`

# Step 4: Initiating parameters

def initialize_with_zeros(dim):
w = np.zeros((dim, 1))
b = 0

assert(w.shape == (dim, 1))
assert(isinstance(b, float) or isinstance(b, int))
return w, b

dim = 2
w, b = initialize_with_zeros(dim)
print (“w = “ + str(w))
print (“b = “ + str(b))

`Output: w= [[ 0.] [ 0.]] b=0`

# Step 5: Forward and Backward Propogation

def propagate(w, b, X, Y):
m = X.shape

# FORWARD PROPAGATION (FROM X TO COST)

A = sigmoid(np.dot(w.T, X) + b) # compute activation
cost = -1./m* np.sum(Y*np.log(A) + (1-Y)*np.log(1-A)) # compute cost

# BACKWARD PROPAGATION (TO FIND GRAD)

dw = 1./m*np.dot(X, (A-Y).T)
db = 1./m*np.sum(A-Y)

assert(dw.shape == w.shape)
assert(db.dtype == float)
#cost = np.squeeze(cost)
#assert(cost.shape == ())

“db”: db}

w, b, X, Y = np.array([[1.],[2.]]), 2., np.array([[1.,2.,-1.],[3.,4.,-3.2]]), np.array([[1,0,1]])
grads, cost = propagate(w, b, X, Y)
print (“dw = “ + str(grads[“dw”]))
print (“db = “ + str(grads[“db”]))
print (“cost = “ + str(cost))

`Output : dw=[[ 0.99845601] [ 2.39507239]] ,db=0.00145557813678 ,cost=5.801545319394553`

# Step 6: Optimization

def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost = False):
costs = []
for i in range(num_iterations):

# Cost and gradient calculation (≈ 1–4 lines of code)
grads, cost =propagate(w, b, X, Y)

# update rule (≈ 2 lines of code)
w = w — learning_rate * dw
b = b — learning_rate * db

# Record the costs
if i % 100 == 0:
costs.append(cost)

# Print the cost every 100 training examples
if print_cost and i % 100 == 0:
print (“Cost after iteration %i: %f” %(i, cost))

params = {“w”: w,
“b”: b}

“db”: db}

params, grads, costs = optimize(w, b, X, Y, num_iterations= 100, learning_rate = 0.009, print_cost = False)

print (“w = “ + str(params[“w”]))
print (“b = “ + str(params[“b”]))
print (“dw = “ + str(grads[“dw”]))
print (“db = “ + str(grads[“db”]))

`w = [[0.19033591] [0.12259159]]b = 1.9253598300845747dw = [[0.67752042] [1.41625495]]db = 0.21919450454067652`

# Step 7 : Prediction

1. Calculate Ŷ =A=σ(wTX+b)Y^=A=σ(wTX+b)
2. Convert the entries of a into 0 (if activation <= 0.5) or 1 (if activation > 0.5), stores the predictions in a vector `Y_prediction`. If you wish, you can use an `if`/`else` statement in a `for` loop (though there is also a way to vectorize this).

def predict(w, b, X):
m = X.shape
Y_prediction = np.zeros((1,m))
w = w.reshape(X.shape, 1)

# Compute vector “A” predicting the probabilities of a cat being present in the picture

A = sigmoid(np.dot(w.T, X) + b)

for i in range(A.shape):

# Convert probabilities A[0,i] to actual predictions p[0,i]

if A[0, i] > 0.5:
Y_prediction[0, i] = 1
else:
Y_prediction[0, i] = 0

assert(Y_prediction.shape == (1, m))

return Y_prediction

w = np.array([[0.1124579],[0.23106775]])
b = -0.3
X = np.array([[1.,-1.1,-3.2],[1.2,2.,0.1]])
print (“predictions = “ + str(predict(w, b, X)))

`predictions = [[1. 1. 0.]]`

# Step 8 : Merge all functions into a mode

def model(train_x, train_y_orig, test_x, test_y_orig, num_iterations = 3000, learning_rate = 0.6, print_cost = False):
w=np.zeros((train_x.shape,1))
b=0
parameters, grads, costs = optimize(w, b, train_x, train_y_orig, num_iterations, learning_rate, print_cost = False)

# Retrieve parameters w and b from dictionary “parameters”
w = parameters[“w”]
b = parameters[“b”]

# Predict test/train set examples
Y_prediction_test = predict(w, b, test_x)
Y_prediction_train = predict(w, b, train_x)

# Print train/test Errors
print(“train accuracy: {} %”.format(100 — np.mean(np.abs(Y_prediction_train — train_y_orig)) * 100))
print(“test accuracy: {} %”.format(100 — np.mean(np.abs(Y_prediction_test — test_y_orig)) * 100))
d = {“costs”: costs,
“Y_prediction_test”: Y_prediction_test,
“Y_prediction_train” : Y_prediction_train,
“w” : w,
“b” : b,
“learning_rate” : learning_rate,
“num_iterations”: num_iterations}
return d

d = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations = 5000, learning_rate = 0.8, print_cost = True)

`train accuracy: 99.99876382512535 %test accuracy: 72.01052229722973 %`

This is a cat image and the image prediction is correct.

This is a non — cat image and the image prediction is incorrect.

This is a non-cat image and the image prediction is correct.

This is a non-cat image and the image prediction is correct.

This is a non-cat image and the image prediction is correct.