Sentiment Analysis of Movie Reviews

A step-by-step guide to building a model that classifies text (e.g., sentiment analysis) using a feed-forward neural network.

Libraries Used

This pipeline uses a few key Python libraries. Here is a quick overview of their roles:

• tensorflow: The core deep learning framework for building the model.
• numpy: Used for numerical operations, especially for handling arrays of text data.
• matplotlib: A plotting library for visualizing the training and validation accuracy/loss.

1. Data Loading and Preprocessing

• What the code does: It loads the popular IMDB movie review dataset, which is pre-tokenized. It then pads the sequences to a uniform length, which is a necessary step for feeding data into a neural network.
• Behind the scenes: Neural networks require numerical inputs of a fixed size. The IMDB dataset is already tokenized into integers, but the reviews have different lengths. Padding adds zeros to the end of the shorter reviews to make them all the same length.
• Abstraction: `tf.keras.datasets.imdb.load_data` abstracts the entire process of downloading and tokenizing the dataset. `tf.keras.preprocessing.sequence.pad_sequences` handles the logic of padding the sequences, saving us from implementing this manually.

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.sequence import pad_sequences
import matplotlib.pyplot as plt

# Define the number of words to consider in our vocabulary
num_words = 10000 
max_len = 256 # The maximum length of each movie review

# Load the IMDB dataset
# The data is pre-tokenized into integers
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.imdb.load_data(num_words=num_words)

# Pad the sequences to a fixed length
X_train_padded = pad_sequences(X_train, maxlen=max_len, padding='post', truncating='post')
X_test_padded = pad_sequences(X_test, maxlen=max_len, padding='post', truncating='post')

print(f"Shape of padded training data: {X_train_padded.shape}")
print(f"Shape of padded testing data: {X_test_padded.shape}")

2. Building the Model

• What the code does: It constructs a simple Sequential model. The architecture consists of an `Embedding` layer, a `Flatten` layer, a `Dense` hidden layer, and a final `Dense` output layer.
• Behind the scenes: The `Embedding` layer is crucial for NLP. It converts each integer token into a dense vector of a fixed size. This is how the model learns a meaningful representation of each word. The `Flatten` layer then prepares these vectors for the standard dense neural network layers that follow.
• Abstraction: The `Embedding` layer abstracts the creation of dense vector representations for each word, which would be a complex process to implement from scratch. The `Sequential` API makes it easy to stack these layers together to form a complete network.

# Define the embedding dimension
embedding_dim = 16

model = models.Sequential()
# The Embedding layer learns a vector representation for each word
model.add(layers.Embedding(input_dim=num_words, output_dim=embedding_dim, input_length=max_len))
# Flatten the output of the embedding layer
model.add(layers.Flatten())
# Add a dense hidden layer
model.add(layers.Dense(32, activation='relu'))
# Final dense layer with a sigmoid activation for binary classification
model.add(layers.Dense(1, activation='sigmoid'))

model.summary()

3. Training and Evaluation

• What the code does: It compiles the model using the `adam` optimizer and `binary_crossentropy` loss, and then trains it on the preprocessed data.
• Behind the scenes: `binary_crossentropy` is the standard loss function for binary classification problems (like positive vs. negative sentiment). The `adam` optimizer is a common and effective choice for training. The `model.fit` method handles the training loop, including backpropagation and weight updates.
• Abstraction: The `model.compile` and `model.fit` functions abstract the entire training process, including the forward pass, loss calculation, backward pass, and weight updates. The `history` object also abstracts the collection of performance metrics (like accuracy and loss) over each training epoch.

# Compile the model
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Train the model
epochs = 10
history = model.fit(X_train_padded, y_train,
                    epochs=epochs,
                    batch_size=32,
                    validation_data=(X_test_padded, y_test),
                    verbose=1)

# Evaluate the model on the test set
loss, accuracy = model.evaluate(X_test_padded, y_test, verbose=0)
print(f"Test Loss: {loss:.4f}")
print(f"Test Accuracy: {accuracy:.4f}")

# Optional: Visualize the training progress
acc = history.history['accuracy']
val_acc = history.history['val_accuracy']
loss = history.history['loss']
val_loss = history.history['val_loss']

epochs_range = range(epochs)

plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()