Advanced Spam SMS Detection

A step-by-step guide to building an advanced SMS spam detection model using word embeddings and an RNN.

Libraries Used

This pipeline uses a few key Python libraries. Here is a quick overview of their roles:

• tensorflow: The core deep learning framework for building the RNN model.
• numpy: Used for numerical operations.
• pandas: For data loading and manipulation from a CSV file.
• nltk: For text cleaning tasks like tokenization and stop word removal.

1. Data Loading and Preprocessing

• What the code does: It loads a sample spam SMS dataset, cleans the text by removing stop words and punctuation, and then tokenizes the text into integer sequences. These sequences are then padded to a uniform length.
• Behind the scenes: Advanced models need dense vector representations of words. This step prepares the text by converting it into a numerical format that can be fed into an Embedding layer, which will learn these dense representations.
• Abstraction: The `Tokenizer` from Keras abstracts the process of creating a vocabulary and converting text to sequences of integers. `pad_sequences` handles the necessary padding to ensure all inputs have the same length.

import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from sklearn.model_selection import train_test_split
from nltk.corpus import stopwords
import string
import os

# --- Dummy Data Generation ---
# In a real scenario, you would load a dataset like 'spam.csv'
data = {'label': ['ham', 'spam', 'ham', 'spam', 'ham'],
        'text': ['Hi there, how are you?',
                 'WINNER! You have won a FREE lottery ticket.',
                 'Meeting at 3pm today.',
                 'FREE MONEY! Claim your prize now!',
                 'Lunch tomorrow?']}
df = pd.DataFrame(data)
df['label'] = df['label'].map({'ham': 0, 'spam': 1})

# Text preprocessing
def preprocess_text(text):
    text = text.lower()
    text = ''.join([char for char in text if char not in string.punctuation])
    stop_words = set(stopwords.words('english'))
    words = text.split()
    words = [word for word in words if word not in stop_words]
    return ' '.join(words)

df['text'] = df['text'].apply(preprocess_text)

# Tokenize and pad sequences
max_words = 1000
max_len = 100
tokenizer = Tokenizer(num_words=max_words, oov_token="")
tokenizer.fit_on_texts(df['text'])
sequences = tokenizer.texts_to_sequences(df['text'])
padded_sequences = pad_sequences(sequences, maxlen=max_len, padding='post', truncating='post')

X = padded_sequences
y = df['label'].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Shape of padded training data: {X_train.shape}")
print(f"Shape of padded testing data: {X_test.shape}")

2. Building the RNN Model

• What the code does: It constructs a `Sequential` model with an `Embedding` layer, a `GRU` (a type of RNN) layer, and a final `Dense` layer for binary classification.
• Behind the scenes: The `Embedding` layer converts the integer sequences into dense vector representations. The `GRU` layer then processes these sequences, capturing the temporal dependencies and context of the words, which is crucial for distinguishing between "ham" and "spam."
• Abstraction: The `Embedding` and `GRU` layers in Keras abstract the complex inner workings of creating word vectors and processing sequences, allowing you to build a powerful model with just a few lines of code.

embedding_dim = 64

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(max_words, embedding_dim, input_length=max_len),
    tf.keras.layers.GRU(64, return_sequences=False),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dense(1, activation='sigmoid')
])

model.summary()

3. Training and Evaluation

• What the code does: It compiles the model with an optimizer and the appropriate loss function for binary classification, and then trains the model on the prepared data.
• Behind the scenes: The model learns to adjust its weights to minimize the `binary_crossentropy` loss, effectively learning to classify SMS messages as spam or not-spam. The GRU layer's internal state is updated at each step of the sequence, allowing it to "remember" important context.
• Abstraction: `model.compile` and `model.fit` handle the entire training loop, including the forward pass, backpropagation, and weight updates. The `metrics=['accuracy']` parameter provides a high-level view of the model's performance without requiring manual calculation.

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Train the model
epochs = 5
history = model.fit(X_train, y_train,
                    epochs=epochs,
                    batch_size=32,
                    validation_data=(X_test, y_test),
                    verbose=1)

# Evaluate the model on the test set
loss, accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"Test Loss: {loss:.4f}")
print(f"Test Accuracy: {accuracy:.4f}")