Naive Bayes Text Classifier

Multinomial Naive Bayes is a fast, surprisingly strong baseline for text classification such as spam detection or ticket routing.

Implement a class NaiveBayes(alpha=1.0):

• fit(docs, labels): tokenise each document by lowercasing and splitting on whitespace. Learn:
  • the class prior $P(c)$ = fraction of documents with label $c$;
  • word likelihoods with Laplace smoothing: $P(w \mid c) = \dfrac{\text{count}(w, c) + \alpha}{\text{total words in } c + \alpha \cdot |V|}$, where $|V|$ is the vocabulary size across all training documents.
  • Return self.
• predict(doc): return the class maximising $\log P(c) + \sum_{w} \log P(w \mid c)$ over the document's words, counting repeats. Ignore words not in the vocabulary. Break ties by the class that sorts first.

Work in log space so long documents don't underflow.