{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dbamman/anlp25/blob/main/2.compare/Log_odds_ratio_TODO.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Log odds-ratio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IIVpdSX0E9Tw"
   },
   "source": [
    "The log odds ratio with an informative (and uninformative) Dirichlet prior (described in [Monroe et al. 2009, Fighting Words](http://languagelog.ldc.upenn.edu/myl/Monroe.pdf)) is a common method for finding distinctive terms in two datasets (see [Jurafsky et al. 2014](https://firstmonday.org/ojs/index.php/fm/article/view/4944/3863) for an example article that uses it to make an empirical argument). This method for finding distinguishing words combines a number of desirable properties:\n",
    "\n",
    "* it specifies an intuitive metric (the log-odds) for the ratio of two probabilities\n",
    "* it can incorporate prior information in the form of pseudocounts, which can either act as a smoothing factor (in the uninformative case) or incorporate real information about the expected frequency of words overall.\n",
    "* it accounts for variability of a frequency estimate by essentially converting the log-odds to a z-score.\n",
    "\n",
    "In this homework you will implement this ratio for a dataset of your choice to characterize the words that differentiate each one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j7sezbdfE9Ty"
   },
   "source": [
    "## Part 1\n",
    "\n",
    "Your first job is to find two datasets with some interesting opposition -- e.g., news articles from CNN vs. FoxNews, books written by Charles Dickens vs. James Joyce, screenplays of dramas vs. comedies.  Be creative -- this should be driven by what interests you and should reflect your own originality. **This dataset cannot come from Kaggle**.  Feel feel to use web scraping (see [here](https://github.com/CU-ITSS/Web-Data-Scraping-S2023) for a great tutorial) or manually copying/pasting text.  Aim for more than 10,000 tokens for each dataset.\n",
    "   \n",
    "Save those datasets in two files: \"class1_dataset.txt\" and \"class2_dataset.txt\"\n",
    "\n",
    "**Describe each of those datasets and their source in 100-200 words.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type your response here:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xO4_1a4EE9Ty"
   },
   "source": [
    "## Part 2\n",
    "\n",
    "Tokenize those texts by filling out the `read_and_tokenize` function below (your choice of tokenizer). The input is a filename and the output should be a list of tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6eYQtYiCE9Tz"
   },
   "outputs": [],
   "source": [
    "def read_and_tokenize(filename: str) -> list[str]:\n",
    "    \"\"\"Read the file and output a list of strings (tokens).\"\"\"\n",
    "    # your code here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "z28xc1Y1E9Tz"
   },
   "outputs": [],
   "source": [
    "# change these file paths to wherever the datasets you created above live.\n",
    "class1_tokens = read_and_tokenize(\"class1_dataset.txt\")\n",
    "class2_tokens = read_and_tokenize(\"class2_dataset.txt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lPHj4k4tE9Tz"
   },
   "source": [
    "## Part 3\n",
    "\n",
    "Now let's find the words that characterize each of those sources (with respect to the other). Implement the log-odds ratio with an uninformative Dirichlet prior. This value, $\\widehat\\zeta_w^{(i-j)}$ for word $w$ reflecting the difference in usage between corpus $i$ and corpus $j$, is given by the following equation:\n",
    "\n",
    "$$\n",
    "\\widehat{\\zeta}_w^{(i-j)}= {\\widehat{d}_w^{(i-j)} \\over \\sqrt{\\sigma^2\\left(\\widehat{d}_w^{(i-j)}\\right)}}\n",
    "$$\n",
    "\n",
    "Where:\n",
    "\n",
    "$$\n",
    "\\widehat{d}_w^{(i-j)} = \\log \\left({y_w^i + \\alpha_w} \\over {n^i + \\alpha_0 - y_w^i - \\alpha_w}) \\right) -  \\log \\left({y_w^j + \\alpha_w} \\over {n^j + \\alpha_0 - y_w^j - \\alpha_w}) \\right)\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\sigma^2\\left(\\widehat{d}_w^{(i-j)}\\right) \\approx {1 \\over {y_w^i + \\alpha_w}} + {1 \\over {y_w^j + \\alpha_w} }\n",
    "$$\n",
    "\n",
    "And:\n",
    "\n",
    "* $y_w^i = $ count of word $w$ in corpus $i$ (likewise for $j$)\n",
    "* $\\alpha_w$ = 0.01\n",
    "* $V$ = size of vocabulary (number of distinct word types)\n",
    "* $\\alpha_0 = V * \\alpha_w$\n",
    "* $n^i = $ number of words in corpus $i$ (likewise for $j$)\n",
    "\n",
    "In this example, the two corpora are your class1 dataset (e.g., $i$ = your class1) and your class2 dataset (e.g., $j$ = class2). Using this metric, print out the 25 words most strongly aligned with class1, and 25 words most strongly aligned with class2.  Again, consult [Monroe et al. 2009, Fighting Words](http://languagelog.ldc.upenn.edu/myl/Monroe.pdf) for more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "lHWahiy8E9T0"
   },
   "outputs": [],
   "source": [
    "def logodds_with_uninformative_prior(tokens_i: list[str], tokens_j: list[str], display=25):\n",
    "    \"\"\"Print out the log odds results given two lists of tokens.\"\"\"\n",
    "    # your code here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Ddu4uK9pE9T0"
   },
   "outputs": [],
   "source": [
    "logodds_with_uninformative_prior(class1_tokens, class2_tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check your work, you can run log-odds on the party platforms from the lab section. With `nltk.word_tokenize` _before_ lower-casing, these should be your top 5 words (and scores, roughly). Depending on your tokenization strategy, your scores might be slightly different.\n",
    "\n",
    "**Democrat**:\n",
    "```\n",
    "president:\t4.75\n",
    "biden:\t4.27\n",
    "to:\t4.11\n",
    "he:\t4.09\n",
    "has:\t4.08\n",
    "```\n",
    "**Republican**\n",
    "```\n",
    "republicans:\t-13.45\n",
    "our:\t-11.23\n",
    "will:\t-10.88\n",
    "american:\t-10.01\n",
    "restore:\t-7.97\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget --no-check-certificate https://raw.githubusercontent.com/dbamman/anlp25/main/data/2024_democrat_party_platform.txt\n",
    "!wget --no-check-certificate https://raw.githubusercontent.com/dbamman/anlp25/main/data/2024_republican_party_platform.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import nltk\n",
    "logodds_with_uninformative_prior(\n",
    "    [w.lower() for w in nltk.word_tokenize(\"2024_democrat_party_platform.txt\")],\n",
    "    [w.lower() for w in nltk.word_tokenize(\"2024_republican_party_platform.txt\")]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}