Threadit — Chrome Extension

A Chrome extension that scrapes a thread on X (Twitter) and summarizes it into a few bullet points.

threadit/
├── extension-frontend/   Vite + React Chrome extension (Manifest V3)
└── backend-api/          FastAPI proxy that talks to OpenAI + PostgreSQL

How it works

1. You open a thread on x.com and click the extension icon.
2. The popup asks the content script (already injected into the page) to read the visible tweets from the DOM.
3. The popup POSTs those tweets to your local FastAPI server.
4. FastAPI builds a prompt, calls OpenAI's gpt-4o-mini, and returns a bullet-point summary.
5. The popup displays it.

---

1. Run the backend

cd backend-api

# start a local Postgres instance (see "Database" section below)
docker compose up -d

python -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate
pip install -r requirements.txt

cp .env.example .env
# open .env and paste your real OpenAI key
# (DATABASE_URL already matches docker-compose.yml, no changes needed)

uvicorn main:app --reload --port 8000

Check it's alive: open http://localhost:8000/health — you should see {"status": "ok"}.

2. Build the extension

cd extension-frontend
npm install
npm run build

This produces extension-frontend/dist/ containing manifest.json, popup.html, the bundled React/JS assets, and content.js.

3. Load it into Chrome

1. Go to chrome://extensions.
2. Turn on Developer mode (top-right toggle).
3. Click Load unpacked.
4. Select the extension-frontend/dist folder.
5. Pin the extension icon for easy access.

4. Try it

1. Make sure the backend from step 1 is still running.
2. Open a thread on x.com.
3. Click the extension icon → Summarize Thread.

If you edit the React code during development, run npm run dev for hot reload of the popup UI in a regular browser tab (useful for styling), but you'll still need npm run build + reload the unpacked extension to test the real content-script/popup flow inside Chrome.

---

Database (PostgreSQL)

On top of the in-memory cache, every summary is persisted to a real PostgreSQL database via SQLAlchemy's async ORM (backend-api/db.py). This is the piece worth highlighting on a resume/report — it's not just an LLM wrapper, it has actual relational data modeling behind it.

Schema:

threads
  id, content_hash (unique), tweet_count,
  created_at, last_accessed_at, access_count
      │
      │ 1-to-many
      ▼
summaries
  id, thread_id (FK -> threads.id), summary_text, model, created_at

request_log   (separate, denormalized — see note below)
  id, content_hash, cache_hit, created_at

• threads / summaries is a standard 1-to-many relationship: a thread can be re-summarized over time without losing earlier summaries.
• request_log deliberately stores content_hash directly instead of a foreign key. It's a write-heavy audit table used only for aggregate analytics, so skipping the join keeps writes cheap — a normalization tradeoff worth mentioning if asked about it.

Request flow for /summarize:

1. Check the in-memory cache (fastest, but wiped on restart).
2. Check Postgres for a previously generated summary (slower, but persistent — survives restarts, works across the whole class if you deploy one shared backend).
3. Only if both miss: call OpenAI, then write the result to Postgres and warm the in-memory cache for next time.

Every call is also logged to request_log, which powers /stats.

Running Postgres locally

The easiest way is Docker Compose (included):

cd backend-api
docker compose up -d      # starts Postgres on localhost:5432

If you don't have Docker, install PostgreSQL locally and create a database named threadit, then set DATABASE_URL in .env to match your credentials.

Tables are created automatically the first time the FastAPI server starts (init_db() in db.py) — no manual migration step needed for this project's scope.

New endpoints

• GET /history?limit=10 — recently generated summaries (a JOIN across threads and summaries). The popup's "View History" button uses this.
• GET /stats — aggregate usage analytics (unique threads, total summaries, cache hit rate) computed with COUNT() queries.
• GET /cache/memory — size of the fast in-memory cache (separate from the Postgres-backed history/stats).

Inspecting the database directly

docker exec -it threadit-db psql -U postgres -d threadit
# then, inside psql:
\dt                          -- list tables
SELECT * FROM threads;
SELECT * FROM summaries ORDER BY created_at DESC LIMIT 5;

Caching

backend-api/cache.py implements the fast first-tier: a small in-memory TTL cache keyed by a SHA-256 hash of the tweet content. Default TTL is 1 hour, configurable via CACHE_TTL_SECONDS in .env. It resets on restart by design — Postgres (above) is the persistent layer.

Notes on the DOM selectors

content.js looks for article[data-testid="tweet"] and div[data-testid="tweetText"]. These data-testid attributes are how X's own frontend tests target these elements, so they're far more stable than the auto-generated CSS class names — but X can still change them. If scraping stops working, open DevTools on a tweet and check whether the data-testid values changed, then update the selectors in src/content/content.js.

---

Deploying so others can use it

Right now everything runs on localhost, which is fine for your own dev machine but won't work for classmates or anyone else who installs the extension — their browser can't reach your localhost:8000.

To make it usable for others, you need to:

A. Deploy the backend somewhere public

Any small host works for a class project:

• Render or Railway (free/cheap tiers, easiest for FastAPI): push this repo, set OPENAI_API_KEY and DATABASE_URL as environment variables in their dashboard (not in a committed .env file), and point the start command at uvicorn main:app --host 0.0.0.0 --port $PORT. Both platforms offer a managed Postgres add-on that gives you a ready-made DATABASE_URL — use that instead of the local Docker one.
• Fly.io or a small VPS (DigitalOcean, AWS EC2) if you want more control — run the same uvicorn command behind a process manager (systemd, or just nohup/screen for a demo), and either run Postgres in a container alongside it or use a managed instance (Supabase, RDS, Neon).

Once deployed, you'll have a public URL like https://your-app.onrender.com.

B. Point the extension at the deployed backend

In src/popup/Popup.jsx, change:

const BACKEND_URL = 'http://localhost:8000/summarize';

to your deployed URL:

const BACKEND_URL = 'https://your-app.onrender.com/summarize';

Then rebuild (npm run build).

C. Tighten CORS (recommended once it's public)

In backend-api/main.py, replace the wildcard CORS origin with your actual extension's origin so random websites can't call your API:

app.add_middleware(
    CORSMiddleware,
    allow_origins=["chrome-extension://<your-extension-id>"],
    allow_methods=["POST", "GET"],
    allow_headers=["*"],
)

You'll get <your-extension-id> from chrome://extensions once loaded (or from the Chrome Web Store listing once published).

D. Distribute the extension itself

Two options:

• Unpacked, for a small group (e.g. classmates/professor): zip the dist/ folder and share it, along with the "Load unpacked" steps above.
• Chrome Web Store, for anyone: create a developer account (one-time $5 fee), zip dist/, and submit it through the Chrome Web Store Developer Dashboard. Review typically takes a few days. You'll need a privacy policy page since the extension reads page content (even though nothing is stored or sent anywhere except your own summarization backend).

E. Cost control

Since your key is billed per API call, consider adding simple rate limiting (e.g. slowapi) on the /summarize endpoint before sharing the deployed link widely, so it can't be spammed.