Social media monitoring platform

Overview

A monitoring platform for a direct client. It watches around fifty accounts, collects their posts on a daily schedule, and turns them into reports the client’s team reads each morning. The accounts are institutional and academic ones, so the value is in completeness and timing rather than volume.

It is the first of four systems I built for this client, and the one that started a relationship lasting close to two years.

What I own

The whole thing. Architecture, the collection layer, the API, the interface, the scheduling, and the deployment.

Complexities tackled

The vendor’s pricing was the hardest technical constraint. The platform’s professional tier and its enterprise tier were both quoted, and both were rejected as unaffordable for what the client would get. Rather than accept that the project was dead, I tested what the cheapest paid tier could actually return, then designed the collection layer around those limits: several authenticated sessions, each with its own stored auth state, with the day’s workload split across them. Fifty accounts to collect meant roughly half from each session rather than all fifty from one.

That split is not an optimisation. Running fifty collections through a single session is exactly the pattern that gets a session flagged and suspended, which would have ended the service rather than slowed it. The design treats the platform’s rate limits and its abuse detection as fixed properties of the environment, not as errors to retry through.

A timezone bug that only lost data at the end of the day. The daily report was assembled using the hosting server’s clock, and the server was not in the client’s timezone. Posts made in the last hours of the client’s day landed on the following server day and silently vanished from that morning’s report. Nothing errored, and the report looked complete. The fix was to resolve the server’s offset explicitly, sort against the client’s day rather than the machine’s, and put the date on the report so a gap would be visible rather than invisible.

Backfill against ongoing collection. Adding a new account has to fetch history; watching an existing one only has to fetch what is new. Those are different jobs with different cost profiles, so a newly added account takes a small initial pull and then hands over to the daily schedule, with a threshold on how far back the backfill is allowed to reach.

Stack

Python and FastAPI for the API, Playwright driving the collection sessions, APScheduler for the daily runs, Firebase for storage and auth, a Vue and TypeScript front end with Tailwind CSS, served behind nginx.