Somewhere between the vendor claim that CI can now run itself and the practitioner reflex that nothing real can be automated, there is a working answer, and it is unusually clean: collection automates, judgement does not, and the productive work is building the seam between them.
What automates well
Registry change detection. A watchlist of trial IDs, re-pulled on a schedule, diffed against stored versions, with material change types routed to an analyst. This is the single highest-yield automation in pharma CI and it is a small script, not a platform — the mechanics are covered in registry signals.
Congress abstract triage. Several thousand abstracts drop on a published date; the set relevant to your assets is two dozen. Filtering by drug name, target, indication and sponsor is mechanical, and machines do mechanical at scale without fatigue. The reading of the twenty that survive is not the machine's job.
Alert consolidation. Publications, press releases, filings and news, deduplicated and routed by competitive set rather than arriving as forty parallel keyword alerts that everyone has learned to delete. Unglamorous, and worth more than most subscriptions.
Entity normalisation. The same asset appears as a code, a generic name and a brand name across sources, usually within the same quarter. Resolving that automatically is what makes every other automation composable.
What LLMs changed — and the failure mode that came with them
Language models made two genuinely new things cheap: extraction of structured fields from unstructured disclosure at scale, and competent first-draft summarisation of a large document pile. Both are real. A monitoring pipeline that reads every amended protocol and drafts the change note for review is buildable today and was not three years ago.
The failure mode is equally specific: fluent output with no chain of custody. CI feeds decisions that are expensive to reverse, and a generated summary that quietly omits the programme the retrieval missed — or invents a detail of one it half-remembers — reads exactly like a correct one. The discipline that contains this is boring and absolute: every machine-drafted claim carries a citation to the underlying document, an analyst signs what goes upstairs, and anything presented as competitive judgement rather than record of disclosure has a human name on it. The moment a hallucinated detail reaches a brand plan, the function's credibility is spending years rebuilding.
What stays human
Materiality. The machine can tell you the eligibility criteria changed; knowing that the exclusion added is a response to a competitor's readout is domain memory. Primary intelligence — the investigator conversations and advisory-board listening where the published record and clinical reality diverge — is human by definition; see primary vs secondary. And the assessment itself: "what should we do" is the sentence the whole apparatus exists to earn, and no reader will accept it from a pipeline.
A realistic starter stack
- Registry watchlist + weekly diff, material changes routed to a named analyst
- Abstract-drop triage for the two or three congresses that matter to the portfolio
- One consolidated alert channel per competitive set, deduplicated, instead of per-source noise
- LLM drafting of the weekly monitoring note — from the week's routed items only, every claim cited, analyst edits and signs
- A maintained record underneath it all, because automation pointed at no data model just produces faster chaos
Build, buy, and the honest denominator
The commercial platforms bundle most of the collection layer, and the make-or-buy question is mostly about whether their change history and coverage fit your indications — the evaluation criteria are in tools and databases. The watchlist-and-diff layer is small enough that internal engineering teams routinely build it; the entity-normalisation layer is harder than it looks and is usually worth buying.
The honest denominator for any of it: an automation is worth what it does to analyst hours. If the pipeline frees the team from monitoring toil and they spend the recovered time on primary work and assessment, it is compounding. If the function's output is now machine-drafted documents that nobody interrogates, the automation did not reduce the cost of intelligence — it reduced the amount of intelligence, at scale, with confidence. The machines are for coverage. The judgement is still the product.