By Greg Miller, Vice President of Marketing & Business Development, Carta Healthcare
There is a tempting way to think about clinical data abstraction that makes it sound easy to automate. In this view, abstraction is data retrieval. A registry asks a question, the answer sits somewhere in the chart, and the job is to find it and copy it into the right field. If that were true, the problem would have been solved a decade ago with pattern matching, and this would not be an interesting topic.
It is not true, and the gap between that description and the actual work explains why a generation of automation tools has fallen short.
Early systems, including some of the earliest at Carta Healthcare, used natural language processing to extract registry data automatically. The approach is reasonable on paper. Map how clinicians document a finding, build extraction logic around those patterns, and scale from there. It works until it meets the variety of real documentation.
The same clinical finding appears as a structured field at one hospital and as buried free text at another. The same concept is phrased a dozen ways by a dozen clinicians. Edge cases multiply rather than settle. Pattern recognition can handle documentation that is predictable, and clinical language rarely is. The conclusion that the field eventually reached is blunt. Pattern recognition cannot replicate clinical judgment, no matter how much pattern you feed it.
Look closely at what a registry actually asks. A question like what was the most recent glucose before the procedure is not a search for a number. It requires knowing the exact procedure start time, then finding a lab value that precedes it, then confirming the value belongs to the right episode. A question like was aspirin prescribed at discharge requires distinguishing a medication the patient was sent home with from one administered during the stay. These are reasoning chains, not field lookups.
It gets harder. Three skilled abstractors can review the same cardiac case and reach different but defensible answers. The physician notes point one way and the imaging points another, and the correct answer requires weighing both. That is not a problem you can rule your way out of, because there is no rule that resolves a genuine conflict of evidence. There is only judgment applied to context.
Stated plainly, an automated abstraction system has to do what a trained abstractor does. It has to read clinical language in context, weigh conflicting evidence across documents, apply temporal logic relative to specific procedure dates, and handle ambiguity rather than break on it. If a weight was recorded after a procedure, a skilled abstractor knows it does not count as a pre-procedure weight. The system has to know that too, and it has to know it as a matter of reasoning, not as one more hard-coded exception in a list that never ends.
This is the dividing line between extraction and reasoning. Extraction asks where the value is. Reasoning asks what the value means, whether it applies, and what to do when the record disagrees with itself. Registries live in the second category.
Once you accept that abstraction is a reasoning task, the choice of model stops being a commodity decision. There is a persistent assumption that any sufficiently capable language model can be pointed at clinical text and made to perform. The evidence from building real systems says otherwise. When Carta Healthcare evaluated several models, the differences in interpreting clinical documentation were not subtle. The capability to understand and reason across messy clinical records varied widely, and only some models held up.
This matters for buyers because it cuts against a comfortable narrative. The narrative says clinical abstraction is on the verge of commoditization, that the underlying intelligence is becoming a utility, and that any vendor can wrap a small model and deliver the same result. A smaller or weaker model can match the easy cases and quietly fail the hard ones, which are exactly the cases that determine whether registry data is trustworthy. The danger is not that it gets the routine question wrong. It is that it gets the routine question right often enough to seem fine while missing the reasoning-heavy cases that matter most.
There is a commercial version of this misunderstanding that buyers should watch for. A smaller or weaker model, wrapped in a clean interface, can look indistinguishable from a frontier reasoning system on the cases people tend to test in a demonstration. The easy cases all resolve the same way. The divergence shows up later, in production, on the reasoning-heavy cases that rarely make it into a sales conversation. By then the cost of the gap is no longer hypothetical, because it has been written into a registry.
Stated as a thesis, clinical data abstraction is a reasoning problem that rewards frontier capability, not a pattern-matching problem that a small model can approximate. The difference is not marketing. It is the difference between a system that can weigh conflicting evidence across documents and apply temporal logic relative to a specific procedure time, and one that retrieves the nearest plausible value and stops. The first is doing the work. The second is doing a convincing impression of the work that holds up until the case gets hard.
The temporal dimension alone illustrates how much reasoning the task hides. Whether a lab value counts depends on when it was drawn relative to a procedure whose exact start time has to be established first. Whether a medication counts as a discharge prescription depends on distinguishing what a patient was sent home with from what was administered during the stay. These are not facts to look up. They are judgments to reason through, and the quality of that reasoning is the whole game.
This has a direct implication for how buyers should evaluate these tools. The instinct is to test a system on a representative sample, which sounds rigorous but quietly favors the easy cases that make up most of any sample. A more honest evaluation deliberately seeks out the hard ones, the conflicting records, the ambiguous timestamps, the findings buried in free text, and asks not only whether the system got them right but whether it reasoned about them the way a clinician would recognize. A tool that performs well on a curated demonstration and poorly on that harder set is telling you exactly where it will fail in production.
None of this argues against automation. It argues for matching the tool to the task. Clinical data abstraction is a high-consequence reasoning problem, and it rewards frontier reasoning capability paired with the clinical expertise to verify it. Treating it as a pattern-matching problem produces tools that demo well and disappoint in production, because the part of the work that is hard to see is precisely the part that pattern matching cannot reach. The organizations that get abstraction right will be the ones that respect what the question is really asking.
