The Research Risk Cascade: Why Even “90% Accurate” AI Tools Break Pipelines

Well over a hundred AI tools and platforms now target customer and UX research alone. They’re marketed under the catch-all “AI” despite representing fundamentally different technologies: automatic speech recognition (ASR), machine translation (MT), large language models (LLMs), etc. The promises are ambitious: 10× productivity, qualitative research at quantitative speed and scale, analysis across 180+ languages—no human translators required. The advances in research support are real. Yet unease accompanies this growth because the market has grown faster than our ability to evaluate it coherently. Not unfounded skepticism, but reasonable concern. As AI tools designed to automate qualitative research proliferate, what are we gaining and what are we losing?

One thing we’re definitely losing is visibility into how these tools work, and how well they work. Visibility is a precondition for evaluation. AI research assistance is sold as modular components—commonly a transcription service, a translation layer, a large language model (LLM) for higher-order “thinking” tasks—or, increasingly, as turnkey solutions. Individual components might come with 90% accuracy claims; bundled platforms might promise trust and reliability. But the claims are often opaque, based on internal benchmarks and proprietary evaluations that can’t be independently verified or replicated in actual research contexts. ASR, MT, and LLMs are fundamentally different technologies, each with distinct failure modes and performance metrics.

No matter the configuration—modular or bundled—the trend across research tech stacks is toward tighter integration and less visibility. Convenience is attractive, but it’s never free. Chaining these technologies together masks the errors that exist at each stage. Research is a pipeline, not a bucket, and errors multiply rather than accumulate. I call this phenomenon the Research Risk Cascade. 

Cascading risk is well documented in fields like systems engineering, finance, public health, and high-reliability organizations. The contexts differ, but the mechanics are shared: in tightly coupled systems, small failures produce outsized downstream effects. Nithya Sambasivan and colleagues documented this pattern in AI and machine learning (ML) as “data cascades.” Their CHI ’21 Best Paper, titled “Everyone wants to do the model work, not the data work,” studied data cascades across AI/ML projects in multiple domains and countries. Failures were hard to detect and expensive to fix because problems surfaced late, long after the original data decisions. Teams got stuck in reactive cycles, repeatedly launching and adjusting based on confusing signals. The ones with fewer cascades shared a pattern: frequent pipeline runs, close collaboration with domain experts, careful documentation, and continuous monitoring. Ninety-two percent of projects they surveyed had cascades, but most could have been prevented. Prevention meant shifting away from reactive cleanup that treats data work as “grunt work” and towards proactive data excellence.

That framing should sound familiar. “Grunt work” has become a common shorthand in the marketing of AI tools, used to promise relief from slow, manual, and repetitive parts of research. The promise is not trivial. Automation does reduce friction. But applying it across the full pipeline with little oversight—or none at all—treats research as a solvable technical problem. It isn’t.

The Research Risk Cascade applies complex risk thinking to qualitative research pipelines. Instead of the sensor data, labeled images, and log files in AI/ML pipelines, the qualitative research pipeline begins with lived experience and converts it into digital artifacts and, ultimately, decisional tools through a familiar chain of transformations:

Capture → Transcribe → Translate → Synthesize → Analyze → Insight

Each step inherits what came before and reshapes it. Loss is possible at every stage—some degree of it nearly certain—and that loss compounds into non-linear impacts that are more than the sum of the original parts (see figure 1). Transcription narrows what is available to translate. Translation filters which meanings survive to synthesis. Synthesis can flatten variance, suppress outliers, and smooth contradictions. Analysis proceeds on outputs that appear fluent and tidy, even when their structure has already thinned. The signal continually degrades, but the degradation becomes harder to detect as it moves downstream toward insight. Et voilà, technical debt becomes insight debt. 

Insights are not merely outputs. Insights require interpretation. Battle and Ottley (2023) define insight as the intersection of two types of knowledge: analytic knowledge derived directly from data, and domain knowledge that provides context beyond the dataset. The more you delegate pipeline stages to AI tools, the harder it becomes to maintain both sides of this intersection. Signal degradation erodes analytic quality, while automation distances you from the domain knowledge needed for interpretation. You get answers, but those answers are easier to produce than they are to trust. Information, yes, but insights? The costs show up later, as distorted understanding and fragile decisions, and as societal harms. 

To track Research Risk Cascades, we need measurement points. The early stages of the research pipeline offer established measurement points and metrics that are imperfect, but long-standing and informative. Synthesis and analysis are harder to measure with standard metrics, but equally important to evaluate. This section introduces several metrics at each stage of the research pipeline and discusses their strengths and caveats.

Transcription can be evaluated using Word Error Rate (WER), which has been the standard metric for ASR since the 1980s. WER counts substitutions, deletions, and insertions at the word level. It’s a blunt instrument that treats all errors as equivalent regardless of their impact on meaning. Features like prosody, hesitation, overlap, or interactional features that qualitative researchers often care about are invisible to WER. But it gives a baseline number for how much of the original signal made it through. More recently, Token Error Rate (TER) has been proposed as a complementary metric for neural and LLM-based ASR systems. TER accounts for tokenization differences and can better reflect how errors propagate in downstream Natural Language Processing (NLP) tasks. TER doesn’t solve WER’s core limitations, but it’s useful when transcription output feeds into further automated processing rather than human-only reading.

Translation quality is often approximated using reference-based metrics such as BLEU, METEOR, chrF, and COMET. BLEU, METEOR, and chrF rely primarily on surface-form overlap between machine output and human translations, comparing shared words or character sequences. These metrics are well known to be limited: they miss meaning shifts, flatten cultural and pragmatic context, and penalize valid alternative translations. BLEU in particular has been criticized for decades, yet it persists because it is fast, reproducible, and gives teams something to compare against. COMET differs in that it is a learned, neural metric trained on human judgments, and while it correlates better with adequacy, it remains constrained by reference translations and inherits their blind spots.

Researchers have developed complementary approaches that attempt to assess meaning preservation more directly. Semantic analysis methods derive meaning representations from text to evaluate whether core semantics survive translation, often using embedding-based comparisons like BERTScore or transformer-based semantic similarity measures. Back-translation (or round-trip-translation) approaches evaluate whether translating content into a target language and back to the source preserves the original message. LLM-as-judge methods are increasingly used to assess correctness. None of these approaches are perfect: embedding-based metrics depend on model quality, back-translation can mask certain error types, and LLM-based evaluation passes on the judge model’s biases. But they often bring out different performance patterns than overlap metrics alone, particularly for morphologically rich languages or those with substantial linguistic distance from English, as we’ll see below.

Why the divergence between overlap scores and semantic measures? Measuring translation quality is orders of magnitude harder than transcription because meaning is contextual, languages are incredibly diverse, and translation itself involves fundamental tradeoffs. A big one is the tradeoff between naturalness and accuracy. Optimizing for one aspect sacrifices the other. Neural Machine Translation (NMT) systems tend to privilege literal accuracy, which often results in stiff or unnatural output. LLMs can produce more natural-sounding translations, but they do so by “interpreting” context, which means outputs that may extrapolate, smooth over ambiguity, or introduce things that weren’t present in the source text. As a longtime Japanese–English translator, I can confirm:  the translational tensions and the choices are real. The choice of translation system bakes in which errors you’ll have to tolerate, not whether errors will occur. All translation tools require cleanup depending on what you’re after: passable gist, idiomatic and elegant prose, or verbatim accuracy for high-stakes contexts like healthcare or legal work.

As research progresses toward synthesis and analysis, measurement becomes more complex still. Unlike transcription and translation, which have established metrics like WER and BLEU, synthesis and analysis lack standardized benchmarks. There’s no simple score for “accurately identified patterns” or “produced valid insights.” However, LLM evaluation benchmarks can provide a directional floor—a lower bound on capability that helps us understand what these tools can and cannot reliably do.

The MMLU (Massive Multitask Language Understanding) dataset and multilingual counterparts like MMMLU (produced by OpenAI) and Global MMLU Lite (produced by Cohere) test “reasoning” capabilities across subject areas and languages, ostensibly similar to tasks synthesis and analysis require. Model builders report their scores on these benchmarks, and independent evaluators use them to track reasoning performance. But MMLU and its variants are severely limited: they test recognition more than generation, favor certain knowledge domains like math, are heavily Western-centric, and can be gamed. MMLU scores have saturated—models now exceed 90% accuracy—driving migration to harder variants like MMLU-Pro, with Pro+ already proposed.

Synthesis and analysis resist definitive measurement because they’re always shaped by interpretation, context, and domain knowledge. No benchmark captures that. But we need to start somewhere, and current LLM benchmarks provide useful anchors. For the modeling exercise that follows, I use MMLU-Pro scores as proxies for synthesis and analysis performance. These scores reflect what model builders currently optimize for, making them relevant—if imperfect—measures of actual system capability.

From WER to BLEU & co. to MMLU-Pro, we now have measurement points across the pipeline. Now we can also  start to track two connected challenges: measurement difficulty increases at each stage, while errors cascade and compound through the pipeline (see figure 2).

These metrics are provisional starting points, not measures of research validity. Content-agnostic metrics like WER and BLEU provide foundations, but domain-specific quality requires context-appropriate evaluation. For translation, newer approaches like semantic analysis and back-translation with LLM-as-judge sit at higher measurement complexity than overlap metrics, as they aim to capture meaning preservation more directly, though with their own limitations and dependencies. The need for context-appropriate evaluation becomes even more critical at synthesis, analysis, and insight. While benchmarks like MMLU-Pro offer directional proxies for reasoning capability, they measure generic performance rather than research validity. Building evaluations specific to your research context—your domain, your questions, your quality standards—is essential work. (More on that soon.)

Even with their limitations, these borrowed metrics are sufficient to model how signal degrades and compounds through research pipelines.