Human--LLM Collaboration Is Transforming Complexity Metrics in Scientific Texts

Since early 2023, scientific writing has seen an unexpected surge in vocabulary size and word turnover. Rather than causing a "collapse" into repetitive, machine-like prose, the widespread adoption of Large Language Models (LLMs) appears to be driving a more volatile linguistic environment. By analyzing millions of arXiv abstracts, researchers have found that the arrival of generative AI has fundamentally altered the statistical structure of scientific discourse.

This study investigates whether AI is simplifying our language or triggering new emergent dynamics. The authors examine a hybrid human–AI world where the rules governing vocabulary growth and word popularity are actively shifting.

The limits of simple diversity metrics

To detect these shifts, one must first define how to measure linguistic complexity. Researchers traditionally rely on two mathematical pillars: Heaps’ law and Zipf’s law. Heaps’ law describes the relationship between the total number of words (tokens) and the number of unique words (types). A higher exponent indicates a vocabulary that continues to expand rapidly as a text grows. Zipf’s law describes the frequency distribution of those words. It typically shows that a few words are extremely common while most remain rare.

Before this study, the debate regarding AI's impact was polarized. One camp argued that LLMs would act as "complexity sinks." They predicted models would recycle training data and produce simplified, repetitive text. The opposing view suggested that human–AI interaction might create unpredictable, emergent behaviors. These behaviors cannot be reduced to the properties of a single model. Current methods often struggle to distinguish between these two possibilities because they fail to account for the rate of change in how words compete for dominance.

Measuring the shift in a mixed ecosystem

The authors move beyond simple counts by treating human–AI interaction as a problem of cultural evolution. They use a massive dataset of over 2.7 million arXiv abstracts. They treat the year 2023—when ChatGPT saw explosive adoption—as a natural experimental landmark. Their methodology relies on three distinct analytical layers:

1. The Stylistic Proxy: Since identifying every AI-edited sentence is difficult, the researchers built a composite LLM-associated style index. This index tracks the rising frequency of specific "tell-tale" markers. These include punctuation patterns like en-dashes, em-dashes, and double or triple hyphens. It also includes specific lexical choices like "significant," "crucial," and "showcase" [Figure 2a].
2. Scaling Laws: The team uses maximum likelihood estimation to determine the exponents of Zipf’s and Heaps’ laws. This allows them to see if the fundamental "shape" of language is warping over time.
3. Top-y Turnover: Most critically, the authors measure "turnover." This is the rate at which new words enter the list of the most frequently used terms. High turnover suggests a system injecting novelty. Low turnover suggests a system dominated by conformity and repetition.

Accelerated growth and volatile rankings

The results reveal a landscape that is more dynamic than previously feared. While the basic exponents of Zipf’s and Heaps’ laws show only subtle changes [Figure 2c, d], the underlying mechanics of word usage have shifted.

The paper reports a sharp increase in the turnover of top-ranked content words since early 2023 [Figure 5a]. Instead of a stable set of scientific terms dominating the discourse, the "leaderboard" of common words is changing much more rapidly. The authors measure this via a power-law relationship $T(y) = ay^b$. Notably, the scaling exponent $b$ shifted from 1.15 in the pre-2023 era to 0.969 in the 2023–2024 period [Figure 5b].

Furthermore, the relationship between the LLM-style index and lexical complexity has fundamentally altered. Before 2023, higher use of LLM-style markers correlated strongly with larger vocabularies and higher Heaps exponents. After 2023, this relationship "flattens" .

Through multivariate regression, the authors demonstrate that the post-2023 surge in vocabulary size is not merely a continuation of old trends. It is a distinct acceleration that coincides with the arrival of generative AI .

What the data does not resolve

Despite these findings, the study does not claim to have mapped the entire future of human knowledge. First, the "LLM-associated style index" is a descriptive proxy, not a definitive detector. While it correlates with the timing of AI adoption, it cannot distinguish between a text that was entirely written by an AI and one that was merely polished by one.

Second, the observed increase in vocabulary size and turnover is a macro-scale observation. The paper does not address whether this "richness" is substantive or superficial. It remains an open question whether the increased variety of words reflects genuine scientific innovation or simply a more varied way of expressing existing ideas. Finally, the study focuses on arXiv abstracts. This is a highly formalized genre of writing. It is unknown if these same complexity shifts occur in more informal or creative human communications.

The verdict: A more volatile equilibrium

The evidence suggests we are not witnessing a simple decay into linguistic mediocrity. Instead, the integration of LLMs into scientific writing has created a "mixed linguistic population" characterized by higher volatility. The sharp increase in word turnover and the accelerated growth of vocabulary suggest that LLMs are acting as agents of change. They inject a specific kind of stylistic noise that prevents the linguistic ecosystem from settling into a static state of conformity.

Whether this volatility is beneficial or detrimental to the robustness of scientific knowledge remains to be seen. If the increased turnover is driven by superficial variety, it could lead to a long-term loss of informational depth. For now, the data indicates that the era of human-only linguistic evolution has ended. It has been replaced by a more turbulent, hybrid regime.

Figures from the paper