How Machine Learning Language Spread Across Journals
A technical account of the Crossref scan used to stratify the REFORMS study: what was counted, where the increase occurred, and which journal-level patterns deserve a closer look.
Technical summary
The increase is large under a conservative lexical definition. Explicit ML language rose from of journal papers in the first year to in the final year. Matching-paper counts rose from to .
The shift is broader than a handful of specialist venues. Among journal-years with enough papers for a stable descriptive rate, the share with no explicit ML match fell from to , and the median reached .
This is a map of language, not a classifier of ML-based science. It is useful for sampling and venue comparison. Paper-level screening is still required before REFORMS reporting can be scored.
Journal papers scanned
Current Crossref journal-article records
Explicit-ML matches
Title or abstract matched the versioned query
Overall six-year density
Paper-weighted across the full period
Journal-year rows
Across exact journal labels
Explicit ML language tripled as a share of journal papers
The paper-weighted density increased every year. Generic prediction, classification, and forecasting terms also increased, but explicit ML language passed that broader task vocabulary near the end of the period. The two series can overlap and should not be added together.
Annual lexical prevalence
Share of all qualifying Crossref journal papers; the series are non-exclusive.
Points are the six annual observations and straight lines are ordinary least-squares trends fitted separately to each series. The two-sided p-values test a zero slope. They are descriptive: six serial annual observations are too few for strong time-series inference, and the fit does not adjust for changes in Crossref metadata coverage.
Exponential fit on a log scale
The same six annual observations with log-scaled prevalence and fitted exponential curves.
The model is fitted by ordinary least squares to log prevalence, and the p-value tests a zero log-slope. R²(y) evaluates the resulting predictions on the original percentage scale and is directly comparable with the linear chart; R²(log y) describes the transformed fit. The explicit-ML curve fits these six observations better than the straight line, but this is descriptive, not a forecast.
The journal distribution moved, but remains strongly right-skewed
The aggregate rise is not only a volume effect inside the largest journals. The zero-density share dropped sharply, while the upper tail expanded: journals with explicit ML language in more than one paper out of ten became much more common. These are journal-weighted results with a minimum annual denominator, unlike the paper-weighted chart above.
Distribution of journal ML density
Journal share by density bin in the first and final years; journals require at least 25 papers in that year.
Each eligible journal label receives equal weight. The leftmost bar includes journals with no matching title or abstract.
Beta-binomial distribution
Zero mass and positive journal densities, observed and fitted, in 2020 and 2025.
The left panel shows the large share of journals with exactly zero matching papers. In the positive-density panel, solid lines and points are observed log-binned shares and dashed lines are beta-binomial predictions averaged over the journals' actual annual paper counts. The beta-binomial was the best of four simple count models by BIC in both years.
Parameter interpretation. The fitted 2020 distribution has α = and β = ; in 2025, α = and β = . β is not a probability. In a beta-binomial model, the latent journal match probability is p ~ Beta(α, β), with mean α/(α+β) and concentration α+β.
The fitted latent mean rises from to , while concentration falls from to . That combination means the journal-level propensity distribution shifts upward and becomes more dispersed in 2025. Because α remains below one and β remains above one, both fitted latent distributions still place their greatest density near zero.
Earlier and current journal density are related, but far from identical
The endpoint comparison contains exact journal labels eligible in both 2020 and 2025. Of these, had no explicit-ML match in either year and moved from zero to a positive density. Overall, increased and decreased; the median change was .
The all-journal Spearman correlation is . Among journals positive in both years it rises to , which is substantial persistence without making the endpoint plot close to a diagonal. A point scatter would overplot tens of thousands of journals, while a conventional KDE would blur the large point masses on the zero axes and leak density outside the bounded 0-100% range.
2020 density versus 2025 density
Full-data hexagonal binning of journals eligible in both years; darker cells contain more exact journal labels.
Both axes use an inverse-hyperbolic-sine display transform with ticks shown as the original percentages. This preserves exact zero, gives low densities more room, and still displays the upper tail. Colour is logarithmic; the dashed diagonal denotes unchanged density.
The fastest rises are concentrated in computing and applied technical venues
The growth ranking uses journal labels with observations in every year and at least one hundred papers in each year. International Journal of Intelligent Systems had the steepest fitted slope at , moving from .
Fastest fitted journal-level rise
Ordinary least-squares slope across six annual density observations; percentage points per year.
The ranking controls noise with a denominator threshold, but it does not normalize venue names or establish why language changed.
Growth also appears in journals that began with little ML language
Restricting the same ranking to journals below five percent in the first year brings a different mix of engineering, education, language, imaging, and environmental venues into view. Bioengineering rose from .
Fastest rise from a low baseline
Same six-year stability rule, restricted to journal labels below 5% in 2020.
This view is designed to surface diffusion beyond journals already saturated with ML terminology at the beginning of the scan.
Low-baseline growth details
Exact endpoints, fitted slopes, fit quality, and minimum annual paper counts.
Journal label
2020
2025
Fitted slope
R²
Smallest annual n
Bioengineering
Applied Mathematics and Nonlinear Sciences
Journal of Emerging Investigators
International Journal of Low-Carbon Technologies
Bitlis Eren Üniversitesi Fen Bilimleri Dergisi
Arab World English Journal
Journal of Engineering Research and Reports
Journal of Computer Assisted Learning
Engineering Research Express
Japanese Journal of Radiology
High density and high contribution are different stories
JMIR AI had the highest sufficiently large journal density at . By contrast, Scientific Reports supplied the largest number of matching papers, , despite a density of .
The matching-paper count is not dominated by one venue: journal labels account for the first quarter of matches, for half, and for four-fifths.
Highest 2025 ML density
Journal labels with at least 100 papers in 2025, ranked by matching-paper share.
This ranking naturally favors specialist AI and machine-learning journals.
Largest 2025 contribution by matching-paper count
Journal labels ranked by the number of records matching explicit ML language.
Large multidisciplinary journals can contribute many matches while having much lower density than specialist venues.
Abstracts add substantial detection, but do not explain the whole rise
Abstract coverage increased from to , so some increase is attributable to better searchable metadata. Yet title-only matches also rose from to . In the final year, matches, or of the explicit-ML numerator, would have been missed by a title-only scan.
Location of explicit ML matches
Composition of the annual title-or-abstract numerator.
Title only, abstract only, and both are mutually exclusive shares of papers matching explicit ML language.
What the density measure includes
The denominator is the current Crossref record for each work classified as a journal article, published from 2020 through 2025, with a non-empty journal name. The numerator is the subset whose normalized title or abstract matches the versioned explicit-ML query.
Paper-weighted density
Matching papers divided by all qualifying papers. This answers how visible ML language is across the literature represented in Crossref.
Journal-weighted distribution
Each eligible journal label receives equal weight after a minimum annual denominator. This answers how widely ML language is distributed across venues.
Explicit ML language
Named methods and families, including machine learning, artificial intelligence, neural networks, random forests, boosting, NLP, computer vision, and related terms.
Generic task language
Prediction, predictive, classification, forecasting, risk-model, screening-model, and prognostic-model language, retained as a separate diagnostic.
Why generic prediction terms were kept out of the numerator
Prediction, classification, and forecasting also occur in conventional statistics, clinical risk modeling, and domain-specific methods. Keeping them separate makes the primary density measure more conservative and easier to reproduce.
Annual scan totals
Exact counts, paper-weighted rates, and abstract coverage.
Year
Journal papers
Explicit-ML matches
Explicit-ML density
Generic-task density
Abstract coverage
How the scan was executed
Take one current record per Crossref work
The source query used current work versions, journal-article records, publication year, and a non-empty journal label.
Normalize title and abstract text
Precomputed normalized fields were preferred, with lower-cased source text as a fallback.
Apply versioned regular expressions
Explicit ML terminology and generic task language were matched separately in both title and abstract.
Aggregate and preserve diagnostics
The scan retained title and abstract availability, field-specific hits, union counts, generic-task counts, and query-version provenance for every journal-year.
Run on the canonical worker and database
The full scan completed in and wrote journal-year rows into the project schema.
What could change the interpretation
Metadata uncertainty dominates sampling uncertainty. This is a census of the local Crossref snapshot under a deterministic lexical rule, so conventional sampling confidence intervals would be misplaced. False positives, false negatives, abstract missingness, journal-label quality, and Crossref coverage are the important uncertainties.
Journal identity is approximate. The analysis uses exact Crossref journal-name strings. Case differences, renamed journals, conference-series metadata, and spelling errors can split one venue into several labels. ISSN normalization is the clearest next robustness check.
Lexical presence is not scientific use. A match can occur in motivation, literature review, or a rhetorical reference. Conversely, papers can use ML without naming one of the explicit terms. The density variable is suitable for stratification, not paper inclusion.
Abstract missingness matters. Abstract coverage changed over time and differs by publisher. The title-only increase is a useful sensitivity check, but a publisher-adjusted analysis would better separate uptake from metadata coverage.
Basic integrity checks passed. The completed table had rows with invalid year ranges, missing journal labels, out-of-range densities, or ML counts exceeding total counts.
Recommended next analyses
Normalize journal identity by ISSN and quantify how much the rankings move after aliases and case variants are merged.
Decompose the trend by publisher, field, and abstract availability to test whether metadata coverage explains part of the increase.
Inspect a stratified sample of lexical matches and non-matches to estimate query precision and recall.
Use the density distribution to draw the study sample, then confirm ML-based science at paper level before scoring REFORMS adoption.
Questions opened by the scan
Does growth in low-baseline journals represent genuine methodological diffusion, looser use of AI terminology, or both? Are education and clinical venues moving faster than comparable fields after controlling for publisher and abstract coverage? And do journals with rising ML density become more likely to require reporting standards before their published papers show stronger reporting?
