Earnings Whisperer

01

Change, not tone

Some management teams are perpetually upbeat; some are congenitally cautious; house style survives multiple CFOs. Scoring tone therefore recovers a lot of company identity and not much else. Differencing against the same company one quarter earlier removes that fixed effect by construction.

For company i at call t, with E a pooled sentence embedding of a section:

Shift(i,t)  = 1 - cos( E(i,t), E(i,t-1) )
ZShift(i,t) = ( Shift(i,t) - mean Shift(i,<t) ) / sd Shift(i,<t)

computed separately for prepared remarks, the full Q&A and management answers only. The normalisation uses strictly earlier calls, so a call never contributes to its own baseline.

02

There is a raw association. It doesn't survive controls.

Unconditionally, the effect is visible and in the expected direction: calls in the top decile of company-historical prepared-remarks shift are followed by 0.297 annualised realized volatility against 0.255 in the bottom decile. That gap is what a paper reporting a positive finding would lead with.

The rest of this page is what happens when you put recent volatility, sector and liquidity in the model alongside it.

03

The central test

Five nested feature groups on a chronological split — train to 2022-12-31, validate to 2023-12-31, test on everything after. Every fitted transform (imputation, scaling, one-hot categories, TF-IDF vocabulary) is fit on training rows only. A versus E is the pre-registered comparison; it was not chosen after seeing which contrast looked best.

04

The interval, and why it is company-clustered

A company appears up to 38 times in this panel and its volatility is persistent, so earnings calls are not independent draws. Resampling calls individually would treat 38 calls from one company as 38 independent facts and produce an interval that is far too narrow. The headline bootstrap resamples whole companies.

The interval contains zero and the point estimate is negative. On this sample, semantic change adds no measurable information beyond financial controls.

05

What actually predicts volatility here

The most uncomfortable number in the study is not any of the model scores. Predict each company's own historical average volatility — using no transcript, no sentiment, no recent returns, just which company this is — and you get MAE 0.0753 and R² +0.123, beating groups A, C, D and E outright.

Most of the predictable variation in 20-day realized volatility across this panel is a stable company-level effect, not something recoverable from one quarter's language or from the last 20 sessions of returns. It also explains the TF-IDF baseline: bag-of-words over a transcript is an excellent company fingerprint, so that row is best read as a company-identity model rather than as evidence that lexical content forecasts risk.

06

Ablations: the shift features are load-bearing — just not enough

Removing one block at a time from the full model. A positive delta means removing it made predictions worse, i.e. the block was carrying information.

Two things stand out. The prepared-remarks shift is the single most load-bearing block in the full model — so the semantic-change features do carry information relative to the other text features. And the financial controls actively hurt inside model E: removing them improves MAE. In this calm test regime, trailing volatility is an upward-biased predictor.

07

Robustness, including where the estimators disagree

Two things could have made the headline an artefact. ZShift is heavy-tailed — a company with very consistent language produces enormous z-scores, and about 1% of calls exceed |z| = 10 — and the choice of estimator could be doing the work. Both are tested rather than assumed away.

Do the feature rankings even agree?

SHAP, permutation importance and linear coefficients are descriptive diagnostics of a fitted model, not causal evidence. They are only worth reading if they agree.

08

Question–answer alignment

For each paired analyst question and the management answer that followed, cos(E_Q, E_A), aggregated per call to mean, median, p10, variance and fraction below threshold — 97,396 pairs, mean 0.521.

09

Building it so the answer could be trusted

Parsing

5,617 of 5,700 transcripts (98.5%) parse cleanly into prepared remarks, analyst questions and management answers. Nothing is silently dropped — every call carries a status. The failure modes that had to be handled were each individually capable of wrecking the study: roster blocks that are structure rather than speech; the opening operator boilerplate "later, we will conduct a question-and-answer session", which a naive boundary match puts at turn 2, discarding every prepared remark; speaker labels like C. Howard Nye - Martin Marietta Materials, Inc.; IR-moderated calls where an investor-relations officer rather than the operator calls on analysts; and executives who greet the analyst first, which breaks positional role assignment.

Chunking

A call is 10k–20k tokens against a 256-token encoder window. Naive truncation would keep about 2.5% of a median call — and would discard the Q&A specifically, which is where the unscripted language lives. Sections are split into overlapping sentence-aligned chunks, embedded, then length-weighted pooled. Measured sentence coverage is 100.0%, with 6 tokens out of ~3.3M irrecoverably lost at final-chunk boundaries.

Leakage

   ... --- 20 trading sessions --- |  D  | --- 20 trading sessions --- ...
           CONTROL WINDOW          |buffer|        TARGET WINDOW
       (last session < D)          |      |     (first session > D)

Controls end strictly before the call date, the target starts strictly after it, and the call date belongs to neither. The vendor's call timestamp has an undocumented timezone, so any rule depending on whether a call started at 08:00 or 17:00 would only be as sound as that timestamp. Excluding day D is correct under every interpretation of it.

The cost is stated rather than hidden: the earnings-reaction day — the highest-variance day in the window — is excluded from the target, which makes the exercise strictly predictive and the target harder to fit.

10

What would change this answer

• A different regime. The held-out period is calm — target dispersion is sd 0.118 against 0.179 in training. An oracle constant predictor scores R² −0.042 there, so R² has very little variance to work with and all MAE differences are small in absolute terms. This study cannot tell you whether the answer holds in a turbulent test window.
• A longer horizon. 20 sessions may simply be the wrong window for language to matter.
• Smaller companies. The corpus is S&P 500 firms with dense analyst coverage, where information reaches the market through many channels. Language may carry more where those channels are thinner.
• A domain-tuned encoder. A general-purpose sentence model measures distributional similarity, not financial meaning. No fine-tuning was done here, by design.

Confounds that remain regardless: management turnover changes vocabulary for reasons unrelated to risk, and sector language drifts collectively — a whole industry can start saying "supply chain" in the same quarter.