Feature Engineering for Financial Data

The ML-in-markets lesson claimed "features beat models." This lesson is the how — the standard transformation pipeline that quant equity researchers apply to nearly every signal, and the reasoning behind each step.

Cross-sectional normalization: compare, don't level

A raw signal (say, earnings yield) mixes information with market-wide level shifts. The cure is normalizing within each date across assets:

• Z-score: subtract the date's cross-sectional mean, divide by its std — "how unusual is this stock today, versus its peers."
• Rank / quantile transform: replace values by their cross-sectional rank. Brutally effective in finance because it's immune to outliers and to the feature's marginal distribution — a rank-transformed signal survives data errors that would let one crazy value dominate a z-score. The information coefficient (rank correlation with forward returns) is the standard signal-quality metric for exactly this reason.

The deeper point: cross-sectional strategies predict relative returns, dodging the near-impossible task (per the time-series lesson) of predicting the market's direction.

Taming the tails

Financial features are fat-tailed (the CLT lesson, again), and least-squares fits chase tails. Standard hygiene: winsorize (clip at, say, the 1st/99th percentiles) or rank-transform; log-transform inherently multiplicative quantities (market cap, volume); and never let a single name contribute unbounded leverage to the fit. The test: refit after deleting the 5 most extreme observations — if conclusions change, the tails were doing the work, and that's fragility, not signal.

Neutralization: subtract what you already know

A promising new signal is usually an old factor in disguise. Neutralize by regressing the signal cross-sectionally on the known factors (market beta, size, value, momentum, industry dummies — the CAPM lesson's cast) and keeping the residual. What survives is the genuinely new content — often disappointingly small, which is precisely the point: better to learn a signal is 90% momentum at research time than after allocating to it. Portfolio-level equivalent: constrain factor exposures to zero at construction (the hedging logic from the factor lesson). Sector-relative features ("cheap versus its industry") are the poor-man's version and often capture most of the benefit.

Time decay and turnover awareness

Signals age. Two engineering responses: exponential decay when combining past observations into a feature (halflife as the tuning knob — the AR(1) lesson's $\phi$ in disguise), and measuring each candidate feature's autocorrelation — a feature that flips daily implies turnover whose costs (execution lesson) may exceed the alpha. A slightly weaker, slower signal frequently wins after costs; feature engineering that ignores the cost model optimizes the wrong objective.

The hygiene layer (non-negotiable)

Point-in-time alignment — each feature computable strictly before the return it predicts, restatements excluded, vendor "history" treated with suspicion (backfilled databases are lookahead in disguise: sin #1 of the backtesting lesson). Missing data handled explicitly (missingness is often informative — a stock with no analyst coverage is a fact, not a NaN to interpolate away). And every transformation fit on training data only — normalizing with full-sample statistics leaks the future through the back door, subtly and constantly.

The interview version

"You have a raw sentiment score per stock per day. Walk me through making it a feature." — Timestamp audit; winsorize or rank cross-sectionally per day; neutralize against momentum and industry (news sentiment is soaked in both); choose a decay halflife; check IC, IC decay, and implied turnover against costs. Six steps, each with a because. That pipeline recital — boring, ordered, reasoned — is what a professional feature process sounds like, and interviewers know it when they hear it.