Engine Glossary
Backtesting terms are not labels. They are failure points. A signal can be correct and still become a wrong position. A stop-loss can be defined and still not protect the same bar. A NAV curve can look smooth and still fail cash, cost or fill reconciliation.
This glossary explains the vocabulary behind auditable backtests: data, indicators, signals, weights, orders, fills, costs, NAV, risk, validation and research packets.
How to use this page
- Use it when a report metric or artifact appears and you want to know what it really means.
- Use it when auditing signal -> weight/order -> fill -> NAV.
- Use it when comparing weight-mode allocation behavior with order-mode execution behavior.
- Use it before trusting a backtest that depends on stops, limits, turnover, costs or out-of-sample validation.
The Backtest Trust Stack
Trust is layered. A strategy can fail at any stage: bad data, unsafe timing, wrong portfolio state, unrealistic fills, missing costs, fragile risk assumptions or unreproducible reports.
Critical distinctions
A signal is intent. A weight is desired exposure. A position is what the portfolio actually holds after shift, rebalance or fills.
Target weights can be shifted, cash-buffered, risk-adjusted and held through drift before becoming actual portfolio state.
An order is a request. A fill is realized execution. A trade record is the audit trail that should reconcile cash and positions.
A daily candle can show that stop and target were both touched, but it cannot prove which came first.
A curve is only an output. Evidence needs data, assumptions, positions, fills, costs, metrics, plots and reproducible artifacts.
Browse by category
How governed data and explicit assumptions become features, signals and an inspectable research packet.
Portfolio StateThe difference between target exposure, actual holdings, cash and marked capital path.
ExecutionOrders, fills, blotter records and the mechanics behind stops, limits, brackets and OCO.
Costs & CapacityThe trading and carrying frictions that turn attractive gross performance into real net performance.
Risk & ExposureHow much risk is deployed, how directional it is and how overlays change the path.
Biases & ValidationThe controls and failure modes behind credible timing, selection and out-of-sample evidence.
Terms
Engine Architecture
How governed data and explicit assumptions become features, signals and an inspectable research packet.
Backtester Engine
The local Python system that turns strategy logic into auditable research evidence.
A backtester engine is not just a loop that creates an equity curve. It is the machinery that reconstructs market data, computes indicators, calls strategy hooks, converts intent into weights or orders, applies risk and execution rules, charges costs, marks NAV and writes artifacts.
In QuantJourney, your strategy class should contain the research logic. The engine should own the boring but critical parts: timestamp alignment, one-bar weight shift, rebalancing, fill handling, cost accounting, reporting and archive state.
Rule: define the timestamp, state transition and artifact where this concept appears.
A serious run should let another researcher answer: what data was used, what signals were produced, what exposure or orders were requested, what was filled, what it cost and what report was generated.
The run produces a chart but not enough state to reproduce or challenge the result.
Open the run archive and verify config, data window, signals, weights/orders, trades, metrics and report files.
Data
The historical market state the strategy is allowed to observe.
Data means more than close prices. It includes open, high, low, close, adjusted close, volume, benchmark series, calendars, missing bars, corporate-action adjustments and sometimes universe membership metadata.
Every backtest inherits the assumptions of its data source. A clean strategy on biased data can be worse than a simple strategy on well-understood data because the false precision is harder to detect.
Rule: define the timestamp, state transition and artifact where this concept appears.
For an execution-sensitive stop-loss test, daily adjusted close is not enough. The engine needs OHLC data, and even then daily OHLC does not reveal the intraday path.
The strategy is tested on adjusted, missing, biased or non-point-in-time data without disclosure.
Check source, adjustment policy, missing bars, duplicate timestamps, universe construction and benchmark alignment.
Point-in-Time (PIT) Data
Data represented as it was actually available at the strategy decision timestamp.
Point-in-time data preserves the historical information set instead of replaying the latest corrected view of history. Observation date, fiscal period end, effective date and publication or availability timestamp are different concepts and must not be treated as interchangeable.
The same rule applies to universes. Historical membership needs effective start and end dates; a current constituent list is not a substitute for the instruments that were eligible on an earlier decision date.
Usable at decision time t only when effective_at <= t and available_at <= t.
A fiscal quarter ending on 31 December but published on 20 February cannot enter a 15 January signal. A later revision also cannot overwrite the value that was known on 20 February.
Latest values, revised history or current universe membership are treated as if they were known at earlier decision dates.
Trace effective and availability timestamps for sample observations, revisions and universe membership at several decision dates.
Data Provenance
The traceable record of where data came from and how it reached a research run.
Data provenance connects a result to its provider, source identifiers, retrieval time, snapshot or version, transformations, adjustment policy, universe definition and temporal assumptions. Hashes can identify exact content, but they do not explain those semantics on their own.
Strong provenance makes corrections auditable. When a vendor revises history or a pipeline changes, the researcher can tell which snapshot produced the original run and whether a comparison still uses equivalent inputs.
Provenance = source + snapshot/version + transformations + temporal contract + identifiers or hashes.
Recording only "daily equities" is insufficient. A useful record identifies the provider, symbols, retrieval timestamp, date coverage, price adjustment, universe rule and snapshot or content hash.
A provider name is recorded, but the exact snapshot, transformations and temporal assumptions cannot be reconstructed.
Verify provider, identifiers, retrieval context, snapshot/version, transformations, adjustment policy and hashes where available.
Research Contract
The explicit set of inputs, timing rules and assumptions under which a result is valid.
A research contract states the strategy parameters, universe, data fields, decision timestamps, execution mode, rebalance policy, costs, risk rules, validation method and output metadata before performance is interpreted.
It creates a stable review boundary: two runs should only be compared as like-for-like evidence when their material contract fields match or the differences are deliberately explained.
Research contract = data + universe + timing + execution + costs + risk + validation + reproducibility metadata.
"Monthly long-only weights, signal at close, eligible from the next return period, adjusted prices, 10 bps slippage and a point-in-time universe" is a contract. "Momentum backtest" is not.
Runs are compared after material data, timing, cost or execution assumptions changed silently.
Compare the declared data, universe, timing, execution, costs, risk and validation fields with the code and generated artifacts.
Indicator
A feature computed from historical bars before strategy logic uses it.
An indicator is a derived variable such as SMA, RSI, volatility, momentum, rolling beta or drawdown. It should be computed from information available at or before the timestamp where it is used.
The safest mental model is: raw bars enter the feature layer, the feature layer writes timestamped columns, and the strategy reads those columns later. This avoids hiding feature logic inside execution code.
Rule: define the timestamp, state transition and artifact where this concept appears.
A 200-day SMA has a warm-up period. Before 200 observations exist, the feature should be NaN or explicitly masked out, not silently treated as a valid trend signal.
A rolling value uses information that would not have existed when the trade decision was made.
Verify rolling windows, warm-up periods, NaN handling and whether the feature is shifted or timestamp-safe.
Signal
Strategy intent before it becomes a portfolio or an order.
A signal answers: what does the strategy want? It can be binary long/flat, signed long/short, a cross-sectional rank, an alpha score, a regime flag or a desired exposure preference.
A signal is deliberately not the same as a position. A signal can say SPY is attractive; the weight function decides how much SPY to hold, and the order function decides what to send to the fill engine.
Signal -> Weight mode: target exposure. Signal -> Order mode: trade instruction logic.
In an SMA strategy, fast SMA > slow SMA might produce signal = 1.0. That still does not mean the portfolio owns SPY until weights are shifted/rebalanced or orders are filled.
A signal is treated as if it were already a filled position.
Trace one date from raw data to feature to signal and confirm the decision could be known at that time.
Corporate Actions & Missing-Data Policy
The declared rules for economic events and absent or stale observations.
Corporate actions can change price basis, position quantity or cash. Splits, dividends, mergers, delistings and symbol changes must be represented consistently with the selected price field and portfolio accounting.
A missing-data policy defines whether an absent or stale price causes carry-forward valuation, warning, exclusion or run failure. Missing observations should not be silently converted into zero prices or fabricated executable returns.
Economic return and NAV must reconcile adjusted prices or explicit quantity and cash-flow events, never a mixture of incompatible treatments.
A 2-for-1 split should not create a 50% loss. A dividend should be included through an adjusted return series or an explicit cash flow, but not counted twice. A suspended asset also needs a declared valuation policy.
Adjusted and raw prices are mixed, or stale and missing observations are handled without an explicit valuation rule.
Reconcile sample splits, dividends and terminal gaps against price basis, position quantities, cash flows, warnings and valuation state.
Research Packet
The complete evidence bundle produced by a backtest.
A research packet includes data assumptions, configuration, indicators, signals, weights or orders, fills, positions, NAV, trades, metrics, plots, archives and reports. It is the difference between a claim and evidence.
The packet should let another person reproduce the run and challenge the assumptions without relying on hidden notebook state or memory.
Rule: define the timestamp, state transition and artifact where this concept appears.
A useful packet contains portfolio_data, instruments_data, blotter or order history, performance report, plots, PDF report and enough config to identify the data window and strategy parameters.
The final output is a screenshot or PDF without reproducible data, config and audit artifacts.
Confirm another researcher could reproduce the run from archived config, data state and artifacts.
Portfolio State
The difference between target exposure, actual holdings, cash and marked capital path.
Weight
The target fraction of portfolio NAV assigned to an instrument.
A weight is portfolio language. A weight of 0.25 means the strategy wants about 25% of NAV exposed to that instrument. A weight of -0.25 means a short exposure of about 25% of NAV.
Weights are best for allocation research because they describe desired exposure without pretending to know the exact order book path. The engine can then shift weights, apply risk overlays, rebalance, charge turnover costs and compute NAV.
Dollar exposure = weight x NAV
NAV = 100,000 and SPY weight = 0.80 means target SPY exposure is 80,000. If SPY is 400, the implied position is roughly 200 shares before rounding and implementation details.
Weights are normalized back to 100% even when the strategy intended to keep a cash sleeve.
Inspect row sums, gross exposure, net exposure, cash and negative weights after engine adjustments.
Target Weight
The desired weight emitted by strategy logic before engine execution rules change it.
Target weight is intent, not final state. In weight mode, `_compute_weights()` returns desired exposures. The engine then shifts those weights by one bar to reduce look-ahead risk, applies cash buffer and risk model logic, and only rebalances when policy allows.
This distinction matters because target weights can look clean while realized weights drift between rebalance dates as prices move.
Target weights -> one-bar shift -> cash buffer -> risk overlay -> rebalance -> actual weights
A strategy can target 50% QQQ on Monday, but if the rebalance policy trades monthly, actual QQQ weight may not snap to 50% until the next rebalance date.
Target weights are confused with actual weights after shift, cash buffer, drift and rebalance rules.
Compare target weights with actual output weights and positions on several rebalance and non-rebalance dates.
Cash Buffer
Capital intentionally not allocated to risky positions.
Cash buffer is the part of NAV held as cash instead of instrument exposure. It can be explicit in strategy weights, or applied by the engine in weight-mode performance through `portfolio_data.cash_buffer`.
A cash buffer is not a formatting detail. It changes CAGR, drawdown, volatility, turnover, capacity and how much capital is exposed during stress.
Invested target = raw target weight x (1 - cash_buffer)
If raw SPY target is 1.00 and engine cash_buffer is 0.05, the invested target becomes 0.95. If the strategy already caps exposure at 0.80, applying another 5% buffer produces 0.76 invested exposure.
A rebalance or normalization step accidentally removes the intended cash reserve.
Check actual weights plus cash after rebalance; confirm the residual cash sleeve is preserved.
Position
The realized holding after weights are converted or orders are filled.
A position is what the portfolio actually holds: shares, contracts or units. It is downstream of strategy intent. A signal can request exposure and an order can request a trade, but the position is the realized state.
Positions are often the easiest way to audit a backtest because they reveal whether the portfolio was long, flat, short, levered or unintentionally stuck in a stale holding.
Approximate shares = weight x NAV / price
If NAV is 100,000, weight is 0.50 and price is 250, the implied position is about 200 shares.
Position changes appear on dates where no rebalance or fill can explain them.
Reconcile position deltas against rebalance flags or fill timestamps.
Rebalance
The event that moves actual portfolio exposure back toward target exposure.
Rebalancing is the trading policy for weight mode. Signals and target weights can change every day, but the rebalance policy decides when the portfolio actually trades toward those targets.
Between rebalance dates, holdings drift as prices move. That drift is realistic and important: a monthly rotation strategy should not silently trade every day just because a target matrix changed.
Trade on rebalance date = new target position - current drifted position
A monthly momentum strategy can compute signals daily but only trade on month-end. Turnover and costs should be charged only when the rebalance actually happens.
The strategy silently trades every day even though the research assumption was weekly or monthly rebalancing.
Inspect rebalance flags, days between rebalances and turnover charged on those dates.
Rebalance Policy
The declarative rule that decides when target weights may become portfolio trades.
A rebalance policy separates signal calculation from trading frequency. The strategy can update target weights on every bar while the policy permits execution only daily, weekly, monthly, at period boundaries or on an explicit schedule.
This separation preserves realistic drift between trading dates and makes turnover, cost and timing assumptions inspectable without embedding calendar logic throughout the strategy.
If policy(date) is true: trade toward target weights. Otherwise: carry positions and let weights drift.
A strategy ranks assets every day but uses a month-end policy. Daily rankings remain research state; only the month-end decision creates rebalance trades.
Calendar logic is hidden inside strategy code, making the actual trading schedule and exceptional triggers hard to audit.
Inspect schedule parameters, timezone and holiday behavior, rebalance flags and any trades outside ordinary policy dates.
Weight Mode
The engine path for allocation research.
Weight mode answers: what should the portfolio hold? The strategy emits signals and target weights. The engine shifts weights by one bar, applies risk/cash/rebalance logic, charges turnover costs and computes NAV.
Use this mode when individual fills, stop levels and order lifecycle are not the research object. It is the right mode for long-only allocation, long-short factors, rotation, risk parity and volatility targeting.
Signals -> weights -> shifted weights -> risk overlay -> rebalance -> positions -> NAV
A top-2 ETF momentum rotation is a weight-mode problem: rank assets, assign 50% to each selected asset, rebalance monthly, inspect NAV and turnover.
Weight mode is used to claim stop-loss or limit-fill realism that it does not simulate.
Confirm the research question is exposure, not fill path, and inspect shifted actual weights.
Execution
Orders, fills, blotter records and the mechanics behind stops, limits, brackets and OCO.
Order Mode
The engine path for execution research.
Order mode answers: how would this strategy trade? Strategy logic submits explicit orders to `FillEngine`. The fill engine owns pending orders, trigger checks, theoretical fill prices, slippage, commission, fills and sibling cancellation.
This mode is necessary when the path of trades matters: stop-loss, take-profit, trailing stop, limit entry, bracket order, OCO exits, order history and realized trade paths.
Pending orders -> OHLC trigger check -> fill price -> slippage/commission -> blotter -> positions/NAV
A breakout strategy that buys on entry and attaches a 5% stop-loss plus 10% take-profit is an order-mode problem, not a weight-mode rebalance problem.
Orders submitted after seeing a bar are assumed to have filled inside that same bar.
Inspect pending orders, order history, fill timestamps, blotter records and fill assumptions.
Order
An instruction submitted to the fill engine.
An order is a request, not an execution. In the current codebase, an `Order` has an instrument, side, quantity, `OrderType`, optional limit/stop/trailing fields, optional bracket spec, optional OCO pair id and metadata such as tag and order id.
An order can remain pending, fill later, be cancelled, expire, or fail to trigger. That lifecycle is exactly why order mode exists.
Rule: define the timestamp, state transition and artifact where this concept appears.
`Order("SPY", OrderSide.BUY, 100, OrderType.LIMIT, limit_price=400)` asks to buy 100 shares only if the engine sees a bar where the limit condition is reachable.
A submitted order is reported as if it were a realized fill.
Check order status, type, side, quantity, price fields, OCO id and creation timing.
Fill
The realized execution produced when an order matches a bar.
A fill is the point where an order changes cash and positions. It records order id, instrument, side, quantity, fill price, slippage, commission and timestamp.
In order mode, fills are the source of truth for realized execution. If a strategy claims it exited at a stop-loss, the blotter and fill history should show that exit.
Cash impact on buy = -(fill_price x quantity) - commission
Buy fill: 100 shares at 400 with 1.00 commission reduces cash by 40,001. Sell fill at 410 with 1.00 commission increases cash by 40,999.
Fill price is assumed to equal the stop or limit level even when gap and slippage rules say otherwise.
Reconcile fills to cash, positions, commission, slippage and blotter rows.
Blotter
The trade audit ledger of the backtest.
A blotter is the readable trade record: timestamp, order id, instrument, side, quantity, price, trade value and transaction cost. It is the place where execution leaves a paper trail.
For order-mode research, the blotter is as important as the equity curve. It tells you what actually happened and whether the strategy traded when expected.
Rule: define the timestamp, state transition and artifact where this concept appears.
If an OCO take-profit filled, the blotter should show the filled exit and order history should show the sibling stop cancelled.
Positions or NAV change without an auditable trade record.
Compare blotter trades with position changes and transaction costs.
Market Order
An order that accepts the engine fill convention instead of specifying a price boundary.
A market order says: execute at the engine assumed market price. In the current FillEngine, market orders fill at bar open by default, or bar close if `fill_at` is configured to close.
This is simple but powerful: a market-at-open backtest and a market-at-close backtest answer different execution questions.
Market theoretical price = bar.open by default
If an order is already pending before a bar with open 100 and close 103, a default market fill uses 100 before slippage and commission.
Market-at-open and market-at-close assumptions are mixed without being documented.
Check fill_at, whether the order was pending before the bar, and whether open or close was used.
Limit Order
An order with a worst acceptable price.
A buy limit says: buy only at this price or better. A sell limit says: sell only at this price or better. In OHLC simulation, the engine checks whether the bar low or high reached the limit.
Limit orders trade off price improvement against fill probability. A strategy can look excellent if every touched limit is assumed to fill perfectly, so the fill convention matters.
Buy limit triggers when bar.low <= limit_price. Sell limit triggers when bar.high >= limit_price.
Close is 100 and you place a buy limit at 98. If the next daily low is 97.5, the limit is reachable; if low is 98.5, it is not.
Every touched limit is assumed to fill perfectly with no queue, volume or priority constraint.
Check high/low trigger, gap behavior, fill price rule and unfilled pending orders.
Stop-Loss
A protective exit triggered when price moves against the position.
For a long position, a stop-loss is usually a sell stop below entry. It is designed to leave the trade when downside reaches a predefined level.
A stop-loss does not guarantee that the exit happens exactly at the stop price. If the market gaps through the stop, a realistic fill may be worse than the stop level.
Long stop-loss example: stop_price = entry_price x (1 - stop_pct)
Entry reference 100, stop_pct 5%. Stop-loss = 95. If the next bar opens at 93 after a gap, the simulated fill rule decides whether the fill is 95, 93 or another convention.
A stop is created after observing the bar and is treated as if it protected that same bar.
Verify the stop existed before the trigger bar and review gap-through fill behavior.
Take-Profit
A profit exit triggered when price reaches a favorable target.
For a long position, a take-profit is commonly represented as a sell limit above entry. It exits the trade when the market reaches a target gain.
Take-profit rules change the distribution of trades. They often increase win rate but cap upside, so they must be studied with loss size, holding period and missed trend behavior.
Long take-profit example: target_price = entry_price x (1 + target_pct)
Entry reference 100, target_pct 10%. Take-profit = 110. If the high reaches 110 before the stop is triggered, the profit exit should fill and cancel the sibling stop in a bracket/OCO setup.
A profit target fills but the sibling stop remains active because OCO/bracket logic was not used.
Confirm the target is linked to its stop sibling through bracket or OCO when appropriate.
Trailing Stop
A stop whose trigger follows favorable price movement.
A trailing stop starts with an anchor price and a distance. For a long position, the anchor ratchets upward as the market makes new highs. The stop level rises with the anchor and does not move down when price pulls back.
Trailing stops are designed to keep upside open while protecting part of accumulated gains. They are useful, but very sensitive to intrabar assumptions.
Long trailing stop = highest_price_since_anchor x (1 - trail_percent)
Trail percent 7%. Price rises from 100 to 120, so stop moves from 93 to 111.60. If price later falls to 111, the trailing stop is triggered under daily OHLC assumptions.
The backtest assumes a favorable intrabar sequence that daily OHLC cannot prove.
Review anchor updates, trail distance and ambiguous high/low sequencing cases.
Bracket Order
An entry order with linked take-profit and stop-loss exits.
A bracket order expresses a complete trade plan: enter, then manage both a profit target and protective stop. In the current FillEngine, a bracket parent is decomposed into an entry plus OCO-linked exit children.
This is cleaner than submitting unrelated exit orders because the engine knows the exits belong to the same trade lifecycle.
Bracket = entry + OCO(take-profit, stop-loss)
Buy 100 SPY, take-profit 110, stop-loss 95. If TP fills first, SL is cancelled. If SL fills first, TP is cancelled.
Bracket exits are assumed to be active before the entry order has filled.
Inspect generated entry, take-profit and stop-loss child orders plus sibling cancellation.
OCO
One-cancels-other order linkage.
OCO means two orders are linked so that filling one cancels the other. It is usually used for paired exits: a take-profit and a stop-loss attached to the same position.
OCO prevents the backtest from accidentally filling both exits after a position should already be closed.
If order A fills -> cancel order B. If order B fills -> cancel order A.
A long trade has sell limit 110 and sell stop 95 with the same OCO pair id. Once 110 fills, the 95 stop should be cancelled.
The same OCO pair id is reused across unrelated trades and cancels the wrong order.
Verify pair ids are unique per trade lifecycle and sibling cancellation appears in order history.
Costs & Capacity
The trading and carrying frictions that turn attractive gross performance into real net performance.
Slippage
The execution penalty between theoretical price and simulated fill price.
Slippage models the fact that you rarely trade at the exact theoretical price seen in a backtest. Market movement, spread, liquidity and order size can all make the fill worse.
In the engine, theoretical price is selected first, then slippage is applied, then commission is charged.
Fill price = theoretical price +/- slippage adjustment
A buy theoretically fills at 100.00. With 5 bps slippage, the simulated fill is about 100.05 before commission.
A high-turnover strategy is evaluated at theoretical prices with no execution penalty.
Run before/after cost comparison and stress slippage bps for high-turnover strategies.
Commission
An explicit fee charged when trades or fills occur.
Commission is the direct cost of trading. It can be modeled per share, per contract, per notional value, flat per trade or with minimum-ticket rules.
Commission does not care whether the signal was good. It compounds mechanically with turnover.
Net PnL = gross PnL - commission - slippage - impact
A strategy making 3 bps gross per trade cannot survive 1 bp commission plus 5 bps slippage.
Costs are charged inconsistently or omitted for strategies with frequent trading.
Check commission model type, minimums and whether charges appear only on real trades/fills.
Borrow, Financing & Margin Costs
The carrying costs and constraints created by shorts, leverage and funded positions.
Borrow fees compensate the lender of a shorted asset. Financing charges apply when exposure is funded rather than fully paid, while margin rules determine collateral requirements and can force de-risking or liquidation.
These inputs vary through time and by instrument. A constant assumption can support a sensitivity test, but hard-to-borrow securities, changing rates and broker-specific margin rules require explicit data or a custom model.
Net performance = gross performance - trading costs - borrow fees - financing charges - other margin effects.
A 150/100 long-short book may be directionally modest but still pay short borrow and finance gross exposure. Omitting both can materially overstate its net return.
A short or leveraged strategy is reported net of trading costs but still excludes material carrying and collateral costs.
Identify short and funded notionals, document the supplied rate and collateral assumptions, and rerun a realistic cost sensitivity.
Market Impact
The price effect caused by the strategy trading size relative to market liquidity.
Market impact is different from fixed slippage. It asks: if this strategy trades a large amount, does its own demand move the price against itself?
Impact is usually nonlinear. Doubling order size should often cost more than double if the trade consumes liquidity aggressively.
A common research approximation is impact proportional to sqrt(order_size / ADV).
Buying 10,000 dollars of SPY and buying 100 million dollars of a thin ETF should not use the same friction assumption.
Institutional-sized trades are tested with retail-sized friction assumptions.
Compare trade value with ADV, participation rate and capacity assumptions.
Turnover
How much the portfolio trades relative to capital.
Turnover measures portfolio change. In weight mode, it is commonly the sum of absolute weight changes on rebalance dates. In order mode, it comes from realized fills and trade value.
Turnover is where many strategies fail. A gross edge that looks large before costs can disappear after commission, slippage, impact and taxes.
Weight turnover on a date = sum(abs(weight_today - weight_yesterday))
Weights move from 50/50 to 80/20. Turnover = abs(0.80 - 0.50) + abs(0.20 - 0.50) = 0.60, or 60% one-way weight change under this convention.
Gross performance is reported without showing how much trading was required to get it.
Break turnover down by date and asset; compare gross versus net performance.
Risk & Exposure
How much risk is deployed, how directional it is and how overlays change the path.
Risk Overlay
A layer that modifies target exposure before execution.
Risk overlays sit between strategy intent and execution. They can cap positions, scale volatility, control gross exposure, de-risk after drawdown or reshape weights through a risk model.
They are not alpha. They do not make a weak signal good, but they can make a strategy path more controlled and easier to survive.
Raw weights -> risk overlay -> adjusted weights -> rebalance
A momentum strategy may select QQQ and SPY, then a volatility-target overlay scales both down when realized volatility rises.
The overlay hides a weak signal by overfitting exposure scaling rules.
Compare weights before and after overlay and measure turnover, vol and drawdown effects.
Volatility Targeting
Scaling exposure to aim for a desired annualized volatility.
Volatility targeting estimates recent realized volatility and scales portfolio exposure so expected risk is closer to a target. If realized vol is high, exposure is cut. If realized vol is low, exposure may be increased, subject to leverage caps.
It makes risk more stable, but it can also chase calm regimes and cut risk after volatility has already spiked.
Scale = target_volatility / realized_volatility, usually capped by max leverage.
Target vol 12%, realized vol 24% -> scale about 0.5. A 100% target book becomes about 50% exposed.
Volatility is estimated using future returns or scales exposure too aggressively after quiet periods.
Check lookback window, realized vol estimate, scale cap and whether estimates use only past returns.
Gross Exposure
The total absolute exposure of the portfolio.
Gross exposure adds long and short exposure without letting them offset. It measures how much total risk is deployed, regardless of direction.
Gross exposure is crucial for long-short and leveraged portfolios. A market-neutral portfolio can still have high gross exposure and therefore high idiosyncratic, liquidity and financing risk.
Gross exposure = sum(abs(weights))
Long 60% and short 40%: gross = abs(0.60) + abs(-0.40) = 1.00, or 100%. Long 150% and short 100%: gross = 250%.
A low net exposure book is described as low risk while gross exposure is high.
Plot gross exposure through time and compare with leverage, costs and liquidity assumptions.
Net Exposure
The signed directional balance after longs and shorts offset each other.
Net exposure tells you the portfolio direction after subtracting shorts from longs. It is useful for asking: if the broad market moves up together, is this book mostly long, mostly short or close to directionally neutral?
It does not measure total risk. A portfolio can have net exposure near zero and still be very risky if it has large long and short books, concentrated sector bets, factor crowding or illiquid names.
Net exposure = sum(long weights) - sum(abs(short weights))
Long 60% and short 40% gives net = 20% and gross = 100%. Long 150% and short 150% gives net = 0% but gross = 300%; direction is neutral, risk is not small.
Net exposure is treated as total risk instead of directional balance.
Read net exposure together with gross exposure, beta, sector exposure and drawdown.
Leverage
Using exposure larger than capital.
Leverage means the portfolio controls more market exposure than its NAV. In weight terms, leverage usually appears when gross exposure is above 100%.
Leverage amplifies both return and loss. It also introduces financing, margin, borrow and liquidity constraints that a simple backtest may not fully model.
Approximate leverage = gross exposure / NAV, expressed as a multiple of capital.
A 150% long portfolio has about 1.5x gross exposure. A 100% long and 100% short market-neutral book has 2.0x gross exposure even though net exposure is 0%.
Borrow, financing, margin or liquidation constraints are ignored while gross exposure exceeds capital.
Check max gross exposure, financing assumptions, borrow constraints and stress losses.
Market-Neutral
A strategy design that tries to reduce broad market directionality.
Market-neutral usually means longs and shorts are balanced so net exposure or beta is near zero. The goal is to earn relative return while reducing dependence on the broad market direction.
Neutral does not mean risk-free. Sector mismatches, factor exposure, crowded trades, borrow costs and liquidity shocks can dominate results.
Simple dollar-neutral condition: long exposure roughly equals short exposure.
Long 100% quality stocks and short 100% low-quality stocks may be dollar neutral, but it can still be exposed to growth, sector, size or liquidity factors.
Dollar neutrality is mistaken for beta, factor, sector or liquidity neutrality.
Test beta, factor, sector and liquidity neutrality separately from dollar neutrality.
Drawdown
Loss from a prior peak in NAV.
Drawdown measures how far the strategy falls from its previous high-water mark. It is path-based, which makes it more useful than volatility for understanding investor pain.
Two strategies can have the same CAGR and completely different drawdown profiles. A strategy with smoother returns and shallower drawdowns is often easier to allocate to even if its top-line return is lower.
Drawdown(t) = NAV(t) / rolling_max_NAV(t) - 1
NAV peaks at 120,000 and later falls to 90,000. Drawdown = 90,000 / 120,000 - 1 = -25%.
Only return metrics are shown while path loss and recovery time are ignored.
Inspect max drawdown, drawdown duration, crisis windows and recovery time.
Biases & Validation
The controls and failure modes behind credible timing, selection and out-of-sample evidence.
Look-Ahead Bias
Using information before it would have been known in real time.
Look-ahead bias is one of the most dangerous backtest errors because it can create excellent historical performance from impossible information. Examples include using today close to trade today close, using future index membership, or using a revised fundamental value before its release date.
The engine reduces one common form by shifting weight-mode target weights by one bar before returns are earned, but strategy code and data construction must still be checked.
Decision at time t may use data available at or before t, not after t.
If a signal is computed from the close on 2024-06-03, the earliest ordinary daily-bar execution assumption should be after that close, not during the same already-known return.
The strategy trades on information from the same bar or a future timestamp.
Audit timestamp alignment, feature shifts, target-weight shift and same-bar execution assumptions.
Survivorship Bias
Testing only assets that survived until the end of the sample.
Survivorship bias happens when the backtest universe excludes failed, delisted, merged or bankrupt assets that would have existed at the time. The strategy is then tested on winners that survived the historical filter.
This bias is especially dangerous for stock selection strategies because removing losers can make almost any ranking rule look cleaner.
Rule: define the timestamp, state transition and artifact where this concept appears.
Testing a 2010 strategy on the current S&P 500 membership ignores companies that were in the index in 2010 but later disappeared or underperformed.
The universe includes only assets that survived to the end of the test.
Confirm whether the universe is point-in-time or disclose that it is current-membership only.
Adjusted Close
A return-friendly price series adjusted for corporate actions.
Adjusted close accounts for splits, dividends and corporate actions so return calculations are more economically meaningful. It is often right for portfolio return math.
Execution logic is more delicate. If stops and limits use unadjusted OHLC while returns use adjusted close, levels may not mean what the strategy author thinks they mean.
Rule: define the timestamp, state transition and artifact where this concept appears.
A split can halve raw price. An adjusted series removes the artificial return jump, but stop and limit prices must be interpreted in the same price basis.
Adjusted returns are combined with unadjusted stop/limit levels.
Check that signal prices, execution prices and NAV prices use compatible adjustment basis.
Intrabar Path
The unknown order of prices inside one bar.
Daily OHLC tells you the open, high, low and close. It does not tell you whether the high happened before the low, whether price crossed a level multiple times, or how much volume traded at that level.
This matters whenever both a profit target and stop-loss are inside the same candle. The backtest must choose a convention because the data does not contain the answer.
OHLC gives range, not sequence.
Daily bar: open 100, high 112, low 94, close 105. A long bracket with TP 110 and SL 95 has both levels touched, but daily OHLC alone cannot prove which filled first.
A daily candle is used to decide an intraday TP/SL sequence it cannot actually reveal.
Find candles where TP and SL are both reachable and document the engine priority rule.
Walk-Forward Validation
Repeated train/test evaluation through time.
Walk-forward validation splits history into chronological folds. A strategy or parameter set is selected on a training window, then evaluated on a later unseen test window. The process repeats across regimes.
The goal is not to find the prettiest in-sample backtest. The goal is to see whether the research process survives new periods without collapsing.
Train window -> choose parameters -> test on next window -> roll forward -> repeat
Train 2010-2012, test 2013. Train 2011-2013, test 2014. Continue through the sample and inspect out-of-sample degradation.
One strong in-sample parameter set is presented as robustness.
Review fold metrics, selected parameters, OOS degradation and regime sensitivity.
Out-of-Sample (OOS)
Evaluation on a time window that was not used to fit or select the tested strategy configuration.
A result is genuinely out-of-sample only when the research choice is made using the training window and then frozen before the corresponding test window is evaluated. In walk-forward research, each fold needs its own training and test boundary.
QuantJourney distinguishes per-fold execution from slice diagnostics. Slicing one full-period NAV into date ranges can test reporting and window geometry, but it does not recreate the strategy with fold-local fitting and is not independent OOS evidence.
Fit or select on IS -> freeze the choice -> execute on unseen OOS -> repeat by fold.
Select SMA parameters on 2018-2021 data, instantiate a fresh strategy with that selection, then run it on 2022 without refitting. Repeat with the next chronological fold.
A slice of one full-sample run is labeled OOS even though parameters or transformations used the test window.
Verify a fresh fold-local strategy, training-only fitting and frozen choices for every OOS window; reject future-aware shared caches.
Purge, pre-OOS extension & embargo
Related but distinct exclusion rules used around train/test boundaries.
Purging removes observations near the train/test boundary when labels or holding periods can overlap the later test window. Classical embargo prevents observations immediately after a test window from entering later training sets.
QuantJourney 0.12.0 implements a fixed purge plus extra_pre_oos_purge_pct, which extends the same exclusion before the current OOS window. The legacy embargo_pct name is a deprecated alias for that pre-OOS extension; it is not a post-test embargo. An optional maximum holding period can increase the fixed purge.
Pre-OOS gap = max(purge_days, max_holding_period_days) + floor(extra_pre_oos_purge_pct × IS length).
If a training label uses a five-day forward return, at least the overlapping boundary observations should be removed before the test period rather than allowed to leak test prices into training labels.
A pre-OOS purge extension is mislabeled classical embargo, adjacent folds share overlapping labels, or the gap is treated as a cure for unrelated leakage.
Compare the pre-OOS purge with label horizon and holding period; if post-test embargo is claimed, inspect later training exclusions separately.
Probability of Backtest Overfitting (PBO)
A canonical CSCV estimator based on symmetric splits of a complete configuration-performance matrix.
Canonical PBO asks whether choosing the best candidate in-sample repeatedly selects configurations that fall into the bottom half out-of-sample. It uses the same full N-configuration performance matrix, symmetric CSCV combinations, IS winner selection and that winner's OOS rank among all N configurations.
QuantJourney 0.12.0 does not label its rolling top-K diagnostic canonical PBO. walk_forward_top_k_rank_failure_rate ranks the IS winner only within a preselected top-K subset on chronological folds. It is useful sensitivity evidence, but it tests a different construction and has no canonical PBO cutoff.
Canonical PBO = fraction of symmetric CSCV combinations where the IS winner's all-N OOS rank logit <= 0.
Build the complete T × N trial-return matrix, enumerate complementary CSCV train/test combinations, select the IS winner in each and rank it OOS among all N trials.
A fold-level ratio or rolling top-K rank diagnostic is labeled canonical CSCV PBO without the complete configuration matrix and symmetric splits.
For canonical PBO, verify the complete trial matrix and symmetric CSCV combinations. Otherwise require the rolling top-K diagnostic to be labeled explicitly and reported without a CSCV cutoff.
Deflated Sharpe Ratio (DSR)
A probability that discounts an observed Sharpe for multiple testing and non-normal returns.
Testing many parameter combinations makes the best observed Sharpe positive even when no candidate has durable alpha. DSR raises the comparison benchmark to reflect the number and dispersion of tried configurations, then accounts for sample length, skewness and kurtosis.
The QuantJourney implementation returns a probability, not another annualized Sharpe value. It needs the selected candidate, the population of trial Sharpes and return moments expressed in consistent per-observation units.
N should represent the effective number of independent trials. When dependence is not estimated, the raw finite completed-trial count is a conservative approximation and both raw and effective counts should be reported.
Sharpe decay is a separate diagnostic: QuantJourney measures the slope of OOS Sharpe across chronological folds. A negative slope can warn that alpha is weakening, while a positive slope is not reassuring when aggregate OOS performance remains negative.
DSR = P(candidate Sharpe exceeds the multiple-testing-adjusted benchmark Sharpe).
A Sharpe of 1.8 found after 500 noisy trials is weaker evidence than the same Sharpe from one pre-specified test. DSR makes that selection burden explicit.
The raw trial count is silently treated as independent, the effective-N assumption is undisclosed, or Sharpe periodicity is inconsistent with the observation count.
Verify the finite completed trial population, raw and effective trial counts, candidate return moments, observation count and consistent per-period Sharpe units.
Overfitting
Learning historical noise instead of a durable effect.
Overfitting happens when strategy rules, parameters or filters are tuned so tightly to the historical sample that they capture accidental noise. The backtest looks good because it was designed around that specific history.
The common symptom is a large gap between in-sample and out-of-sample performance, or a result that disappears when parameters move slightly.
Robust idea: broad parameter region works. Overfit idea: one tiny parameter island works.
A moving average pair of 47 and 213 works beautifully, but 45/210 and 50/200 fail. That is a warning sign, not precision.
The strategy works only in one narrow parameter island or one historical regime.
Perturb parameters and verify performance is not concentrated in a single fragile point.
Final audit checklist
- What data source, adjustment policy, universe and benchmark were used?
- Were indicators computed using only information available at the timestamp?
- Were target weights shifted before returns were earned?
- Do actual positions reconcile with target weights, cash and NAV?
- Were orders submitted before the bars that filled them?
- Were commissions, slippage and impact charged only on real trades or fills?
- Could daily OHLC hide an ambiguous stop-loss / take-profit sequence?
- Is turnover consistent with rebalance dates or fill records?
- Was performance tested out-of-sample or through walk-forward folds?
- Can another researcher reproduce the run from the archived research packet?