Why 40 Trades Are Not Enough to Trust a Backtest

For third-party reproduction, fix the estimand as “the true mean return after finite-sample uncertainty and confidence in its sign.” Do not substitute net profit, win rate, or a visually smooth curve for that target. Declare the horizon, account currency, included frictions, and operating-stop boundary before calculation. Any post-result change creates a new hypothesis and version, preventing the question from being selected after the answer is known.

Reconstruct the observation unit as “an approximately independent trade cluster rather than every consecutive signal counted separately” before treating rows as independent evidence. Report raw rows, parent trades, decisions, event clusters, and the denominator used for each average or standard error. Recompute Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity under more than one defensible aggregation rule so that a larger export is not mistaken for a larger information set.

Preserve the hash of the TradingView export and the symbol, timeframe, session, timezone, order-processing settings, costs, account currency, and Pine version. For finite-sample risk with forty trades, estimation error, heavy tails, and the loss of effective sample size from serial dependence directly affects reproducibility. Keep immutable source, normalized, and analysis layers separate, with every join, deletion, imputation, and conversion recorded in a transformation ledger.

The export identifies only what can be rebuilt from recorded time, price, quantity, and P&L. rare gaps outside the sample, missing regimes, and changes in the sampling rule requires additional evidence. Mark each causal link as observed, bounded by assumption, or externally unverified. This prevents estimation error, heavy tails, and the loss of effective sample size from serial dependence from being presented as a confirmed fact when the available data support only an interval or conditional conclusion.

Do not adopt the platform summary as ground truth. Independently state wins, losses, mean R, and rare-loss counts explicitly, then compute binomial intervals, mean intervals, quantile uncertainty, and the probability of observing no rare event. Reconcile total and row-level differences by sign, date, symbol, and order type. If discrepancies concentrate in the exact state associated with finite-sample risk with forty trades, treat that concentration as a primary finding rather than dismissing it as rounding.

Report Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity with intervals or resampling distributions, not point estimates alone. Match the uncertainty method to sample size, skewness, heavy tails, censoring, and selection history. If normal, quantile, and dependence-aware methods disagree on the sign, classify the edge as unidentified and show the minimum detectable effect and lower decision bound.

Do not narrow uncertainty with an IID shuffle alone. Resample the short trade sequence under several conservative block lengths, with IID results shown only as a benchmark using several fixed block lengths and stationary bootstrap. Preserve random seed, repetition count, wrap rule, and missing-data treatment. For each block specification, report the distribution of Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity, the rejection-side tail mass, and the rate at which the verdict changes sign.

Interrogate the mechanism “estimation error, heavy tails, and the loss of effective sample size from serial dependence” with lower quantiles, expected shortfall, influence, cluster length, and boundary-hitting measures. Historical maximum loss is not a loss cap. Define several absorbing or operating boundaries—capital, margin, mandate drawdown, and recovery time—and record which boundary fails first under each stress.

A flat commission deduction is not an execution model for finite-sample risk with forty trades. Allocate spread, slippage, financing, borrow, roll, conversion, rounding, and rejected orders to the relevant unit. Recompute Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity under base, upper-quantile, and crisis states while preserving the possibility that costs and losses worsen together.

Count the complete population of periods, symbols, timeframes, parameters, exits, filters, and metrics that were tried. Do not detach the attractive result for finite-sample risk with forty trades from rejected candidates, interim changes, or repeated validation reviews. Where appropriate, use PBO, SPA, and a Deflated Sharpe Ratio, and treat an unrecorded trial count as a material audit limitation.

Test whether finite-sample risk with forty trades is concentrated in one trend, volatility, liquidity, rate, or session state. Define regimes prospectively or on training data only. Report statewise Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity, occupancy, transition probabilities, and costs, then reweight the mixture to adverse but realistic future compositions.

For the forty-trade finite-sample risk case, the same trade set can follow different capital paths under another inception date, order, initial balance, rounding rule, or stop condition. Separate fixed quantity, fixed R, and percentage sizing, then use circular shifts and block orderings to recompute drawdown, recovery, and boundary hits. Equal terminal P&L does not imply equal path risk.

Perturb “sample size, rare-loss rate, tail thickness, and confidence level” one axis at a time before creating a joint sensitivity surface. Add the negative control “simulate only forty observations from a zero-expectancy process and measure how often attractive win rate, profit factor, and net profit arise by chance.” Predefine the grid and crisis rule so that neither the most favorable nor the most damaging cell is selected after inspection. Save the slope, curvature, and exact point where the decision boundary is crossed.

Have a second implementation state wins, losses, mean R, and rare-loss counts explicitly, then compute binomial intervals, mean intervals, quantile uncertainty, and the probability of observing no rare event, then compare critical row-level outputs. Regression fixtures should include empty files, duplicate timestamps, extreme costs, reverse ordering, missing values, and boundary cases. Agreement between implementations is insufficient if they share the same bad input, so separate data construction and review roles where feasible.

Predeclare the decision rule. This case passes only if “the interval lower bound—not the point estimate—clears the threshold and the probability of missing the assumed rare-loss rate is acceptable.” Near a boundary, disclose interval width and economic materiality rather than a binary badge. If only one favorable block length, cost state, or implementation passes, classify the result as assumption-sensitive rather than robust.

The evidence ledger must store the input hash, code version, settings, exclusions, “sample size, rare-loss rate, tail thickness, and confidence level,” block lengths, random seed, repetition count, and every scenario output. Keep exploratory and confirmatory results in separate namespaces and retain failed trials. When new TradingView data arrive, create a new version and track interval lower bounds, cumulative rare-loss counts, and departures from predeclared predictive intervals rather than overwriting the old result.

Translate statistical changes into capital consequences. A shift in expectancy, lower quantile, recovery time, or boundary risk caused by finite-sample risk with forty trades should be mapped to trade count, capital, margin, and continuation. A small per-trade difference can compound under high turnover, while a rare loss can be decisive near an absorbing boundary.

Separate hypothesis design, implementation, independent recalculation, and approval where practical. Stop automatically on material reconciliation error, unresolved missing data, non-reproducibility, or a predeclared threshold breach. Audit the chain “small sample → accidental win/loss composition → precise-looking decimals → overconfidence → reversal when an unseen loss arrives,” and monitor interval lower bounds, cumulative rare-loss counts, and departures from predeclared predictive intervals prospectively without turning a historical pass into a promise of future profit.

The estimand “the true mean return after finite-sample uncertainty and confidence in its sign” is identified only within the columns present in the TradingView export and the stated assumptions. If rare gaps outside the sample, missing regimes, and changes in the sampling rule cannot be observed, report bounds rather than a false point estimate.

Past estimates of finite-sample risk with forty trades need not belong to the same population after changes in rules, participants, volatility, costs, or data specifications. Track interval lower bounds, cumulative rare-loss counts, and departures from predeclared predictive intervals in rolling and regime-specific windows.

For forty-trade uncertainty, repeatedly applying the same diagnostic battery and editing until it passes turns verification into another optimizer. Every post-audit change therefore creates a new model version and requires untouched evidence.

Even if the interval lower bound—not the point estimate—clears the threshold and the probability of missing the assumed rare-loss rate is acceptable, the analysis does not establish tax, regulatory, behavioral, liquidity, order-size, or systems suitability. Separate statistical diagnosis from live-operating approval.

Deleting observations related to estimation error, heavy tails, and the loss of effective sample size from serial dependence may improve the result. Compare no deletion, conservative imputation, and worst-case imputation, and display how Wilson interval, mean-R interval width, rare-event miss probability, minimum detectable effect, and lower-tail data scarcity changes.

Run the control “simulate only forty observations from a zero-expectancy process and measure how often attractive win rate, profit factor, and net profit arise by chance.” If the control performs similarly, suspect processing rules or common market drift before attributing performance to the strategy.

After a provisional pass, log interval lower bounds, cumulative rare-loss counts, and departures from predeclared predictive intervals sequentially and stop on persistent departures from the predeclared predictive range. Diagnose implementation drift before reoptimizing history.

Multiple methods can agree because they share the same bad input or the same mechanism “estimation error, heavy tails, and the loss of effective sample size from serial dependence.” Give lower-tail outcomes, failed scenarios, and unresolved mismatches the same visual prominence as favorable results; test count is not proof of correctness.