How to Backtest a Trading Strategy and Trust the Result

Every guide to backtesting tells you to split your data, avoid curve fitting and watch the profit factor. Almost none of them mentions the setting that determines whether the numbers mean anything at all. Before comparing any two results, it is worth establishing what a drawdown figure does and does not tell you, because the sampling interval alone changes the number.

A backtest is a measurement, and like any measurement it depends on the instrument. Change how the tester generates prices inside each bar and the same rules on the same history produce a different result, with no warning that anything has changed. A rule that classifies a level as valid only after seeing what price did next belongs in this category, which is why rules that cannot be scored in advance have to be rewritten prospectively before any test means anything.

What follows covers the parts that decide whether a result is worth acting on: how prices are modelled, how the data is divided, what the costs assume, and which strategies cannot be tested at all.

What a Backtest Can and Cannot Prove

A backtest applies a fixed set of rules to historical prices and records what would have happened. That is the whole of it, and the limits follow directly from the definition.

It can prove that a rule set was profitable on a specific sample, under specific cost assumptions, at a specific price resolution. It can also prove the opposite, which is the more useful outcome.

It cannot establish that the behaviour that made the rules work will persist. History contains one realised path out of the many that were possible, and a strategy tuned to that path is describing it rather than explaining it.

The distinction worth holding onto is between rejecting and confirming. A backtest is good at rejecting a rule set that loses money across a long sample under honest costs. It is much weaker as confirmation, because a result can be manufactured by adjusting parameters until the past cooperates. Stepping through history one bar at a time is a different exercise again, and our page on replaying bars by hand covers what that can and cannot settle.

Define the Rules Before You Touch the Data

The order of operations matters more than any setting. Rules written after looking at the chart are fitted to what you already know happened.

A testable rule leaves no decision to the person running the test. Entry, exit, stop placement, position size and the handling of edge cases all have to be specified in advance, in terms a machine could follow without asking a question.

Most rules fail this before testing begins. A condition such as a strong breakout on rising momentum is not a rule until strong and rising are defined as numbers, and choosing those numbers after seeing the chart is where fitting starts.

Position sizing belongs in the specification too, because it changes the equity curve and therefore the drawdown. Testing entries with a fixed lot and then trading them with a percentage of equity is a different strategy, so the sizing method from your risk management rules should be in the test.

If the rules came from an idea rather than from a chart, the work of building a strategy from rules is already most of the way to a testable specification.

Tick Generation: The Setting That Decides What You Measured

This is the part the popular guides omit, and it does more damage than curve fitting because it is invisible in the output.

Historical data is stored as bars. A bar records an open, a high, a low and a close, but says nothing about the order in which the high and the low occurred. The software must therefore decide what the market touched, and in what sequence, inside every bar.

The MetaTrader 5 Strategy Tester exposes that decision as a setting. MetaQuotes documents the modes as Every tick, Every tick based on real ticks, 1 minute OHLC, Open prices only and Math calculations.

The differences are substantial. Every tick based on real ticks uses real ticks accumulated by brokers with no simulation, which MetaQuotes describes as providing conditions close to real ones. Every tick simulates all ticks and is described as the most accurate simulated mode but the slowest.

At the other end, 1 minute OHLC emulates only the four prices of each minute bar, and Open prices only models OHLC prices but uses only the open price for testing. Math calculations does not download history or generate ticks at all.

Here is the consequence that matters. Suppose a position holds a stop below and a target above, and during one bar the price reaches both.

Which one was hit first determines whether the trade was a loss or a win, and a mode that models only a handful of prices per bar cannot answer that question. It resolves the ambiguity by a rule of its own, and reports a result either way.

Nothing in the output marks that trade as uncertain. The equity curve looks the same, the profit factor prints a clean number, and a strategy whose entire edge lives inside the bar has been measured by something that cannot see inside the bar.

The practical instruction is short. If any part of the logic can trigger between the open and the close of a bar, test on real ticks. If the result changes materially when you switch modes, the strategy depends on intrabar path and the low-resolution number was never valid. The same resolution question explains why a live alert and a tested signal disagree on identical data.

Data provenance matters as much as tick generation. A continuous futures series is stitched from separate contracts, which our page on currency futures explains, so its older levels are adjusted rather than traded. The same concern applies to charts whose bar close was never a traded price, since a Renko brick or range bar is printed only after price has already covered the distance.

Splitting In-Sample From Out-of-Sample Data

The standard defence against fitting is to develop on one part of the history and test on another that you have not looked at.

The development portion is the in-sample data, where rules are chosen and parameters set. The remainder is out-of-sample, held back untouched, and the result there is the closest thing available to a blind test.

Fitting works by absorbing the specific accidents of a sample, and those accidents do not repeat, so performance surviving on unseen data is likelier to be structural.

The weakness is procedural rather than statistical. Nothing stops you looking at the out-of-sample result, going back to adjust a parameter, and running it again. Do that a few times and the held-back data has been used for fitting, and there is no longer any blind test left.

This is why the discipline matters more than the ratio. A split you can revisit at will provides the appearance of validation without the substance.

Built-In Forward Testing and Why It Beats a Manual Split

The comparable guides all describe splitting the data by hand. MetaTrader 5 removes the temptation by making the split part of the run configuration.

MetaQuotes documents the forward testing options as No, meaning forward testing is not used, then 1/2, 1/3 and 1/4, which allocate that fraction of the specified period to the forward test, and Custom, where the forward start day is set manually.

The documentation also states that the second and most recent portion of the total period is consistently used for the forward test. So the held-back data is always the more recent stretch, which is the correct orientation.

The advantage is that the division is declared before the optimisation runs rather than applied afterwards. The optimiser searches the in-sample period, and the forward result is produced by the same run as a separate output.

That does not make fitting impossible, since anyone can re-run with different settings. It removes the casual version, where the split is notional and the boundary moves whenever the answer disappoints.

If you are choosing between the two platforms for this work, the tester is one of the clearer differences covered in MetaTrader 4 and MetaTrader 5 compared.

Cost Assumptions: Spread, Commission and Slippage

A backtest is a model of trading, and the costs in the model are assumptions. Getting them wrong is the most common way a losing strategy appears profitable.

Spread is the first. Testing on a fixed spread assumes a cost that does not widen, when in practice spreads widen around news, at session changes and in thin conditions, which are precisely the moments many strategies choose to act.

Commission has to be included where the account charges it, and per lot per side is not the same as per lot round turn. A strategy trading frequently for small gains can be turned negative by this alone.

Then there is the gap between the requested price and the achieved price. A tester generally fills at the modelled price, whereas a live order can fill away from it, and the mechanics of that are covered in slippage between the expected and filled price.

Handle all three by testing under deliberately unfavourable assumptions. If the strategy survives costs worse than you expect, the estimate has margin; if it only works on optimistic costs, the edge is the assumption.

One further caution applies to anything driven by scheduled data. Figures in an economic calendar are revised after publication, so a test run today may be reacting to numbers that were not the numbers available at the time.

Reading the Result Without Borrowed Thresholds

Several of these statistics reappear on a live account export, where they are computed the same way but from executed trades rather than simulated ones, as the same fields on a live account statement sets out.

Trading blogs circulate a set of numbers that sound authoritative: a profit factor above a certain level is solid, a strategy needs a minimum number of trades, live performance degrades by a fixed percentage.

None of the readable comparables sources any of these. They are conventions passed between articles, and treating them as standards imports someone else’s arbitrary line into your decision.

What actually governs the reliability of the estimate is simpler and less quotable. A result from very few trades tells you almost nothing, because a handful of outcomes can be produced by chance regardless of whether an edge exists.

The spread of outcomes matters as much as the average. Profit concentrated in a few exceptional trades is a weaker proposition than the same total earned steadily, because it depends on those trades repeating. Reducing that spread to a single comparable figure is what a Sharpe ratio attempts, subject to its own conditions.

The most informative question is usually about stability rather than magnitude. If the result collapses when a parameter is nudged, when the period shifts, or when the tick mode changes, the number was never describing an edge.

Strategies That Cannot Be Backtested at All

Some approaches are not hard to backtest. They are impossible to backtest, and the exercise produces a number that means nothing.

A discretionary strategy is the clearest case. If entry depends on the trader judging that a setup looks convincing, there is no rule to encode, and manually clicking through history measures your memory of what happened next rather than the method.

Anything depending on information not in the price series is in the same position. If a decision uses order book depth, positioning data or a read of the news, and the tester holds only bars, the test is running a different strategy from the one you trade.

Strategies conditioned on rare events are technically testable and practically uninformative. If the trigger has occurred a handful of times, no sample can distinguish a real edge from coincidence.

The honest response to all three is not to force a backtest but to say what the evidence is instead: a recorded forward test, a documented log of decisions taken in advance, or an acknowledgement that the approach rests on judgement. Reporting a fabricated backtest for a discretionary method is worse than reporting none, because it dresses judgement as measurement.

Strategies whose payoff depends on a counterparty honouring it belong in the same group. Our page on forex arbitrage covers why a result captured in a test can still be cancelled under a client agreement.

Frequently Asked Questions

What is backtesting in trading?

Backtesting means applying a fixed set of trading rules to historical price data and recording what the result would have been. It measures how one specific rule set performed on one sample of history, under whatever cost and price-resolution assumptions the test was configured with, which is narrower than it is usually presented as being.

How much historical data does a backtest need?

Enough to produce a number of trades large enough that chance is not the likely explanation, which depends on how often the strategy trades rather than on a fixed span of years. A short period covering hundreds of trades can be more informative than a decade producing twenty. The circulated minimum trade counts are conventions with no source behind them.

What is the difference between in-sample and out-of-sample testing?

In-sample data is the portion used to develop rules and choose parameters. Out-of-sample data is held back untouched and used once, as a blind check on whether the result survives on data that did not shape it. The value depends entirely on not revisiting the out-of-sample portion after seeing the outcome.

Does a good backtest mean the strategy will work live?

No. A backtest describes one realised path of history under modelled costs and a chosen price resolution. It is far more reliable at rejecting a rule set that loses money than at confirming one that makes money, because a profitable result can be produced by adjusting parameters until the sample cooperates.

Can a discretionary strategy be backtested?

Not meaningfully. A backtest needs rules a machine can follow without judgement, so if the entry depends on a setup looking convincing there is nothing to evaluate. Clicking through history by hand measures your recollection of what followed rather than the method, because you already know how each chart resolved.