Was It Just Hindsight? Our Forward Validation Delivers a Decisive Conclusion

A beginner-friendly summary of the verification: “Was It Just Hindsight? Our Forward Validation Delivers a Decisive Conclusion”.

What’s the idea?

Today, we’re diving into a crucial experiment to see if we can find a truly sustainable edge in algorithmic FX trading. The big question is: can we pick a winning portfolio of currency pairs for the coming year solely based on their past performance, without any “peeking” into the future? To answer this, I designed what’s called a complete Out-of-Sample (OOS) testing approach. Think of it like this: imagine you’re picking a fantasy sports team for the upcoming season. You’re only allowed to look at player stats from previous seasons. You pick your best team, and then you see how they perform in the new, unseen season. That’s exactly what we did, but with currency pairs and trading strategies.

How I tested it

My goal was to eliminate any trace of selection bias – that sneaky tendency to pick what looks good after you already know the outcome (also known as hindsight). Here’s the rigorous process: For each year we tested (from 2020 through 2024):

I looked only at the data available before that specific year. So, when testing for 2020, I only used data up to the end of 2019.
Based on that historical data, I identified the top 6 best-performing currency pairs. The idea here was to find “past winners” using a specific set of standard technical indicators.
Then, and this is the critical part, I tested the combined performance of these 6 chosen pairs on the unseen data of that specific year. For example, the pairs chosen based on data before 2020 were then tested on the actual market data for 2020.
This process was repeated for each subsequent year, always making selections based only on prior data and testing on entirely fresh, unobserved market conditions. This is the essence of walk-forward testing, where your strategy constantly adapts to the latest historical data before being deployed into the future. This method ensures that any results we see are genuinely reflective of a strategy’s ability to predict future performance, not just fit past data.

What happened?

Spoiler alert: the results were quite definitive, and not in the way we might have hoped. Overall, across the five years of testing (2020-2024), this strategy resulted in a total loss of -8.3%. Ouch! Let’s break it down year by year:

2020: -7.6%
2021: -0.8%
2022: -5.9%
2023: +3.3%
2024: +2.7% As you can see, only two out of the five years showed a positive return, and those gains were modest, nowhere near enough to offset the earlier losses.

The Myth of Past Winners

This outcome delivered a critical blow to my earlier research (studies 20 and 21), which had shown promising results of +17.7% over 6-7 years. The stark reality revealed by this rigorous test is that those earlier good results were almost certainly a product of selection bias – or put simply, hindsight. We were inadvertently picking strategies and pairs that had performed well, but there was no guarantee they would continue to do so. In other words, simply picking “past winners” doesn’t mean they’ll keep winning in the future. The “edge” – that sustainable advantage in the market – wasn’t actually sustainable. Even with diversification (combining multiple pairs, which typically helps reduce drawdown, or the peak-to-trough decline in your capital), the strategy failed to generate consistent profits. Diversification is great for managing risk, but it can’t create a positive return if there isn’t an underlying positive expectancy (an average profit per trade) to begin with. It’s like having a very diverse portfolio of losing lottery tickets – you still lose!

What I learned

This research has led to some pretty strong, even definitive, conclusions:

Standard Technical Analysis: The Search Is Over

After years of rigorous testing, including clean data, walk-forward analysis, and complete forward testing, I can confidently state that standard price-based technical indicators (whether used individually, specialized, or combined) do not provide a robust, forward-stable edge sufficient for consistent, withdrawable profits. The search for such an edge within standard technical analysis is, for me, complete and definitively closed.

Where True Edge Lies

If standard technical analysis isn’t the answer, then what is? The only remaining genuine sources of a sustainable edge appear to come from:

User-specific ideas: Your own unique insights and hypotheses about market behavior.
Market views: A deep, nuanced understanding of economic forces and market psychology.
Non-price data: This is a big one! Information beyond just price and volume – think fundamental data, sentiment analysis, news processing, or even satellite imagery. This is where the truly unique advantages might be found.

The Unwavering Achievement: A Bulletproof Testing Platform

Despite the disappointing results for standard indicators, there’s a huge silver lining: I’ve built an incredibly powerful and disciplined testing framework. This platform incorporates:

Clean data: Ensuring our backtests aren’t skewed by bad information.
Walk-forward testing: Constantly adapting and testing on fresh data.
Complete forward testing: The rigorous method we just discussed, leaving no room for hindsight.
M1 intraday data: Using highly granular 1-minute chart data for precise analysis.
Monte Carlo simulations: Running many random variations of potential trade sequences to see how robust a strategy is under different market conditions. This multi-stage system is designed to ruthlessly reject “fake edges” and confirm “real ones.” It means that any new hypothesis, no matter how promising, can now be strictly judged: is it a genuine edge, or just another product of hindsight? This platform is an invaluable tool for any future exploration into the complex world of algorithmic trading.

How this connects

This verification builds on earlier ones (what failed before and what I tried this time, comparisons between approaches).