Index EA on M1: Can It Survive Intraday Shocks? Unveiling Hidden Risks!
## What's the idea?
A beginner-friendly summary of the verification: “Index EA on M1: Can It Survive Intraday Shocks? Unveiling Hidden Risks!”.
What’s the idea?
You know how sometimes a trading strategy looks amazing on paper, or when you backtest it on daily charts, but you worry about what happens when things get really fast and furious during the trading day? That’s exactly the challenge we tackled recently! We’ve been working on an exciting strategy (let’s call it “Research 59”) that showed great potential for trading stock indices. But there was one big question mark, a “blocker” preventing us from fully adopting it: we’d only tested it on daily (D1) data. We hadn’t checked how it would handle the minute-by-minute (M1) ups and downs, especially those sudden gaps or sharp drops that can happen intraday. Our main concern was whether these rapid moves could cause the strategy to hit our internal “proprietary daily -5% loss limit.” It’s like having a race car that’s proven on the highway, but you need to see if it can handle the tight turns and sudden stops of a city race track!
How I tested it
To put Research 59 through its paces on an intraday basis, we needed detailed 1-minute (M1) data for major stock indices. We sourced this data from the FutureSharks/financial-data GitHub repository, which provided us with M1 OHLCV (Open, High, Low, Close, Volume) data for SP500 (US500), DAX (DE40), and Nikkei (JP225) from 2010 to 2018. This data represented 24-hour CFD (Contract For Difference) trading, which is perfect for our purposes.
Now, a quick note on data: While other sources like HistData.com offer more recent data (2019+), their download process uses dynamically generated JavaScript tokens, making fully automated data acquisition a bit brittle for continuous research. So, for future tests covering 2019-2025, we’ll likely turn to Dukascopy, which provides reliable tick and M1 data for free.
A Backtesting Engine Bug!
Before we could even look at the results, we hit a snag – and a pretty big one! While setting up the tests, we discovered a crucial bug in our backtesting engine, specifically in the function that reconstructs intraday daily performance (btengine.intraday.reconstruct_intraday_daily).
Here’s what was happening: When we combined the “unrealized P&L” (that’s your floating profit or loss on open positions) from different instruments, our system was making a mistake. Stock indices trade in different sessions (Asian, European, US), meaning there are times when one index is active and receiving price “ticks” (updates), but another might be closed or have no recent ticks. Our engine was mistakenly treating the unrealized P&L of the inactive positions as zero during these tick-less periods.
Imagine you’re tracking the total weight of three people, but one person temporarily steps off the scale. Instead of remembering their last known weight, you record them as weighing zero! This would cause the total weight to suddenly drop and then jump back up when they step back on. In our backtesting, this led to “flickering” unrealized P&L, creating what looked like “fake intraday sharp drops.”
Because of this bug, our initial (and incorrect!) reports showed that the strategy erroneously hit the -5% daily limit 10 times for indices, with the worst being a -5.99% loss. But after digging in, we found the actual swing in position value was a mere 0.31%! Talk about a false alarm!
The fix was to ensure that for each open position, we “forward-fill” (ffill) its last known unrealized P&L onto a unified timeline before combining everything. This way, we always use the most recent value, even if there are no new ticks for a moment. This correction was vital for accurate results. Interestingly, this bug had minimal impact on our FX core strategies (P&L changed from 4.05% to 4.00%), primarily because FX markets trade 24/7, so tick alignment is less of an issue.
What happened?
With the backtesting engine bug squashed and our data ready, we ran the strategy on SP500, DAX, and Nikkei M1 data for the period of 2015-2018. This period is particularly interesting because it includes some pretty wild market events: the August 2015 Flash Crash, the Brexit vote, and the February 2018 “Volmageddon” event (a massive spike in volatility). If the strategy could handle these, it would be a strong sign! And the results? Drumroll, please…
- Worst daily loss: A manageable 2.60% (this occurred on the day of the Feb 2018 crash!).
- Days breaching our -5% daily loss limit: A fantastic 0 days!
- Days breaching our -10% daily loss limit: Also 0 days! In other words, the strategy, when applied to these indices with a risk setting of 0.005, successfully passed our proprietary intraday risk rules for this period! The “long trend” strategy for indices demonstrated healthy intraday risk management. That’s a huge relief!
What I learned
This is a really positive preliminary result for the “promotion” of Research 59. Our initial concern about intraday gaps and sharp drops for indices didn’t materialize in the 2015-2018 data, even through some significant market turbulence. This means we’ve cleared a major hurdle! However, we’re not quite done yet. This is an important step, but there are still a couple of crucial areas we need to cover:
- The missing period: The biggest untested stress period is still ahead: 2019-2025, especially the infamous March 2020 COVID crash. That event saw “limit down” moves and -10% daily drops, and it’s absolutely critical to see how the strategy performs under such extreme conditions.
- More instruments: We also need to test it on the full “US3” suite of indices: US500 (S&P 500), US100 (Nasdaq), and US30 (Dow Jones). The Nasdaq (US100) in particular is known for its higher volatility, so that will be an important test. We’ll need to acquire the 2019+ M1 data (likely from Dukascopy, as mentioned) to tackle these remaining verification steps. For now, there are no definite system changes based solely on these results, but it’s a very encouraging sign that we’re on the right track!
How this connects
This verification builds on earlier ones (what failed before and what I tried this time, comparisons between approaches).