Making "the final call is discretionary" testable

Since building the pipeline that extracts trading rules from YouTube videos, I have verified quite a few video strategies. Every time, I hit the same wall: “the final call is discretionary.”

You can mechanize the setup conditions, but once the presenter says “whether to take it comes down to experience,” verification stops there. Discretion works as an unfalsifiable escape hatch.

So I built a framework that puts discretion itself on the test bench.

Goal: turn “it’s discretionary” into a numeric claim

The idea is simple — answer two questions with numbers:

1. If the discretion were perfect, how much would this method make? (measuring the ceiling)
2. How much judgment skill is needed to hit the advertised results? (back-solving the required skill)

With those two, “you win with discretion” becomes a testable proposition.

Method 1: bracketing the value of discretion

Backtest only the mechanizable part (the setup conditions) and treat every resulting trade as a “candidate.” Then measure three outcomes:

• Floor: no discretion — take every candidate
• Ceiling: an oracle who knows the future keeps only the best third
• Control: a dice roll keeps the same fraction (the baseline for comparison)

The ceiling is the key. If even future-knowing discretion cannot make the method attractive, no human expert can save it — and the verification can stop right there.

Method 2: back-solving the required skill

Model discretion as a decision-maker who holds real information with probability κ (kappa) and guesses otherwise. κ=0 is pure guessing; κ=1 means knowing the future.

Sweeping κ from 0 to 1 in simulation gives the minimum κ needed for any target performance. A claim like “80% win rate thanks to discretion” becomes a concrete question: “that requires κ=0.6 — can you prove you have it?”

Method 3: mechanizing the usual discretionary phrases

“Only when the trend is clear,” “only on strong candles,” “near a key level,” “only when there is room to run” — I implemented twelve such phrases as objective conditions computed purely from price data.

Applying all of them to the candidate trades shows which, if any, actually improve results. One caveat matters: try twelve conditions and some will look good by luck. A multiple-testing correction (FDR) filters out the lucky ones, keeping only real improvements.

What the measurements showed

Case 1: a video method advertising an “80% win rate”

An earlier verification had already shown its mechanizable core is a coin flip (51.9% win rate, PF 0.78, monthly -0.03%).

• Ceiling (perfect discretion keeping the best third): the win rate becomes 100%, but monthly return tops out at +0.10% (at 0.5% risk per trade). The setup appears only 79 times in 11 years — perfect selection cannot fix scarcity.
• Skill required for the advertised 80% win rate: κ=0.60 — the informational equivalent of knowing the future on two out of three calls. Not realistic.
• All twelve mechanized discretionary phrases: none distinguishable from chance (the best one only reached PF 1.09).

So the method fails twice over: (a) the discretion it demands is implausible, and (b) even if real, it barely pays. What used to end with “maybe it works with discretion” now has a definitive answer.

Case 2: a live mean-reversion logic (a surprising finding)

As a control, I ran the same tests on a logic I actually run live (floor: PF 1.44, monthly +0.22%).

The surprise: “cherry-picking only the good-looking trades” is itself harmful. Randomly keeping a third of the trades leaves PF at 1.45 but cuts monthly return to +0.07%. Recovering the lost volume through quality requires κ≥0.3 — and none of the twelve conditions came close.

If a logic already has an edge, do not layer human “selectivity” on top of it. That is a quantitative answer to a very common temptation.

Bonus: measuring your own discretion

The same framework measures human judgment directly. A quiz tool shows historical charts one at a time — symbol and dates hidden, no future bars — and asks “take or pass?” With enough answers, your personal κ can be estimated statistically (proving κ=0.3 takes about 80 answers).

If you want to succeed at discretionary trading, the first step is not hunting for methods or practicing — it is measuring your κ before risking money.

Takeaways

• “The final call is discretionary” becomes falsifiable via the ceiling (perfect-discretion performance) and the required skill (κ)
• The video method tested cannot be saved even with discretion included — now proven with numbers
• Adding “selectivity” to a logic that already works makes it worse
• Judgment skill can be measured before any money is at stake

Verifications cover only the publicly disclosed rules of published videos. All figures are measured on this site’s backtesting platform (real data, 2015–2026).