13,500 Model Fits Later: What Actually Works in Crypto ML (2026)

Most crypto prediction claims are hand-waving. "Our AI uses deep learning" or "powered by neural networks" — with zero evidence that any of it works. We took a different approach: 13,500+ individual model fits across 30+ systematic experiments, testing everything from gradient boosting to Transformers, on-chain metrics to macro-economic indicators. XGBoost with aggressive tuning and macro features won, hitting 54-56% accuracy with walk-forward validation. That's not a flashy number, but in trading, 55% accuracy with proper risk management compounds into real edge.

Every experiment used walk-forward validation — the model only ever sees past data when making predictions. No lookahead bias. No cherry-picked results. Over 40+ hours of compute time, we tested what actually predicts crypto price direction and what's just noise dressed up as signal.

Here's what survived.

What Machine Learning Models Did You Test?

We tested 8 different ML architectures in a controlled experiment — same features, same 10 cryptocurrencies (BTC, ETH, SOL, and 7 altcoins), same train/test splits across 3 timeframes (1-hour, 4-hour, daily). Binary classification: predict whether price moves up or down by at least 0.5% over the next 20 candles.

| Model | Avg Accuracy | Training Speed | Verdict | |-------|-------------|----------------|---------| | XGBoost (aggressive) | 54.9% | Seconds | Winner | | XGBoost (conservative) | 53.1% | Seconds | Baseline | | Random Forest (100 trees) | 52.3% | Fast | Replaced | | LSTM (seq=10) | 53.6% | Minutes/fold | Rejected | | GRU (seq=10) | 52.4% | Moderate | Rejected | | BiLSTM | 51.9% | Slow | Rejected | | TCN (32 filters) | 51.2% | Moderate | Rejected | | Transformer (1 layer) | 51.4% | Slow | Rejected | | Transformer (2 layers) | 50.1% | Very slow | Rejected | | LSTM+XGB Hybrid | 52.1% | Very slow | Not worth it |

XGBoost won across the board. Not by a dramatic margin on any single test, but consistently across every coin and every fold. A 2024 survey in the Journal of Machine Learning Research confirmed this pattern — gradient boosting methods outperform deep learning on tabular financial datasets with fewer than 10,000 training samples in roughly 80% of benchmarks.

The deep learning results surprised us. LSTMs, GRUs, TCNs, and Transformers all underperformed on our dataset size (2,000 training samples per walk-forward window). Deep learning needs 50,000+ samples to shine. With 2,000, a Transformer with just 35,000 parameters is already prone to overfitting. We wrote a detailed breakdown in XGBoost vs LSTM for Crypto: 8 Models Tested.

The data engineering behind all this was non-trivial. Coordinating feeds from multiple exchanges, aligning timestamps across different API conventions, and handling missing data across 10 coins and 3 timeframes requires solid extraction infrastructure. My background building 200+ data extraction actors on Apify — spanning financial data, government APIs, and web intelligence — meant we had the pipeline expertise to get this right from day one.

Which Features Actually Predict Crypto Prices?

The three most important feature categories for crypto ML prediction are volatility indicators (ATR, Bollinger Band Width), momentum indicators (RSI rate-of-change, MACD histogram), and macro-economic data (bond yields, VIX, dollar index). Feature selection from 100+ candidates down to the top 15 improved accuracy by 2-4%.

Here's what we found across the feature engineering rounds:

Top 15 feature selection was a big win. We extract 100+ technical indicators from price data. Using all of them gave ~52% accuracy. Selecting only the top 15 by importance pushed it to 54-56%. Less noise, better signal. The model was drowning in irrelevant inputs.

Lag and rate-of-change features added +1-2%. Instead of just feeding RSI = 72, we also feed "RSI 3 candles ago was 65" and "RSI rate-of-change = +7." This gives the model context about whether momentum is building or fading. Thirteen lag/ROC features made the final cut.

Macro data was the single biggest improvement: +3-5%. Bond yields, VIX, dollar index, gold prices, central bank meeting proximity — these added more predictive power than any individual technical indicator. A 2023 paper from the Federal Reserve Bank of St. Louis found that macro variables explain 25-35% of Bitcoin's variance at the daily timeframe, which aligns with what we saw. The daily timeframe showed the strongest improvement from macro features, which makes sense — macro data is daily, so there's natural alignment. Full details in How Macro Indicators Predict Crypto Prices.

On-chain metrics didn't move the needle enough. Only available for BTC/ETH with free APIs, and the improvement was marginal. Not worth the data pipeline complexity for now.

How Does Walk-Forward Validation Work?

Walk-forward validation trains a model on a fixed historical window, tests on the next unseen period, then slides both windows forward and repeats. It prevents lookahead bias by ensuring the model never sees future data during training. It's the most honest form of backtesting for time-series prediction.

Our setup: train on 2,000 candles, test on the next 500, slide forward, repeat. Over 10 coins and 3 timeframes, this produced 13,500+ individual model fits. Each fold is an independent test of the model's ability to predict on data it hasn't seen.

This matters because most crypto ML claims use random train/test splits — which means future data leaks into training. A model might learn that "when RSI is 30 and it's March 2024, price goes up" not because RSI at 30 is predictive, but because March 2024 happened to be bullish. Walk-forward prevents this entirely.

The downside is brutal honesty. Walk-forward results are always lower than random-split results. If someone tells you their crypto ML model gets 70% accuracy, ask them if they used walk-forward validation. If they didn't, those numbers are meaningless.

What About Signal Quality Filtering?

Raw predictions aren't enough. A model saying "buy" with 51% confidence and "buy" with 65% confidence are very different signals, but the raw output treats them the same.

Meta-labeling was the clear winner. This is a second model that predicts whether the first model's prediction will be correct. Think of it as a confidence filter. At our chosen threshold, only ~44% of signals pass — but those that do are significantly more accurate. We covered this in depth: Meta-Labeling: Filtering Bad Trades Before They Happen.

Drift detection was too aggressive. It checks if market conditions have changed since training. Good idea in theory, but with macro features included, it filtered out 75-100% of signals. The macro environment is always "drifting" — that's the point.

Regime-based filtering — adjusting for trending vs ranging markets — showed inconsistent results. Some coins benefited, others didn't. Not reliable enough to include.

Results by Timeframe

| Timeframe | Baseline | With Full Pipeline | Signals That Pass | |-----------|----------|-------------------|-------------------| | 1 Hour | ~53% | ~56% | 44% | | 4 Hour | ~52% | ~53% | 41% | | 1 Day | ~51% | ~56% | 39% |

Per-coin results varied a lot. Some assets are more predictable than others. The improvement from our pipeline ranged from negligible (+0.2% for some altcoins) to substantial (+15% for BTC on the daily timeframe in the best fold).

The daily timeframe is interesting — it had the lowest baseline but the highest improvement from macro features. Makes sense. Intraday noise drowns out macro signals on the 1-hour chart, but at the daily level, bond yields and dollar moves are directly relevant.

What Didn't Work

Honesty is part of the process. These approaches failed or weren't worth the complexity:

• Deep learning models — LSTMs, GRUs, TCNs, Transformers all underperformed XGBoost on 2,000-sample datasets. Not close.
• On-chain metrics — Only free for BTC/ETH, marginal improvement
• Order book data — Too noisy at hourly resolution. The order book is useful for real-time trading, not ML features
• Google Trends — Promising for BTC but the data source is unreliable (API frequently broken)
• 3-class prediction (bullish/neutral/bearish) — The neutral class dominates, destroying effective accuracy
• Exponential decay weighting — Recent data isn't always more relevant. Markets cycle.
• Sentiment data from social media — Too noisy and lagging. By the time Reddit is buzzing about a coin, the move is usually half done

Honest Caveats

These are backtest results. Walk-forward validation is the most honest form of backtesting, but real-world performance includes slippage, execution timing, and market microstructure effects that historical data can't capture.

56% accuracy means 44% wrong. That's a lot of incorrect predictions. The edge is slim. Meta-labeling helps by withholding signals the model isn't confident about, but nothing eliminates uncertainty. For context, Renaissance Technologies' Medallion Fund — arguably the most successful quant fund ever — reportedly hits around 50.75% accuracy but makes money through position sizing and risk management. Edge is about consistency, not being right most of the time.

No transaction costs included. The effective edge after exchange fees and slippage will be smaller than these numbers suggest. On most crypto exchanges, that's 0.1% per trade round-trip — which eats into a 56% win rate meaningfully.

We track live prediction accuracy separately on our How Our AI Works page and update it with real-world results as they accumulate. Nydar's AI signals feature uses this exact pipeline in production.

The Full Research Series

This is the overview. For specific findings:

1. This post — Overview of all 13,500 experiments
2. Why We Chose XGBoost Over LSTM — Deep learning vs gradient boosting
3. How Macro Indicators Predict Crypto Prices — Yields, VIX, dollar index, and gold
4. Meta-Labeling: Filtering Bad Trades — Two-stage prediction and signal quality

All methodology details are on the How Our AI Works page. If you want to explore options strategies that complement ML signals, or see how our Pine Script engine lets you build custom indicators on top of the same data, those guides cover it.

---

AI trading signals are probabilistic predictions, not financial advice. Past performance does not guarantee future results. Never trade with money you cannot afford to lose.

Last updated: March 2026