I’m building a Bitcoin quantitative analysis system with ChatGPT. What am I doing wrong?

I’ve been experimenting with building a Bitcoin quantitative analysis system using ChatGPT, and I’d like to get some honest feedback from people who have more experience with quantitative trading.

I’m not trying to build a “predict Bitcoin and get rich” bot.

My current approach is to build the system step by step:

data collection → data validation → market analysis → forward predictions → backtesting → paper trading → eventually, maybe, automated trading

Right now I’m still working mainly on the data and analysis side.

I’m collecting and testing different types of data such as price/volume, order books, funding rates, open interest, liquidations, derivatives data, macro data, news/events, and potentially on-chain data.

One thing I’m trying to be strict about is not trusting a data source just because an API works. I want to test data quality, historical coverage, stability, update frequency, limitations, and whether the data is actually useful.

I also want every prediction to be recorded before the outcome is known, so I can’t fool myself by changing the interpretation afterwards.

The first experiment I’m considering is something simple: predicting BTC’s market direction/state over the next 4 hours, recording every prediction, and evaluating it over a large number of samples.

If that shows any real value, I would move on to proper backtesting and paper trading. I don’t plan to put real money behind it until the system survives those stages.

I’m using ChatGPT heavily throughout the process — not just for coding, but also for researching data sources, designing tests, reviewing assumptions, finding weaknesses, and documenting results.

I know there are plenty of ways an AI-assisted project like this can go wrong: overfitting, data leakage, bad data, false confidence, transaction costs, regime changes, etc.

That’s actually why I’m posting here.

If you have experience in quant trading, crypto market data, or systematic trading, what would you tell me to watch out for?

What mistakes do beginners usually make when building something like this?

And if you think the whole approach is flawed, I’d genuinely like to hear why.

I’m here to learn, not to sell anything.