CCXT Websocket Integration

[tolgakaratas]

Describe your environment

(if applicable)

• Operating system: ____
• Python Version: 3.12.3
• CCXT version: 4.2.91
• Freqtrade Version: docker-2024.4-dev-094bbb3a

Describe the enhancement

Hi Freqtrade Team,

We've got an idea we'd love to share with you! How about adding the Websocket feature from CCXT PRO to Freqtrade? We think it could really boost our experience by speeding up how we process real-time data, especially when we're juggling multiple bots or crunching indicator numbers.

You know, the Websocket feature keeps data flowing in real-time, with minimal delays. Having that could mean we make our trades faster and more accurately.

Getting data quickly and acting on trading signals promptly is key, right? We reckon integrating CCXT PRO's Websocket feature could make Freqtrade even better, helping us navigate the markets with more agility and precision.

Adding this feature might just make managing our trading strategies smoother and help us respond quicker to market shifts.

We hope you'll consider our suggestion and maybe even give it a spin in development.

Thanks a bunch,
DigiTuccar

[xmatthias]

Well it's neither a new idea, nor will it improve performance as significant as you'd think.

In my tests, endpoints like "fetch_ohlcv" for a 50 pair whitelist provide you the data for all pairs ~3-4 seconds after the candle starts.
Changing this to the websocket implementation, you can get to roughly 1.5-2s (if you're playing it safe and assume a slight clock drift) (tested on binance spot).

Now i've only tested this for ohlcv data for now - which is a branch that will need more testing - and where this functionality will be limited to exchanges where behavior was tested.

---

While there's "probably" bigger advantages for other endpoints (like orderbook data) - the way ccxt currently works, implementing that is currently not feasible / reasonable.
The data transferred over the network becomes big pretty fast even for one pair.
While that can be neglected to some degree as "i'm in a position, so it does need more data" - this is made worse by the fact that ccxt doesn't allow unsubscribing from a websocket stream (you can no longer consume it - but the data in the background will still be sent to your system).

This means that for a bot with a 50pair whitelist, with a decent amount of trades and an average trade duration, you can probably expect to be subscribed to the orderbook stream for all pairs within a week (or max a month).

While the same will be true for volumepairlists, where you'll "drag along" all streams for old data - that's (probably) neglectible from a data transfer perspective - whereas for orderbooks, it's not.

So until unsubscribing is implemented, i don't see this as a feature we'll be going to add.

---

For the feature itself (websockets for ohlcv) - it's available and i'm testing it currently (already for quite some time, actually).
It's still missing a few documentation parts - and for sure tons of testing - but i can see to make that available as a PR soonish.

[aleksandar-babic]

This might be useful not just for performance improvements, but to avoid the rate limiting in certain exchanges (e.g Binance) when multiple instances of freqtrade are running from the same external IP.

[dnuno89]

This might be useful not just for performance improvements, but to avoid the rate limiting in certain exchanges (e.g Binance) when multiple instances of freqtrade are running from the same external IP.

I have this same exact issue but ... Would two freqtrade instances connected to the same exchange through WebSocket share the data feed?

[xmatthias]

Would two freqtrade instances connected to the same exchange through WebSocket share the data feed?

No, freqtrade instances are seen as independent and i don't see this changing anytime soon.

there's no guarantee that the pairlist or exchange is the same - and you can stop each instance at any moment.