Probabilistic estimation of remote channels balance

[DerEwige]

I'm realy excited for this new feature in the upcomming v0.8.0 version:

Whenever an outgoing payment is attempted, eclair will use the result to keep track of the balance of channels in the path. The path-finding algorithm doesn't yet use these estimates: we want to gather real-world data and improve the model before we can actually use it. But a future release of eclair will leverage this data to improve payments reliability. See #2272 for more details.

I know it will only be passive for now.
But once the version is released (I do not plan to use the master branche but rather wait for the release), is there an option for me to share the information my node gathers with you?
To speed up the development?

[t-bast]

But once the version is released (I do not plan to use the master branche but rather wait for the release), is there an option for me to share the information my node gathers with you?

I don't think sharing your raw data with us would help, but you'll be able to analyze the estimates your node generates and check whether the payments you actually make are consistent with it or not: that would help figure out issues with the probabilistic model. @thomash-acinq should be able to share more info to explain how to analyze that data.

[thomash-acinq]

If you use Kamon, you can look at router.balance-estimates.remote-edge-relay to see how good is our prediction. It tells you for each edge in each payment attempt how confident we were that the payment would pass through this edge and if it actually did or not.
Sharing this data would help judge the model. Ideally, when our confidence is X%, we should have X% of payments succeeding.

[DerEwige]

@t-bast
So I've set up Promethus today and tried to get it working.

I've configured eclair to publish the info

eclair {
  ..........  
  enable-kamon=true
}

kamon {
  modules {
    apm-reporter {
      enabled = false
    }
  }
  
  prometheus {
    start-embedded-http-server = yes
    embedded-server {
      hostname = 0.0.0.0
      port = 6969
    }
  }
}

This works. I can acces http://localhost:6969/metrics and see the data

I've configured prometheus to connecto to itselfe and eclair to collect data

The selfe connection works fine, but eclair connectiong gives an error.
After googling a bit I came to this conclusion.

Some of the values in the eclair endpoint have to be not formated correctly, which leads to a parsing error
Any hints on how to figure out the error?
Or have you already found this bug and might have a fix in the master branche

[thomash-acinq]

Well, there is 3b2eb96 which fixes the metric we're interested in here. It may not be the cause of this error but you will need it anyways to get the router.balance-estimates.remote-edge-relay metric.

[DerEwige]

Ok I found the problem and will open an issue

[DerEwige]

#2532

The issue is with kamon, but I found a workaround and i can now scrap the eclair metrics from prometheus

[DerEwige]

I can no plot the data from eclair, just have to understand which metrics are for what and how to plot them in the way that is usefull