CPU consumption on driver station causing packet loss.

[Crossle86]

This bug was completely on the WPILib side, so packet loss wouldn't have been affected. At this point, I wonder if there is something else running on your driver station causing that much packet loss. But glad to hear joysticks are working again. That was very much not what I would have expected the bug to be, especially being a runtime bug.

Originally posted by @ThadHouse in wpilibsuite/allwpilib#3896 (comment)

[Crossle86]

I wanted to continue this discussion a bit. As you see in the DS log posted in #3896 we have very high packet loss. The suggestion of excessive cpu consumption on DS triggered me on a long standing issue with Shuffleboard. We see very high cpu consumption by the OpenJDK process that runs shuffleboard. We also see high cpu by the DS process was well. Historically shuffleboard as very slow to load and the DS PC is pretty much useless during SB load. However when SB finishes loading, the behavior of SB on the DS and robot and the controls etc appear to be normal. So while cpu was observed unusually high, since things worked, this issue kept getting pushed down and eventually became "accepted" as normal. Now our DS PC is very small and not that powerful we have been using it for years and we accepted startups could be slow and as long as it worked when operating the bot we ignored it.

Now I'm wondering if this is the explanation for the packet loss. I am going to install 2022.2.1 as it has a shuffleboard fix and do dome testing on a different more powerful PC, revisiting this cpu consumption issue and will post the results.

[Daltz333]

It might be worthwhile to have minimum recommend specs required for Shuffleboard. Due to it's complex usage of JavaFX objects and it's NT polling, it's not a CPU/RAM friendly application (but not a particularly "demanding" one at that).

Additionally, it's different per computer.

On my computer, running the ElevatorSimulation example in WPILib. I maintain a steady 1-3% (spiked to ~13% on moving widgets or opening UI) when connected via simulation. A static 500-700MB in RAM.

However, it does seem to be using a decent amount of GPU performance (4-13% GPU). I am on a desktop with a dedicated GPU, so that GPU overhead could definitely be a bottleneck on some laptops. This is likely due to the rendering in the JavaFX objects as that is hardware accelerated.

Can you share the system specs of the computer running Shuffleboard?

CPU (Name and Clock)
GPU (Dedicated or Integrated)
How much RAM?

Additionally, any screenshots of task manager when Shuffleboard is running is appreciated (A full screenshot please, not just Shuffleboard!). Even with CPU usage near maxed, I wouldn't expect that to interfere with robot control. What is your disk usage looking like? A common failure point of laptops and desktops alike is the hard disk failing (when this is maxed out, it will destroy network communications!)

[Crossle86]

After reading this it does support the idea that our DS PC is too lightweight for Shuffleboard. Prior to 2022, SB was slow at points but worked acceptably and ran our robot without issues.

Now looking at things today I have proven (using DS log) that SB is the cause of the high packet loss and latency. So that question is answered. Again, even with the loss and latency, the robots operates correctly. So while not ideal, the situation is understood and stable.

With the arrival of SB 2022 (both in beta and now 2022.2.1) SB will not load reliably and if it does load may run fine or may crash (just disappears) after a minute or so. This suggests that SB has increased its load and maybe we are just getting beyond what our DS PC can handle. To explore this, I will substituting a more powerful PC and we will see if it can handle 2022 SB.

The specs for our small DS PC are:
Lenovo Celeron N3350 1.1 GHZ
2 GB memory

This PC handled the old labview dashboard ( which we used before SB) fine and we switched to SB because the labview DB was too much trouble to maintain. SB has functioned for us last few years but we may be getting to the end of that. I expect the packet loss and latency was there all along but as I said, things worked so as is typical in a small team, that was put aside for another day so we could address whatever was currently on fire.

If a better DS PC runs SB without problems, then this issue is on us. If not, then there may more to this than our PC. I will report back on testing this next few days.

[ThadHouse]

Do you use any graphs in shuffleboard? One of the big changes this year was in the graphing libraries, which resulted in a reduction in CPU usage, but at the cost of memory usage. We bumped the minimum recommended RAM requirements to 4GB this year, as did NI for the DS. 2GB of RAM is very difficult to work in nowadays. So its possible a memory upgrade would help, as that CPU isn't actually that bad, relative to the current KOP laptops.

[PeterJohnson]

Note you can always choose to run an older version of Shuffleboard, e.g. 2019. It won't have a couple of newer features, but it really hasn't changed that much in the last few years and will interoperate fine with 2022 robot code.

[Crossle86]

No graphs, but, yeah, our little DS PC is probably not up to the job. Given what you have said, I will probably fall back to 2021 SB for the short term. A completely new DS with new PC is on our list of projects anyway. Thanks for the help.