Possible Enhancement: How difficult woud it be to add Syncthing to Lakka?

[jdalmanza]

This is really just a question / discussion:

Ive been trying to get the RetroArch native Cloud Sync feature to work and while i can make it work on my own self-hosted webdav server, i'm looking at all of the cloud storage providers and none of the mainstream vendors with a free option support webdav. Box, Dropbox dropped webdav functionality some time ago. You can kind of hack webdav into Google Drive but it is not permanent (3rd party plugin, files only last for 24 hours). I think this feature in and of itself is great, but I wish it was compatible with the popular cloud storage providers

I spend a lot of time in the RetroArch subreddit and one topic that seems to come up almost daily is how to sync data across multiple devices, rarely do I see people talking about the native Cloud Sync feature in RetroArch but most people are using (or are convinced by the community to use) Syncthing (https://github.com/syncthing/syncthing)

I have been using RetroArch forever and even I did not know that Cloud Sync was added into the 1.19.1 builds until @warmenhoven informed me in the lakka discord.

So, all that said, since Syncthing is open source and seems to run on just about anything, my question is: How Difficult would it be to add a Syncthing client, either as an option (protocol) in the native cloud sync feature, or as a separate menu item unique to lakka?

I think this would be a big win for the community. If I was a skilled programmer, I would try to port / incorporate / etc. myself, but unfortunately I am not.

What do y'all think? (I won't get my feelings hurt if I get shot down ... well maybe, but at least i won't complain about it)

[warmenhoven]

I wish it was compatible with the popular cloud storage providers

In principle this should be relatively straightforward but in practice the authentication part becomes quite complicated as most of the services require some form or derivative of oauth and RetroArch doesn't have a great way of supporting that in a cross platform way. Any services that don't require oauth (like webdav, or s3) are relatively trivial to implement.

most people are using (or are convinced by the community to use) Syncthing

Syncthing was designed and intended for use cases like this, and has far more configurability and functionality than RetroArch could reasonably be expected to provide. Using RetroArch's native cloud sync feature should be a last resort, for platforms that can't handle something like syncthing (tvOS in particular comes to mind).

How Difficult would it be to add a Syncthing client [...] as an option (protocol) in the native cloud sync feature

I looked into this a bit and it would be very difficult. The native cloud sync assumes that it's operating on a dumb CRUD file system abstraction, and it wants to own and manage all of the logic around what needs to be synchronized. Syncthing wants to own all of that. That's even before getting into the viability of integrating a large library in a language RetroArch's build doesn't already natively support.

... or as a separate menu item unique to lakka

I only looked into this extremely cursorily and I think this is a reasonable direction, though I would caution against integrating syncthing configuration/usage too deeply into RetroArch menus, and instead rely on its own web based configuration tool.

[achadwick]

👍 I'd really like Syncthing support as well. I use it in loads of other places I run RetroArch, and I already synchronize ROMs, states, saves, BIOSes and thumbnails. I think it'd be a natural match for portable devices that won't always be able to mount those things from a LAN's NAS.

Turns out it's "fairly" easy as a user to cross-compile Syncthing for Lakka, and then integrate it via systemd. It's written in Go, so the output is a static binary, which helps a lot.

Installing a Syncthing service on Lakka 5.0

This is just a brief HOWTO for power users. Doing this requires a Linux device you can compile on, and some command-line and home network setup skills to get it going.

Instructions: cross-compile and deploy

# Install build prereqs (Debian testing)
sudo apt install golang-1.22-go golang-1.22-src golang-go golang-src

# Grab source
cd ~/src
git clone https://github.com/syncthing/syncthing.git

# Cross-compile Syncthing and deploy it on Lakka 5..0
cd ~/src/syncthing
go run build.go -goos linux -goarch arm64 build
ssh root@lakka 'mkdir -p ~/bin'
scp ./syncthing root@lakka:/bin/

You can run the resulting binary on the target device out of ~/bin or wherever you put it, for testing and initial configuration. If you want it to run permanently, it'll need to be integrated with systemd so it runs automatically as soon as there's some networking available.

File: systemd integration .service file

# Put this in /storage/.config/system.d/syncthing.service
# For Lakka 5.0, syncthing git 2024-12-17 v1.29.0-rc.1.dev.10.gce3248ce
# Adapted from syncthing's etc/linux-systemd/system/[email protected]

[Unit]
Description=Syncthing for Lakka
Documentation=https://docs.syncthing.net/
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=4

[Service]
User=root
Environment=HOME=/storage
ExecStart=/storage/bin/syncthing serve --no-browser --no-restart --logflags=0
Restart=on-failure
RestartSec=1
SuccessExitStatus=3 4
RestartForceExitStatus=3 4

# Hardening
ProtectSystem=full
PrivateTmp=true
SystemCallArchitectures=native
MemoryDenyWriteExecute=true
NoNewPrivileges=true

[Install]
WantedBy=multi-user.target

Don't forget to reload systemd's services list, and enable syncthing.service if you want it to come up at startup.

(Click the details foldouts to reveal...)

See also

• I wrote up a fuller explanation of this with proper steps on my disroot. It contains notes on how to configure Syncthing itself.
• There's an old Syncthing thread on libretro's Lakka discourse.

Future Lakka integration ideas

It'll need a knob in Settings → Services, alongside SSH, Samba, Bluetooth etc. Off by default, naturally. Hopefully that's easy to do, and all it'd need to do is systemctl {enable|disable|status} syncthing.service.

I don't think that Lakka should present the Syncthing web UI on the device's screen. It'd be absolutely horrible with remotes and joypads, trust me.

Syncthing will need a canned configuration to make its web UI come up on 0.0.0.0 and [::], on a known port with a known username and password, like Lakka does for SSH. The default is localhost-only and no user/pass. It should use TLS/SSL by default, even if the user has to click through SSL certificate warnings to get in.

For the Syncthing web UI, maybe it's possible to to turn off the scary messages about running as root. It's not as if there's another option in Lakka/LibreELEC, after all, so it'd be nice not to bother the user about it.

Performance could be an issue, since Syncthing can be a trifle memory hungry. I ought to test this out on my spare Pi 3B, really, to see if it impacts emulation.

It could be a support issue too, and it'd need a bit of documentation for new users telling them how to set up synchronized folders for use in RetroArch under Lakka and some random RetroArch running on something else. Most existing Syncthing users are pretty OK with experimenting and changing configs, luckily!

[jdalmanza]

that's good stuff, i'm going to try to test that this weekend

[achadwick]

Updated the HOWTO to remove the --depth=1 parameter to git clone. Syncthing's a bit sensitive to its own git history during build: it needs to see tags to calculate the version string properly, and defaults to just a hash if it's in a shallow clone. Symptoms are syncthing failing at runtime:

Lakka:~ # bin/syncthing --version   # without tags in history
2024/12/17 18:44:21 Invalid version string "ce3248ce";
	does not match regexp ^v\d+\.\d+\.\d+(-[a-z0-9]+)*(\.\d+)*(\+\d+-g[0-9a-f]+|\+[0-9a-z]+)?(-[^\s]+)?$

Contrast with a correctly built syncthing:

Lakka:~ # bin/syncthing --version    # updated, with tags in git history
syncthing v1.29.0-rc.1.dev.10.gce3248ce "Gold Grasshopper" (go1.23.4 linux-arm64) andrewc@jackalope 2024-12-16 11:04:24 UTC