"Add non-steam game" script does not set compatibility tool #964
Comments
cchulo
changed the title
|
Good catch! Sorry you ran into that. The good news is I have some ideas on the problem. The tl;dr: the AppID we're creating is probably not unique enough as I can reproduce the issue if I have similar names and/or paths, but not if they are sufficiently unique, which they seem to have been in testing up to this point. I have some ideas on generating a more unique AppID, and I'll try to push it later today. I suspect we're not generating unique enough AppIDs and something is going wrong. An AppID is a unique identifier for a game on Steam. Games on the store have a fixed AppID, but Non-Steam Games can have any AppID so long as the format is correct. The format is most likely correct, as this has worked in my and others' testing, but if the AppID is not unique then Steam will do weird things. I confirmed that the AppID that STL generates is not always unique. I can re-create the problem if I use similar app names and paths: "Test Game 1" and "Test Game 2", in mostly the same path but with one or two directories different at the end, give the same AppID. So I suspect games in your I agree with you that it looks like it should have succeeded, however if AppIDs are the same, there will be a conflict and so the compatibility tool will be set incorrectly. In my tests up to this point, I guess the paths have all been sufficiently different so it has worked. Adding "Test Game 1" worked and set the compat tool, but "Test Game 2" generated the same AppID, and partially caused a conflict that Steam probably wouldn't fully resolve until the game launched, so the compat tool was not set. I have some ideas for how to generate a random number from a seed better, using an md5 hash, so I'll work on this hopefully later tonight. It should preserve the AppID being the same when the name is the same (so prior artwork/compat tools can be preserved) but also ensure the AppID is unique at least among games added by SteamTinkerLaunch and when the name and EXE path are unique. It is conceivable that with enough shortcuts, there could be overlap, but we're talking about millions of shortcuts 😅 Unfortunately, you will need to re-add your Non-Steam Games, though hopefully removing it from Steam (optional, just keeps the library cleaner) and re-running the command you posted should result in things working, once I get the fix up. This kind of commandline usage is the kind of use-case I envisioned for As a heads-up, though you may already know, if you have your artwork in the same folder as the exe and you name them If you need to bulk-remove shortcuts, ProtonUp-Qt from git has a shortcut editor which you can access from the games list. Though I'm very confident this is the issue, if you're interested, you could try to verify this: You can check your Then in Steam, from the Properties dialog, edit the compatibility tool for your Biohazard shortcut, and then go back to your Steam uses this to map what games (by AppID) are using which compatibility tool. If you see a block below it or a different block somewhere else being updated, then this means STL is inserting the block correctly, but since the AppID that STL creates is not unique, it is not reading this compatibility tool mapping entry. Thanks! |
sonic2kk
added
Non-Steam Game
|
2be16b3 contains the updated AppID generation code, let me know if that resolves the issues for you :-) I try to be fast when investigating problems but I'll be out for a bit tonight so the issue may persist for a while longer if the above commit didn't fix it. But hopefully that fixes it, in my tests the AppIDs are always unique and the cases before where I could reproduce the AppID overlap (my suspected root cause of this issue) were fixed by this commit. |
|
wow thank you very much for the speedy turn-around on this, wasn't expecting it! :) will check it out |
|
@sonic2kk it seems to work, but its a bit weird I readded it and it was assigned attached additional log above in the OP It might be working because this entry is now in my config.vdf permanently, it does not get cleaned up automatically after removing the non-steam game:
Side note, i kinda like how the random number generation is deterministic, not sure if that was a feature before The other thing I'm noticing is this file is edited/generated every time steam launches, so it might be getting information for what to set compatibilitytool elsewhere :\ |
At least there is a little bit of good news here :-) But that is a very odd quirk.
It is editing it because there are a lot of things in this file, and I don't think this file is being edited because other tools like ProtonUp-Qt edit this file to update the compatibility tool in use by a game, as well as for the bulk-update feature. It could have changed recently I suppose, but I think this file is working. You can check if it works by renaming Steam does seem to look elsewhere than
This is also odd... I thought Steam was meant to clean this up when you remove the entry from Steam. I'll test this now in fact. If you remove this entry from your
And on a related note, can you reproduce this behaviour generally? I will also test this from my side :-) If we can't reproduce the issue again I'm willing to chalk it up to some weirdness and leave it be.
Thank you! It all looks like it should be set properly from the logging: But it was useful to see and document nonetheless, thanks for attaching an updated log.
Heh, it was present before and was something that I didn't do intentionally at first but when I noticed it I thought it was useful since Steam will generate a seemingly random AppID. The original behaviour was due to the |
|
I can't re-create the problem, if you can't recreate the problem again either maybe this was some one-off weirdness. Something that is very weird, if I'm understanding your description correctly:
SteamTinkerLaunch actually won't update the I am not too sure what caused the quirk, if removing and re-adding the game worked, maybe for some reason the AppID for the game was picked up wrong. Because the AppIDs are deterministic, re-adding it should have used the same AppID, and so Steam would pick it up correctly. But if STL was able to re-insert the compatibility tool block into I did confirm just now that Steam does not remove these compatibility tool mapping entries from |
|
I reported this |
|
I'll leave this open for a bit to allow for further discussion and testing, as it can understandably take time to test and collate findings. But if the issue is resolved for you now you're absolutely free to close the issue :-) |
|
thanks I appreciate it, give me until Saturday, I got a bit tied up at work and haven't had time to come back to this quite yet. Will retest as soon as I can, Saturday the latest! |
|
There is absolutely no rush, real life should always come first. When an OP is active at trying to help with issues and has done their bit to help, I'm a lot more inclined to leave issues open compared to blank ones :-) |
|
okay i got around to retesting, this time using Biohazard 3, and still same quirky behavior as before: command used: ./steamtinkerlaunch ansg \
--appname="Biohazard 2" \
--exepath="/home/cchulo/Games/Biohazard3/BIOHAZARD(R) 3 PC.exe" \
--startdir=/home/cchulo/Games/Biohazard3 \
--iconpath=/home/cchulo/Games/Biohazard3/steamgrid/icon.ico \
--launchoptions="WINEDLLOVERRIDES=\"ddraw.dll=n,b\" %command%" \
--compatibilitytool=GE-Proton8-19 \
--hero=/home/cchulo/Games/Biohazard3/steamgrid/hero.png \
--logo=/home/cchulo/Games/Biohazard3/steamgrid/logo.png \
--boxart=/home/cchulo/Games/Biohazard3/steamgrid/boxart.png \
--tenfoot=/home/cchulo/Games/Biohazard3/steamgrid/tenfoot.jpg \
--copyRestarted steam and:
The log is attached on the original issue. I am sure that if i force the compatibility tool on, remove the game, re-run the command it will "stick" because the I can keep tinkering with it to see if i can figure out a workaround, i can report back if I find anything |
|
also, i realized that i accidentally left the Biohazard 2 label, see im creating these type of self-install .sh scripts, forgot to edit that part out for Bio3, im sure the label does not actually matter |
|
That is odd, I can't reproduce the problem. Your log also looks fine. I'll investigate some more tomorrow though. Before running anything agin, by the looks of the log, it created a backup of your I tested with the same label name and a similar path (on my home drive, different username ofc) and didn't have any trouble. Is the artwork being set correctly for your inserted game, and when inserted, before you open Steam, does the |
|
I don't have config.vdf.bak, i have a config_steamtinkerlaunch.vdf though and I do not see 3610055705 in either one of these btw there are a couple of more bugs i have since discovered around this feature, making it not viable for my purposes, do you want me to open two additional issues or should i go ahead and add them here? One issue: is that the --exepath has to be a fullpath, otherwise --startdir is ignored and will default to "." I can add these issues separately. |
Ah, my bad, yes it's called that (I think the first name was one I used briefly during development)
If it gets inserted into the new VDF file, under
This is intentional, this is true for any SteamTinkerLaunch path. All paths must be absolute, as noted on the help screen:
The reason
Steam should be closed when running steamtinkerlaunch ansg <args blah blah>
steamtinkerlaunch ansg <args blah blah>
steamtinkerlaunch ansg <args blah blah>If you're seeing this problem with Steam closed, then please open a separate issue. It sounds very strange as well. |
|
With Steam open, I can reproduce the behaviour of the compatibility tool not being set correctly. I just tested adding a game with Steam opened, and the compatibility tool was not set. I suspect Steam, when closing itself, overwrites the Removing the shortcut, closing Steam, and re-running the exact same command adds the shortcut with the compatibility tool selected as expected. If this resolves the issue for you, perhaps we just need better documentation that Steam should be closed when adding shortcuts. I was just thinking last night that the "Add Non-Steam Game" button should be removed from the main menu because it won't quite work as expected with Steam running. Especially when we start modifying I guess I incorrectly assumed that users would close Steam before performing these actions. It is probably not obvious that this should be done unless you're pretty close to how the whole thing works. |
|
The Add Non-Steam Game GUI does already note that the Steam Client should be closed before adding shortcuts. A note should probably be added to the help menu as well as the wiki. I will update both now. |
|
Updated the Add Non-Steam Game wiki page, and added a note to the help screen that the Steam Client should be closed before adding any shortcuts in 9b56004. |
|
ah yea i just discovered that before seeing your reply haha, sounds good! Will disregard the other two non-issues then. Will keep tinkering with the original issue for now. |
|
If the double-entry thing persists please feel free to open an issue, I'd be very interested in trying to fix that. I appreciate the tinkering too, the more testing for these features, the better! And in future if I find a safe way to expand the paths in a way that has minimal risk, I can implement. Or if someone opens a PR for it I'll accept that too :-) If your paths always start with the home directory, you could set them as And also, in the case of the command you showed, you can leave The reason echo "~/Games" # Will return ~/Games
echo "$HOME" # Will return /home/gaben/Games
[ ! -d "~/Games" ] && echo "Path '~/Games' does not exist"
[ -d "$HOME/Games" ] && echo "Path '$HOME/Games' does exist" # Note how the echo here still expands '$HOME'If you run into any more trouble please don't hesitate to ask or open another issue, because this feature is one I'm really excited about and am really happy to improve wherever possible, or explain shortcomings where there are some. A couple of other tricks you may be interested in with the
There are also a couple of shortcomings to note:
|
|
Did closing Steam before adding the entries resolve the issues? :-) |
|
oh oops idk why it appears like i closed this, i might've been distracted. Yes still experimenting with my install scripts, and I do suspect that having steam closed actually has an effect, though i need to retest with other non-steam games |
|
No problem, GitHub has a keyboard shortcut for closing issues that I end up doing by accident all the time :-) |
|
Seems like ensuring Steam is closed has resolved the problem, I can't reproduce anymore. Assuming this is fixed now, please re-open if the issue persists while Steam has been closed - ensuring the client has fully closed, sometimes it takes a few moments after the window disappears. Once the tray icon disappears, that's generally a good indication (if you have a system tray configured for your desktop). Thanks! |
|
@sonic2kk i found out that from time to time, valve updates their format slightly. What makes it difficult is the vdf binary format is hard, as well as the ascii vdf format since it uses tabs rather than spaces. I ended up writing my own python based tool that handles automating adding shortcuts using python with a pypi library that handles all the vdf reading/writing for me: https://github.com/cchulo/steam-conductor I do realize this might not be appealing solution to you, i think you keep everything in a single bash file so it is lightweight and as portable as it can be. My python program is portable since it bundles everything, including the python interpreter, into a single exe with pyinstaller, but it is a bit of a chonky .exe as a result. Still I thought i'd share since i did run into the same problem i ran into with your script. I figured out that valve, on occasion, updates the objects inside binary VDFs, breaking automation tools like this one, it takes a while to figure out what changed, but on the bright side, its python, and so its a bit easier to read and debug than a bash script. |
|
I don't think the Can SteamTinkerLaunch no longer set compatibility tools? The issue here was that the Steam Client was open when trying to write to The only format that should have changed with the Steam Client Beta is the
There was no call for this... |
|
Adding a shortcut with SteamTinkerLaunch and setting the compatibility tool still works fine on the latest Steam Client Beta, I just tested. If you tried to add a shortcut again and it didn't update the tool it's probably because there was an existing entry in The So the Python project you have linked, while indeed cool, doesn't affect this project, and I won't be switching over to using a Python utility when Bash works just fine for this project. We will most likely already soon have to introduce a small C++ dependency to parse Steam categories using LevelDB which I am not pleased about (I would like to figure out how to write a LevelDB parser in Bash for our purposes) and it would be considerably smaller than bundling a Python project. |
|
its not when I originally wrote steam-conductor, compattools were mapped under an object just called "CompatToolMapping", and it was working for a while, then suddenly it stopped, turns out that now CompatToolMapping is nested under 'InstallConfigStore.Software.Valve.Steam.CompatToolMapping' Like i had it working, then suddenly it stopped working after an update |
|
For the last while at least, it has always been under SteamTinkerLaunch parses for the first heading matching a given indent level, so even if the format did change, it would be a matter of SteamTinkerLaunch just bumping which indent level it should look at, if it can't guess the initial indent level (which it probably can). |
|
also understood, was not suggesting that you move to using my script, mine is going to have more specialized automation built into it for my purposes, but wanted to show you how the add_shortcut works, could give you clues as to why yours compat tool mapping does not work |
|
The
|
|
was not my intention to start an argument, simply put this issue was started because that feature never worked for me, it did add shortcuts, but quite simply it never mapped compat tools for me. i simply wanted to share that i created a python utility for doing the exact same thing that works, and was sharing it with you to see if maybe you could check it out, im glad it works for you, it does not work well for me, for whatever reason. |
|
Well it has worked for at least a couple of other users (can't find the exact comments but there were two users that were particularly interested in Add Non-Steam Game that tested it, one even uses it downstream for an FFXII script iirc), and has worked across a couple of devices for me including a very recent one. Sorry it doesn't work for you, but the solution here seems to work across devices and at least a handful of other users. If it never worked for you, it wasn't due to a format change at least. The format has been
So even if it doesn't work for you, the reason for why is very different than what you might have expected coming from Python. It does not work in the same way, as Bash does not have the same concept of data structures. The file is treated as a string that we run |
System Information
Issue Description
Writing some automation to automatically reinstall games to freshly imaged desktop and steam deck.
I run the following command, and while it mostly works, it failed to set the compatibility tool:
I did not get any errors, and inspecting the log, from first glance, looks like it would've worked.
I made sure compatibility tool was spelled correctly, as
GE-Proton8-19Logs
steamtinkerlaunch.log
steamtinkerlaunch-10-31.log
steamtinkerlaunch-11-4.log
The text was updated successfully, but these errors were encountered: