Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cant install gintro on Arch. Using libsoup2 and libsoup3 in the same process is not supported. #190

Open
gavr123456789 opened this issue Nov 28, 2021 · 39 comments

Comments

@gavr123456789
Copy link

gavr123456789 commented Nov 28, 2021

First we try generating bindings for GTK4, this may fail when GTK4 is not properly installed
on your computer. But don't worry, you can still use GTK3
gtk4....................: Remaining delayed methods: 0 (Fine!)
gdk4....................: Remaining delayed methods: 0 (Fine!)
gdkx114.................: Remaining delayed methods: 0 (Fine!)
gsk.....................: Remaining delayed methods: 0 (Fine!)
graphene................: Remaining delayed methods: 0 (Fine!)
gobject.................: Remaining delayed methods: 0 (Fine!)
glib....................: Remaining delayed methods: 0 (Fine!)
gio.....................: Remaining delayed methods: 0 (Fine!)
gdkpixbuf...............: Remaining delayed methods: 0 (Fine!)
gmodule.................: Remaining delayed methods: 0 (Fine!)
Failed to load GtkSource
Requiring namespace 'Gtk' version '3.0', but '4.0' is already loaded
Maybe for your OS you have to install additional GTK related packages?
We continue with the remaining packages...
atk.....................: Remaining delayed methods: 0 (Fine!)
pango...................: Remaining delayed methods: 0 (Fine!)
pangocairo..............: Remaining delayed methods: 0 (Fine!)
pangoft2................: Remaining delayed methods: 0 (Fine!)
fontconfig..............: Remaining delayed methods: 0 (Fine!)
freetype2...............: Remaining delayed methods: 0 (Fine!)
harfbuzz................: Remaining delayed methods: 0 (Fine!)
rsvg....................: Remaining delayed methods: 0 (Fine!)
xlib....................: Remaining delayed methods: 0 (Fine!)
notify..................: Remaining delayed methods: 0 (Fine!)
adw.....................: Remaining delayed methods: 0 (Fine!)
soup....................: Remaining delayed methods: 0 (Fine!)

(process:69675): libsoup-ERROR **: 22:27:13.714: libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported.
stack trace: (most recent call last)
/tmp/nimblecache-4098377420/nimscriptapi_3780256293.nim(199, 29)
/tmp/nimble_69566/githubcom_stefansalewskigintro/gintro.nimble(77, 7) installBefore
/tmp/nimble_69566/githubcom_stefansalewskigintro/gintro.nimble(64, 7) prep
/usr/lib/nim/system/nimscript.nim(273, 7) exec
/usr/lib/nim/system/nimscript.nim(273, 7) Error: unhandled exception: FAILED: /tmp/gintrosalewski/gen 1 [OSError]
       Tip: 4 messages have been suppressed, use --verbose to show them.
     Error: Exception raised during nimble script execution

And yes, I have installed libsoup 2.74.2-1 and libsoup3 3.0.3-1

@gavr123456789
Copy link
Author

related #183

@StefanSalewski
Copy link
Owner

Strange. I had the hope we fixed that.

I assume that you did some GTK update?

Well, if I remember correctly, we had a discussion that a really clean fix would need a very large rewrite of the whole generator script?

@StefanSalewski
Copy link
Owner

For me testing would imply installing latest GTK4 from git sources, which unfortunately is some initial work. And indeed I did not found any motivation doing some Nim work at all in the last 4 weeks, with the exception shipping some sections of the Nim book which I mostly had already on my local box or at least in my head.

Are you already able to do fixes on the gen.nim file and then run it? If you can do that, you may move the line with

main("Soup", "3.0") # process Soup before Nice, preventing the load of not matching libsoup version

upwards, that was what we did last time. Maybe adw or another lib is loading libsopup2.0. When that moving up does not help, then someone would have to do a drastically rewrite.

Have to watch AnneWill in German TV now, about latest Covid news.

@StefanSalewski
Copy link
Owner

Sorry, can not reproduce.

First I tested my current GTK4 install --works.

Then I have completely reinstalled GTK4 from git sources with all the corresponding modules like adw, libnice, libhandy and that, all from git sources. I leave out webkitgtk for GTK4, because that takes an hour to build and needs a lot of free space on the SSD. But I think that can not be the problem. So I assume that you have at least one lib that is built before libsoup3 and that loads libsoup2. So it would be good if you can test yourself.

For the rewrite, I think we would have to split the gisub files then into many, one for each module, maybe integrated into the modules. And then we have to process each module fully isolated on its own, in its own process. Of course we have to modify the mconnect macro again, as it has not to search in the current gisup files but in the new location then. I would guess that we ca get the filename for the gisup content from the first parameter of the arguments of connect -- e.g. for connect("clicked", myButton) owner(myButton) would give us gtk4, and so we search for the gisup content in gtk4 module then.

What do you think, may that work?

@gavr123456789
Copy link
Author

gavr123456789 commented Nov 29, 2021

So, as I understand, you cant reproduce issue, yesterday itwas fixed for me with commented both libsoup 2 and 3.
Now Im in office with different pc, will try to reinstall gintro, update gtk and reinstall gintro again to check will this reproduce.
For the rewrite, ofc I agree that processing of every GIR file in separate process is a good idea.

@gavr123456789
Copy link
Author

yes, all as I expected, before package update all works, after -- I got save error about libsoup.
Also I wanna mention the difference between this error and error from #183
In 183 there was GLib-CRITICA, and now its
(process:3852): libsoup-ERROR **: 14:48:29.760: libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported.

@gavr123456789
Copy link
Author

If I comment in gen.nim only first soup

    # main("Soup", "2.4") # process Soup before Nice, preventing the load of not matching libsoup version
...
  main("Soup", "3.0")

I still got the error libsoup3 symbols detected. Using libsoup2 and libsoup3 in the same process is not supported
So, may be some of other libs start to use libsoup 2, and didnt do that before.
Now the temp solution for me is comment both soup in gen.nim

@StefanSalewski
Copy link
Owner

From your initial post the issue should be in

  else:
    echo "First we try generating bindings for GTK4, this may fail when GTK4 is not properly installed"
    echo "on your computer. But don't worry, you can still use GTK3"
    main("Gtk", "4.0") # the 3 new 4.0 releases
    main("Gdk", "4.0")
    main("GdkX11", "4.0")
    main("Gsk") # and two new ones for gtk4
    main("Graphene")
    main("GObject") # and the old common onces. GObject before Glib!
    main("GLib")
    main("Gio")
    main("GdkPixbuf")
    main("GModule")
    main("GtkSource")
    main("Atk")
    main("Pango")
    main("PangoCairo")
    main("PangoFT2")
    main("fontconfig")
    main("freetype2")
    main("HarfBuzz")
    main("Rsvg")
    main("xlib")
    # main("Vte") # not yet available for GTK4
    main("Notify")
    # main("Handy") # not yet available for GTK4
    main("Adw") # replaces libhandy for GTK4
    main("Soup", "3.0") # process Soup before Nice, preventing the load of not matching libsoup version

So commenting the last line of that block should fix it, and hopefully moving that line up may fix it also? That is moving up multiple steps.

Maybe you can use ldd to find which lib uses libsoup. For me it is only libnice.so

When you should have adw use of libsoup, then maybe exchanging the last two lines would work. If not, you would have to move main("Soup", "3.0") more upwards.

But yes, all that is ugly.

@trisub
Copy link

trisub commented Jan 8, 2022

I think the problem might (also) be related to different versions of WebKit. I have webkit2gtk and webkit2gkt-4.1 installed and they pull in libsoup2 and libsoup3 respectively. When I comment out the main("Soup", "3.0") I can install gintro, but I get a warning:

adw.....................: Remaining delayed methods: 0 (Fine!)                                                                                                                                 
nice....................: Remaining delayed methods: 0 (Fine!)                                                                                                                                 
cairo...................: Remaining delayed methods: 0 (Fine!)       
Failed to load WebKit2                                                                                                                                                                         
Typelib file for namespace 'WebKit2', version '5.0' not found                                                                                                                                  
Maybe for your OS you have to install additional GTK related packages?                                                                                                                         
We continue with the remaining packages...                                                                                                                                                     
Failed to load JavaScriptCore                                                                                                                                                                  
Typelib file for namespace 'JavaScriptCore', version '5.0' not found                                                                                                                           
Maybe for your OS you have to install additional GTK related packages?                                                                                                                         
We continue with the remaining packages...                                                                                                                                                     
Failed to load WebKit2WebExtension
Typelib file for namespace 'WebKit2WebExtension', version '5.0' not found
Maybe for your OS you have to install additional GTK related packages?
We continue with the remaining packages...

Still, the install goes through ("Success: gintro installed successfully.") and I can also run the example from here: https://github.com/StefanSalewski/gintro/blob/master/examples/gtk3/webkitgtk.nim

I then tried to install webkit2gtk-5.0 and to reinstall gintro. But this completely breaks it. I get the same libsoup2 and libsoup3 conflict even though I commented out all Soup related lines in gen.nim.

@StefanSalewski
Copy link
Owner

Thanks for reporting. Will think about it this evening or tomorrow, have to do some other work now.

@AS400JPLPC
Copy link

hello idem problem "manjaro" this trisub

@StefanSalewski
Copy link
Owner

Yes, I will investigate it. Was busy with #198 in the last two weeks.

@StefanSalewski
Copy link
Owner

hello idem problem "manjaro" this trisub

Have just shipped the PEGs section of the book, so maybe I can investigate this issue again this weekend. But it may be very hard and may enforce a larger rewrite of the gen.nim generator script.

But I just had an interesting idea: I think Nim loads dynamic libs with dlopen, so may it be possible to use dlclose to close it, and then reopen in a virgin state? What do you think?

@StefanSalewski
Copy link
Owner

Have you made some progress?

And did you saw my comments at https://discourse.gnome.org/t/will-dlclose-reset-internal-state-of-libgirepository/8963

Maybe I have to actually install Arch Linux to verify where exactly the problem arises from.

@AS400JPLPC
Copy link

Thank you, I read correctly, I tried to make a basic XFCE manjaro thinking that the addition of some software could have an action, results the generation is not done and always blocks libsoup and libsoup3 moreover webkit2gtk and webkit2gkt-4.1 are installed automatically.
webkit2gtk use libsoup
webkit2gkt-4.1 and webkit2gtk5 use libsoup3

@StefanSalewski
Copy link
Owner

I just took a brake from book writing, and so did some investigations of this issue again, and I had an idea, which turns out not working. But for my Gentoo box, with additional ONLY libsoup installed from git sources to /opt, all works fine. I can even modify the order of installs, and for example put libsoup to the bottom. All works fine. For details see https://discourse.gnome.org/t/libnice-should-have-libsoup-as-depency/9320/6

So I will hope that for ArchLinux it will work now, or soon also. Unfortunately nimble allows no parameters, so there is no good solution for people with problem to pass a parameter to skip libsoup or libnice, maybe even webkit. Maybe we can use environment variables instead of parameters?

I just tried to fix the latest issue with #200 which resulted from a rename of uref to unref of latest gobject. For me it works again, I hope my fix will not break it for people with older gobject. I will push that change tomorrow.

@StefanSalewski
Copy link
Owner

We have just applied the uref/unref fix which was necessary for latest gobject, and we removed libnice from the GTK4 process. Seems to work now. The only disadvantage is, that users can not use libnice with GTK4 currently. But we have not that many libnice users at all. And when libnice may much much later decide to use libsoup3, we get new trouble. Another possible solution is to use g_typelib_free() when processing libsoup and libnice, but that has other issues, and may not work for Windows and Mac, see https://discourse.gnome.org/t/libnice-should-have-libsoup-as-depency/9320/13. Please test with

nimble uninstall gintro
nimble install gintro@#head

@gavr123456789
Copy link
Author

it works! thanks for this investigation

...
gstapp..................: Remaining delayed methods: 0 (Fine!)
gstpbutils..............: Remaining delayed methods: 0 (Fine!)
Failed to load GtkLayerShell
Typelib file for namespace 'GtkLayerShell' (any version) not found
Maybe for your OS you have to install additional GTK related packages?
We continue with the remaining packages...
  Verifying dependencies for gintro@#head
 Installing gintro@#head
   Success: gintro installed successfully.

@beef331
Copy link

beef331 commented Dec 8, 2022

It's not urgent for me but just thought I'd report this issue still persists on EndeavourOS(Arch derivative)

@teras
Copy link

teras commented Dec 9, 2022

It's not urgent for me but just thought I'd report this issue still persists on EndeavourOS(Arch derivative)

I can confirm that I have the same problem in vanilla Arch

@StefanSalewski
Copy link
Owner

So this issue popped up again? We had the impression that we solved it in spring this year? The libsoup2 and libsoup3 issue is a very ugly one, in principle caused by how GTK and Gobject-introspection works internally. I have the impression that libsoup3 will replace libsoup2 soon totally, so this issue may vanish. I will try a a fresh gintro install on my Gentoo box soon (not this weekend, here in Germany our fiber cable is damaged since a few days, damage will persist at least for two day...) , to test if I can reproduce any issue. If not, and is again a poor Arg Linux issue, then I may not be able to help, sorry.

@StefanSalewski
Copy link
Owner

I have some confidence that I may be able to reproduce your issue.

https://packages.gentoo.org/packages/net-libs/webkit-gtk

webkit-gtk-2.38 seems to use libsoup3 now, while it used libsoup2 for 2.36. I should be able to fix that in the gen.nim install script. Unfortunately that will break it then for people who have still installed webkit-gtk-2.36. A parameter for nimble to influence the install process would be nice to solve such issues, but as you know, I have good reasons not to ask for things like that :-)

Currently my box is busy, Gentoo has to update a lot of packages including webkit-gtk-2.38. Maybe I can start more detailed investigations in a few hours.

@StefanSalewski
Copy link
Owner

I have been able to install latest webkit versions:

ls -lt /usr/share/gir-1.0/Web*
-rw-r--r-- 1 root root 1728408 Dec 12 21:52 /usr/share/gir-1.0/WebKit2-5.0.gir
-rw-r--r-- 1 root root 2357784 Dec 12 21:52 /usr/share/gir-1.0/WebKit2WebExtension-5.0.gir
-rw-r--r-- 1 root root 1732055 Dec 12 21:18 /usr/share/gir-1.0/WebKit2-4.1.gir
-rw-r--r-- 1 root root 2360529 Dec 12 21:18 /usr/share/gir-1.0/WebKit2WebExtension-4.1.gir
-rw-r--r-- 1 root root 1732056 Dec 11 18:14 /usr/share/gir-1.0/WebKit2-4.0.gir
-rw-r--r-- 1 root root 2360530 Dec 11 18:14 /usr/share/gir-1.0/WebKit2WebExtension-4.0.gir

I think 4.1 and 5.0 are using libsoup3 -- that combinations seems to be shipped for Arch Linux. But most others may have still 4.0 which uses libsopup2.4. It is not easy to install all that combinations. I will try with autodetect, and when that fails, maybe controlled by environment variables? Another problem is, that libnice still needs libsoup2.4, and we had a libnice user some years ago. I can not remember what libnice really is? Without libnice, we could just drop WebKit2-4.0 and use libsoup3 only.

@StefanSalewski
Copy link
Owner

I just tested an "nimble install gintro@#head" and its works without any problems for me.

I get

ls -lt webk*
-rw-r--r-- 1 salewski salewski 820928 Dec 13 09:57 webkit2webextension.nim
-rw-r--r-- 1 salewski salewski 460709 Dec 13 09:57 webkit2.nim
-rw-r--r-- 1 salewski salewski 821350 Dec 13 09:55 webkit2webextension5.nim
-rw-r--r-- 1 salewski salewski 457630 Dec 13 09:55 webkit25.nim

so the maybe problematic webkitgtk 4.1 is just ignored. This is the same behaviour, which we had two years ago, when webkitgtk5 was not shipped by most distributions, and we had to install it from sources to /opt.

Well, my box has currently no libnice installed, maybe I should install that one and test again. Will do.

@StefanSalewski
Copy link
Owner

StefanSalewski commented Dec 13, 2022

After libnice install, still no issues. Are you sure you actually tried

nimble uninstall gintro
nimble install gintro@#head

So what is the actual issue? Maybe Arch Linux ships broken Gobject-Introspection files? But to test that, I would have to install Arch Linux on my box, and learn to use it.

Or is it that you are just missing webkitgtk 4.1? Well, we could try to create that.

@beef331
Copy link

beef331 commented Dec 13, 2022

jason@jason-desktop ~> ls -lt /usr/share/gir-1.0/Web*
-rw-r--r-- 1 root root 1728289 Nov 30 16:52 /usr/share/gir-1.0/WebKit2-5.0.gir
-rw-r--r-- 1 root root 2357784 Nov 30 16:52 /usr/share/gir-1.0/WebKit2WebExtension-5.0.gir
-rw-r--r-- 1 root root 1731936 Nov 30 16:52 /usr/share/gir-1.0/WebKit2-4.1.gir
-rw-r--r-- 1 root root 2360529 Nov 30 16:52 /usr/share/gir-1.0/WebKit2WebExtension-4.1.gir
-rw-r--r-- 1 root root 1731937 Nov 30 16:29 /usr/share/gir-1.0/WebKit2-4.0.gir
-rw-r--r-- 1 root root 2360530 Nov 30 16:29 /usr/share/gir-1.0/WebKit2WebExtension-4.0.gir

I did not have gintro installed prior, and I am trying to install head. No clue if I can provide any more insight though.

@StefanSalewski
Copy link
Owner

Thanks for reporting. So there seems to be really again an issue for Arch Linux. But I have no idea for the reason.

@StefanSalewski
Copy link
Owner

Hello Jason B.,

as I had the feeling that you are indeed some of the few people who are still using Nim, I finally decided to install Arch Linux myself to find and fix your issue. So yesterday, I took an old laptop of my deceased mom and installed Arch Linux, Gnome, Nim, and latest gintro. The obvious issue is, that latest Arch Linux ships vte 3.91 for GTK4, while most other distributions including my Gentoo still have only v2.91 for GTK3. I was not aware that a VTE for GTK4 is already available, otherwise the issue would have been obvious. I guess fixing this is not that difficult, I will tell you when the fix is available at Github.

But note that I very seriously intend to remove gintro from Github soon. This issue proves again, how lazy (or stupid) users are. Five minutes test for someone with Arch Linux installed, vs. a whole day for me. Well, I know from IRC logs, that recently some "them" complained about this issue too.

For Arch Linux, I still prefer Gentoo. Installing Arch is a bit simpler at first, but still more complicated than Ubuntu. Of course, on an old laptop with only 2 cores and only 4 GB RAM a source based distribution like Gentoo makes not much sense, I would be not able to install Firefox from sources, and GCC or CLang would be problematic already. So Ubuntu or Arch Linux are candidates, I will continue with Arch now, as we have it now. But well, I do not really use that device, it is an Lenovo Thinkpad R500, twelve years old.

Best regards,

Dr. Stefan Salewski

@StefanSalewski
Copy link
Owner

Actually, there seems to be at least one more issue. When generating the GTK3 bindings, at least one of the Arch libs seems to pull in libsoup3 now, without really advertising. Have to comment them out one by one, to find the culprit. Maybe Arch Linux has switched to libsoub3 fully already, so that we could drop libsoup2. That would simplify things a lot. Will continue tomorrow with testing.

Maybe, we could fully drop GTK3? I think GTK4 is available for more than two years now, so why would someone still use GTK3? GTK4 has much more interesting bugs :-)

@StefanSalewski
Copy link
Owner

Actually Arch Linux has a libnice that uses libsoup3 now, that seems to be the main issue. Arch is ahead to most other distributions, we have to find a way to fix it for Arch, without breaking it for old distributions. Problem is, libnice does not really advertise libsoup use, see https://discourse.gnome.org/t/libnice-should-have-libsoup-as-depency/9320

@beef331
Copy link

beef331 commented Dec 20, 2022

But note that I very seriously intend to remove gintro from Github soon

You could archive the repo and explain why in the readme.

I do also have to say do not feel pressured to fix this just cause of me, I just updated this issue do to the person on discord not doing such.

@teras
Copy link

teras commented Dec 20, 2022

Hello @StefanSalewski and thank you for your efforts.

I am a dedicated Arch user, especially because I can see and validate "things to come"

If you believe that we can drop GTK4 on recent distributions (like Arch), I think it's a good idea anyway. There's no reason to have backwards compatibility forever, especially on Linux world.

@StefanSalewski
Copy link
Owner

If you believe that we can drop GTK4 on

Actually drop GTK3 support. But maybe dropping libsoup2.4 would be good enough. My feeling is, that Arch Linux does not need old libsoup any longer, all seems to be libsoup3 now. Other distributions should follow soon. And libsoup is needed only for webkitgtk, which no one really uses currently. And if someone wants webkitgtk, they can use the gtk4 version with libsoup3. All that would be easier, if "nimble install" would allow parameters to influence the install process. Maybe environment variables will do.

You could archive the repo and explain why in the readme.

Would make not really much sense. An archived version without any support would get bitrotting soon, and no one would be willing to fix it, in the same way as no one beside me has touched it in the last eight years. Maybe I could re-upload it with a new name, maybe as sgb for "Salewski's gobject bindings". Then all the fools would not discover it easily, the issue with too high version number would be solved (v0.99 already), and all the old issues would be gone also. And actually I was never very happy with the old name, was suggested by Mr. Rumpf seven years ago. Will try to fix the issues in the next days.

@teras
Copy link

teras commented Dec 20, 2022

Actually drop GTK3 support.

Yes sorry, I had in my mind "drop GTK3 and keep only GTK4", but my brain was faster than my fingers :)

Still, the rest of your thoughts look valid to me. And don't get it wrong: I think the community wants and needs what you are doing 😃

@StefanSalewski
Copy link
Owner

I think the community wants and needs what you are doing

Not really. We had one user, maybe in 2017 or 2018, who created some game unlocking tool with gintro. But not more. Maybe we have a few unpublished toy tools, but for toy tools the 20 other Nim GUI libs should be good enough. Actually, my feeling is, that GTK has no more users at all, for C and all the languages with bindings, the appearance of GTK4 has not improved the situation. When you go to Github and search for new serious GTK projects, there is nothing. With Gnome, GTK has still a passive, consumer community, asking for new features or complaining from time to time.

@StefanSalewski
Copy link
Owner

Hello Jason B.,

as you may have already noticed, we have just uploaded the latest fix for the libsoup issue, from which currently Arch Linux users and others with latest libnice version may suffer. You may test with

nimble uninstall gintro
nimble install gintro@#head

For me it installs fine now for Gentoo and Arch Linux. For tiny restrictions see #209 (comment)

Currently we have two known issues: The webkitgtk2.nim test for gtk4 does not compile, and for Arch Linux gstBasicTutorial1.nim does not compile. But these issues should be unrelated to our current fix. I will investigate these issues in the next days, and will do some more tests. We will see which new issues the upcoming Nim 2.0 may bring.

Best regards,

Stefan Salewski

@teras
Copy link

teras commented Dec 23, 2022

I'd like to confirm that this works for me now. Thank you @StefanSalewski

@StefanSalewski
Copy link
Owner

OK, I have just uploaded the fix for webkitgtk2 and for gst. The examples/gtk4/webkitgtk.nim should run now and display the Nim home page, and examples/gtk3/gstBasicTutorial1.nim should play the video.

@beef331
Copy link

beef331 commented Dec 23, 2022

Also can confirm works here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants