Adapt to new structure of better-bibtex.sqlite

[ailintom]

The current version of zotsite appears to be incompatible with the current version of better bibtex (6.7.140), for the database structure appears to have changed. This patch attempts to adapt to the new database structure of better-bibtex.sqlite

[plandes]

Hi @ailintom,

BetterBibtex recently broke for me as well, and a change was made based on its new schema already. I am also on BetterBibtex version 6.7.140 and the program works for me.

Will you please confirm you have the latest version of the program by explicitly first uninstalling the version you have, and then getting the latest by doing something like this:

pip uninstall zensols.zotsite
pip install --no-cache zensols.zotsite
zotsite --version

It should show the current version (0.9.0).

[ailintom]

Hi, thanks for your response. I have zotsite 0.9.0. It seems that it must have worked in 0.8.2, but was then broken again with this commit 896145b
The main table in better-bibtex.sqlite, created by BetterBibtex version 6.7.140 , is called citationkey and it contains plain records, however the mentioned commit reverted the name to better-bibtex, used in the old versions of BetterBibtex and containing JSON values.
However, the code used in 0.8.2 also required correction, since it does not filter the records in the better bibtex database by library id.

[plandes]

I understand, but I cannot reproduce any error. On my side, citation keys are rendering correctly and I can copy http links by citation key, and then go directly to the web site.

Will you please give a stack trace and describe in detail issue/error? Does the program fail on the command line or do you see some issue with the actual rendering in the browser of what it generates?

[ailintom]

I run zotsite 0.9.0 (zotsite export --config zotsite.conf) with the following line in my zotsite.conf file:

id_mapping = betterbib

and I get the following error message

  File "C:\Python311\Lib\site-packages\zensols\zotsite\betterbib.py", line 31, in data
    rows = tuple(conn.execute("""select * from betterbibtex.`better-bibtex`"""))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlite3.OperationalError: no such table: betterbibtex.better-bibtex

[ailintom]

Are you sure that you run BetterBibtex version 6.7.140 ?

[plandes]

@ailintom Re BetterBibtex version: ....yes :)

[plandes]

Hi @ailintom,

After making your modification and regenerating the site, citation keys were generated except for those new entries I added since upgrading BetterBibtex. The program still worked, but reverted to using the item unique keys.

As you mentioned,citationkey table changed after the schema change release. But even before that, but even before switching to the better-bibtex table, it was still getting the keys from a JSON blob--so that must have changed. I'm not sure why it works for me (and I imagine others in the two months after this change) but not you, but I have added your change and released the fix. My guess is BetterBibtex now keeps multiple places to store the keys (the large blob table still exists in my SQLite file).

Please update to the latest and test to see if it works. Thanks for your pull request.

[ailintom]

I guess there is a difference between fresh installations of Better Bibtex (as in my case) and old databases converted to the new version. I've tested version 0.9.1, and it works fine for me, thank you

[plandes]

@ailintom Great to hear it work. It's much simpler as it avoids having to use JSON and navigate through a tree to get the data. Thanks again.