Make EvaP pip-installable and publish to PyPI

[niklasmohrin]

Closes #2303

Since we don't want to use nix in production (for example because security updates might take a long time to propagate through nixpkgs / through our github release cycle), pip install . or pip install evap seems to be the most canonical way. This allows us to install security patches via pip install --upgrade evap.

If we really need to override a dependency version beyond the allowed range in pyproject.toml, we could also clone the repository on the production server, perform the change, and then run pip install . - I hope this won't be necessary though.

The new release workflow is:

1. Add translations and commit them.
2. Bump version in pyproject.toml, commit it, tag it, push it.
3. Create Github Release.
4. Github Actions will be triggered when the Github release is published and publish to PyPI.

The versioning scheme we agreed on is year-month-patch, for example "2024.11.0". I suppose we should take care that we don't necessarily push breaking changes like updating the Python version in a new patch.

Since upgrading to the next stable revision of EvaP is handled by pip, there is no need to the release branch anymore.

See

• https://packaging.python.org/en/latest/tutorials/packaging-projects/
• https://docs.pypi.org/trusted-publishers/using-a-publisher/

TODO:

• compiled typescript / scss / translations are not included in the published package
• the situation around static files is unclear (read: I haven't thought about / tested them yet). Probably something like python -m evap collect_static to generate the static files for apache to serve should do the trick. Not sure how bootstrap fits into this yet though
• it's unclear how localsettings fit into this

[niklasmohrin]

See https://test.pypi.org/project/evap/ and https://pypi.org/project/evap/

[niklasmohrin]

@richardebeling Thoughts? Furthermore, if you create a PyPI account, I can add you as a co-owner of the package. (I also requested the creation of an e-valuation organization on PyPI, not sure if it will be nicer / needed, we'll see once it's processed)

[niklasmohrin]

Should we add the organization members here? Right now, PyPI displays "EvaP Developers" under unverified details, I suppose we could also just add the names which will hopefully be deduplicated then

[niklasmohrin]

Looks like it won't be deduplicated, I suppose no one really cares about this, see https://test.pypi.org/project/evap/2024.10.0/ or https://pypi.org/project/black/

[richardebeling]

I must be blind -- what section exactly would contain duplicated names? I see 5 unique user names on the black project page under "Maintainers"

I'd be fine with just putting "EvaP Developers", which will always remain correct.

Do we even need this in the pyproject.toml? I would expect that for username mapping, you need to configure this on PyPI separately anyway?

[niklasmohrin]

"Ambv" is the same person as the unverified "author" (=> big projects don't care about about unverified data) and also on the evap release my name appears twice (=> it doesn't get deduplicated). Therefore, "evap developers" also seems fine to me.

We need to add something because otherwise poetry will complain :/

[richardebeling]

Ah, I see -- I was looking for duplication inside one list. Yeah, "evap developers" 👍

[niklasmohrin]

One thing we discussed in person was that we now don't really need a full checkout of the EvaP repository on production. In fact, this is a chance to move most of the (apache-)deployment files out of the main repository and into a separate repository in the e-valuation organization. @janno42 agreed that this would be a nice (while not really needed) change. The only drawback seems to be that testing the backup process in the EvaP repository will be more complicated. Currently, I believe that we would want to pin a commit of the deployment repository, for example through flake.lock, and run the tests here. I believe that we could add the backup process test logic through nix in the deployment repository and then import this test in the EvaP repository to use the updated source of EvaP. In particular, this would allow us to share the test definition and "easily" run the same tests in both repos. I am not too settled on this approach though.

Thoughts @richardebeling ?

[richardebeling]

@richardebeling Thoughts? Furthermore, if you create a PyPI account, I can add you as a co-owner of the package. (I also requested the creation of an e-valuation organization on PyPI, not sure if it will be nicer / needed, we'll see once it's processed)

While certainly unexpected, I think it's a funny and efficient solution, and I don't have any reasons against doing this.

One thing we discussed in person was that we now don't really need a full checkout of the EvaP repository on production. In fact, this is a chance to move most of the (apache-)deployment files out of the main repository and into a separate repository in the e-valuation organization. janno42 agreed that this would be a nice (while not really needed) change. The only drawback seems to be that testing the backup process in the EvaP repository will be more complicated.

Why? "Backup" essentially is "Dump database via dumpdata", isn't it? This would be separate from apache and all the other deployment aspects. I can see that the "Upgrade" part of the script wouldn't be testable -- we can compromise on that, I think. It's important that backup and especially restore is tested.

---

Re (separate repo, version pinning): Adds complexity, but if there is a need for this or it helps in any way, I guess we won't die 🤷‍♂️

A proper release procedure with some github magic action could also be used to deliver compiled TS and SCSS files automatically, something that I dislike a bit about our current approach (necessity to have npm somewhere). (We should keep in mind though: Automation is always nice up to the point where it stops working)