Name collision which renders repo formally broken and causes several side effects

[ink-splatters]

Describe the problem

What

Two versions of tests/demo/doc/mañana.txt existed in the repository at the same time, each of them with unique filename.

This was possible because of git ill-formed approach to utilize filesystem-dependent filename representations by default: likely, the mentioned file was added twice: both on Linux and macOS and due to filename encoding differences between the systems git permitted it.

This default can be changed on macOS using:

$ git config --global core.precomposeunicode true
$ git config --global core.quotePath false

and if followed by re-adding affected files, results in fixing the issue.

Affected software

the mentioned software was installed from nixpkgs rev 355cbc482f33f5b07a6bc0d72be862b1ccdb99dd

• git version 2.44.0
  minor:

  • prints warning
  • after fixing the issue in a child branch as per described above, it becomes impossible to switch back to parent branch without hard reset

• gitoxide: gix 0.35.0

  • showstopper: fails to clone repo properly

• nix (Nix) 2.21.2 with flakes enabled

  • showstopper: fails to install copier using its flake:

$ nix profile install 'https://flakehub.com/f/copier-org/copier/*.tar.gz'
error:
       … while fetching the input 'https://flakehub.com/f/copier-org/copier/%2A.tar.gz'

       error: creating file '/nix/store/p1k8x1b6arrk91qrva2l3yzga9d1iqhw-source/tests/demo/doc/mañana.txt': File exists

Template

N/A

To Reproduce

Besides what's been already described, please see aggregated versions info below.

Also see pinned versions in your flake.lock

$ sw_vers
ProductName:		macOS
ProductVersion:		14.4.1
BuildVersion:		23E224

$ nix --version
nix (Nix) 2.21.2

$ cat ~/.config/nix/registry.json
{
  "flakes": [
    {
      "from": {
        "id": "nix",
        "type": "indirect"
      },
      "to": {
        "lastModified": 1712161137,
        "narHash": "sha256-ObaVDDPtnOeIE0t7m4OVk5G+OS6d9qYh+ktK67Fe/zE=",
        "owner": "NixOS",
        "repo": "nix",
        "rev": "355cbc482f33f5b07a6bc0d72be862b1ccdb99dd",
        "type": "github"
      }
    }
  ],
  "version": 2
}

$ git --version
git version 2.44.0

$ gix --version
gix 0.35.0

Logs

$ nix profile install 'https://flakehub.com/f/copier-org/copier/*.tar.gz'
error:
       … while fetching the input 'https://flakehub.com/f/copier-org/copier/%2A.tar.gz'

       error: creating file '/nix/store/p1k8x1b6arrk91qrva2l3yzga9d1iqhw-source/tests/demo/doc/mañana.txt': File exists

Expected behavior

it should not happen :)

Screenshots/screencasts/logs

No response

Operating system

macOS

Operating system distribution and version

14.4.1

Copier version

used nix installation instructions from copier current master branch (ae49a4)

Python version

N/A

Installation method

pipx+pypi

Additional context

nix flakes (not in the list)

[sisp]

Thanks for the thorough analysis and report of this problem, @ink-splatters! 🙏

Besides fixing the file, should we set this Git configuration you show above (without --global) in the Nix environment initialization? Perhaps here (I'm not a Nix expert, is this the right place? /cc @yajo):

copier/flake.nix

Lines 58 to 69 in ae49a49

	installCheckPhase = ''
	patchShebangs tests
	env \
	[email protected] \
	GIT_AUTHOR_NAME=copier \
	[email protected] \
	GIT_COMMITTER_NAME=copier \
	PATH=$out/bin:$PATH \
	POETRY_VIRTUALENVS_PATH=$NIX_BUILD_TOP/virtualenvs \
	PYTHONOPTIMIZE= \
	pytest --color=yes -m 'not impure'
	'';

[ink-splatters]

@sisp hi, thanks!

No, actually you'd be just fine once the repo has been fixed (which is what I hope my PR does).

It's good to have this config as part of your dev environment so that if another file with name containing diacritics comes in, git would encode it consistently across platforms.

As for quotePath, disabling it seems to ensure first-class UTF-8 support, otherwise git shows crappy escape codes in git status for staged files with non-ASCII names.

[sisp]

It's good to have this config as part of your dev environment so that if another file with name containing diacritics comes in, git would encode it consistently across platforms.

That why I was thinking to add it to the Nix environment initialization, so everybody using Nix to contribute to this project will have those settings.

[ink-splatters]

It's good to have this config as part of your dev environment so that if another file with name containing diacritics comes in, git would encode it consistently across platforms.

That why I was thinking to add it to the Nix environment initialization, so everybody using Nix to contribute to this project will have those settings.

Sounds like an idea! Let me get back to you early next week