How do people feel about the use of unicode in Oceananigans' API?

[glwagner]

On slack we recently had a conversation with @robertwhelber and @aramirezreyes about the downsides of unicode in our API. There are really two issues:

1. Some unicode symbols are not supported by some fonts. When a font does not support a certain symbol, readers see ⍰ instead. There's even a julia-specific font called JuliaMono that has "reasonable Unicode support" to at least partly solve this problem.

2. Not all text editors support Unicode entry. For example, vim requires a plugin (I use Vundle with Plugin 'JuliaEditorSupport/julia-vim'). @robertwhelber also reported a problem with VSCode on his machine.

This two issues mean that for some users, getting started with Oceananigans requires more than just installing Julia and typing pkg> add Oceananigans.

I think replacing Unicode in the source code is not only a gargantuan task but would also severely impact readability of the source code. We make widespread use of Unicode to produce easily readable, distinguishable operators like δxᶜᵃᵃ, ℑxᶠᵃᵃ, ∂xᶜᶜᶠ and we feel that unicode has solved some tough issues (eg making sense of the combinatorial explosion of operators required for staggered grids).

However, it may be possible to

1. Reduce unicode in our API somewhat, for example Simulation.dt instead of Simulation.Δt
2. Provide a "Unicodeless" API somehow... nu rather than ν, dx instead of ∂x, phi, lambda instead of φ λ?

The problem with 1 is that it's just a half-measure that might reduce the quality of our API without really solving anyone's problem.
As @simone-silvestri said

You're going to encounter Unicode eventually if you use Oceananigans

As for 2, it would require work and maintainence. However it's possible that we (or even someone else) could support option 2 as an external package, by redefining some key constructors and functions and re-exporting the already-acceptable versions.

My personal opinion that that using Unicode is important in making "code look like math" and writing "scripts that look like papers", and therefore a cornerstone in achieving one of our missions. However, I think we're also going to fail at achieving anything if lots of people are turned away at the door by Unicode.

[iuryt]

From my perspective, using Unicode is important. The code will look more like math and physics expressions, which makes easier to understand and convince people to use and adapt it, even if they are not very used to program in Julia.

[francispoulin]

I agree that using Unicode is very nice and hope we keep on doing it. This unfortuantely means that some problems will arise. Maybe we can suggest a default editor in case people have problems that is more fool proof than others?

[glwagner]

Probably VSCode is the recommended editor. Docs look decent too https://www.julia-vscode.org/docs/dev/gettingstarted/

[francispoulin]

Sadly, I could not get VSCode to do Unicode and have since gone the way of atom. I did post an issue but did not get a response.

But I hope that I am the outlier and that VSCode will work for others.

[navidcy]

But I hope that I am the outlier and that VSCode will work for others.

Works for me

[robertwhelber]

If I follow just the instructions here: https://www.julia-vscode.org/docs/dev/gettingstarted/
will Unicode characters also work? Or, will I need to install other VSCode packages to get that working?
I appreciate tutorials like this, thanks.

[glwagner]

I'd think so but I don't know for sure. @navidcy can you confirm? It's mentioned in the changelog: https://www.julia-vscode.org/docs/dev/changelog/#[0.14] a while ago now. Unicode / tab completion does work for me in VSCode (though it's a little slower than vim) so that's consistent with my experience.

Nothing is mentioned here: https://www.julia-vscode.org/docs/dev/userguide/editingcode/

I think a request for some information about Unicode and backslash completion in those docs might be appropriate for an issue here: https://github.com/julia-vscode/docs, though the top of the docs does mention they are currently being revamped so that might be in the works somehow.

[MasonProtter]

There's nothing wrong with providing a Unicode API. Unicode is great. It is quite problematic though to provide APIs that require unicode.

There should always be ugly ASCII fallbacks.

[glwagner]

I wrote below:

I'm worried that it's not feasible / too much effort to maintain "two APIs". I second grave issue is that this produces "two languages"; ie users may have trouble reading scripts written using the alternate API that they don't use. I think we'll have to choose one.

[MilesCranmer]

Hey all,

From an outsider's perspective who just recently started trying out Oceananigans.jl and other related packages (really enjoying them overall!), the use of unicode was the biggest annoyance for me. Thus I would highly recommend option (2) above. For the record, this is the only Julia package I've encountered which uses a unicode API. I think unicode in scripts is fine, but for APIs it is too subjective to enforce.

While there are plugins I can install for IDEs to support it, there are so many things which break:

1. GitHub search/VSCode search/vim search. Most plugins don't help with this: try to search for a unicode symbol with the GitHub search console - tab completion does not work. If I want to search for occurrences of a particular function/symbol, I need to copy-paste the symbol which is (when compounded) a lot of unneeded effort.
2. github editor/github.dev/nano/other minimal editors which do not have any Julia plugins do not support unicode. (Sometimes I just want to make an edit on my phone...)
3. Bash. If I want to use grep, I need to know how to write the symbol in bash, or, again, copy-paste it to my terminal (assuming my terminal supports unicode).
4. IDE movement commands. vim's f command is very useful for navigation (see this), but I can't use it to jump to a unicode key. I also use vim-sneak for quickly moving around code, but it is also incompatible with unicode.
5. More keystrokes, especially for super/subscripts.

The argument I've seen in favor of unicode is that it looks better, but I can't see this either. LaTeX math looks good, but unicode math looks terrible (subjective opinion!). It would be great if _psi was rendered as full-blown LaTeX $\psi$ - then I could use ascii to navigate, and see the LaTeX rendering when reading. But unicode math seems like the worst of both worlds... It also makes it so much harder to read. e.g., δxᶜᵃᵃ - I have to increase the zoom setting on my window (which is calibrated for normal fontsize) just to read the full variable name.

Anyways, this is all subjective. I think the point is that some users (like myself 🙂) are very anti-unicode - and thus an API should not enforce its use.

Again, awesome package! Happy either way you choose to go.
Best,
Miles

[MilesCranmer]

Up to this point we have prioritized the "readability" of user scripts. Our philosophy is that a user script should be as readable as a paper; therefore we would users to learn unicode so that they write readable scripts meant to be openly shared with an academic community.

I think unicode just isn't expressive enough for this, so it ends up being an awkward middle ground which is more confusing than if code was written normally, and then annotated with fully rendered LaTeX in a Jupyter notebook.

Take for example this snippet:

u, v, w = model.velocities
ζ = Field(∂x(v) - ∂y(u))
compute!(ζ)

If I pretend I am a naive user, I might look at this and be unsure if ∂x(v) here means $\frac{\partial x}{\partial v}$ or $\frac{\partial v}{\partial x}$ or $d(x v)$ (i.e., the full derivative). It would be great if unicode had a character for $\frac{\partial}{\partial x}$, but it unfortunately does not, so I think the only way to be clear is to use something like

u, v, w = model.velocities
# $$\zeta = \frac{\partial v}{\partial x} - \frac{\partial u}{\partial y}$$
ζ = Field(∂(v, direction=:x) - ∂(u, direction=:y))
compute!(ζ)

and either write that comment in a jupyter notebook markdown cell, or use a TeX renderer like https://marketplace.visualstudio.com/items?itemName=vs-publisher-1305558.VsTeXCommentsExtension or https://orgmode.org/manual/LaTeX-fragments.html.

Being new to Julia myself, sometimes I get confused whether unicode is a function or part of a variable name. For example, if someone writes ∇f or f', those characters are actually functions – gradient and transpose, respectively – and not variable names. But with the current API, the code grid.Δzᵃᵃᶜ does not mean grid.Δ(z)[:a, :a, :c], i.e., the laplacian of a field z indexed at (:a, :a, :c), which it could very well be, but instead an actual variable named called Δzᵃᵃᶜ. So there's this ambiguity here which is confusing unless you are already comfortable with the code.

[glwagner]

It'd be interesting to hear if other users think that our API is awkward and confusing. If so, then we certainly have work to do.

Our intention is for users to read ∂x as the operator

$$ \partial_x $$

so that ∂x(v) - ∂y(u) is

$$ \partial_x v - \partial_y u $$

which is the notation for partial derivatives that most of the developers prefer (I think). We opted against sub/super scripts because they can be difficult to read, as noted, and don't seem necessary here.

grid.Δzᵃᵃᶜ isn't part of the user API. We are currently missing an API for grid spacings, areas, and volumes (though we have one for nodes). We should use the syntax Center() and Face() to refer to locations when we get around to designing this interface, similar to how we have

z = znodes(Center(), Center(), Center(), grid)

[glwagner]

Also, it seems like the Julia vscode doesn't patch the search bar - e.g.,

This could be something to raise over at https://github.com/julia-vscode/julia-vscode

[tomchor]

Up to this point we have prioritized the "readability" of user scripts. Our philosophy is that a user script should be as readable as a paper; therefore we would users to learn unicode so that they write readable scripts meant to be openly shared with an academic community.

I think unicode just isn't expressive enough for this, so it ends up being an awkward middle ground which is more confusing than if code was written normally, and then annotated with fully rendered LaTeX in a Jupyter notebook.

@MilesCranmer you're raising some very interesting points, so thanks for the discussion! That said, I definitely have the opposite opinion here.

I remember when I first started with Oceananigans, part of what I liked was that the code (and the math in particular) was really easy and intuitive to read. So while I agree that unicode isn't perfect, it's way better than pure ascii code. LaTeX code rendered in Jupyter is definitely more legible, but that also requires extra work (and extra line for each expression). I never use Jupyter notebooks, for example, and work purely out of Vim. I don't know if there's even a way that I could render LaTeX comments there.

Being new to Julia myself, sometimes I get confused whether unicode is a function or part of a variable name. For example, if someone writes ∇f or f', those characters are actually functions – gradient and transpose, respectively – and not variable names. But with the current API, the code grid.Δzᵃᵃᶜ does not mean grid.Δ(z)[:a, :a, :c], i.e., the laplacian of a field z indexed at (:a, :a, :c), which it could very well be, but instead an actual variable named called Δzᵃᵃᶜ. So there's this ambiguity here which is confusing unless you are already comfortable with the code.

It's my impression that this isn't a characteristic of Oceananigans per so, but of Julia as a whole. It seems to me that Julia gives users and developers a lot of freedom with characters and names, and other packages take a advantage of that freedom too, just like Oceananigans does. I think in the end lots of it is subjective, but for me personally it just took a bit of getting used to and now it feels as natural as python syntax (but more readable :) )

[MilesCranmer]

It'd be interesting to hear if other users think that our API is awkward and confusing.

Sorry, that's not what I meant at all! I actually find the Oceananigans.jl API to be by far the most intuitive one for setting up hydro simulations I have seen out there. (I've even tweeted about how much I love the API!). But if there is any weakness to improve on, I think unicode is it.

I am trying to say that I am doubtful about the statement "a user script should be as readable as a paper" - I don't think unicode is expressive enough to do this, and trying to write out full equations using unicode seems like it will lead to confusion versus using full function names with annotations. I do find some of the examples in the docs fairly difficult to parse because of this. e.g., in this example, I can't tell what superscript or bar is just part of the variable name, or an operator:

I can still figure it out, but it's just a higher mental load to do so. I suppose eventually this would become intuitive, but it certainly seems like a steeper learning curve. (I guess this note is about the examples; not the API itself)

Luckily the current Oceananigans.jl API doesn't rely heavily on unicode, which is good! Although I do think having non-unicode versions of existing unicode (in API + examples) would be even better. (So 👍 for We should use the syntax Center() and Face() to refer to locations when we get around to designing this interface). Btw, grid.Δzᵃᵃᶜ appears in the Ocean mixing example - not sure if there's a user API equivalent or if it's just demo'ing some internals.

Regarding ∂x(v) - ∂y(u) in particular, I can see why others like it, but I must confess I am really not a fan of this style because to my eyes it is ambiguous - both for the reasons noted earlier and because ∇x(v) in Julia would typically mean gradient(x)(v) in autodiff packages, rather than a directional gradient_x. Maybe I will get used to it eventually but with fresh new-user eyes, I just can't say I found it easy to understand at first pass, though. Whereas the rest of the API, with full ASCII names, is super easy to understand and non-ambiguous.

@tomchor: It's my impression that this isn't a characteristic of Oceananigans per so, but of Julia as a whole. It seems to me that Julia gives users and developers a lot of freedom with characters and names, and other packages take a advantage of that freedom too

For the record, I've never seen unicode used this extensively in any other Julia package's examples over the past ~two years of using Julia. I've also only seen a few examples of unicode being used in the API itself, and for those it was very minimal (e.g., a single operator like ∇) and there were also ascii equivalents (noted earlier).

e.g., take a look at a random sampling of packages in, say, JuliaMath: Interpolations.jl, NFFT.jl, Combinatorics.jl, FFTW.jl, Polynomials.jl, SpecialFunctions.jl. Each of these packages could choose to use unicode math extensively, but in actuality there is either no unicode, or extremely minimal use of unicode in both the examples and the API.

(Not that I think using other packages as examples is a good idea, though. There's always better ways to do things!)

---

At the end of the day though, it's up to you! Just wanted to give my honest feedback in the hopes of making this awesome package even better. Take from it as you will.

[tomchor]

I'm not sure how relevant this is gonna be, but I'm gonna leave this here in case it can help someone. In Linux you can easily(-ish) have your keyboard type unicode. There are a few ways listed here. I do this (and set specific bindings for more common characters like ∂) and thus I rarely need to rely on any editor to type any of my code.

The easiest thing, however, is to follow what's described here and add a greek layout to your keyboard, with a shortcut to change layouts. I do that too, and by pressing Ctrl+Alt+k I can swap back and forth between English and Greek, which allows me to type all the Greek letters effortlessly, not having to rely on any editor. (This obviously doesn't deal with all the other unicode characters, but it's a start!) I've only tried this on Manjaro Linux, but I'm pretty sure this method can be used on any OS.