Database support #63
Replies: 25 comments 21 replies
-
Regarding quiver specifically, I am very happy to discuss how to enable external tools to interact with quiver (e.g. exposing an internal representation for the diagrams, or possibly thinking about some sort of API). I also think interactive learning and proving using a visual editor is worth pursuing. However, I do not see this as a design goal of quiver itself, and I personally am unlikely to spend any time on these sorts of features. I think @AviCraimer is interested in similar ideas to you, so it might be worth starting a conversation with them. |
Beta Was this translation helpful? Give feedback.
-
Awesome. I'm doing some reading on subgraph isomorphism algorithms as they
apply to modern graph db's. IIRC, Neo4j doesn't even have a
straightforward subgraph isomorphism search. So we should have some fun
writing one of these in terms of breadth or depth-first search abilities of
ArangoDB.
I think graph search should be comprehensive in its options. For example,
do we only match two arrows if they have the same hook/vee/split tail
setting? That should be a checkbox. As I said before, this is a
humanistic math project, so whatever we do doesn't have to be formally
perfect and rigorous, as it will be a humans job in the end to verify
correctness of some diagram or some relationship to another diagram.
A brainstorm here seems like the right place for it.
A harder problem to think about is do we try to autodetect variable names
in a user's LaTeX label, or do we say that every string is a variable and
so every string can be replaced by any other string in a morphism that
describes a search matching.
If you detect variables for the purpose of forming a regular expression,
then what do you do with things like \lim etc which is everywhere in
category theory. This is another reason why the search options should be
comprehensive and detailed because maybe either way is sometimes preferable.
I think the first goal should be just a simple search and display of an
exactly matching graph (or whatever these papers I'm reading lead us), that
is also a morphism of labels. That last part is quite important, you
wouldn't want 10,000 copies of a commuting square to accomodate all latin
and greek variable strings (with sub / super scripts, etc.). I believe
these papers don't reallly address that way of doing it and instead they
look for an injection of the label sets between the respective query graph
and the graphs residing in the DB.
…On Mon, Jan 4, 2021 at 11:27 AM Avi Craimer ***@***.***> wrote:
Maybe we should have a brainstorming / planning session sometime soon. I'm
moving house this month so I can't commit to any coding work until
February. I'm available to share ideas though.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#63 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMIF57IYXNFGG4TX53U7JLSYIJB3ANCNFSM4VQXSSGQ>
.
|
Beta Was this translation helpful? Give feedback.
-
*injection = inclusion of the label sets
On Mon, Jan 4, 2021 at 12:08 PM Luna Tuna <[email protected]>
wrote:
… Awesome. I'm doing some reading on subgraph isomorphism algorithms as
they apply to modern graph db's. IIRC, Neo4j doesn't even have a
straightforward subgraph isomorphism search. So we should have some fun
writing one of these in terms of breadth or depth-first search abilities of
ArangoDB.
I think graph search should be comprehensive in its options. For example,
do we only match two arrows if they have the same hook/vee/split tail
setting? That should be a checkbox. As I said before, this is a
humanistic math project, so whatever we do doesn't have to be formally
perfect and rigorous, as it will be a humans job in the end to verify
correctness of some diagram or some relationship to another diagram.
A brainstorm here seems like the right place for it.
A harder problem to think about is do we try to autodetect variable names
in a user's LaTeX label, or do we say that every string is a variable and
so every string can be replaced by any other string in a morphism that
describes a search matching.
If you detect variables for the purpose of forming a regular expression,
then what do you do with things like \lim etc which is everywhere in
category theory. This is another reason why the search options should be
comprehensive and detailed because maybe either way is sometimes preferable.
I think the first goal should be just a simple search and display of an
exactly matching graph (or whatever these papers I'm reading lead us), that
is also a morphism of labels. That last part is quite important, you
wouldn't want 10,000 copies of a commuting square to accomodate all latin
and greek variable strings (with sub / super scripts, etc.). I believe
these papers don't reallly address that way of doing it and instead they
look for an injection of the label sets between the respective query graph
and the graphs residing in the DB.
On Mon, Jan 4, 2021 at 11:27 AM Avi Craimer ***@***.***>
wrote:
> Maybe we should have a brainstorming / planning session sometime soon.
> I'm moving house this month so I can't commit to any coding work until
> February. I'm available to share ideas though.
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#63 (reply in thread)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAMIF57IYXNFGG4TX53U7JLSYIJB3ANCNFSM4VQXSSGQ>
> .
>
|
Beta Was this translation helpful? Give feedback.
-
I'm back to evaluating Neo4j. ArangoDB didn't work out very well. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer this is how easy Neo4j. I got this to run in a few minutes (compared with Arango, which took hours). You'll need to install Neo4j Desktop and create a database named as above with a node with a LaTeX property with value "A". The code prints "A", successfully! I'm in the process of learning Cypher (the query language of Neo4j), which is the string containing "MATCH" above. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer |
Beta Was this translation helpful? Give feedback.
-
See this SO post on how to call a function on your flask app (server-side) from JS. I'm thinking we don't want to try and hack too much of Quiver, as it will have to be merged later with updates of Quiver. I'm guessing there's some way to embed Quiver into a subframe of the page, and have our own app parts (JS/HTML) running in other frames. We might add a button say to Quiver, but that's easy. According to a comment by @varkor in an issue thread here, we should subclass QuiverImportExport into QuiverImportExport.databaseService just like QuiverImportExport.base64. Althernatively we could somehow decode the base64 output, and that would be even more modular! |
Beta Was this translation helpful? Give feedback.
-
Also, note that creating many databases is out-of-the-question and represents a major logistics / db management headache (I asked on the Neo4j forums). Therefore, I propose a single database for everything! Two nodes can be connected by a morphism if and only if they lie in the same Diagram. The Diagram node connects to all its node "children" via ":ContainsObject" (colon means Node/Edge label type). The :Object nodes will link to the Diagram as well via the inverse relationship ":InhabitsDiagram". The Diagram is then given a category assignment and also has a bidirectional arrow for ":AdmitsDiagram" / ":InhabitsCategory". |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I've switched to a Django app and have read through the Neomodel documation. Here's where I'm at, defining the models for the graph database:
Feel free to recommend any way of structuring things logically. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I have been working on getting a Django site hosted on heroku, it's at: Clear database In that order. Note that it works! Took 2-3 days to get hosted, and get a dev environment setup. We can git push straight to heroku and the changes will appear on that site. The hosting is $8 / month though. We should slightly monetize to pay for a good Neo4j host eventually (such as GrapheneDB). |
Beta Was this translation helpful? Give feedback.
-
The only way to embed the quiver app is with iframe. I tried to integrate the file structure with Django, but that turns out way too difficult. This method however, worked almost immediately. So we will have to edit parts of quiver to add in things such as "Save to Database" button. The methods I googled about for getting the iframe's URL as it changes did not work. The above screenshot has quiver served from my local drive. All it takes is to put entire quiver folder inside of |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I'm wondering now how you're doing the Front-End design because how I do things in Django depends on it. I've come up with a pipeline for UI dev. We first layout bootstrap templates using LayoutIt: There's one I worked on. Django interacts nicely with bootstrap forms through a plugin called crispy forms. Try going to Preview on that page, and resizing your window. You'll see that because we used Bootstrap (A CSS / JS library), it responds nicely to page size changes. So eventually we may be accessible from a phone screen. |
Beta Was this translation helpful? Give feedback.
-
The home page etc is not ready, but you can view this: This is essentially the only way to do nice-looking, responsive, Django front-end templates. I.e. using Django + Bootstrap 3 + X-Editable. Try clicking on an editable text, as you can see X-editable gives us this ability to edit text items in-place. It's not connected to the database yet. I will work on that next. Crispy forms with Django didn't work very well (had errors with it). Plus, I don't like how the frontend code is put on the python side (Crispy Form "Helpers"). So please learn some Bootstrap 3 and X-Editable. I used Layout-It to get a basic boilerplate template for that Rule Editor page: The rule page essentially says, if you have a diagram drawn that matches the Key Diagram, then you can get to the Result Diagram logically. So when the user is drawing, we might populate a button widget as soon as a match is made. If the Rule Title is "Take Functor Image". Then that is what the button will read. The functor name / target category is supplied by the user per the rule editors options (Prompt The User) and the fact that a diagram is drawn containing in its text the variable "F", which is unassigned, so the prompt will read "F = ?". The other variables appearing the diagram are gotten from the key diagram (the bound variables). Those will get replaced by the variables actually used in the final user's drawing. Seamlessly! |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I've put together a live demo for you. It doesn't do any graph stuff quite yet. To demo it, click "Sign in" with these credentials: TestEditor Then click "Rule editor". When you edit text, it's actually updating in the Neo4j database the corresponding node property. Note that if the site loads slowly, that is only a one-time thing and only in the dev environment. Heroku runs code on a "dyno" and when say you press Sign In and it's spinning forever, that's the dyno starting up. I think they sleep them after 30 minutes of inactivity. In production this won't happen and the site is actually very fast. Once you wake up the dyno, you can see how fast it is on those DB writes. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I'm switching over the templates I have so far to Jinja2. With the new site structure, I have "Create New Diagram" pages embedded into a Rule Editor page, so that those Create New Diagram pages then open up a Blank Diagram once you have it named. I'm making Diagrams, Categories, DiagramRules all uniquely named within the database. This should encourage people to pick descriptive names. It's a while before I have it ready for you to clone from github though. Maybe 3 days. But I've come up with design to finished product pipeline (I've gone from design to nearly finished pages), so we should stick with it only adding 3rd party javascript or django plugins. Right now JS plugins are: Bootstrap 3 (not 4) for nicely scaling responsive layouts, X-editable (for in-place editing of text widgets), KaTeX as part of Quiver for rendering (so we should also use it for text in the rest of the site as well), and of course the quiver CD editor is embedded as an iframe. I tried twice breaking up the quiver files into a Django app hierarchy and failed. This is the simplest route. We should share screens some time and I can show you what's happening in my dev setup and also how to set up your own. Don't be afraid to learn some Python, it's a lot easier than JS in my op. |
Beta Was this translation helpful? Give feedback.
-
Here is the Latest Demo Sign in with: TestEditor How's it going? Are we still going to code together on this? |
Beta Was this translation helpful? Give feedback.
-
Update: tried triggering events inside the iframes to no avail. So I tried making a django app out of quiver, and I finally got KaTeX to load, which I think was the main issue. Here's a screenshot: This means we're going to replace all those JS buttons with our site's buttons outside of the render view. This also means we don't have to worry about triggering events across iframe boundary. We can call javascript functions directly from the HTML template now. Notice that we even have the URL with the diagram data string (see address bar of screenshot). We can use URL for now for passing data, but as the diagrams grow we have to keep in mind that URL length limits might get exceeded. @varkor could you show me how to expose some functions out of the Toolbar class so that I can call them from index.html? |
Beta Was this translation helpful? Give feedback.
-
Here's a video of the code succsesfully saving the diagram to the database: |
Beta Was this translation helpful? Give feedback.
-
Looks like you've been making great progress. I've been moving into my new house this week so I haven't had time to follow this in detail. I will have some time Sunday afternoon. I'll take a look at it, and try to get the code running on my system. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I had to switch back to Django Template Library from Jinja2-based plugin. Ran into errrors that google couldn't solve which shouldn't have occured. Anyhow, I have the site back in working order (not live but on my local dev machine) using DTL. I had to switch from Jinja2 because I couldn't get a basic Crispy Form to work (Django plugin "Crispy Forms"), without erring out on some library's internal {% load x %} usage, which Jinja2 doesn't support, so not sure how things got crossed. None the less, I've got us back to the same state we were in that you can now see in the live demo, except with the DTL being used. When we connect. We need to exchange emails her in a code-talker like way to keep them secret, then we'll delete posts. Once we've done that, you need to install TeamViewer (so I can RDP into your machine to help you get the dev env setup). Once we do that, we exchange connection details over Skype. Let's connect over skype first, We'll need a private chat channel for certain parts of the dev. I need to install: Heroku (what we will first host our live site at) looks for a requirements.txt or a setup.py. In our case we just use requirements.txt because it's simple: It's just a newline separated list of known packages with optional "== version number" on each line. So for instance: You do
You will also sometimes need to do So any time you make edits to a models.py file, run the above makemigrations command. Then run The code in settings.py has a switch:
You can code settings.py, and when you are debugging code in WingWare, changes to settings.py or anything non-model related (even .js or .css files in static) are usually reflected immediatly in your browser without restarting the debugger / django server's process. You just have to save the file, so if it shows a "*" in the tab name, then hit Ctrl-S to save it. If the change isn't reflected, then hover over tab titles in your IDE's an make sure the directories are correct. Also you might want to do a Chrome > "..." icon > Clear Browsing Data > Use default settings for common dev files. To clear out your browser's cache of .css / .js or image data that has actually changed but the browser didn't detect it. There's lots to learn but I can get you up to speed more quickly than going at it alone and reading tutorials. So you'll be where I'm at in a a week of coding together. I'm a big copy / paste coder. If it's already coded perfectly to suite my needs, I just copy / paste code and if it's slightly off I adjust settings or go to other examples, etc. It has been hectic trying to code login / logout / account signup views as a new person to Django. Each tutorial is slightly not just what I need, so I have to keep jumping around... Maybe you would be good at coding the signup / login(out) / password recovery system. Remember we have a $8/month linux host we can use if we need an email system, but maybe Heroku too has a cheap option. So there are two servers: Has an exact copy of the code that we push with:
Once you have a remote added to git for heroku. The second server is: (which hosts our graph database). But for you to deploy to to the live site you'll need the secret keys and to know where I store them. So we need a private Skype conversation going on. I will show you how I make changes to both servers. If second logon costs money, I'll lend you my login for Heroku. I think though I should create a user for the linux server that has some admin priveleges like you can start / stop neo4j. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer There have been some changes to the README.md page of the QuiverDatabase repo. https://github.com/enjoysmath/QuiverDatabase It now gives instructions on setting up your environment. Let's move future discussion over to the issues section of that repo, or someone can show me how to set up the Discussions tab there. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer Here's a screenshot of where the code is at: I'm implementing Rule Search as we speak. This allows the user to match an input diagram with the key (or result) of a DiagramRule. Then hitting "Apply" will take the user to the new diagram that is the old diagram with the found rule applied to it. In theory. Having some issues with regexes and Neo4j querying, so made a Neo4j community forum post about it here: https://community.neo4j.com/t/regex-matching-text-hom-does-not-work/33295 Are you still interested in working on this together? |
Beta Was this translation helpful? Give feedback.
-
I got the Rule Search feature + Rule Application (with Variable Substitution) features to work properly, and here is a video: |
Beta Was this translation helpful? Give feedback.
-
I think I'm about to re-open this project from the backburner. I thought there must be more rigorous ways of doing this CD database, but what I have so far is pretty far along, so I'm going to stick with it and try to get it fully working. |
Beta Was this translation helpful? Give feedback.
-
@AviCraimer I'm getting back into Quiver Database code. Are you interested in collaborating on it? I'm thinking we fork Quiver and make it as close as possible to original Quiver except set / get graph ability and anything else needed complete the design. I did a lot of hacking of JS directly into my copy of Quiver. But that makes updating from this official repository not so easy. One thing I would like however is better mobile support. The tool buttons are not fitting on my screen and when you zoom out on the iPhone the buttons will then be within the screen, but too small to click on comfortably. Other than that the mobile support is amazing. You can draw diagrams with your finger in such a way that it's much easier than using a mouse. So for example, to decouple the code more than what it is currently, the search results would not be a popup within Quiver JS code, but a panel rendered in the Django template itself, outside of Quiver's iframe. See above screenshot for what I mean. |
Beta Was this translation helpful? Give feedback.
-
This will sound strange at first, but the usage's for this are manifold.
I'm looking into ArangoDB. Ran into trouble with Neo4j. ArangoDB has a JS driver so that's how it would connect into Quiver.
ArangoDB's AQL language supports regex matching, so for instance if you wanted F(A) to match G(B) as long as the substitution is common to the whole diagram, then that would handle that issue (having too many graphs, with all possible combinations of letters / variable strings). However, if the user starts to draw with F in mind, while matching a graph with G in the database, the user can stick with F and the substitution is applied transparently.
So every diagram can be stored in a graph in an arrango db (even if it has loops). Why do this? It would allow us to create a sort of gigantic Finite State Machine of high-level maths for one.
The user's drawing has a Category drop-down in which they can select what category they're working in, e.g. Ab. A functor button will allow the user to essentially relabel all object and arrow main labels L to F(L) if F is the given functor name.
I've thought about doing all this using a formal proof system but the code for that is voodoo to me. I've tried many ad hoc solutions that just don't meet the criteria you'd expect them to.
I do my work mostly in PyQt5. Here are some screenshots, actually of a previous attempt which involved Neo4j:
https://math.stackexchange.com/questions/2849246/category-theorists-would-you-use-a-software-tool-for-diagramming-chasing
What could you do with such a tool? Speed up learning mathematics (at your own pace) because the database would enable us to "gameify" category theory. "Fill in the missing homomorphism" e.g.
It could be used as a research tool. Think of it as a humanistic / not formal proof assistant. Everything is human checked, though there could be some ways of issuing an error via graph matching. If the logical error is present in the diagram when read by a human, then store that diagram with an error discription.
Commutative diagrams should highlight the faces of commuting squares, rectangles, etc in unique colors when you hover over them. This should all be done in 3D however, Quiver is a great start in this direction.
A user can step through a proof, with all uneeded info dimmed or hidden, all with clicks of a button. They can search for their diagram, or add it to the global database. Posted diagrams are rated just like any other site e.g. the MSE voting system.
All-in-all it's a tool to speed up learning about abstract mathematics, mainly maths which involves CDs. That is where the bottleneck lies. The best sources to learn from have been written for the past couple hundred years. It is however, our own learning inabilities that stop us from learning the material at a "quick enough" pace.
A tool like this could blow wide-open the doors of accademia to anyone with a PC or mobile phone. Why write a paper, when you can digitize your findings in a much more sophisticated tool than digital ink & paper (PDF / LaTeX documents). Or it can be used in conjunction with the old ways.
Discuss!
Beta Was this translation helpful? Give feedback.
All reactions