dissemination smartsky plugins renku

[volodymyrss]

two plugins:

• graph visualization https://github.com/oda-hub/renku-graph-vis/
• aqs plugin explaining how we detect connection to external resources and impact on reproducibility

to prepare:

• we can add a note/reference https://renku.readthedocs.io/en/stable/renku-python/docs/plugins.html @burnout87 can do it. https://github.com/SwissDataScienceCenter/renku-python/blob/develop/docs/plugins.rst
  • check that it is deployed
• we have some manual here https://odahub.io/docs/guide-development/
• make few demo videos/gif showing full flow , 1 min
• add to contributed renku template and new renku features contributed-project-templates#4 mmoda/smartsky templates (with icons!) SwissDataScienceCenter/contributed-project-templates#16
• prepare dissemination post text

dissemination actions:

• series of renku blog posts https://blog.renkulab.io/ https://medium.com/the-renku-blog https://github.com/oda-hub/renku-blog can add a post there in our fork, need to mention @rokroskar
• try to make a talk on Linked Data meeting
• try to disseminate in unige DSCS
• add to astroordas and mmoda website
• find conferences
• can make a discourse post renku.discourse.group , we post it, share draft in advance.
• put own social media (linkedin, mastodon)

[burnout87]

The 'Edit on github' link on the page https://renku.readthedocs.io/en/stable/renku-python/docs/plugins.html is not working, probably outdated

[rokroskar]

Right... that's because the renku-python docs are embedded in the other repo 🙈

Here's the correct place: https://github.com/SwissDataScienceCenter/renku-python/blob/develop/docs/plugins.rst

[burnout87]

Thanks

[burnout87]

Changes with notes on the two plugins have been merged

https://github.com/SwissDataScienceCenter/renku-python/blob/develop/docs/plugins.rst have been merged

[burnout87]

A first draft of the post about the plugins.

• What do you think?
  • Is it too focused on the workflow-development aspect?
  • should it be a bit more general or a but more technical?
  • is too detailed ? or is it too short?

Also, not sure this is the best place for talking about it, feel free to suggest a more appropriate location.

---

Support the workflow development via renku plugin

To support the development of workflows in Renku, a set of dedicated functionalities, provided as Renku plugins, are made available. Specifically, these plugins are:

• renku-graph-vis: to offer a visualization of the project's Knowledge Graph.
• renku-aqs-annotation: to Intercept calls to astroquery functions and store them in the project's Knowledge Graph.

renku-graph-vis

This plugin provides a graphical representation of the renku repository's knowledge graph in two ways:

• From within the renkulab session
• Via a set of CLI commands.

In this way, the ability to have a live overview of the ongoing development for real-time monitoring is introduced.
The animation below shows the automatic updates of the graph during a Renkulab session, with information about the execution of a notebook (papermill command), upon its completion.

To initiate or access the live graph visualization during your session, simply click on the Graph icon located on the main page, as shown in the image below.

As mentioned above, two CLI commands are also offered:

• display: to generate a representation of the graph over an output image.
• show-graph: to start an interactive visualization of the graph over the browser.

More technical details are presented in the README of the repo page: https://github.com/oda-hub/renku-graph-vis/

renku-aqs-annotation

This plugin enables the tracking of the accesses to astronomical archives and services in the project Knowledge Graph.
In particular, the plugin intercepts several key astroquery methods and stores annotations containing information about the calls to these methods (like the arguments used in the call itself) to the project's Knowledge Graph.

In the image below, the graph produced by the renku-graph-vis plugin during a renkulab session, exposes the information added to the project Knowledge Graph as a result of the interception of the astroquery methods. Specifically, it can be seen that during a papermill run of the test-notebook.ipynb notebook (that produced out.ipynb as an output notebook) a call to the astroquery method query_object, using the SimbadClass, has been detected. This notebook is requesting the object Mrk 421 object. The highlighted labels on the edges provide information about the relationship between the two nodes: during the papermillexecution, a call to the query_object method is executed (call label) and in turn, this requests the Astrophysical Object Mrk 421.

More technical details can be found in the README of the repo page: https://github.com/oda-hub/renku-aqs-annotation

[burnout87]

First version of two short video demo.

• installation

demo_video_plugins_installation.mp4

• demo

demo_video_plugins_usage.mp4

[volodymyrss]

First version of two short video demo.

* installation

demo_video_plugins_installation.mp4

* demo

demo_video_plugins_usage.mp4

Nice! But we are waiting for some 1.5min for renku session start. Could you please fast-forward them?

[burnout87]

Nice! But we are waiting for some 1.5min for renku session start. Could you please fast-forward them?

I did a 10x increase in speed of that part of video, what do you think?

demo_video_plugins_usage_faster.mp4

[volodymyrss]

Nice! But we are waiting for some 1.5min for renku session start. Could you please fast-forward them?

I did a 10x increase in speed of that part of video, what do you think?
demo_video_plugins_usage_faster.mp4

It's not a bad way! Could you maybe add some ">>>" label when acceleration happens?
Also there are other a bit slow parts.

[burnout87]

I added a "fast-forward" image/logo when speeding up the vide...What do you think? @volodymyrss

demo_video_plugins_usage_faster_labelled.mp4

[volodymyrss]

I added a "fast-forward" image/logo when speeding up the vide...What do you think? @volodymyrss
demo_video_plugins_usage_faster_labelled.mp4

The fast-forward looks good I think, thanks.

[burnout87]

I made a faster version also of the installation video

demo_video_plugins_installation_faster_labelled.mp4

[burnout87]

* [ ]  series of renku blog posts https://blog.renkulab.io/ https://medium.com/the-renku-blog https://github.com/oda-hub/renku-blog can add a post there in our fork, need to mention @rokroskar

SwissDataScienceCenter/renku-blog#17