[REVIEW]: ProbabilityBoundsAnalysis.jl: theory and algorithms #96

whedon · 2021-10-22T10:06:44Z

Submitting author: @AnderGray (Ander Gray)
Repository: https://github.com/AnderGray/ProbabilityBoundsAnalysis.jl
Branch with paper.md (empty if default branch):
Version:
Editor: @carstenbauer
Reviewers: @dpsanders, @baggepinnen
Archive: Pending

Status

Status badge code:

HTML: <a href="https://proceedings.juliacon.org/papers/41644cd3e3844116d843e5efa1a5cd30"><img src="https://proceedings.juliacon.org/papers/41644cd3e3844116d843e5efa1a5cd30/status.svg"></a>
Markdown: [![status](https://proceedings.juliacon.org/papers/41644cd3e3844116d843e5efa1a5cd30/status.svg)](https://proceedings.juliacon.org/papers/41644cd3e3844116d843e5efa1a5cd30)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@dpsanders & @baggepinnen, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @carstenbauer know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @dpsanders

Conflict of interest

As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

Repository: Is the source code for this software available at the repository url?
License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
Authorship: Has the submitting author (@AnderGray) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

Authors: Does the paper.tex file include a list of authors with their affiliations?
A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
Page limit: Is the page limit for full papers respected by the submitted document?

Content

Context: is the scientific context motivating the work correctly presented?
Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
Results: are the results presented and compared to approaches with similar goals?

Review checklist for @baggepinnen

Conflict of interest

As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

Repository: Is the source code for this software available at the repository url?
License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
Authorship: Has the submitting author (@AnderGray) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

Authors: Does the paper.tex file include a list of authors with their affiliations?
A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
Page limit: Is the page limit for full papers respected by the submitted document?

Content

Context: is the scientific context motivating the work correctly presented?
Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
Results: are the results presented and compared to approaches with similar goals?

The text was updated successfully, but these errors were encountered:

whedon · 2021-10-22T10:06:47Z

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @dpsanders, @baggepinnen it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon · 2021-10-22T10:06:54Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

carstenbauer · 2021-10-22T10:07:39Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2021-10-22T10:07:42Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2021-10-22T10:07:48Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

whedon · 2021-11-05T10:07:02Z

👋 @dpsanders, please update us on how your review is going (this is an automated reminder).

whedon · 2021-11-05T10:07:02Z

👋 @baggepinnen, please update us on how your review is going (this is an automated reminder).

AnderGray · 2021-11-08T16:47:20Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2021-11-08T16:47:23Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2021-11-08T16:47:31Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

AnderGray · 2021-11-08T16:50:11Z

@dpsanders @baggepinnen, not sure why it's not finding the paper. I changed the title in the header.tex, could this perhaps have led to some issues? Was it ok to change the title?

Otherwise, a pdf of the paper is uploaded to the branch

carstenbauer · 2021-11-14T16:15:55Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2021-11-14T16:15:58Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2021-11-14T16:16:04Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

carstenbauer · 2021-11-14T16:16:13Z

@whedon generate pdf

whedon · 2021-11-14T16:16:21Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

carstenbauer · 2021-11-14T16:25:15Z

Perhaps the issue here is that the branch information ("JuliaConPaper") is already part of the repo URL (see OP). @vchuravy, can you drop that part and make the repo here simply point to https://github.com/AnderGray/ProbabilityBoundsAnalysis.jl? (I think I can't do it, right?)

@AnderGray In any case, it is better to not have a paper.pdf in the paper/ folder of the JuliaConPaper branch. On the one hand, to not confuse whedon, and on the other hand since the paper isn't published yet.

AnderGray · 2021-12-14T11:39:59Z

Hi all, is this still being considered for review?

baggepinnen · 2021-12-14T11:46:19Z

I have started my review, but I've been reading the pdf that was deleted from the https://github.com/AnderGray/ProbabilityBoundsAnalysis.jl repo, I couldn't find any other pdf?

AnderGray · 2021-12-14T12:18:15Z

Thanks @baggepinnen, I removed it as was suggested.

Here's an upload: PBApaper.pdf

carstenbauer · 2022-01-07T12:41:36Z

@whedon generate pdf

whedon · 2022-01-07T12:41:47Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

carstenbauer · 2022-01-07T12:42:29Z

@whedon generate pdf

whedon · 2022-01-07T12:42:41Z

PDF failed to compile for issue #96 with the following error:

 Can't find any papers to compile :-(

carstenbauer · 2022-01-07T12:48:57Z

While I try to fix the whedon issue here, @dpsanders and @baggepinnen please let me know how your reviews are going. Would be great if you could finish them within the next few weeks at the latest. Thanks in advance!

dpsanders · 2022-02-07T02:42:07Z

👋 @AnderGray: It's not clear to me why Eduardo Patelli is an author of the paper, since he does not seem to be an author of the package, according to the docs.

AnderGray · 2022-03-04T15:51:28Z

@dpsanders @baggepinnen Thanks both for this review, and for the great suggestions which will improve the quality of package substantially. Apologies I'm taking so long to get back. I'm working on two PRs which address your comments. I've completed David's, and will hopefully finish Fredrik's in the coming week

AnderGray · 2022-03-04T16:07:43Z

👋 @AnderGray: It's not clear to me why Eduardo Patelli is an author of the paper, since he does not seem to be an author of the package, according to the docs.

Edoardo Patelli provided funding sourcing and mentorship, without which this package would have not been created, and I believe he should be an author of the paper. I will change the authorship in the Docs

carstenbauer · 2022-03-31T11:00:29Z

What's the status here? @AnderGray I see there are still open issues and PRs related to the reviews by @dpsanders and @baggepinnen. Am I right in assuming that you just need more time to improve / fix things?

AnderGray · 2022-03-31T15:51:25Z

@carstenbauer Thanks for following this up. Yes, most of the issues have been addressed. @dpsanders's questions have all be addressed I believe, and although the PR was merged (since we wanted this functionality). I've addressed some of @baggepinnen's, the biggest is related to testing, which is taking time. Nearly there

AnderGray · 2022-04-15T11:06:38Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2022-04-15T11:06:41Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2022-04-15T11:06:48Z

PDF failed to compile for issue #96 with the following error:

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
===========Latexmk: Missing input file: 'subfig.sty' from line
  '! LaTeX Error: File `subfig.sty' not found.'
Latexmk: Missing input file: 'subfig.sty' from line
  '! LaTeX Error: File `subfig.sty' not found.'
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF

AnderGray · 2022-04-15T11:14:43Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2022-04-15T11:14:45Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2022-04-15T11:14:53Z

PDF failed to compile for issue #96 with the following error:

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.out'
------------
Run number 3 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF

AnderGray · 2022-04-15T11:41:00Z

@baggepinnen @dpsanders apologies for the wait, I think we're ready for another look. Unsure why why the pdf isn't compiling, but I believe I've incorporated the requests. I've added all the DOI's I could find, and shortened the paper.

For @baggepinnen, the PR can be seen in pull/37:

Testing improved
Documentation improved (included added community guidelines)
Added comparisons

Note that unlike intervals, which always return Boolean values (perhaps erroneously), comparisons between p-boxes return interval probabilities, which give the probability that the imprecisely characterise random variables meet the condition. More details can be found in the new docs

Example between p-box and scalars:

julia> X = uniform(0, 1)
julia> X <= 0.7
[0.695, 0.705001]

julia> X >= 0.4
[0.594999, 0.605]

julia> X = normal(interval(-0.5, 0.5), interval(1, 1.5))
julia> X >= 1
[0.0649999, 0.37]

julia> X = uniform(0, 1)
julia> X <= 2
true

julia> X >= 2
false

Example between p-box and intervals

julia> X = uniform(0, 1)
julia> Y = interval(0.7, 2)
julia> X <= Y
[0.695, 1]

julia> X >= Y
[0, 0.305]

julia> X <= interval(2, 3)
true

julia> X >= interval(2, 3)
false

Example between p-boxes

julia> X = uniform(0, 1)
julia> Y = uniform(0.5, 1.5)
julia> X <= Y
[0.5, 1]

julia> X <= uniform(2, 3)
true

julia> X >= uniform(2, 3)
false

Note that this will cause your foo function above to fail due to using a non-Boolean ... I do however believe that this is correct in this context. Evaluate this with intervals [0, 1] <= 0.5 returns false by default, but it could be argued that the interval Boolean [0, 1] is the correct return here, since we cannot say if the interval is above or below the scalar. There is some discussion about this in the introduction in the paper. ProbabilityBoundsAnalysis.jl returns the probability that the condition is met, e.g., uniform(0,1) <= 0.5 gives a tight probability interval [0.494999, 0.505001] (the interval comes from discretisation error bounding).

A warning about this has been added to the docs. Ideally I would like to modify the Julia warning message when an interval (probability) is used in a Boolean context, saying why this is happening. Do you know if it's possible to do this?

For @dpsanders the PR was already merged, but is pull/35. I've also responded to your issues individually, which are still open.

dpsanders · 2022-04-15T19:41:10Z

There is a massive overhaul of IntervalArithmetic.jl underway which will fix the X < Y issue for intervals X and Y, by allowing the user to specify the behaviour they prefer: see JuliaIntervals/IntervalArithmetic.jl#271 and JuliaIntervals/IntervalArithmetic.jl#514

@AnderGray : Thanks, I'll take a look!

AnderGray · 2022-05-02T10:33:39Z

@baggepinnen @dpsanders Hi guys, any updates with this? :)

AnderGray · 2022-06-07T19:33:46Z

@baggepinnen @dpsanders @carstenbauer hi fellas, are we still considering this for review?

carstenbauer · 2022-06-09T09:36:09Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2022-06-09T09:36:12Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2022-06-09T09:36:22Z

PDF failed to compile for issue #96 with the following error:

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.out'
------------
Run number 3 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF

AnderGray · 2022-10-27T15:54:50Z

Hi @carstenbauer @baggepinnen @dpsanders, been over a year since submission. This still being considered, or is it better to resubmit to another journal?

carstenbauer · 2022-12-28T09:20:29Z

@whedon generate pdf from branch JuliaConPaper

whedon · 2022-12-28T09:20:32Z

Attempting PDF compilation from custom branch JuliaConPaper. Reticulating splines etc...

whedon · 2022-12-28T09:20:40Z

PDF failed to compile for issue #96 with the following error:

 Latexmk: This is Latexmk, John Collins, 17 Jan. 2018, version: 4.55.
Rule 'pdflatex': Rules & subrules not known to be previously run:
   pdflatex
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.aux'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'paper.out'
------------
Run number 3 of rule 'pdflatex'
------------
------------
Running 'pdflatex  -recorder  "paper.tex"'
------------
Failure to make 'paper.pdf'
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'paper.log' for details
Looks like we failed to compile the PDF

carstenbauer · 2022-12-28T09:24:14Z

@AnderGray I apologise sincerely and must admit that I completely forgot about your submission / this issue. Are you willing to continue and move this over the finish line?

If so, it would be great if you could debug the PDF compilation error (Does it work locally? Maybe "rebase" the paper folder to the current version of the submission template.). Afterwards, I'll ask the reviewers to give this another / a final look.

Sorry again.

carstenbauer · 2023-03-20T16:10:41Z

Since there hasn't been any response to my last message, I'm closing this. If you want to reopen, please let me know!

lucaferranti · 2024-04-25T17:24:18Z

@editorialbot reject

editorialbot · 2024-04-25T17:24:20Z

Paper rejected.

whedon added the review label Oct 22, 2021

whedon assigned carstenbauer Oct 22, 2021

whedon mentioned this issue Oct 22, 2021

[PRE REVIEW]: ProbabilityBoundsAnalysis.jl: theory and algorithms #90

Closed

This comment has been minimized.

Sign in to view

dpsanders mentioned this issue Feb 7, 2022

Clashes with Plots.jl AnderGray/ProbabilityBoundsAnalysis.jl#32

Open

AnderGray mentioned this issue Mar 8, 2022

Baggepinnen review AnderGray/ProbabilityBoundsAnalysis.jl#37

Merged

carstenbauer closed this as completed Mar 20, 2023

editorialbot added the rejected label Apr 25, 2024

[REVIEW]: ProbabilityBoundsAnalysis.jl: theory and algorithms #96

[REVIEW]: ProbabilityBoundsAnalysis.jl: theory and algorithms #96

Comments

whedon commented Oct 22, 2021 • edited by editorialbot Loading

Status

Reviewer instructions & questions

Review checklist for @dpsanders

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Paper format

Content

Review checklist for @baggepinnen

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Paper format

Content

whedon commented Oct 22, 2021

whedon commented Oct 22, 2021

carstenbauer commented Oct 22, 2021

whedon commented Oct 22, 2021

whedon commented Oct 22, 2021

whedon commented Nov 5, 2021

whedon commented Nov 5, 2021

AnderGray commented Nov 8, 2021

whedon commented Nov 8, 2021

whedon commented Nov 8, 2021

AnderGray commented Nov 8, 2021

carstenbauer commented Nov 14, 2021

whedon commented Nov 14, 2021

whedon commented Nov 14, 2021

carstenbauer commented Nov 14, 2021

whedon commented Nov 14, 2021

carstenbauer commented Nov 14, 2021

AnderGray commented Dec 14, 2021

baggepinnen commented Dec 14, 2021

AnderGray commented Dec 14, 2021

carstenbauer commented Jan 7, 2022

whedon commented Jan 7, 2022

carstenbauer commented Jan 7, 2022

whedon commented Jan 7, 2022

carstenbauer commented Jan 7, 2022

This comment has been minimized.

This comment has been minimized.

dpsanders commented Feb 7, 2022

AnderGray commented Mar 4, 2022

AnderGray commented Mar 4, 2022

carstenbauer commented Mar 31, 2022

AnderGray commented Mar 31, 2022

AnderGray commented Apr 15, 2022

whedon commented Apr 15, 2022

whedon commented Apr 15, 2022

AnderGray commented Apr 15, 2022

whedon commented Apr 15, 2022

whedon commented Apr 15, 2022

AnderGray commented Apr 15, 2022 • edited Loading

dpsanders commented Apr 15, 2022 • edited Loading

AnderGray commented May 2, 2022

AnderGray commented Jun 7, 2022

carstenbauer commented Jun 9, 2022

whedon commented Jun 9, 2022

whedon commented Jun 9, 2022

AnderGray commented Oct 27, 2022

carstenbauer commented Dec 28, 2022

whedon commented Dec 28, 2022

whedon commented Dec 28, 2022

carstenbauer commented Dec 28, 2022

carstenbauer commented Mar 20, 2023

lucaferranti commented Apr 25, 2024

editorialbot commented Apr 25, 2024

whedon commented Oct 22, 2021 •

edited by editorialbot

Loading

AnderGray commented Apr 15, 2022 •

edited

Loading

dpsanders commented Apr 15, 2022 •

edited

Loading