Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reporting Dummy Pull Request #43

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ docs/source/images/*_dag.png
/docs/source/config.yaml
/workflows/mapping/fastq_screen_config.txt
/workflows/mapping/Snakefile.log
.DS_Store
9 changes: 7 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,14 @@ tests:
source activate lcdb-workflows-$(USER)-env && python -m unittest discover -s 'lcdb' -p "*_test.py"
source activate lcdb-workflows-$(USER)-env && python lcdb/interface.py

report-test:
source activate lcdb-workflows-$(USER)-env && python -m unittest lcdb/test/reporting_test.py
source activate lcdb-workflows-$(USER)-env && python lcdb/reporting.py

workflowtest: $(workflow)
test/run_test.py . --build-env --clean --workflow=$(workflow)

osx-dev:
dev:
# pyinotify can be installed with: conda install -c conda-forge pyinotify
clear
fswatch -o -0 -r lcdb | xargs -0 -n 1 bash -c "clear && make tests"
python -m pyinotify -e IN_MODIFY -r lcdb/ -c "clear && make report-test"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To run my tests for this just run
make report-test

75 changes: 75 additions & 0 deletions lcdb/reporting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/usr/bin/env python
""" Reporting System """

import os
import sys
import base64
from textwrap import dedent

import markdown
import jinja2

TEMPLATES_DIR = os.path.join(os.path.dirname(__file__), 'templates')
loader = jinja2.FileSystemLoader(TEMPLATES_DIR)
ENV = jinja2.Environment(loader=loader)

def report(string, **kwargs):
""" Build a report """
md = markdown.Markdown(output_format='html5')
tp = ENV.get_template('base.html')
return tp.render(content=md.convert(dedent(string).format(**kwargs)))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still trying to decide how to build table and figure html representations. Right now I am doing it with a rule and saving the HTML chunk. Then importing the report rule. If we think this is best then probably should add a file importing here instead of assuming **kwargs is a string ready for injection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note that de-indention is huge, because if tabs are not removed then Markdown.convert thinks you are trying to do a code block. I think this should take care of it, but something to look for as we write real reports.

Copy link
Contributor

@daler daler Aug 1, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify: in this system, each logical chunk of a report will have its own rule to create some HTML. The snakemake infrastructure will identify which rules will need to run and will thereforel be used to determine whether a chunk needs to be built. The final report rule in turn will essentially stitch together the various chunks into one final HTML report. Is that right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think the cleanest/easiest way is to just have a rule for each HTML blob and then use the report rule to write the text and insert the blobs in the correct order.


class Report(object):
""" Basic report class """
pass


class Section(object):
""" Basic section class """
pass


class Image(object):
""" Image class """

def __init__(self, png, name='', caption=''):
with open(png, 'rb') as fh:
uri = base64.b64encode(fh.read()).decode('utf-8').replace('\n', '')
self.image = 'data:image/png;base64,{uri}'.format(uri=uri)
self.image_html = '<a href={0}><img src="{0}"/></a>'.format(self.image)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I am saving both a raw string and an HTML image representation of the image. For the Panel class/jinja templating I am using the self.image, but for straight image injecting into markdown I am using the `self.image_html.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable and useful. If size becomes an issue we can move image and image_html to properties (@property decorator) so that the URI is in memory only once rather than 3x here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know much about the @property decorator. But probably a good idea.

self.name = name
self.caption = caption

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any other attributes that would be useful for images. Perhaps a thumbnail size?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about image format? Is everything we're working with PNG? JPEG might be useful. Can just detect on extension.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense. We should come up with a list of image formats and test this out. I would guess {png, jpeg, eps, svg} could cover 99% of cases. Don't know know if eps and svg would encode or not.

We may also want to do bokeh chunks, which should not be too bad. I think bokeh will dump the html blob.

def __repr__(self):
return self.image

def __str__(self):
return self.image


class JinjaPanel(object):
""" Image panel class """

def __init__(self):
self.tp = ENV.get_template('panel.html')
self.panel = ''

def add_row(self, images, caption=''):
self.panel += self.tp.render(caption=caption, images=images)

def __repr__(self):
return self.panel

def __str__(self):
return self.panel


if __name__ == '__main__':
import doctest
from textwrap import dedent
from unittest import TestCase

def assertRaises(*args, **kwargs):
return TestCase.assertRaises(None, *args, **kwargs)
2 changes: 2 additions & 0 deletions lcdb/static/css/bokeh-0.11.1.min.css

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions lcdb/static/css/bootstrap-theme.min.css

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions lcdb/static/css/bootstrap-theme.min.css.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions lcdb/static/css/bootstrap.min.css

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions lcdb/static/css/bootstrap.min.css.map

Large diffs are not rendered by default.

105 changes: 105 additions & 0 deletions lcdb/static/css/dashboard.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
/*
* Base structure
*/

/* Move down content because we have a fixed navbar that is 50px tall */
body {
padding-top: 50px;
}


/*
* Global add-ons
*/

.sub-header {
padding-bottom: 10px;
border-bottom: 1px solid #eee;
}

/*
* Top navigation
* Hide default border to remove 1px line.
*/
.navbar-fixed-top {
border: 0;
}

/*
* Sidebar
*/

/* Hide for mobile, show later */
.sidebar {
display: none;
}
@media (min-width: 768px) {
.sidebar {
position: fixed;
top: 51px;
bottom: 0;
left: 0;
z-index: 1000;
display: block;
padding: 20px;
overflow-x: hidden;
overflow-y: auto; /* Scrollable contents if viewport is shorter than content. */
background-color: #f5f5f5;
border-right: 1px solid #eee;
}
}

/* Sidebar navigation */
.nav-sidebar {
margin-right: -21px; /* 20px padding + 1px border */
margin-bottom: 20px;
margin-left: -20px;
}
.nav-sidebar > li > a {
padding-right: 20px;
padding-left: 20px;
}
.nav-sidebar > .active > a,
.nav-sidebar > .active > a:hover,
.nav-sidebar > .active > a:focus {
color: #fff;
background-color: #428bca;
}


/*
* Main content
*/

.main {
padding: 20px;
}
@media (min-width: 768px) {
.main {
padding-right: 40px;
padding-left: 40px;
}
}
.main .page-header {
margin-top: 0;
}


/*
* Placeholder dashboard ideas
*/

.placeholders {
margin-bottom: 30px;
text-align: center;
}
.placeholders h4 {
margin-bottom: 0;
}
.placeholder {
margin-bottom: 20px;
}
.placeholder img {
display: inline-block;
border-radius: 50%;
}
269 changes: 269 additions & 0 deletions lcdb/static/js/bokeh-0.11.1.min.js

Large diffs are not rendered by default.

7 changes: 7 additions & 0 deletions lcdb/static/js/bootstrap.min.js

Large diffs are not rendered by default.

40 changes: 40 additions & 0 deletions lcdb/templates/base.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags -->
<title>{% block title %}{% endblock %}</title>

<!-- Bootstrap -->
<link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" rel="stylesheet">
<link href="static/css/dashboard.css" rel="stylesheet">

<!-- Bokeh CSS -->
<link rel="stylesheet" href="static/css/bokeh-0.11.1.min.css" type="text/css" />

<!-- jQuery (necessary for Bootstrap's JavaScript plugins) -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.jst">
<script src="static/js/bokeh-0.11.1.min.js"></script>

<!-- Bokeh Scripts go here -->
{% block bokeh %}{% endblock %}

</head>
<body>
{%block navbar %}{% endblock %}
<div class="container-fluid">
<div id="content">
{% block content %}{% endblock %}
{{ content }}
</div>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally jinja2 uses the content block to extend the base template. But this would complicate things with our current setup, so I added a {{ content }} for injecting converted markdown string into the base package.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about providing the template with an iterable of chunks? The final report rule can add an ID for each chunk, for things like populating a TOC:

<div id="content">
    {% for chunk in chunks %}
    <div id="{{chunk.id}}">
        {{ chunk.html }}
    </div>
    {% endfor %}
</div>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.

<div id="footer">
{% block footer %}
&copy; Copyright 2016 by LCDB
{% endblock %}
</div>
</div>
</body>
</html>
10 changes: 10 additions & 0 deletions lcdb/templates/panel.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<div class="row">
<div class="legend">{{ caption }}</div>
{% for image in images %}
<div class="col-sm-3 thumbnail figure">
<div class="legend">{{ image.name }}</div>
<a href="{{ image.image }}"><img src="{{ image.image }}" /></a>
<div class="legend">{{ image.caption }}</div>
</div>
{% endfor %}
</div>
Loading