Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add docs CI and fix some bugs in it #90

Merged
merged 4 commits into from
Sep 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# Circle CI configuration file
# https://circleci.com/docs/

---
version: 2.1

#######################################
# Define some common steps as commands.
#

commands:
check-skip:
steps:
- run:
name: Check-skip
command: |
export git_log=$(git log --max-count=1 --pretty=format:"%B" |
tr "\n" " ")
echo "Got commit message:"
echo "${git_log}"
if [[ -v CIRCLE_PULL_REQUEST ]] && ( \
[[ "$git_log" == *"[skip circle]"* ]] || \
[[ "$git_log" == *"[circle skip]"* ]]); then
echo "Skip detected, exiting job ${CIRCLE_JOB} for PR ${CIRCLE_PULL_REQUEST}."
circleci-agent step halt;
fi

merge:
steps:
- run:
name: Merge with upstream
command: |
if ! git remote -v | grep upstream; then
git remote add upstream https://github.com/matplotlib/cycler.git
fi
git fetch upstream
if [[ "$CIRCLE_BRANCH" != "main" ]] && \
[[ "$CIRCLE_PR_NUMBER" != "" ]]; then
echo "Merging ${CIRCLE_PR_NUMBER}"
git pull --ff-only upstream "refs/pull/${CIRCLE_PR_NUMBER}/merge"
fi

pip-install:
description: Upgrade pip to get as clean an install as possible
steps:
- run:
name: Upgrade pip
command: |
python -m pip install --upgrade --user pip

cycler-install:
steps:
- run:
name: Install Cycler
command: |
python -m pip install --user -ve .[docs]

doc-build:
steps:
- restore_cache:
keys:
- sphinx-env-v1-{{ .BuildNum }}-{{ .Environment.CIRCLE_JOB }}
- sphinx-env-v1-{{ .Environment.CIRCLE_PREVIOUS_BUILD_NUM }}-{{ .Environment.CIRCLE_JOB }}
- run:
name: Build documentation
command: |
# Set epoch to date of latest tag.
export SOURCE_DATE_EPOCH="$(git log -1 --format=%at $(git describe --abbrev=0))"
mkdir -p logs
make html O="-T -j4 -w /tmp/sphinxerrorswarnings.log"
rm -r build/html/_sources
working_directory: doc
- save_cache:
key: sphinx-env-v1-{{ .BuildNum }}-{{ .Environment.CIRCLE_JOB }}
paths:
- doc/build/doctrees

doc-show-errors-warnings:
steps:
- run:
name: Extract possible build errors and warnings
command: |
(grep "WARNING\|ERROR" /tmp/sphinxerrorswarnings.log ||
echo "No errors or warnings")
# Save logs as an artifact, and convert from absolute paths to
# repository-relative paths.
sed "s~$PWD/~~" /tmp/sphinxerrorswarnings.log > \
doc/logs/sphinx-errors-warnings.log
when: always
- store_artifacts:
path: doc/logs/sphinx-errors-warnings.log

##########################################
# Here is where the real jobs are defined.
#

jobs:
docs-python39:
docker:
- image: cimg/python:3.9
resource_class: large
steps:
- checkout
- check-skip
- merge

- pip-install

- cycler-install

- doc-build
- doc-show-errors-warnings

- store_artifacts:
path: doc/build/html

#########################################
# Defining workflows gets us parallelism.
#

workflows:
version: 2
build:
jobs:
# NOTE: If you rename this job, then you must update the `if` condition
# and `circleci-jobs` option in `.github/workflows/circleci.yml`.
- docs-python39
66 changes: 66 additions & 0 deletions .circleci/fetch_doc_logs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
"""
Download artifacts from CircleCI for a documentation build.

This is run by the :file:`.github/workflows/circleci.yml` workflow in order to
get the warning/deprecation logs that will be posted on commits as checks. Logs
are downloaded from the :file:`docs/logs` artifact path and placed in the
:file:`logs` directory.

Additionally, the artifact count for a build is produced as a workflow output,
by appending to the file specified by :env:`GITHUB_OUTPUT`.

If there are no logs, an "ERROR" message is printed, but this is not fatal, as
the initial 'status' workflow runs when the build has first started, and there
are naturally no artifacts at that point.

This script should be run by passing the CircleCI build URL as its first
argument. In the GitHub Actions workflow, this URL comes from
``github.event.target_url``.
"""
import json
import os
from pathlib import Path
import sys
from urllib.parse import urlparse
from urllib.request import URLError, urlopen


if len(sys.argv) != 2:
print('USAGE: fetch_doc_results.py CircleCI-build-url')
sys.exit(1)

target_url = urlparse(sys.argv[1])
*_, organization, repository, build_id = target_url.path.split('/')
print(f'Fetching artifacts from {organization}/{repository} for {build_id}')

artifact_url = (
f'https://circleci.com/api/v2/project/gh/'
f'{organization}/{repository}/{build_id}/artifacts'
)
print(artifact_url)
try:
with urlopen(artifact_url) as response:
artifacts = json.load(response)
except URLError:
artifacts = {'items': []}
artifact_count = len(artifacts['items'])
print(f'Found {artifact_count} artifacts')

with open(os.environ['GITHUB_OUTPUT'], 'w+') as fd:
fd.write(f'count={artifact_count}\n')

logs = Path('logs')
logs.mkdir(exist_ok=True)

found = False
for item in artifacts['items']:
path = item['path']
if path.startswith('doc/logs/'):
path = Path(path).name
print(f'Downloading {path} from {item["url"]}')
with urlopen(item['url']) as response:
(logs / path).write_bytes(response.read())
found = True

if not found:
print('ERROR: Did not find any artifact logs!')
58 changes: 58 additions & 0 deletions .github/workflows/circleci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
name: "CircleCI artifact handling"
on: [status]
jobs:
circleci_artifacts_redirector_job:
if: "${{ github.event.context == 'ci/circleci: docs-python39' }}"
permissions:
statuses: write
runs-on: ubuntu-latest
name: Run CircleCI artifacts redirector
steps:
- name: GitHub Action step
uses: larsoner/circleci-artifacts-redirector-action@master
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
artifact-path: 0/doc/build/html/index.html
circleci-jobs: docs-python39
job-title: View the built docs

post_warnings_as_review:
if: "${{ github.event.context == 'ci/circleci: docs-python39' }}"
permissions:
contents: read
checks: write
pull-requests: write
runs-on: ubuntu-latest
name: Post warnings/errors as review
steps:
- uses: actions/checkout@v3

- name: Fetch result artifacts
id: fetch-artifacts
run: |
python .circleci/fetch_doc_logs.py "${{ github.event.target_url }}"

- name: Set up reviewdog
if: "${{ steps.fetch-artifacts.outputs.count != 0 }}"
uses: reviewdog/action-setup@v1
with:
reviewdog_version: latest

- name: Post review
if: "${{ steps.fetch-artifacts.outputs.count != 0 }}"
env:
REVIEWDOG_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REVIEWDOG_SKIP_DOGHOUSE: "true"
CI_COMMIT: ${{ github.event.sha }}
CI_REPO_OWNER: ${{ github.event.repository.owner.login }}
CI_REPO_NAME: ${{ github.event.repository.name }}
run: |
# The 'status' event does not contain information in the way that
# reviewdog expects, so we unset those so it reads from the
# environment variables we set above.
unset GITHUB_ACTIONS GITHUB_EVENT_PATH
cat logs/sphinx-deprecations.log | \
reviewdog \
-efm '%f\:%l: %m' \
-name=examples -tee -reporter=github-check -filter-mode=nofilter
5 changes: 2 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,13 @@ jobs:
python-version: ${{ matrix.python-version }}
allow-prereleases: true

- name: Install Python dependencies
- name: Upgrade pip
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade pytest pytest-cov pytest-xdist

- name: Install cycler
run: |
python -m pip install --no-deps .
python -m pip install .[tests]

- name: Run pytest
run: |
Expand Down
2 changes: 1 addition & 1 deletion doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# html_static_path = []

# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
Expand Down
23 changes: 11 additions & 12 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,10 @@ composition and iteration logic.
Base
----

A single entry `Cycler` object can be used to easily
cycle over a single style. To create the `Cycler` use the :py:func:`cycler`
function to link a key/style/kwarg to series of values. The key must be
hashable (as it will eventually be used as the key in a :obj:`dict`).
A single entry `Cycler` object can be used to easily cycle over a single style.
To create the `Cycler` use the :py:func:`cycler` function to link a
key/style/keyword argument to series of values. The key must be hashable (as it
will eventually be used as the key in a :obj:`dict`).

.. ipython:: python

Expand All @@ -53,7 +53,7 @@ hashable (as it will eventually be used as the key in a :obj:`dict`).
color_cycle = cycler(color=['r', 'g', 'b'])
color_cycle

The `Cycler` knows it's length and keys:
The `Cycler` knows its length and keys:

.. ipython:: python

Expand Down Expand Up @@ -97,7 +97,7 @@ create complex multi-key cycles.
Addition
~~~~~~~~

Equal length `Cycler` s with different keys can be added to get the
Equal length `Cycler`\s with different keys can be added to get the
'inner' product of two cycles

.. ipython:: python
Expand Down Expand Up @@ -180,7 +180,7 @@ matrices)
Integer Multiplication
~~~~~~~~~~~~~~~~~~~~~~

`Cycler` s can also be multiplied by integer values to increase the length.
`Cycler`\s can also be multiplied by integer values to increase the length.

.. ipython:: python

Expand Down Expand Up @@ -331,8 +331,7 @@ the same style.
Exceptions
----------


A :obj:`ValueError` is raised if unequal length `Cycler` s are added together
A :obj:`ValueError` is raised if unequal length `Cycler`\s are added together

.. ipython:: python
:okexcept:
Expand Down Expand Up @@ -401,6 +400,6 @@ However, if you want to do something more complicated:

ax.legend(loc=0)

the plotting logic can quickly become very involved. To address this and allow easy
cycling over arbitrary ``kwargs`` the `Cycler` class, a composable
kwarg iterator, was developed.
the plotting logic can quickly become very involved. To address this and allow
easy cycling over arbitrary ``kwargs`` the `Cycler` class, a composable keyword
argument iterator, was developed.
13 changes: 13 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,19 @@ keywords = ["cycle kwargs"]
homepage = "https://matplotlib.org/cycler/"
repository = "https://github.com/matplotlib/cycler"

[project.optional-dependencies]
docs = [
"ipython",
"matplotlib",
"numpydoc",
"sphinx",
]
tests = [
"pytest",
"pytest-cov",
"pytest-xdist",
]

[tool.setuptools]
packages = ["cycler"]

Expand Down