Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scrape project board items (experimental) #514

Merged
merged 4 commits into from
Sep 9, 2024
Merged

Scrape project board items (experimental) #514

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2d47494
Scrape project board items (experimental)
rithviknishad Sep 9, 2024
616556e
skip try catching
rithviknishad Sep 9, 2024
450a946
Support for overriding GITHUB_TOKEN
rithviknishad Sep 9, 2024
2fa7cdf
fix typo
rithviknishad Sep 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/scraper-dry-run.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,26 +37,27 @@ jobs:
working-directory: scraper

- name: Scrape data from GitHub
run: pnpm start ${{ github.repository_owner }} ../data/github
run: pnpm start ${{ github.repository_owner }} ../data/github
working-directory: scraper
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.SCRAPER_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
PROJECTS_BOARD_ID: PVT_kwDOA7JD8804yA

- run: mkdir contributors

- name: Generate markdown files for new contributors
run: node scripts/generateNewContributors.js
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- uses: actions/upload-artifact@v4
with:
name: output
retention-days: 5
path: |
data
contributors
- name: Get pnpm store directory
shell: bash
run: |
Expand Down
10 changes: 9 additions & 1 deletion .github/workflows/scraper.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,14 @@ on:
type: string
default: main
description: A specific branch of scraper that is to be used.
projects-board-id:
required: false
type: string
description: Experimental; Works only with particular project board structure. Requires `read:project` scope for the SCRAPER_GITHUB_TOKEN. Disabled if not configured.
secrets:
SCRAPER_GITHUB_TOKEN:
required: false
description: Allows overriding the default GITHUB_TOKEN for providing scopes beyond what GITHUB_TOKEN can offer.
GIT_ACCESS_TOKEN:
required: true
SLACK_API_TOKEN:
Expand Down Expand Up @@ -47,7 +54,8 @@ jobs:
run: pnpm start ${{ github.repository_owner }} ../../data/github
working-directory: scraper
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.SCRAPER_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
PROJECTS_BOARD_ID: ${{ inputs.projects-board-id }}

- uses: actions/setup-python@v3
with:
Expand Down
Loading
Loading