Commit 2174f137 authored by Cyril Matthey-Doret's avatar Cyril Matthey-Doret
Browse files

Merge branch 'console-entrypoint' into 'master'

* Migrate package metadata to setup.cfg
* Use click command group to make subcommands for each script
* update license {GPLv3 -> MIT} in package metadata
* update readme to show examples with subcommands

See merge request !2
parents c896e437 99692118
Pipeline #388573 passed with stage
in 4 minutes and 14 seconds
......@@ -22,7 +22,37 @@ cd advanced-teaching-automation
pip install -e .
```
Most script below require a Gitlab API token. By default, you will be prompted for your token upon running the scripts, but you can also provide it as a plain text file with the `--token` option.
Installing the package this way allows you to modify / extend the code to match your personal needs.
If you don't need to modify it, you can also install it directly from the URL without cloning:
```
pip install git+https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation
```
## Usage
Once the package is installed, the command `teach-auto` will be available globally on your system. Each functionality is available as a subcommand. For example `teach-auto collect-forks`. Each command stands in its own python script which can also be executed directly, e.g. `python teach_auto/collect_forks.py`. Each command provides a `--help` flag to print the list of available options and usage information:
```sh
➜ teach-auto --help
Usage: teach-auto [OPTIONS] COMMAND [ARGS]...
teach-auto: Classroom automation utilities for Gitlab + Renku.
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
clone-forks Clone all forks from input JSON into target directory.
collect-forks Gather metadata about all forks of input project into...
create-groups Given a parent-group URL and a CSV file of students...
invite-students Send invitations to join input gitlab group to all...
send-feedback Send feedback to repositories in input JSON by opening...
```
Most functionalities below require a Gitlab API token. By default, you will be prompted for your token upon running the scripts, but you can also provide it as a plain text file with the `--token` option.
## Content
......@@ -45,13 +75,13 @@ The manual process of inviting the whole class to the group can be automated usi
<details>
<summary> <b>Read more...</b> </summary>
script: [teach\_utils/invite\_students.py](teach_utils/invite_students.py)
script: [teach\_auto/invite\_students.py](teach_auto/invite_students.py)
**usage**:
Invite students to `class-group`:
`python invite_students.py emails.txt https://gitlab-instance.com/class-group`
`teach-auto invite-students emails.txt https://gitlab-instance.com/class-group`
</details>
......@@ -66,13 +96,13 @@ One way to achieve this is to have the teacher create all private student groups
<details>
<summary> <b>Read more...</b> </summary>
script: [teach\_utils/moodle\_to\_student\_groups.py](teach_utils/moodle_to_student_groups.py)
script: [teach\_auto/create\_groups.py](teach_auto/create_groups.py)
**usage**:
Create student (sub)groups inside `class-group`:
`python moodle_to_student_groups.py students_moodle.csv https://gitlab-instance.com/class-group`
`create-groups students_moodle.csv https://gitlab-instance.com/class-group`
</details>
......@@ -99,15 +129,16 @@ The teacher can then use the Gitlab API to keep track of student forks. We provi
> Note: the script assumes Gitlab commit timestamps and the deadline are in the UTC timezone.
script: [teach\_utils/collect\_forks.py](teach_utils/collect_forks.py)
script: [teach\_auto/collect\_forks.py](teach_auto/collect_forks.py)
**usage**:
Collect all group-owned forks of upstream-project, and write the JSON list to `forks.json`. Commits and autostart URLs will point to the last commit before 23h59 on March 15, 2022:
```sh
python collect_forks.py \
teach-auto collect-forks \
--deadline "2022-03-15T23:59" \
--group-only \
https://gitlab-instance.com/namespace/upstream-project \
> forks.json
```
......@@ -147,19 +178,19 @@ In some cases, the student projects may have to be cloned locally by the teacher
<summary> <b>Read more...</b> </summary>
script: [teach\_utils/clone\_forks\_from\_json.sh](teach_utils/clone_forks_from_json.sh)
script: [teach\_auto/clone\_forks.py](teach_auto/clone_forks.py)
**usage**:
Clone all forks listed in `forks.json` into `clone_dir`:
`./clone_forks_from_json.sh forks.json clone_dir`
`teach-auto clone-forks forks.json clone_dir`
Or reading directly from stdin:
```
python ./collect_forks.py https://gitlab-instance.com/namespace/upstream-project \
| ./clone_forks_from_json.sh clone_dir
teach-auto collect-forks https://gitlab-instance.com/namespace/upstream-project \
| teach-auto clone_forks clone_dir
```
</details>
......@@ -173,13 +204,13 @@ After grading student assignments, the teacher may want to send student groups t
<details>
<summary> <b>Read more...</b> </summary>
script: [teach\_utils/send\_feedback.py](teach_utils/send_feedback.py)
script: [teach\_auto/send\_feedback.py](teach_auto/send_feedback.py)
**usage**:
By default, this script opens issues in all forks listed in `forks.json`. For each fork, fields {grades}, {comments} in the issue description are filled using column values in a CSV file provided with `--feedback`. The issue description uses a default template defined in the script:
`python ./send_feedback.py --feedback grades.csv forks.json`
`teach-auto send-feedback --feedback grades.csv forks.json`
where grades.csv is:
......@@ -194,9 +225,9 @@ A custom template can also be provided, and fields will be replaced by values fr
```
echo ":warning: Your fork is public, please make it private!" > visibility.md
./collect_forks.py https://renkulab.io/gitlab/class/homework \
teach-auto collect-forks https://renkulab.io/gitlab/class/homework \
| jq '.[] | select(.visibility == "public")' \
| ./send_feedback.py --template visibility.md
| teach-auto send-feedback --template visibility.md
```
> Note: `jq` is a tool to process JSON data. It can be used to reformat or filter the output of `collect_forks.py` in many ways. A number of useful one liners are [shown here](./docs/oneliners.md).
......
[metadata]
name = teach_auto
version = attr: teach_auto.__version__
author = Cyril Matthey-Doret
author_email = cyril.matthey-doret@epfl.ch
license = MIT
description = Utilities to automate classroom management with Gitlab and Renku.
keywords = automation, git, gitlab, teaching
url = https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation
classifiers =
Development Status :: 3 - Alpha
Programming Language :: Python :: 3.9
Programming Language :: Python :: 3.10
Intended Audience :: Education
Topic :: Software Development :: Version Control :: Git
License :: OSI Approved :: MIT License
......@@ -5,8 +5,7 @@ with open("requirements.txt", "r") as f:
setup(
name="teach_auto",
version="0.1",
packages=find_packages(),
install_requires=REQUIREMENTS,
license="GPLv3",
entry_points={"console_scripts": ["teach-auto=teach_auto.cli:cli"]},
)
from operator import inv
import click
from teach_auto import __version__
from teach_auto.clone_forks import clone_forks
from teach_auto.collect_forks import collect_forks
from teach_auto.create_groups import create_groups
from teach_auto.invite_students import invite_students
from teach_auto.send_feedback import send_feedback
@click.group()
@click.version_option(version=__version__)
def cli():
"""
teach-auto: Classroom automation utilities for Gitlab + Renku.
"""
...
cli.add_command(clone_forks, name="clone-forks")
cli.add_command(collect_forks, name="collect-forks")
cli.add_command(create_groups, name="create-groups")
cli.add_command(invite_students, name="invite-students")
cli.add_command(send_feedback, name="send-feedback")
......@@ -23,10 +23,15 @@ def clone_fork(url: str, out_path: Path, commit: str):
@click.command()
@click.argument("out_dir", type=click.Path(file_okay=False, exists=False))
@click.argument("forks", type=click.File(mode="r"), default=sys.stdin)
def main(out_dir, forks, token):
def clone_forks(out_dir, forks, token):
"""Clone all forks from input JSON into target directory. The input json should be produced by collect-forks."""
forks = json.load(forks)
for fork in forks:
url, group, commit = [fork.get(key) for key in ["url", "group", "commit"]]
path = Path(out_dir) / group
clone_fork(url, path, commit)
if __name__ == "__main__":
clone_forks()
......@@ -15,8 +15,8 @@ import requests
import click
from datetime import datetime
import pytz
from class_auto.common_requests import parse_repo_url, get_project_id
from class_auto.utils import ingest_token
from teach_auto.common_requests import parse_repo_url, get_project_id
from teach_auto.utils import ingest_token
def validate_iso_date(date: str) -> str:
......@@ -28,7 +28,7 @@ def validate_iso_date(date: str) -> str:
raise ValueError("Deadline must be in ISO-8601 format.")
def collect_forks(project_url: str, header=Dict[str, str]) -> List[Dict]:
def request_forks(project_url: str, header=Dict[str, str]) -> List[Dict]:
"""Retrieve the metadata from all forks of input project"""
base, namespace, repo = parse_repo_url(project_url)
upstream_id = get_project_id(project_url, header)
......@@ -139,7 +139,13 @@ def format_fork_metadata(
type=click.Path(exists=True),
help="Path to a file containing the Gitlab API token. If not provided, you will be prompted for the token.",
)
def main(repo_url, token_path, deadline=None):
@click.option(
"--group-only",
is_flag=True,
help="Only consider forks belonging to a group (and not an individual user).",
)
def collect_forks(repo_url, token_path, deadline=None, group_only=False):
"""Gather metadata about all forks of input project into a JSON file."""
token = ingest_token(token_path)
# Check for valid deadline format
if deadline is not None:
......@@ -147,8 +153,9 @@ def main(repo_url, token_path, deadline=None):
# Get metadata of all forks from input project
header = {"PRIVATE-TOKEN": token}
forks = collect_forks(repo_url, header)
forks = request_forks(repo_url, header)
# Only keep those which belong to a group
if group_only:
forks = filter_group_forks(forks)
# Reformat metadata for convenience
......@@ -157,4 +164,4 @@ def main(repo_url, token_path, deadline=None):
if __name__ == "__main__":
main()
collect_forks()
......@@ -57,7 +57,7 @@ def generate_student_code(first_name: str, last_name: str) -> str:
type=click.Path(exists=True),
help="Path to a file containing the Gitlab API token. If not provided, you will be prompted for the token.",
)
def main(student_table, parent_url, token_path):
def create_groups(student_table, parent_url, token_path):
"""Given a parent-group URL and a CSV file of students exported from Moodle,
create one private subgroup for each student in the parent group and invite students
in their personal group. The student table must contain columns: "Email address", "First name"
......@@ -85,4 +85,4 @@ def main(student_table, parent_url, token_path):
if __name__ == "__main__":
main()
create_groups()
......@@ -5,8 +5,8 @@ import re
from typing import Dict
import requests
import click
from class_auto.common_requests import parse_repo_url, get_group_id
from class_auto.utils import ingest_token
from teach_auto.common_requests import parse_repo_url, get_group_id
from teach_auto.utils import ingest_token
def invite_student_email(
......@@ -52,7 +52,7 @@ def find_email(line: str) -> str:
type=click.Path(exists=True),
help="Path to a file containing the Gitlab API token. If not provided, you will be prompted for the token.",
)
def main(emails_file, group_url, token_path):
def invite_students(emails_file, group_url, token_path):
"""
Send invitations to join input gitlab group to all emails in the emails_file.
The file should have one email per line."""
......@@ -71,4 +71,4 @@ def main(emails_file, group_url, token_path):
if __name__ == "__main__":
main()
invite_students()
......@@ -15,8 +15,8 @@ import requests
from pathlib import Path
import click
import pandas as pd
from class_auto.common_requests import parse_repo_url
from class_auto.utils import ingest_token
from teach_auto.common_requests import parse_repo_url
from teach_auto.utils import ingest_token
FEEDBACK_TEMPLATE = """
......@@ -114,7 +114,7 @@ def gather_fill_data(fork: Dict, feedback_csv: Optional[Path] = None) -> Dict:
@click.option(
"--dry-run", is_flag=True, help="Print feedback to stdout instead of opening issues"
)
def main(forks, feedback_csv, token_path, template, dry_run):
def send_feedback(forks, feedback_csv, token_path, template, dry_run):
"""
Send feedback to repositories in input JSON by opening issues.
"""
......@@ -149,4 +149,4 @@ def main(forks, feedback_csv, token_path, template, dry_run):
if __name__ == "__main__":
main()
send_feedback()
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment