README.md 9.73 KB
Newer Older
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
1
# Advanced teaching automation :robot: :mortar_board:
2

3
Leverage Renku and Gitlab to automate classroom management.
4

5
## Target audience
6

7
This repository is intended for technically inclined teachers who use Renku and/or Gitlab in their class and wish to automate repetitive tasks related to student and project management.
8

9
## Purpose
10

11
We showcase the potential of Renku + Gitlab through a few scripts covering common use-cases.
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
12
These scripts can be used as-provided, but are also meant to be extended and adapted by teachers for their specific needs.
13

14
15
## Setup

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
16
17
18
19
20
21
22
23
The repository is meant to be setup as a local python package.
You can clone and install it in editable mode using:

```sh
git clone https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation
cd advanced-teaching-automation
pip install -e .
```
24

25
26
27
28
29
30
31
32
33
34
Installing the package this way allows you to modify / extend the code to match your personal needs.

If you don't need to modify it, you can also install it directly from the URL without cloning:

```
pip install git+https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation
```

## Usage

35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
Once the package is installed, the command `teach-auto` will be available globally on your system. Each functionality is available as a subcommand. For example `teach-auto collect-forks`. Each command stands in its own python script which can also be executed directly, e.g. `python teach_auto/collect_forks.py`. Each command provides a `--help` flag to print the list of available options and usage information:

```sh
➜ teach-auto --help
Usage: teach-auto [OPTIONS] COMMAND [ARGS]...

  teach-auto: Classroom automation utilities for Gitlab + Renku.

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  clone-forks      Clone all forks from input JSON into target directory.
  collect-forks    Gather metadata about all forks of input project into...
  create-groups    Given a parent-group URL and a CSV file of students...
  invite-students  Send invitations to join input gitlab group to all...
  send-feedback    Send feedback to repositories in input JSON by opening...
  ```
54
55

Most functionalities below require a Gitlab API token. By default, you will be prompted for your token upon running the scripts, but you can also provide it as a plain text file with the `--token` option.
56

57
## Content
58

59
The following tasks are automated:
60

61
62
63
64
65
* [:email: Inviting students to the class group](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation#email-inviting-students-to-the-class-group)
* [:lock: Creating private student groups](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation#lock-creating-private-student-groups)
* [🗂️ Gathering all forks of a project](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation#%EF%B8%8F-gathering-all-forks-of-a-project)
* [💾 Cloning all forks of a project](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation#-cloning-all-forks-of-a-project)
* [:loudspeaker: Sending feedback](https://renkulab.io/gitlab/learn-renku/teaching-on-renku/advanced-teaching-automation#loudspeaker-sending-feedback)
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
66

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
67

68
---
69
### :email: Inviting students to the class group
70

71
72
73
74
75
76
Creating a single group for the class provides a centralizd place to store all materials, student repositories and exercises. In some cases, one may also want to make this group private.

The manual process of inviting the whole class to the group can be automated using Gitlab's invitation API. Given a list of student email addresses (e.g. exported from moodle) and the URL to a an existing gitlab group, we provide a script to automatically send an invitation email to each student.

<details>
<summary> <b>Read more...</b> </summary>
77

78
script: [teach\_auto/invite\_students.py](teach_auto/invite_students.py)
79

80
**usage**:
81

82
Invite students to `class-group`:
83

84
`teach-auto invite-students emails.txt https://gitlab-instance.com/class-group`
85

86
87
88
</details>

---
89

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
90
### :lock: Creating private student groups
91

92
93
94
When students work on graded assignments, they usually have to work in private groups. The groups must be accessible to the teachers in order to grade the assignments.

One way to achieve this is to have the teacher create all private student groups (The teacher will therefore own the groups) and then invite students to their respective groups. We provide a script to automate the group creation and invitation process. It creates one private group for each student based on their information in a CSV file exported from Moodle.
95

96
97
<details>
<summary> <b>Read more...</b> </summary>
98

99
script: [teach\_auto/create\_groups.py](teach_auto/create_groups.py)
100

101
102
103
**usage**:

Create student (sub)groups inside `class-group`:
104

105
`create-groups students_moodle.csv https://gitlab-instance.com/class-group`
106
107
108
109

</details>

---
110

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
111
### 🗂️ Gathering all forks of a project
112

113
Homework assignment can be provided by the teachers in the form of a repository containing instructions and depencencies. Students can then fork this project into their own private groups (e.g. created with the aforementioned script).
114

115
116
117
118
The teacher can then use the Gitlab API to keep track of student forks. We provide a script to gather metadata about all student-group forks of a given project, with the option to retrieve the last commit before a deadline. That script will output the metadata as a JSON structure containing the following fields for each fork:

<details>
<summary> <b>Read more...</b> </summary>
119

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
120
121
122
123
124
125
126
127
128
* `id`: The gitlab project ID of the fork
* `url`: The https URL to reach the fork
* `group`: The student group name
* `members`: The members of the group. For each member the following fields are provided:
  + `username`: The gitlab username
  + `name`: The full name (First_name Last_name)
  + `email`: The email address of the student
* `commit`: The hash of the last commit before the deadline, used for grading the assignment.
* `autostart_url`: A URL to directly start a Renku session at the last commit before the deadline.
129

130
131
> Note: the script assumes Gitlab commit timestamps and the deadline are in the UTC timezone.

132
script: [teach\_auto/collect\_forks.py](teach_auto/collect_forks.py)
133

134
**usage**:
135

136
Collect all group-owned forks of upstream-project, and write the JSON list to `forks.json`. Commits and autostart URLs will point to the last commit before 23h59 on March 15, 2022:
137

138
```sh
139
teach-auto collect-forks \
140
  --deadline "2022-03-15T23:59" \
141
  --group-only \
142
143
144
145
  https://gitlab-instance.com/namespace/upstream-project \
  > forks.json
```

146
147
148
149
150
151
152
153
154
155
156
**sample output:***

```
[
    {
        "id": 379,
        "url": "https://renkulab.io/gitlab/demo-group/assignment.git",
        "visibility": "private",
        "group": "demo-group",
        "members": [
            {
157
158
159
                "username": "smcstudent",
                "name": "Student McStudent",
                "email": student.mcstudent@email.here
160
161
162
163
164
165
166
167
168
169
170
171
172
            }
        ],
        "commit": "0175894cf61fe820fdb84e5c52fc7ee0259a3c71",
        "autostart_url": "https://renkulab.io/projects/demo-group/assignment/sessions/new?autostart=1&commit=0175894cf61fe820fdb84e5c52fc7ee0259a3c71&branch=master"
    },
    ...
]
```

</details>

---

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
173
### 💾 Cloning all forks of a project
174

175
176
177
178
179
180
In some cases, the student projects may have to be cloned locally by the teacher. We provide a shell script to automate this process. It reads the JSON output from `collect_forks.py` and clones all the forks at the deadline commit into a target directory.

<details>
<summary> <b>Read more...</b> </summary>


181
script: [teach\_auto/clone\_forks.py](teach_auto/clone_forks.py)
182

183
**usage**:
184

185
186
Clone all forks listed in `forks.json` into `clone_dir`:

187
`teach-auto clone-forks forks.json clone_dir`
188
189
190
191

Or reading directly from stdin:

```
192
193
teach-auto collect-forks https://gitlab-instance.com/namespace/upstream-project \
  | teach-auto clone_forks clone_dir
194
195
196
197
198
```

</details>

---
199

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
200
### :loudspeaker: Sending feedback
201

202
203
204
205
After grading student assignments, the teacher may want to send student groups their grades, as well as positive or critical comments on their solution. This can be automated as well, using the Gitlab issues API. We provide a script to read a csv file containing grades and comments for each student group and open issues in the corresponding repositories automatically.

<details>
<summary> <b>Read more...</b> </summary>
206

207
script: [teach\_auto/send\_feedback.py](teach_auto/send_feedback.py)
208
209

**usage**:
210

211
By default, this script opens issues in all forks listed in `forks.json`. For each fork, fields {grades}, {comments} in the issue description are filled using column values in a CSV file provided with `--feedback`. The issue description uses a default template defined in the script:
212

213
`teach-auto send-feedback --feedback grades.csv forks.json`
214

215
216
217
218
219
220
221
222
223
where grades.csv is:

```
id,grade,comments
379,6,great use of list-comprehension!
424,5.5,Missing imports
```

A custom template can also be provided, and fields will be replaced by values from columns with corresponding names in `--feedback`. One could also use this script to warn all groups that their fork is public for example:
224
225
226
227

```
echo ":warning: Your fork is public, please make it private!" > visibility.md

228
teach-auto collect-forks https://renkulab.io/gitlab/class/homework \
229
  | jq '.[] | select(.visibility == "public")' \
230
  | teach-auto send-feedback --template visibility.md
231
232

```
233
> Note: `jq` is a tool to process JSON data. It can be used to reformat or filter the output of `collect_forks.py` in many ways. A number of useful one liners are [shown here](./docs/oneliners.md).
234
235
236
237
238
239
240
241
242
243

</details>

## Contributing

Contributions related to common use-cases, or improvements are welcome.
Testing is done with `pytest --doctest-modules` and can be run with `make test`.

## License

244
These scripts are provided under the [MIT license](./LICENSE).