README.md 7.86 KB
Newer Older
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
1
# Advanced teaching automation :robot: :mortar_board:
2

3
Leverage Renku and Gitlab to automate classroom management.
4

5
## Target audience
6

7
This repository is intended for technically inclined teachers who use Renku and/or Gitlab in their class and wish to automate repetitive tasks related to student and project management.
8

9
## Purpose
10

11
We showcase the potential of Renku + Gitlab through a few scripts covering common use-cases.
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
12
These scripts can be used as-provided, but are also meant to be extended and adapted by teachers for their specific needs.
13

14
15
16
17
18
## Setup

The repository is setup as a local python package.
You can install it with: `pip install -e .`

19
20
Most script below require a Gitlab API token. By default, you will be prompted for your token upon running the scripts, but you can also provide it as a plain text file with the `--token` option.

21
## Content
22

23
The following tasks are automated:
24

25
* [:email: Inviting students to the class group](#email-inviting-students-to-the-class-group)
Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
26
27
28
29
30
* [:lock: Creating private student groups](#lock-creating-private-student-groups)
* [🗂️ Gathering all forks of a project](#%EF%B8%8F-gathering-all-forks-of-a-project)
* [💾 Cloning all forks of a project](#-cloning-all-forks-of-a-project)
* [:loudspeaker: Sending feedback](#loudspeaker-sending-feedback)

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
31

32
---
33
### :email: Inviting students to the class group
34

35
36
37
38
39
40
Creating a single group for the class provides a centralizd place to store all materials, student repositories and exercises. In some cases, one may also want to make this group private.

The manual process of inviting the whole class to the group can be automated using Gitlab's invitation API. Given a list of student email addresses (e.g. exported from moodle) and the URL to a an existing gitlab group, we provide a script to automatically send an invitation email to each student.

<details>
<summary> <b>Read more...</b> </summary>
41

42
script: [teach\_utils/invite\_students.py](teach_utils/invite_students.py)
43

44
**usage**:
45

46
47
Invite students to `class-group`:
`python invite_students.py emails.txt https://gitlab-instance.com/class-group`
48

49
50
51
</details>

---
52

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
53
### :lock: Creating private student groups
54

55
56
57
When students work on graded assignments, they usually have to work in private groups. The groups must be accessible to the teachers in order to grade the assignments.

One way to achieve this is to have the teacher create all private student groups (The teacher will therefore own the groups) and then invite students to their respective groups. We provide a script to automate the group creation and invitation process. It creates one private group for each student based on their information in a CSV file exported from Moodle.
58

59
60
<details>
<summary> <b>Read more...</b> </summary>
61

62
script: [teach\_utils/moodle\_to\_student\_groups.py](teach_utils/moodle_to_student_groups.py)
63

64
65
66
67
68
69
70
71
**usage**:

Create student (sub)groups inside `class-group`:
`python moodle_to_student_groups.py students_moodle.csv https://gitlab-instance.com/class-group`

</details>

---
72

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
73
### 🗂️ Gathering all forks of a project
74

75
Homework assignment can be provided by the teachers in the form of a repository containing instructions and depencencies. Students can then fork this project into their own private groups (e.g. created with the aforementioned script).
76

77
78
79
80
The teacher can then use the Gitlab API to keep track of student forks. We provide a script to gather metadata about all student-group forks of a given project, with the option to retrieve the last commit before a deadline. That script will output the metadata as a JSON structure containing the following fields for each fork:

<details>
<summary> <b>Read more...</b> </summary>
81

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
82
83
84
85
86
87
88
89
90
* `id`: The gitlab project ID of the fork
* `url`: The https URL to reach the fork
* `group`: The student group name
* `members`: The members of the group. For each member the following fields are provided:
  + `username`: The gitlab username
  + `name`: The full name (First_name Last_name)
  + `email`: The email address of the student
* `commit`: The hash of the last commit before the deadline, used for grading the assignment.
* `autostart_url`: A URL to directly start a Renku session at the last commit before the deadline.
91

92
93
94
> Note: the script assumes Gitlab commit timestamps and the deadline are in the UTC timezone.

script: [teach\_utils/collect\_forks.py](teach_utils/collect_forks.py)
95

96
**usage**:
97

98
Collect all group-owned forks of upstream-project, and write the JSON list to `forks.json`. Commits and autostart URLs will point to the last commit before 23h59 on March 15, 2022:
99
100
```sh
python collect_forks.py \
101
  --deadline "2022-03-15T23:59" \
102
103
104
105
  https://gitlab-instance.com/namespace/upstream-project \
  > forks.json
```

106
107
108
109
110
111
112
113
114
115
116
**sample output:***

```
[
    {
        "id": 379,
        "url": "https://renkulab.io/gitlab/demo-group/assignment.git",
        "visibility": "private",
        "group": "demo-group",
        "members": [
            {
117
118
119
                "username": "smcstudent",
                "name": "Student McStudent",
                "email": student.mcstudent@email.here
120
121
122
123
124
125
126
127
128
129
130
131
132
            }
        ],
        "commit": "0175894cf61fe820fdb84e5c52fc7ee0259a3c71",
        "autostart_url": "https://renkulab.io/projects/demo-group/assignment/sessions/new?autostart=1&commit=0175894cf61fe820fdb84e5c52fc7ee0259a3c71&branch=master"
    },
    ...
]
```

</details>

---

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
133
### 💾 Cloning all forks of a project
134

135
136
137
138
139
140
141
In some cases, the student projects may have to be cloned locally by the teacher. We provide a shell script to automate this process. It reads the JSON output from `collect_forks.py` and clones all the forks at the deadline commit into a target directory.

<details>
<summary> <b>Read more...</b> </summary>


script: [teach\_utils/clone\_forks\_from\_json.sh](teach_utils/clone_forks_from_json.sh)
142

143
**usage**:
144

145
146
147
148
149
150
151
152
153
154
155
156
157
158
Clone all forks listed in `forks.json` into `clone_dir`:

`./clone_forks_from_json.sh forks.json clone_dir`

Or reading directly from stdin:

```
python ./collect_forks.py https://gitlab-instance.com/namespace/upstream-project \
  | ./clone_forks_from_json.sh clone_dir
```

</details>

---
159

Cyril Matthey-Doret's avatar
Cyril Matthey-Doret committed
160
### :loudspeaker: Sending feedback
161

162
163
164
165
After grading student assignments, the teacher may want to send student groups their grades, as well as positive or critical comments on their solution. This can be automated as well, using the Gitlab issues API. We provide a script to read a csv file containing grades and comments for each student group and open issues in the corresponding repositories automatically.

<details>
<summary> <b>Read more...</b> </summary>
166

167
168
169
script: [teach\_utils/send\_feedback.py](teach_utils/send_feedback.py)

**usage**:
170

171
By default, this script opens issues in all forks listed in `forks.json`. For each fork, fields {grades}, {comments} in the issue description are filled using column values in a CSV file provided with `--feedback`. The issue description uses a default template defined in the script:
172

173
`python ./send_feedback.py --feedback grades.csv forks.json`
174

175
176
177
178
179
180
181
182
183
where grades.csv is:

```
id,grade,comments
379,6,great use of list-comprehension!
424,5.5,Missing imports
```

A custom template can also be provided, and fields will be replaced by values from columns with corresponding names in `--feedback`. One could also use this script to warn all groups that their fork is public for example:
184
185
186
187

```
echo ":warning: Your fork is public, please make it private!" > visibility.md

188
./collect_forks.py https://renkulab.io/gitlab/class/homework \
189
  | jq '.[] | select(.visibility == "public")' \
190
  | ./send_feedback.py --template visibility.md
191
192

```
193
> Note: `jq` is a tool to process JSON data. It can be used to reformat or filter the output of `collect_forks.py` in many ways. A number of useful one liners are [shown here](./docs/oneliners.md).
194
195
196
197
198
199
200
201
202
203

</details>

## Contributing

Contributions related to common use-cases, or improvements are welcome.
Testing is done with `pytest --doctest-modules` and can be run with `make test`.

## License

204
These scripts are provided under the [MIT license](./LICENSE).