Automate creating readme and tracking issue (#460)

Note at @fediverse/FEP-Editors - Once this is merged please check it does the right thing. - If you are ok with automating these things, I'll update the description of a facilitators role to just be checking and mergin - finally, can we rename the codeberg group, I just addressed, to FEP-Facilitators Final note: I already configured Woodpecker and the repository. This is to ensure that merging this pull request does not lead to a broken CI, second I might forget the steps ... - [x] Force myself to document the automated process as a FEP Reviewed-on: https://codeberg.org/fediverse/fep/pulls/460 Reviewed-by: silverpill <silverpill@noreply.codeberg.org> Co-authored-by: Helge <helge.krueger@gmail.com> Co-committed-by: Helge <helge.krueger@gmail.com>
2026-08-08 21:45:52 +00:00 · 2025-01-20 10:24:00 +00:00
parent c5d44a7cc5
commit 854ef33092
8 changed files with 281 additions and 54 deletions
@@ -0,0 +1,24 @@
+when:
+  - branch: [main]
+    event: push
+
+steps:
+  build:
+    image: python:3.11-alpine
+    commands:
+      - ./scripts/update_issues.py
+      - ./scripts/create_readme.py
+    environment:
+      CODEBERG_API_TOKEN:
+        from_secret: codeberg_api_token
+
+  push:
+    image: appleboy/drone-git-push
+    settings:
+      branch: main
+      remote: ${CI_REPO_CLONE_SSH_URL}
+      force: false
+      commit: true
+      commit_message: Automatic update [skip ci]
+      ssh_key:
+        from_secret: deploy_key
@@ -0,0 +1,107 @@
+---
+slug: "7d8c"
+authors: Helge <@helge@mymath.rocks>
+status: DRAFT
+dateReceived: 1970-01-01
+discussionsTo: https://socialhub.activitypub.rocks/t/fep-repo-rights/4797/3
+---
+# FEP-7d8c: Documentation: Automation of FEP
+
+## Summary
+
+This FEP discusses scripts and woodpecker configuration used to automate parts of the FEP process. The FEP process is described in [FEP-a4ed][a4ed].
+
+As [FEP-a4ed][a4ed], this is a living document, and should be updated as the FEP process evolves.
+
+## Vision
+
+Automation has the goal of reducing the burden on facilitators by:
+
+- allowing people without deep knowledge of git to become facilitators. Basically, the job will boil down to read a document and ensure it meets our standards, then click "merge".
+- performing as many checks as possible via automation
+
+Once one has simplified this job, one can seek broader adoption of the FEP process..
+
+### Todos
+
+This is a living document. So let's have a todo list
+
+- [ ] Check for broken links
+- [ ] Simplify creating FEPs. Does one really need git and a codeberg account?
+- [ ] Code documentation to enable more people to write tools, e.g. an automatic generation of which FEP is used where
+
+The first two items have been on my personal todo list for a long time, and might stay there ... so feel free to pick them up.
+
+## Automation
+
+Automation for the FEP repository uses the [codeberg woodpecker](https://ci.codeberg.org). There are two jobs:
+
+- test runs basic checks on the contained FEPs
+- readme updates the `README.md` file and creates the tracking issue
+
+Automation code is written in python and is contained in the [script folder](https://codeberg.org/fediverse/fep/src/branch/main/scripts).
+
+## Configuration variables
+
+To be able to do these configuration tasks, you need to be in the __admin__ group. See the
+first discussion [here](https://socialhub.activitypub.rocks/t/fep-repo-rights/4797/2).
+
+The configuration [.woodpecker/readme.yml](https://codeberg.org/fediverse/fep/src/branch/main/.woodpecker/woodpecker.yml) uses two secrets:
+
+- codeberg_api_token, a codeberg API token used to create the tracking issues.
+- deploy_key, an SSH private key used to push to the codeberg repository.
+
+Furthermore, the environment variable `CI_REPO_CLONE_SSH_URL` is used see [woodpecker documentation](https://woodpecker-ci.org/docs/usage/environment).
+
+We will now discuss how to obtain these secrets.
+
+### The api token
+
+API tokens can be generated on [codeberg](https://codeberg.org/user/settings) under user settings -> `applications` -> `select permissions` -> `issue`: `Read and Write` and giving it a name. They are hexstrings with 40 characters (if I counted correctly), e.g.
+
+```text
+33104dd6847e87ef3d6925effdfc852349233034
+```
+
+### Deploy key
+
+As already mentioned the deploy key is an ssh key. One can generate these on linux via
+
+```bash
+ssh-keygen -t ed25519 -f deploy_key
+```
+
+Do not specify a passphrase. Examples:
+
+``` bash
+$ cat deploy_key
+-----BEGIN OPENSSH PRIVATE KEY-----
+b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAAAMwAAAAtzc2gtZW
+QyNTUxOQAAACCQ5XqPCdvHPHsukhPS+RMDpMnOCCLW0csGGoFgijQRiwAAAJDpM7fx6TO3
+8QAAAAtzc2gtZWQyNTUxOQAAACCQ5XqPCdvHPHsukhPS+RMDpMnOCCLW0csGGoFgijQRiw
+AAAEB2ofBDBwUcZs5FHsaFMvyO95Qrvn8+PO3BchraucMRwJDleo8J28c8ey6SE9L5EwOk
+yc4IItbRywYagWCKNBGLAAAADWhlbGdlQHJhaW5ib3c=
+-----END OPENSSH PRIVATE KEY-----
+$ cat deploy_key.pub
+ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJDleo8J28c8ey6SE9L5EwOkyc4IItbRywYagWCKNBGL helge@rainbow
+```
+
+The public key `deploy_key.pub` should be added to the codeberg repo under `settings` -> `deploy keys` -> `add deploy key` and checking the `enable write access` checkbox.
+
+See [here](https://codeberg.org/fediverse/fep/settings/keys).
+
+### Adding the secrets to woodpecker
+
+One can add these secrets on [woodpecker](https://ci.codeberg.org/repos/12388), then selecting the settings wheel -> `secrets` -> `add secret`. The secrets only need to available for the `push` action (corresponding to a merged pull request).
+
+## References
+
+- pukkamustard, [FEP-a4ed: The Fediverse Enhancement Proposal Process][a4ed], 2020
+
+[a4ed]: https://codeberg.org/fediverse/fep/src/branch/main/fep/a4ed/fep-a4ed.md
+
+## Copyright
+
+CC0 1.0 Universal (CC0 1.0) Public Domain Dedication
+
+To the extent possible under law, the authors of this Fediverse Enhancement Proposal have waived all copyright and related or neighboring rights to this work.
@@ -1 +1 @@
-config.json
+config*.json
@@ -1,64 +1,16 @@
 #!/usr/bin/env python

 from argparse import ArgumentParser
-from datetime import date, timedelta
-from tools import FepFile
+from tools.issue import create_issue

 import json
-from urllib.request import Request, urlopen
-
-DRAFT_FEP_LABEL = 149758

 parser = ArgumentParser("Create tracking issue for FEP")
 parser.add_argument("fep", help="slug of the FEP")
 args = parser.parse_args()

-fep_file = FepFile(args.fep)
-
-if "trackingIssue" in fep_file.parsed_frontmatter:
-    print("File already has trackingIssue")
-    exit(1)
-
-title = f"[TRACKING] FEP-{args.fep}: {fep_file.title}"
-
-date_received = date.fromisoformat(fep_file.parsed_frontmatter["dateReceived"])
-
-date1 = date_received.isoformat()
-date2 = (date_received + timedelta(days=365)).isoformat()
-
-body = f"""
-The [proposal](https://codeberg.org/fediverse/fep/src/branch/main/{fep_file.filename}) has been received. Thank you!
-
-This issue tracks discussions and updates to the proposal during the `DRAFT` period.
-
-Please post links to relevant discussions as comment to this issue.
-
-`dateReceived`: {date1}
-
-If no further actions are taken, the proposal may be set by the facilitators to `WITHDRAWN` on {date2} (in 1 year).
-"""
-
 with open("scripts/config.json") as f:
    config = json.load(f)

-request = Request(
-    f"https://codeberg.org/api/v1/repos/{config['owner']}/{config['repo']}/issues"
-)
-request.add_header("Content-Type", "application/json; charset=utf-8")
-request_body = json.dumps(
-    {"title": title, "body": body, "labels": [DRAFT_FEP_LABEL]}
-).encode("utf-8")
-request.add_header("authorization", f"Bearer {config['token']}")
-request.add_header("Content-Length", len(request_body))
-request.data = request_body
-response = urlopen(request)
+create_issue(config['owner'], config['repo'], config['token'], args.fep)

-
-issue_url = json.loads(response.read())["html_url"]
-
-fep_file.frontmatter.append(f"trackingIssue: {issue_url}")
-if "discussionsTo" not in fep_file.parsed_frontmatter:
-    fep_file.frontmatter.append(f"discussionsTo: {issue_url}")
-fep_file.write()
-
-print(f"Issue url: {issue_url}")
@@ -0,0 +1,89 @@
+from datetime import timedelta, date
+from urllib.request import Request, urlopen
+
+import json
+
+from .fep_file import FepFile
+
+DRAFT_FEP_LABEL = 149758
+
+
+def create_body(filename: str, date_received: date):
+    date1 = date_received.isoformat()
+    date2 = (date_received + timedelta(days=365)).isoformat()
+
+    body = f"""
+The [proposal](https://codeberg.org/fediverse/fep/src/branch/main/{filename}) has been received. Thank you!
+
+This issue tracks discussions and updates to the proposal during the `DRAFT` period.
+
+Please post links to relevant discussions as comments to this issue.
+
+`dateReceived`: {date1}
+
+If no further actions are taken, the proposal may be set by the facilitators to `WITHDRAWN` on {date2} (in 1 year).
+    """
+
+    return body
+
+
+def create_codeberg_issue(owner, repo, token, title, body):
+    request = Request(f"https://codeberg.org/api/v1/repos/{owner}/{repo}/issues")
+    request.add_header("Content-Type", "application/json; charset=utf-8")
+    request_body = json.dumps(
+        {"title": title, "body": body, "labels": [DRAFT_FEP_LABEL]}
+    ).encode("utf-8")
+    request.add_header("authorization", f"Bearer {token}")
+    request.add_header("Content-Length", len(request_body))
+    request.data = request_body
+    response = urlopen(request)
+
+    issue_url = json.loads(response.read())["html_url"]
+
+    return issue_url
+
+
+def parse_and_update_date_received(input_date: str) -> date:
+    try:
+        parsed = date.fromisoformat(input_date)
+
+        if parsed < date.today() - timedelta(days=30):
+            return date.today()
+
+        return parsed
+    except Exception:
+        return date.today()
+
+
+def update_fep_file_with_date_received(fep_file: FepFile, date_received: date):
+    fep_file.frontmatter = [
+        x for x in fep_file.frontmatter if not x.startswith("dateReceived")
+    ]
+    fep_file.frontmatter.append(f"dateReceived: {date_received.isoformat()}")
+
+
+def create_issue(owner, repo, token, slug):
+    fep_file = FepFile(slug)
+
+    if "trackingIssue" in fep_file.parsed_frontmatter:
+        print("File already has trackingIssue")
+        exit(1)
+
+    title = f"[TRACKING] FEP-{slug}: {fep_file.title}"
+
+    date_received = parse_and_update_date_received(
+        fep_file.parsed_frontmatter["dateReceived"]
+    )
+    update_fep_file_with_date_received(fep_file, date_received)
+
+    body = create_body(fep_file.filename, date_received)
+
+    issue_url = create_codeberg_issue(owner, repo, token, title, body)
+
+    fep_file.frontmatter.append(f"trackingIssue: {issue_url}")
+    if "discussionsTo" not in fep_file.parsed_frontmatter:
+        fep_file.frontmatter.append(f"discussionsTo: {issue_url}")
+
+    fep_file.write()
+
+    print(f"Issue url: {issue_url} for {title}")
@@ -1,14 +1,14 @@
 import datetime
 import pytest

+from urllib.parse import urlparse
+
 from scripts.tools import get_fep_ids, FepFile, title_to_slug


@pytest.mark.parametrize("fep", get_fep_ids())
-def test_fep(fep):
+def test_fep_front_matter(fep):
    fep_file = FepFile(fep)
-
-    content = fep_file.content
    parsed_frontmatter = fep_file.parsed_frontmatter

    assert "status" in parsed_frontmatter
@@ -18,6 +18,12 @@ def test_fep(fep):
    assert "dateReceived" in parsed_frontmatter
    assert "discussionsTo" in parsed_frontmatter

+    discussions_to = parsed_frontmatter["discussionsTo"]
+
+    assert not urlparse(discussions_to).netloc.endswith(
+        ".example"
+    ), "Update discussionsTo to a valid URL for a discussion topic"
+
    if parsed_frontmatter["status"] == "FINAL":
        assert "dateFinalized" in parsed_frontmatter
    if parsed_frontmatter["status"] == "WITHDRAWN":
@@ -27,6 +33,13 @@ def test_fep(fep):
        if field_name in parsed_frontmatter:
            datetime.datetime.strptime(parsed_frontmatter[field_name], "%Y-%m-%d")

+
+@pytest.mark.parametrize("fep", get_fep_ids())
+def test_fep_content(fep):
+    fep_file = FepFile(fep)
+
+    content = fep_file.content
+
    assert "## Summary" in content
    assert "## Copyright" in content

@@ -0,0 +1,24 @@
+from datetime import date, timedelta
+
+from .issue import parse_and_update_date_received
+
+
+def test_parse_and_update_date_received():
+    today = date.today()
+    result = parse_and_update_date_received(today.isoformat())
+
+    assert result == today
+
+
+def test_parse_and_update_date_received_parse_error():
+    today = date.today()
+    result = parse_and_update_date_received("incorrect")
+
+    assert result == today
+
+
+def test_parse_and_update_date_received_outdated():
+    today = date.today()
+    result = parse_and_update_date_received((today - timedelta(days=60)).isoformat())
+
+    assert result == today
@@ -0,0 +1,18 @@
+#!/usr/bin/env python
+
+from tools import get_fep_ids, FepFile
+from tools.issue import create_issue
+import os
+
+
+owner = os.environ.get("CI_REPO_OWNER")
+repo = os.environ.get("CI_REPO_NAME")
+token = os.environ.get("CODEBERG_API_TOKEN")
+
+for slug in get_fep_ids():
+    fep_file = FepFile(slug)
+    tracking_issue = fep_file.parsed_frontmatter.get("trackingIssue")
+    if not tracking_issue:
+        print(slug, tracking_issue)
+
+        create_issue(owner, repo, token, slug)