flipdot-gschichtler =================== sterni November 2020 `flipdot-gschichtler` allows to display text on the OpenLab's flipdot panel in an user- and machine-friendly way without fear of interfering with others and without requiring direct connectivity to the panel. Components ---------- `flipdot-gschichtler` consists of the following three components: - `warteraum` manages a queue of text snippets which is accessible and manipulable via a public REST API. Users will directly or indirectly use it to send text to the flipdot panel. - `anzeigetafel` is a daemon which runs on a machine with direct connectivity to the flipdot panel. It monitors the queue of `warteraum` and displays new incoming text on the panel and deletes it from the queue afterwards. - `bahnhofshalle` is a web frontend which allows users to view and manipulate the queue of `warteraum` from their browser. API Documentation ----------------- Caveats ~~~~~~~ * The `v2` API may support `application/json` request bodies in the future. This is not yet the case. * Any endpoint may also return different status codes than the ones listed here in unusual cases. These would typically be codes like 500 or 502. * ids are not guaranteed to never be reassigned. However, ids only start to get reassigned after the queue has been fully emptied. API `v2` ~~~~~~~~ GET `/api/v2/queue` ^^^^^^^^^^^^^^^^^^^ |============================================= | Request Content-Type | none | Response Content-Type | `application/json` | Authentication | no |============================================= This endpoint returns the current queue in the following JSON format. `length` corresponds with the length of the queue array. `queue` is ordered by time of each item's addition, i. e. the first item is next up to be displayed. The supplied `id` may be used to delete the entries. ------------------------- { "queue": [ { "id": 0, "text": "hello" }, { "id": 1, "text": "world" } ], "length": 2 } ------------------------- |==================================== | HTTP Status | Meaning | 200 | Success |==================================== POST `/api/v2/queue/add` ^^^^^^^^^^^^^^^^^^^^^^^^ |============================================= | Request Content-Type | `application/x-www-form-urlencoded` | Response Content-Type | `application/json` | Authentication | no |============================================= This endpoints allows you to add text to the queue, it requires no authentication as it is a feature available to the public. There is also no rate limiting as of yet, but it may be implemented should it become necessary (hopefully not). The endpoint expects an urlencoded form as request body with the following field. |============================================= | `text` | text to be added to the queue |============================================= The response contains the queue entry consisting of its `text` and `id`: ---------------------- { "id": 3, "text": "hello" } ---------------------- |============================================= | HTTP Status | Meaning | 200 | Success, text added | 400 | Illegal method or malformed request | 415 | Request body too big or text field longer allowed (usually 512 bytes) | 503 | The queue is full, i. e. the max id has been reached |============================================= DELETE `/api/v2/queue/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^ |============================================= | Request Content-Type | `application/x-www-form-urlencoded` | Response Content-Type | none | Authentication | yes (see below) |============================================= This endpoint can be used to delete queue entries, e. g. after they have been displayed on the panel. The request should send a form with the following fields as request body in order to authenticate itself: |============================================= | `token` | API token of the application |============================================= |============================================= | HTTP Status | Meaning | 204 | Success, queue entry deleted | 400 | Illegal method or malformed request | 401 | Unauthorized API token | 404 | No queue entry with given `id` | 415 | Request body too big |============================================= API `v1` ~~~~~~~ GET `/api/v1/queue` ^^^^^^^^^^^^^^^^^^^ Same as `/api/v2/queue`, since v2 didn't introduce any changes. POST `/api/v1/queue/add` ^^^^^^^^^^^^^^^^^^^^^^^^ |============================================= | Request Content-Type | `application/x-www-form-urlencoded` | Response Content-Type | `text/html` | Authentication | no |============================================= This is the legacy endpoint to add text to the queue. It enabled interacting with it via a `
` in the old web app. The form sent as part of the request should have the following fields: |============================================= | `text` | text to be added to the queue |============================================= The response format has been changed since the previous implementation. I sincerly hope that nobody scraped the resulting page. |============================================= | HTTP Status | Meaning | 200 | Success, text added | 400 | Illegal method or malformed request | 415 | Request body too big or text field longer allowed (usually 512 bytes) |============================================= DELETE `/api/v1/queue/del/` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Same as `/api/v2/queue/`, `v2` only changed the endpoint URL. Bug Bounty ---------- https://github.com/sternenseemann[I]'ll gift anyone who finds serious bugs or exploits in `flipdot-gschichtler` a crate of _Mate_ (or another beverage if you don't like it) as a reward. Such issues would be remotely triggering crashes or segfaults, bypassing authentication, remote code execution etc. I won't bother making an exhaustive list or a precise list of criteria in the hopes it doesn't come back to haunt me… Contributing ------------ Help is welcome! Some things that remain to be done: * More “funny” bits for the web frontend (hint: see `const subjects` in `main.es6`) * Important: Documentation. Annoying sterni into doing it is also helping. * Make `warteraum` accept `application/json` request bodies for the `v2` API using http://www.catb.org/~esr/microjson/[microjson] (?). * Polish the web frontend, test across browsers * Refresh queue regularly in the web frontend * Write more tests * Full Unicode support by using Unifont on the flipdots * A completely new feature you thought of Building -------- warteraum ~~~~~~~~~ Requirements: * a C99 compiler * GNU make * https://www.tarsnap.com/scrypt.html[`libscrypt-kdf`] ------------------------ cd warteraum make ------------------------ bahnhofshalle ~~~~~~~~~~~~~ To build `bahnhofshalle` you need GNU make and https://esbuild.github.io/[esbuild]. One way to obtain both is to run `nix-shell -A bahnhofshalle` from the repository's root. To build, use the following commands: ------------------------ cd bahnhofshalle make firefox index.html # for local development where only js needs to be rebuilt make dist firefox dist/index.html # properly minified distribution ------------------------ Note that all requests are sent using a `same-origin` policy, so you need to configure a reverse proxy to serve the web frontend and API simuntaneously for testing. You may take inspiration from the `nginx` configuration in `nixos/flipdot-gschichtler.nix`. A note on vendoring ~~~~~~~~~~~~~~~~~~~ To ease the submodule hassle, dependencies that are inconvenient to handle via a package manager are vendored or added as a https://www.atlassian.com/git/tutorials/git-subtree[git subtree]. To avoid confusion these are located under `third_party` exclusively. Also be aware that different licensing terms may apply to code under this directory. Nix packages ~~~~~~~~~~~~ `default.nix` provides the following nix derivations which are ready to be installed: * `warteraum`: standard clang/glibc build of warteraum * `warteraum-static`: statically linked build of warteraum using gcc and musl (used for the systemd service so we can restrict file system access) * `bahnhofshalle` * `anzeigetafel` Configuration ~~~~~~~~~~~~~ `nixos/flipdot-gschichtler.nix` provides a NixOS module which defines `services.flipdot-gschichtler` to conveniently set up the server side with `warteraum` and `bahnhofshalle` behind a nginx reverse proxy. A minimal `configuration.nix` utilizing it could look like this: --------------- { pkgs, ... }: { imports = [ /path/to/flipdot-gschichtler/nixos/flipdot-gschichtler.nix ]; services.flipdot-gschichtler = { enable = true; virtualHost = "flipdot.openlab-augsburg.de"; tokensFile = "/var/secrets/flipdot-gschichtler/tokens"; saltFile = "/var/secrets/flipdot-gschichtler/salt"; # if you want to change the derivations to use # packages = { # warteraum = …; # bahnhofshalle = …; # }; }; services.nginx.enable = true; security.acme = { .... }; } --------------- warteraum ^^^^^^^^^ `warteraum` is configured via environment variables (which the NixOS module utilizes): * `WARTERAUM_SALT_FILE`: A file containing random data to use as salt * `WARTERAUM_TOKENS_FILE`: API tokens hashed using `scrypt` To generate the tokens file, `warteraum` ships a utility tool. Setting up auth works like this: ------------------- $ head -c 512 /dev/urandom > $WARTERAUM_SALT_FILE $ hashtoken $WARTERAUM_SALT_FILE token1 >> $WARTERAUM_TOKENS_FILE $ hashtoken $WARTERAUM_SALT_FILE token2 >> $WARTERAUM_TOKENS_FILE ------------------- Now `warteraum` would accept “token1” and “token2” when authenticating. Note that `hashtoken` only supports appending tokens in a convenient fashion at the moment. Removing tokens is quite cumbersome and only possible with a knowledge of `warteraum` internals. Changelog --------- 2.1.0 (WIP) ~~~~~~~~~~~ * `warteraum` ** Limit size of request bodies to prevent DoS attacks ** Trim whitespace on input text ** Instead of compiling in salt and tokens, read them from the files specified via the `WARTERAUM_SALT_FILE` and `WARTERAUM_TOKENS_FILE` environment variables. * NixOS module ** Reflect change to `warteraum` by using `saltFile` and `tokensFile` respectively over the previous `salt` and `tokens`. ** Fix sandboxing in `nixos/flipdot-gschichtler.nix`: Now only the secret files and the nix store will be readable to the `warteraum` process. ** Allow changing `bahnhofshalle` and `warteraum` derivation to use via `packages`. ** Don't require `flipdot-gschichtler` to be passed as module argument, instead import directly from file system (unless a non-default derivation is configured). * `bahnhofshalle` ** Switch to `esbuild`, requiring ES6 support in the browser as a result. 2.0.0 ~~~~~ https://github.com/openlab-aux/flipdot-gschichtler/tree/2.0.0[Browse code] * Replace `admin` and `web` frontends with pure EcmaScript frontend `bahnhofshalle` * Replace `web` API implementation with `warteraum` * Rename `flipper` to `anzeigetafel`, port to Python 3 * API: ** Move endpoints from `/` to `/api/v1/` ** `/api/v1/queue/add` HTML response changes, since no longer used by the frontend (except when no JavaScript is available) ** Add cleaned up version of the API as `/api/v2`. This one is used by `bahnhofshalle` and `anzeigetafel` and should be utilized by clients going forward. * Deployment: ** Implement API/Frontend deployment as a NixOS service 1.0.0 ~~~~~ https://github.com/openlab-aux/flipdot-gschichtler/tree/1.0.0[Browse code] Initial Version: Flask and Python 2.7 based web interface.