# CircleCI :circleci: in daily life: how I improved Matrix stickerpicker

[CircleCI](https://circleci.com) is a great tool that everyone seems to skip and I don't know exactly why.
I'm gonna quickly show you how I've automated adding stickers to the [Matrix](https://matrix.org) messenger.

## What is CircleCI?
`CircleCI` is a cloud platform that can run automated jobs when different events occur. The most common use-case is
running tests in the cloud when you push a commit to the `GitHub` repo. But the possibilities are endless,
you can literally automate everything with it - I even used it to compile `.exe` file **in the cloud** and include
a **zipped binary** into the release package. Don't ask me why I had to compile `.exe`, better ask why the guys from my
University **made a deal with** [**the Devil**](https://microsoft.com).

## Matrix
[Matrix](https://matrix.org) is a messaging protocol which I'm currently trying to replace **Telegram** with.
It's in itself a separate topic for discussion, but what's important for today is that **Matrix** lacks such
a huge collection of stickers as there is in **Telegram**. I'm a big fan of **Wojak** stickers and many other ones,
so I had to do something about it.

![stickerpicker](/public/stickerpicker.png)

# Matrix Stickerpicker
Stickers work in a slightly weird way in **Matrix**.  Basically you have to deploy your own sticker "server"
(*don't worry, it's just a static site*) where you can add your stickers - it will be used as a widget inside your client.

There's an amazing tool for that - [stickerpicker](https://github.com/maunium/stickerpicker). It's written in `Python` and the author
of course took care of the import feature - you can provide a link to a **Telegram** sticker pack - it will re-upload it
to your `Matrix` server and then generate a web UI using GitHub Pages that you can integrate as a widget in your client. Your friends
can use it too.

The problem is that each time you want to add new sticker pack to your collection you have to run a `Python` script.
I'm lazy and I don't wanna do that - so let's automate it!

## Automation
Main idea of my solution is to have a single plaintext file `telegram-packs.txt` which will contain links to all
sticker packs I import from **Telegram**. Then every time I update it `CircleCI` will run a job that will run this `Python` script for me. Deadly simple.

This approach ships with few bonuses by default:
 - You can edit the file from anywhere (even from phone :laughing:)
 - If someone from your friends uses your stickers collection and wants to add a new pack, they simply have to open a PR which adds the pack URL to `telegram-packs.txt`.

There are two problems of running this `Python` script automatically:
 - You have to log into your `Matrix` account
 - You have to log into your `Telegram` account

## Config
Here's my fork of the repo: https://github.com/eug-vs/stickerpicker

`CircleCI` is fully controlled by a config file `.circleci/config.yml`. It uses `YAML` which is really picky about indentation, so be careful with that.

Ok so let's start outlining our job, we'll be using default CircleCI `Python` image:
```YAML
defaults: &defaults
  working_directory: ~/repo
  docker:
    - image: cimg/python:3.9.5

jobs:
  upload_stickers:
    <<: *defaults
    steps:
```

The `defaults` part might look scary, but you could just move everything from defaults directly under `upload_stickers` job.
Defaults are useful when you have multiple jobs, and I planned to have 2 initially. Let's now move to the `steps`:
```YAML
- checkout

- run:
    name: Install python dependencies
    command: pip3 install .
```
Obviously we checkout our code first, and then install all the dependencies for our `Python` script.

Now let's solve the first problem we found. Instead of logging into `Matrix` manually, you can supply a `config.json` file with your credentials. So I prepared a template for this file (`config.template.json`):
```JSON
{"homeserver": "$HOMESERVER", "user_id": "$USER_ID", "access_token": "$ACCESS_TOKEN"}
```
You might think how am I gonna put the actual variables in here? Well, there's a tool for that called `envsubst`. It does exactly what you think - substitutes environment variable names with the actual values.
We will put the actual values of `$HOMESERVER`, `$USER_ID` and `$ACCESS_TOKEN` in our `CircleCI` environment (as secrets), and the template can be safely pushed to the GitHub repo.

There's a little problem - `envsubst` is not installed in the image that we use (`cimg/python:3.9.5`). Solution to it is [orbs](https://circleci.com/developer/orbs) - basically plugins for your CircleCI needs.
Let's add `envsubst` orb at the top of our config:
```YAML
orbs:
  envsubst: sawadashota/envsubst@1.1.0
```

Now back to the steps. Let's install the tool and use it to generate our `config.json`:
```YAML
- envsubst/install

- run:
  name: Create config.json from template
  command: envsubst < config.template.json > config.json
```

Ok so we can now login into `Matrix`, but what about `Telegram`? The script uses `Telethon` package under the hood which works with a `sticker-import.session` file. This is basically an `SQL` database, and I don't really
wanna mess around with creating it from scratch (*although I could!*). Instead, I will just encrypt it with `GPG` and push to the repo. That's just me being lazy, but you know, that is what automation is about!
Of course, someone could try to decrypt my `Telethon` session since it's now publicly available, but I can just revoke it at any time. I will probably improve this at some point, but now it's not a big deal for me.

Our next steps in config will be to decrypt the session and finally pipe our `telegram-packs.txt` to the script using `xargs`:
```YAML
- run:
    name: Decrypt telethon session
    command: gpg --batch --passphrase $PASSPHRASE --decrypt sticker-import.session.gpg > sticker-import.session

- run:
    name: Reupload and install stickers
    command: cat telegram-packs.txt | xargs sticker-import
```

At this point, the stickers are uploaded to the server, but the UI is updated via it's own `.json` file which contains information about uploaded stickers. It's already updated by the script, we just have to push it to our repo.
For that I've created a new SSH key and added it to `CircleCI` secrets so that it can push commits to my repository. So the final steps would be:
```YAML
- add_ssh_keys

- run:
    name: Add github to known_hosts
    command: |
      mkdir -p ~/.ssh
      ssh-keyscan github.com >> ~/.ssh/known_hosts
- run:
    name: Commit stickerpack JSON's
    command: |
      git config --global user.email "eug-vs@keemail.me"
      git config --global user.name "eug-vs"
      git add .
      git commit -m "feat: add stickerpacks with CircleCI [ci [skip](skip.md)]"
      git push -u
```

The `[ci skip]` flag tells Circle to avoid running on this commit, otherwise it would have to run once again since we pushed a new commit with it.

One thing that I didn't mention is that I did not actually
need to check the **new** packs in the `telegram-packs.txt` - the script is actually smart enough to skip already existing ones. Otherwise I would have to use `git diff` to find only the new ones.

And that's it! Now every time I add new URLs to the `telegram-packs.txt`, they will automatically appear in my widget within a minute or so. Enjoy your stickers!

![wojak-brain-chair](/public/wojak-brainchair.png)

You can find full version of the config here: https://github.com/eug-vs/stickerpicker/blob/master/.circleci/config.yml