1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
---
layout: ../../layouts/BlogLayout.astro
---
# CircleCI :circleci: in daily life: how I improved Matrix stickerpicker
[CircleCI](https://circleci.com) is a great tool that everyone seems to skip and I don't know exactly why.
I'm gonna quickly show you how I've automated adding stickers to the [Matrix](https://matrix.org) messenger.
## What is CircleCI?
`CircleCI` is a cloud platform that can run automated jobs when different events occur. The most common use-case is
running tests in the cloud when you push a commit to the `GitHub` repo. But the possibilities are endless,
you can literally automate everything with it - I even used it to compile `.exe` file **in the cloud** and include
a **zipped binary** into the release package. Don't ask me why I had to compile `.exe`, better ask why the guys from my
University **made a deal with** [**the Devil**](https://microsoft.com).
## Matrix
[Matrix](https://matrix.org) is a messaging protocol which I'm currently trying to replace **Telegram** with.
It's in itself a separate topic for discussion, but what's important for today is that **Matrix** lacks such
a huge collection of stickers as there is in **Telegram**. I'm a big fan of **Wojak** stickers and many other ones,
so I had to do something about it.
# Matrix Stickerpicker
Stickers work in a slightly weird way in **Matrix**. Basically you have to deploy your own sticker "server"
(*don't worry, it's just a static site*) where you can add your stickers - it will be used as a widget inside your client.
There's an amazing tool for that - [stickerpicker](https://github.com/maunium/stickerpicker). It's written in `Python` and the author
of course took care of the import feature - you can provide a link to a **Telegram** sticker pack - it will re-upload it
to your `Matrix` server and then generate a web UI using GitHub Pages that you can integrate as a widget in your client. Your friends
can use it too.
The problem is that each time you want to add new sticker pack to your collection you have to run a `Python` script.
I'm lazy and I don't wanna do that - so let's automate it!
## Automation
Main idea of my solution is to have a single plaintext file `telegram-packs.txt` which will contain links to all
sticker packs I import from **Telegram**. Then every time I update it `CircleCI` will run a job that will run this `Python` script for me. Deadly simple.
This approach ships with few bonuses by default:
- You can edit the file from anywhere (even from phone :laughing:)
- If someone from your friends uses your stickers collection and wants to add a new pack, they simply have to open a PR which adds the pack URL to `telegram-packs.txt`.
There are two problems of running this `Python` script automatically:
- You have to log into your `Matrix` account
- You have to log into your `Telegram` account
## Config
Here's my fork of the repo: https://github.com/eug-vs/stickerpicker
`CircleCI` is fully controlled by a config file `.circleci/config.yml`. It uses `YAML` which is really picky about indentation, so be careful with that.
Ok so let's start outlining our job, we'll be using default CircleCI `Python` image:
```YAML
defaults: &defaults
working_directory: ~/repo
docker:
- image: cimg/python:3.9.5
jobs:
upload_stickers:
<<: *defaults
steps:
```
The `defaults` part might look scary, but you could just move everything from defaults directly under `upload_stickers` job.
Defaults are useful when you have multiple jobs, and I planned to have 2 initially. Let's now move to the `steps`:
```YAML
- checkout
- run:
name: Install python dependencies
command: pip3 install .
```
Obviously we checkout our code first, and then install all the dependencies for our `Python` script.
Now let's solve the first problem we found. Instead of logging into `Matrix` manually, you can supply a `config.json` file with your credentials. So I prepared a template for this file (`config.template.json`):
```JSON
{"homeserver": "$HOMESERVER", "user_id": "$USER_ID", "access_token": "$ACCESS_TOKEN"}
```
You might think how am I gonna put the actual variables in here? Well, there's a tool for that called `envsubst`. It does exactly what you think - substitutes environment variable names with the actual values.
We will put the actual values of `$HOMESERVER`, `$USER_ID` and `$ACCESS_TOKEN` in our `CircleCI` environment (as secrets), and the template can be safely pushed to the GitHub repo.
There's a little problem - `envsubst` is not installed in the image that we use (`cimg/python:3.9.5`). Solution to it is [orbs](https://circleci.com/developer/orbs) - basically plugins for your CircleCI needs.
Let's add `envsubst` orb at the top of our config:
```YAML
orbs:
envsubst: sawadashota/envsubst@1.1.0
```
Now back to the steps. Let's install the tool and use it to generate our `config.json`:
```YAML
- envsubst/install
- run:
name: Create config.json from template
command: envsubst < config.template.json > config.json
```
Ok so we can now login into `Matrix`, but what about `Telegram`? The script uses `Telethon` package under the hood which works with a `sticker-import.session` file. This is basically an `SQL` database, and I don't really
wanna mess around with creating it from scratch (*although I could!*). Instead, I will just encrypt it with `GPG` and push to the repo. That's just me being lazy, but you know, that is what automation is about!
Of course, someone could try to decrypt my `Telethon` session since it's now publicly available, but I can just revoke it at any time. I will probably improve this at some point, but now it's not a big deal for me.
Our next steps in config will be to decrypt the session and finally pipe our `telegram-packs.txt` to the script using `xargs`:
```YAML
- run:
name: Decrypt telethon session
command: gpg --batch --passphrase $PASSPHRASE --decrypt sticker-import.session.gpg > sticker-import.session
- run:
name: Reupload and install stickers
command: cat telegram-packs.txt | xargs sticker-import
```
At this point, the stickers are uploaded to the server, but the UI is updated via it's own `.json` file which contains information about uploaded stickers. It's already updated by the script, we just have to push it to our repo.
For that I've created a new SSH key and added it to `CircleCI` secrets so that it can push commits to my repository. So the final steps would be:
```YAML
- add_ssh_keys
- run:
name: Add github to known_hosts
command: |
mkdir -p ~/.ssh
ssh-keyscan github.com >> ~/.ssh/known_hosts
- run:
name: Commit stickerpack JSON's
command: |
git config --global user.email "eug-vs@keemail.me"
git config --global user.name "eug-vs"
git add .
git commit -m "feat: add stickerpacks with CircleCI [ci [skip](skip)]"
git push -u
```
The `[ci skip]` flag tells Circle to avoid running on this commit, otherwise it would have to run once again since we pushed a new commit with it.
One thing that I didn't mention is that I did not actually
need to check the **new** packs in the `telegram-packs.txt` - the script is actually smart enough to skip already existing ones. Otherwise I would have to use `git diff` to find only the new ones.
And that's it! Now every time I add new URLs to the `telegram-packs.txt`, they will automatically appear in my widget within a minute or so. Enjoy your stickers!
You can find full version of the config here: https://github.com/eug-vs/stickerpicker/blob/master/.circleci/config.yml
|
