Shell Scripting on mm-dev

End Results and Bonus

Thu, 06 Mar 2025 12:26:15 +0000

Good

I love that it’s silent
Apple Silicon is as good as I’d hoped — the bang-per-watt is great for my circumstances with solar power etc
The annoyances with not knowing when it’s turned on are largely alleviated with my scripts, as long as I’m using it on my LAN (which, realistically I always am)

Bad

MacOS, when I do have to use it (sorry Mac fans… I do at least prefer it to Windows)
The battery does not hold power at all well when turned off (if I turn it off — yes off, not asleep — with a full battery, a few days later it’s completely dead and needs to be charged before I can turn it on)

I’m never going to use this as my main machine, and that was never my plan. I’m into minimal Linux with invisible tiling window managers like DWM, and dragging windows around, being forced to watch animations etc won’t cut it for my preferences.

I do know about Asahi Linux but that makes power usage much less special and I’m not sure how it plays with the development stack I need for Flutter. More crucially, at time of writing DP Alt Mode is not working, which means monitors plugged in to the USB-C port won’t work. Having cut the display off, this scuppers me.

For me this was an investment into being able to responsibly build software for the Apple ecosystem and its users who have different preferences than mine. It’s a tool for a specific sub-set of tasks, and under those conditions it does what I need.

Bonus: E-ink Macbook

Remember I mentioned that I had a Boox Max Lumi which could function as a monitor for the Macbook? Here it is in full effect.

Macbook using Boox Max Lumi as an E-ink display

After tweaking some settings in Accessibility/Display, MacOS can be made to look half-decent on e-ink.

All the usual caveats about e-ink apply re refresh rates and ghosting of course, but I could easily see myself working quite happily in Vim on it in some imaginary future dystopia where I’m trapped in Apple’s walled garden, all that exists, protected by a multi-trillion dollar Reality Distortion Field that they managed to power up just before the bombs dropped…

Running Headless: What I've Learned

Thu, 06 Mar 2025 12:26:14 +0000

The lid

Removing the lid turns it on (hall sensors in the base are activated by magnets in the lid)
Replacing the lid leaves the device visible on the network, but it can no longer be SSHed into (some form of ‘sleeping’)
During a user-initiated shutdown process (which takes some time, up to 20-30 seconds], replacing the lid re-wakes the machine — so wait 30 seconds or more before replacing the lid As the lid is no longer attached, ‘opening/closing’ makes no sense so I’ll instead use ‘removing/replacing’

Power button

Holding the power button for 5 seconds will begin the shutdown process, but the machine will still be visible on the network for a few seconds, allow at least 15-20 seconds for shutdown to complete
Holding the power button for 2 seconds will turn the machine on but it takes maybe 20 seconds to show up on the network As mentioned in previous parts of this series, ‘showing on the network’ is taken as a proxy for ‘machine is turned on’

From dead (no/low power)

A couple of times I’ve not used the machine for a week or more and the battery has got very flat — honestly it’s disappointing that it isn’t engineered to handle this situation better.

For a while I struggled to charge it, trying all of the following with no success:

Charging for over 10 mins in first USB-C port
Charging for over 10 mins in second USB-C port
Tried powering up several times during the charge periods above
After each attempt at powering up, waited 2 minutes for sign of life via the Macbook being seen on the LAN Charging units are dedicated USB PD/QC chargers wired directly to DC 12V/20V feeds, very stable/reliable and with ample amperage

What eventually got past this problem:

Unplugged all cables
Plugged one end of a charging cable into the Macbook first (top/corner port)
THEN plugged the other end of the cable into the USB charger
30 seconds later I was in (Macbook showed on router as a client and could be sshed into etc)

This could be a red herring eg the order of plugging in cables was not important but instead the battery had accumulated a charge and the final successful attempts just pushed it above some threshold that would allow it to boot. The fact that the response was so quick (30 secs after plugging in the cables in this order the machine was on) makes me think this is unlikely.

The only other explanation I can think of is that which end of the charging cable is plugged in first makes a difference, possibly due to something related to USB-C handshake/power negotiations that I’m ignorant of. This is a hunch and might make no sense in reality. I leave the info/observation here with very little weight attached to it, it is what it is.

Remote Access

Thu, 06 Mar 2025 12:26:13 +0000

Now I’m past the novelty of plugging in monitors this is how I actually use the machine.

Remote builds over SSH

I’m working on an Android tablet most of the time, using Termux, Proot-distro and Termux/X11 to run Debian Bookworm arm64. This setup has shortcomings but I find it very comfortable and manage to get 90% of my work done on it.

One current problem is that the arm64 version of the Android SDK can’t build arm64 APKs for Android. During main development I just build/run the Linux version of whichever app I’m working on — Flutter is multi-platform and suprisingly consistent between platforms. But of course I need to build APKs for Android devices at some point.

When I just need to quickly build some APKs for a given project, I use something like the following script (this one is for auDAV, my WebDAV audiobook player app):

#!/bin/sh

# Script to be run from Termux/proot-distro
# - Connects to build machine AIR (host defined in `~/.ssh/config`)
# - Logs in to working directory
# - Builds APKs
# - Syncs APKs to local machine


remote_commands=$(cat << 'EOF'
echo "~~~~~~~ Remote: Source local environment ~~~~~~~"
source ~/.zprofile
source ~/.zshrc

echo "~~~~~~~ Remote: Change working directory ~~~~~~~"
cd ~/development/audav

echo "~~~~~~~~~ Remote: Pull latest from git ~~~~~~~~~"
git pull

echo -e "\n~~~~~~~~~~~~~~ Remote: Build APKs ~~~~~~~~~~~~~~"
flutter build apk --split-per-abi
EOF
)

ssh AIR "bash --login -c '${remote_commands}'"

echo "\n~~~~~~~~~~~ Local: Sync APKs to local ~~~~~~~~~~"

rsync -r --mkpath --progress AIR:development/audav/build/app/outputs/flutter-apk $HOME/000-WORK/audav/build/app/outputs/

Fromhttps://codeberg.org/mm-dev/shell-scripts/raw/branch/termux-proot-ubuntu/audav-build-on-mac

Full remote graphical access with NoMachine

Remoting into MacOS with S8 tablet, trackball and split keyboard

To build apps for MacOS, iOS and iPadOS I must endure the torturous window management, tedious animations, and terrible file manager of MacOS (and that’s before we even get to XCode).

To ease my suffering I can at least use MacOS via the OLED screen of a Galaxy Tab S8 Ultra, a DEFT PRO trackball and a Ferris Sweep split mechanical keyboard.

NoMachine works well for remoting into MacOS and over the LAN it performs well with very little lag. There’s client software for Android and arm64 Linux meaning it’s quick and easy for me to just pop in to MacOS from my favoured Linux/DWM environment.

Is This Thing On?

Thu, 06 Mar 2025 12:26:11 +0000

One of the most difficult problems I’ve had with using this machine headless, is knowing whether it’s turned on or not. Seriously.

If I connect the external monitor (ok, not headless in that case), no image on the screen might mean:

The Macbook is not turned on
The Macbook is asleep
There is a problem with the monitor or connection (eg bad cable)

If I try to connect over NoMachine or VNC and fail, that might mean:

The Macbook is not turned on
The Macbook is asleep
The Macbook is not connected to the network
NoMachine/VNC is not started on the Macbook
I’ve configured something wrongly with NoMachine/VNC

I really dislike the millions of blinking LEDs designers love to stick on electronic devices. They constantly nag or distract, I think they are bad. But I’d kill for one one this Macbook!

Things I’ve tried

USB-C SD card reader with LED indicator

I have a small USB-C SD card reader. It has a red LED which lights up when it is receiving power. For a while I used this, but it’s impractical. Although it’s small, it still sticks out to be breakage risk. A nice bit of leverage on the USB port, just waiting to wrench it off if it catches on something (a bit like the 1st-gen Apple Pencil when it was charging).

Capslock button with LED indicator

At some point I noticed that the caps lock button on the keyboard had an LED which lit when caps was locked. And (remembering I’d usually be using a Bluetooth keyboard and pointing device), the caps lock could be applied to the built-in keyboard without affecting the Bluetooth keyboard. I could just leave caps lock on and the LED would stay lit.

The showstopper problem with both of the above ideas was that they both only worked when the Macbook was fully booted and ready to rock. I guess the ports and caps lock LED are powered off at other times. Most of my periods of confusion as to whether it was powered on or not were happening when it was half-on, during boot, sleep, shutdown etc.

My current-best solution

I noticed that my router, in its list of ‘attached clients’ was providing relatively quick/up-to-date information about whether the Macbook was connected or not. It seemed to connect to Wi-Fi quite early in the boot process, and disconnect soon after I initiated a shutdown.

Following on from that, I started pinging the Macbook in a terminal window, which I could keep nearby while I worked. This was getting close enough for my purposes.

The obvious next step was to script it properly. My aim was to get a brief status line telling me if the Macbook was connected or not, and its battery charge level. The following script did the job:

#!/bin/bash

# Ping a Mac on the LAN
# If it's alive, check its battery level using a seperate script `battery-check` (present on the
# Mac)
# Return the result with coloured text to indicate off/on status, and rough battery level
# Loop repeatedly, overwriting the results text each time onto the same single line

ip_to_check=192.168.8.123
hostname=AIR

wht='\033[1;15m'
red='\033[1;31m'
ylw='\033[1;33m'
ong='\033[38;5;208m'
grn='\033[1;32m'
nc='\033[0m' # No Colour

update_battery () {
  # Log in with SSH and call script to get battery charge info.
  battery_check_output=$(ssh AIR "bash --login -c battery-check")

  # `battery-check` returns a string like `battery: 18%`, we want to extract the number.
  percentage=$(echo "${battery_check_output}" | tr -dc '0-9')

  if [[ $percentage -lt 15 ]]; then
    battcolor=$red
  elif [[ $percentage -lt 30 ]]; then
    battcolor=$ong
  elif [[ $percentage -lt 80 ]]; then
    battcolor=$ylw
  else
    battcolor=$grn
  fi 
}

# Hide cursor
tput civis

# Sometimes (too soon after boot?) the first call fails
# Machete through it
update_battery
sleep 1
update_battery
sleep 1


i=0
while :
do

  # -c 1 ... count 1 (only ping once)
  if ping -c 1 ${ip_to_check} &> /dev/null; then
    # Temporarily go white to indicate battery has just been checked/updated (2 * 5 = 10secs)
    if [[ $i -lt 2 ]]; then
      color=$wht
    else
      color=$battcolor
    fi

    echo -ne "  ${hostname} ${grn}    ON     ${nc}${color}${percentage}%${nc}\r"

    # i ends up incrementing every 5secs or so
    ((i++))
    
    # Increment and every n counts update battery
    # 5sec * 36 = 180secs = 3mins
    if [[ $i -eq 36 ]]; then
      i=0
      update_battery
    fi
  else
    echo -ne "  ${hostname} ${red}   OFF       ${nc}\r"
  fi

  sleep 5
done

Fromhttps://codeberg.org/mm-dev/shell-scripts/raw/branch/termux-proot-ubuntu/mac-get-status

I already had a little script on the Mac to report the battery level (a very simple wrapper around a Mac built-in system_profiler, just to shorten the output).

#!/bin/sh

percent=$(system_profiler SPPowerDataType | grep "State of Charge (%)" | awk '{print $5}')
echo "battery: ${percent}%"

Monitoring script in a tiny floating window with close-up

Then I wanted to be able to float the mac-get-status script in a tiny window.

This next script opens up a terminal in a new window, gives it a specific title macmonitor, and runs mac-get-status:

#!/bin/bash

$TERMINAL --title macmonitor -e "mac-get-status" &

Fromhttps://codeberg.org/mm-dev/shell-scripts/raw/branch/termux-proot-ubuntu/mac-monitor

Then I tell my window manager (DWM) to apply a special rule to windows with that specific title. This is how it’s done in DWM but other window managers may have ways of achieving the same thing.

static const Rule rules[] = {
    // If an Xfce4-terminal window has title 'macmonitor':
    // - Make it float
    // - Make its dimensions 150px x 22px
    {"Xfce4-terminal", NULL, "macmonitor", 0, 1, -1, 0, 0, 150, 22, 1},

    // other rules...
};

Building Apps and Getting Notified

Tue, 30 Jul 2024 17:11:49 +0100

If you’ve followed along with all of the articles in this series you’ll now have:

A complete F-Droid build environment set up in a container
A Telegram bot raring to go

Now we’ll put it all together, add some bash scripts to make everything easy to work with, and see how to get notified in Telegram when the local F-Droid build process has finished.

We use a couple of shell scripts to facilitate this, they’re available here in my Codeberg repo.

I’ll refer to the Alpine container as the host as all of these instructions are based on that environment, and from that context the F-Droid docker container will be the guest.

Alpine container directory structure

So far in our Alpine container we should have a couple of directories which are our local clones of the F-Droid repos fdroiddata and fdroidserver.

We also need to add a directory where we’ll store our shell scripts:

mkdir shell-scripts

So now we should have the following structure (I named my Alpine container fdroid):

fdroid:~$ ls -1

fdroiddata
fdroidserver
shell-scripts

The scripts

In ~/shell-scripts we’ll have two scripts. Don’t forget to make sure they’re executable with chmod +x.

docker-run-fdroidserver.sh

Run this on the Alpine host to start the container.

This is really just a docker run command, based on the example given in the F-Droid docs, then heavily commented for people like me who aren’t experts in docker and need to be reminded of the basics.

Here is the script:

#!/bin/sh

# Based on F-Droid's guide at https://f-droid.org/en/docs/Submitting_to_F-Droid_Quick_Start_Guide/

# The only amendment to this script from the above URL are:
# - Mount an additional volume, the `shell-scripts` directory (parent directory of this script and `fdroid-prepare-env.sh`)
# - Change the entry point to `fdroid-prepare-env.sh`
# - Pass a couple of environment variables, IDs to be used to interact with Telegram so we can get a message pushed when a process finishes

# Explanation of command:
# `run`		creates a container from the image (image is given as final argument to the command)
# --rm 		when the container is exited, automatically remove it (the container, not the image)
# -i		`--interactive`, meaning you will be put into a `/bin/bash` (from the later command) session in the container
# -t		`--tty` container output gets sent to a virtual tty, meaning it will be formatted nicely and behave in specific expected ways
# -u		`--user` so in this case we are running as user 'vagrant'
# --entrypoint	overrides the default ENTRYPOINT of the image
# -v		mount a volume, the arg:arg format mounts a local (1st arg) volume inside the container at (2nd arg)
# 		:z at the end means Docker labels the content with a shared content label. Shared volume labels allow all containers to read/write content
# 		:Z at the end means Docker to label the content with a private unshared label. Only the current container can use a private volume
# -e		environment variable to be passed
# [final arg]	the image from which the container will be created

# Notes on setting up credentials for Telegram bot
# Based on tips from https://www.techrepublic.com/article/how-to-safely-store-passwords-linux-server/
# sudo apk add pass gnupg gnupg2 pinentry-tty
# gpg2 --full-generate-key
# pass init m@mm-dev.rocks
# pass insert telegrambot/botid
# pass insert telegrambot/chatid
# gpg-connect-agent reloadagent /bye

sudo docker run --rm -itu vagrant --entrypoint /home/vagrant/shell-scripts/fdroid-prepare-env.sh \
  -v ~/fdroiddata:/build:z \
  -v ~/fdroidserver:/home/vagrant/fdroidserver:Z \
  -v ~/shell-scripts:/home/vagrant/shell-scripts:Z \
  -e TELEGRAM_BOTID="$(pass telegrambot/botid)" \
  -e TELEGRAM_CHATID="$(pass telegrambot/chatid)" \
  registry.gitlab.com/fdroid/fdroidserver:buildserver

Fromhttps://codeberg.org/mm-dev/fdroid-shell-scripts/raw/branch/master/docker-run-fdroidserver.sh

In addition to the F-Droid original functionality, this version does a few other important things:

Pass in, as environment variables, some credentials which are required to access a Telegram bot via the Telegram API
- API token for the bot
- An ID for the chat channel that the message will be delivered to
Mount our ~/shell-scripts directory in the container so that the fdroid-prepare-env.sh script is available to it
Set the fdroid-prepare-env.sh script as the entry point for the container so that it automatically runs

As explained earlier in this series of articles, the Telegram credentials are stored on the host via GPG passwords and passed to the container as environment variables.

This means the credentials will be accessible in the clear to the container user so if this isn’t secure enough for your needs you should use another method.

fdroid-prepare-env.sh

This script is run inside the container to finish setting it up.

fdroid-prepare-env.sh is stored on the Alpine host in the ~/shell-scripts directory
But that ~/shell-scripts directory is mounted as a volume inside the container as /home/vagrant/shell-scripts
vagrant is the name of the user inside the container
So the script is accessible from inside the container

As it is passed (by the docker-run-fdroidserver.sh script) to the docker run command as the entry point, it gets run automatically when the container starts.

The full script is below:

#!/bin/bash

# #################################
#
# F-Droid build tools helper script
#
# #################################
#
# Docker image housekeeping, stuff to be done whenever the F-Droid container is creaated.
# Based on F-Droid's guide at https://f-droid.org/en/docs/Submitting_to_F-Droid_Quick_Start_Guide/
#
# - Several commands need to be run inside the container every time it starts, meaning constandly having to refer to the above URL
# - Instead, now this script is passed as the entry point to `docker run`
# - Alongside the F-Droid recommendations, export a function `send_tg_alert`, which allows a Telegram message to be sent when a command has completed execution


# Bring in system-wide settings and add some environment variables
# #################################
. /etc/profile
export PATH="$fdroidserver:$PATH" PYTHONPATH="$fdroidserver"
export JAVA_HOME=$(java -XshowSettings:properties -version 2>&1 > /dev/null | grep 'java.home' | awk -F'=' '{print $2}' | tr -d ' ')


# Create the message text to be sent to Telegram
# #################################
#
# - The message contains the shell command (from history) last entered (ie the command that was run and we are informing the recipient about)
# - Use MarkdownV2 syntax, see notes at https://core.telegram.org/bots/api#markdownv2-style
# - Use heredoc style to create the message and allow sending of newlines to survive the following pipeline:
#     bash | curl (GET, urlencode) | Telegram API
function get_msg_text {
	# Enable history, as it's not usually available from within bash scripts
	set -o history

	cat <<EOF
\`\`\`
# F\\-Droid command finished at:
# $(date)

# Command as entered:
\$ $(history | cut -c 8-)
\`\`\`
EOF
}

# Send a Telegram message via a bot
# #################################
function send_tg_alert {

	# Use heredoc style to create $MSG_TXT and allow sending of newlines to survive:
	# bash | curl (GET, urlencode) | Telegram API
	MSG_TXT=$(get_msg_text)

	echo "$MSG_TXT"

	# $TELEGRAM_CHATID and $TELEGRAM_BOTID should already exist in this container as environment variables,
	# having been passed in via the `docker run` command
	curl \
		--get \
		--data-urlencode "chat_id=${TELEGRAM_CHATID}" \
		--data-urlencode "parse_mode=MarkdownV2" \
		--data-urlencode "text=${MSG_TXT}" \
		https://api.telegram.org/bot${TELEGRAM_BOTID}/sendMessage
}

# Make functions available to the container and shell
# #################################
#
# If functions calls other functions, all functions in the chain need to be exported
export -f send_tg_alert
export -f get_msg_text


# Finish up...
# #################################
#
# Show some helpful reminders
cat << EOF

Example commands:

fdroid readmeta
fdroid rewritemeta com.example
fdroid checkupdates --allow-dirty com.example
fdroid lint com.example
fdroid build com.example


To send a Telegram message after the command has completed, append '; send_tg_alert', eg:

fdroid build com.example ; send_tg_alert

EOF


# Enter the directory which is most likely to be useful when the F-Droid commands are run
cd /build


# By default, the container would exit at the end of this ENTRYPOINT script --- start a shell instead.
/bin/bash "$@"

Fromhttps://codeberg.org/mm-dev/fdroid-shell-scripts/raw/branch/master/fdroid-prepare-env.sh

The effects of running the script in the container are:

Sets up some environment variables as per F-Droid’s instructions
Creates and exports our send_tg_alert function which can be called to send a message to a Telegram bot when a shell command has finished
The notification message sent by the function will include the command which was run/finished

The Telegram messaging is useful because the F-Droid build command can take a long time to complete. The function is just appended as an extra command eg:

# Build the 'com.example.appname'
# then send a Telegram message
fdroid build com.example.appname ; send_tg_alert

The F-Droid command will run, and when it ends a Telegram message will be sent to the specified channel, saying something like:

# F-Droid command finished at:
# Mon Jul 29 17:34:42 UTC 2024

# Command as entered:
$ fdroid build com.example.appname ; send_tg_alert

The F-Droid build commands

Now from inside the docker container (the entry point script should have put us inside the /build directory already) we can run the fdroid commands.

You can run fdroid --help to get a list of subcommands such as:

# Build a package from source
fdroid build

# Read all the metadata files and exit
fdroid readmeta

# Rewrite all the metadata files
fdroid rewritemeta

# Warn about possible errors in the metadata
fdroid lint

For each command you can append --help for more details, eg fdroid build --help, but check the official documentation for full details.

Creating a Telegram Bot

Tue, 30 Jul 2024 17:11:44 +0100

Create the bot

This is done via the Telegram app, so we need to open that up (if you don’t have it installed yet you’ll need to do that first).

Use search (magnifying glass icon) to find the bot called ‘BotFather’. This is an official Telegram bot, used for setting up new bots. Tap it to start a chat with it.
In the chat with ‘BotFather’, use the /newbot command, this is just a matter of typing the command as a message just like you’d send a message to a person
The slash is part of the command so make sure you include it
Enter a name for your bot, this is the name which will be displayed in chats
Enter a username for the bot, this is for Telegram’s use and must end in ‘_bot’ (the prompts will tell you this)

This creates the bot, then you will receive a message with some info about your new bot. The main thing you need here is the new bot’s token for accessing the HTTP API, this will be a long string something like 7901112608:AAD9fDpa9kgw3lZ10o6qANkXnB_D2ZvwuPc

This is your BOT TOKEN, keep a note of it as you need it to do stuff with your bot

Check that everything is working so far…

In a browser, visit a special URL including your bot token (hitting it in a browser like this is one simple way of accessing the API).

The format is https://api.telegram.org/bot<BOT_TOKEN>/getMe
So using the example ID above https://api.telegram.org/bot7901112608:AAD9fDpa9kgw3lZ10o6qANkXnB_D2ZvwuPc/getMe
Make sure the bot part is there just before the token as it’s an essential part of the URL
You should see a page in JSON format with some basic data about your new bot

Create a channel

To interact with the bot (eg to receive messages from it), you need to create a channel.

In Telegram, tap the pencil icon
The ‘New Message’ dialog will appear, tap ‘New Channel’
Give the channel a name (and description if you wish)
Choose whether it should be public or private
Choose subscribers — here you need to add your new bot. If you can’t see it in the list, tap the blank area at the top to search and type part of its name, it should appear.
The list of subscribers can be edited at any time by tapping its icon in Telegram

Now you can use the API again to get the ID for the channel (or ‘chat’ as it is referred to). The URL is similar to the one you used above to get the token ID, but the last part changes from getMe to getUpdates, so https://api.telegram.org/bot<BOT_TOKEN>/getUpdates.

Like before, you will see some JSON data. You are looking for a channel_post.chat.id, something like -1002179830041 the dash is part of the ID.

This is your CHAT ID, keep a note of this too as it will allow your bot to send messages to this chat/channel

Test your bot

Now you have what you need to send automated messages via your bot. It’s as simple as:

# Send a message from a Telegram bot to a specific chat/channel
curl 'https://api.telegram.org/bot<BOT_TOKEN>/sendMessage?chat_id=<CHAT_ID>&text=<YOUR_MESSAGE>'

Storing Secret Credentials

Tue, 30 Jul 2024 17:11:44 +0100

If you’ve already set up a Telegram bot and added it to a channel, you only need 2 things to be able to write code that uses that bot:

The BOT TOKEN - when you create a bot on Telegram you are given this
A CHAT ID - this points to a specific chat/channel on Telegram

These bits of info are used to send messages via the Telegram API. Anybody who has them can use your bot, so you want them to be secret, meaning you won’t want to hard-code them in any scripts.

Storing bot credentials using GPG

GNU Privacy Guard is a set of popular command line tools for working with encrypted secrets. We’ll use this to manage our Telegram IDs/keys.

First, make sure all the dependencies are installed

# Some distros might use gnupg, some have gnupg2
# some have both
sudo apk add pass gnupg gnupg2 pinentry-tty

Set up your user ready for key storage

Run the following commands and choose your options:

The default options are usually fine, but check out man pass to learn more
Use any email address you want, it doesn’t have to be real, it will be used as an ID to work with your keystore
Keep a note of whatever address you used

gpg2 --full-generate-key
pass init <YOUR_EMAIL>

Store your secrets

Use pass insert and give it an argument which you’ll use like a label for your secrets so you can edit/delete/retrieve them. You can organise them by using slashes so you can create something like a path or namespace for related secrets.

You will be prompted to enter the secret you want to store, then again to confirm it.

# I want to store a couple of IDs related to a Telegram bot,
# so start both with the same `telegrambot/` string
pass insert telegrambot/bottoken
pass insert telegrambot/chatid

Retrieve your secrets

pass telegrambot/bottoken 
# Output eg > t3l3GRAMbotIDiJustEnT3red

pass telegrambot/chatid 
# Output eg > teleGR4MchatIDiJustEnT3red

# So eg to set environment variables
# !WARNING ANYONE WHO CAN READ YOUR VARIABLES
# WILL BE ABLE TO READ THE SECRET!
export TELEGRAM_BOT_ID=$(pass telegrambot/bottoken)

To let go of your cached password

After you enter your password GPG hangs on to it for a while, which can make testing difficult while you are getting things set up. If you enter the command below it will be cleaned up and you’ll be prompted to enter your password again.

gpg-connect-agent reloadagent /bye

Setting Up the LXC Container

Tue, 30 Jul 2024 17:11:22 +0100

I’m using LXC containers on Proxmox. If you have a different method of working with containers I’m pretty sure you’ll be able to use the info here and repurpose it to your preferences.

Create an Alpine container

I won’t go into the details of how to do this as it’s specific to Proxmox and no different to setting up any other LXC container in Proxmox.

I’m using Alpine Linux as the container distro as it’s very lightweight and the F-Droid tools don’t need anything fancy.

~~In terms of resources, it doesn’t need much, the following has been more than enough for me so far:~~

~~512MB RAM~~
~~512MB swap~~
~~8GB disk~~

How naive of me, Gradle is much more greedy than I originally thought and I was getting build errors due to low resources. So I set the container up with:

6GB RAM
1GB swap
16GB disk

Set up the container for F-Droid builds

While logged into my Alpine container, first I set up some basics:

# Ensure everything is up to date
apk update
apk upgrade

# Install and configure git
apk add git vim
git config --global user.name "<YOUR_NAME>"
git config --global user.email "<YOUR_EMAIL>"
ssh-keygen
cat .ssh/id_rsa.pub 
# ... then copy key to gitlab SSH keys

F-Droid build instructions depend on docker, so:

apk add docker

# Start docker on boot
rc-update add docker boot

# Start docker now (or reboot so the above command can take effect)
service docker start

Now get the F-Droid stuff installed:

git clone --depth=1 git@gitlab.com:<YOUR_ACCOUNT>/fdroiddata.git ~/fdroiddata
cd ~/fdroiddata
git checkout -b com.example
cp templates/build-gradle.yml metadata/com.example.yml



# If (like me) you already created a fork before it's a little different 
# and you have to do this instead of the above

# git clone --depth=1 git@gitlab.com:<YOUR_ACCOUNT>/fdroiddata.git ~/fdroiddata
# cd fdroiddata/

# Shallow cloning makes other branches inaccessible, fix that
# git remote set-branches origin '*'
# git fetch -v --depth=1

# And now the branch can be checked out
# git checkout com.example

The container is set up now, later on on we’ll get around to how to actually use it…

Vanilla JS

Fri, 05 Apr 2024 12:18:03 +0100

Toolchain

I enjoy using plain/vanilla JS where possible. For my own projects I try to avoid the sprawl and long build times of NPM.

I do like to split my code up into separate modules though, and to be able to benefit from bundling/minification etc, for which I use esbuild. I also want to be able to use SCSS and JSDoc for type annotations. So I do allow myself some minimal tooling (the cone and Flake).

My setup looks like this…

esbuild - Download a build

JS bundler
Combine separate modules into a single JS file, minifying/compressing if required
Tree shaker
Avoid bloat by making sure only those modules/functions which are used get added

I use esbuild for most JS projects, so I installed a standalone build which can be run from anywhere — but it can also be installed using npm or other methods.

SASS - How to Install SASS

Compiler and bundler
Converts SASS/SCSS files into CSS

JSDoc - Documentation

Automated documentation creation
HTML docs for your methods, classes and variables will be created based on your code comments
Type annotations
Comments in your code can tell JSDoc which data types your code expects

Type annotations you provide (in specially-formatted comments) will be used to automatically create detailed documentation for your project.

Docs are created as HTML files, with lots of clickable cross-references. The files can be styled based on a templating system and CSS styling.

Lots of modern editors/IDEs will also use your type annotations to provide type hints and draw attention to potential problems eg if you try to pass an argument of the wrong type you’ll get a warning.

Example of Project Setup Using Tmux/Vim

I usually write a simple bash script for most projects. I’ll run this script when I want to start a session of working on a specific project.

Below is an example build script (for my ‘Pipe Dream’ game) which does the following:

Enters the project directory
Uses tmux to open a few window panes/splits:
- esbuild watching for changes to JS
- SASS watching for changes to CSS
- JSDoc watching the src directory (in reality I’d often skip this and run it manually on-demand, as it can be slow)
- git status, pane left open for commits etc
- A simple Python server so I can test the app in a browser
Opens a vim session with the relevant source files ready for editing

#!/bin/bash

# Open a JS project using esbuild, SASS and vim

work_dir="$HOME"/pipe-dream/

'tmux' \
    new-window -c "$work_dir" "esbuild src/main.js --outfile=www/js/pipe-dream.js --target=es6 --bundle --minify --sourcemap --global-name=PIPEDREAM --format=iife --watch" \; \
    split-window -c "$work_dir" "sass --watch src/scss/:www/css/" \; \
    split-window -c "$work_dir" "jsdoc src/ -c jsdoc.config" \; \
    split-window -c "$work_dir" "git status; bash -i" \; \
    split-window -c "$work_dir" "python -m http.server --directory www/" \; \
    select-layout even-vertical \; \
    new-window -c "$work_dir" "vim -S vim.Session" \; \

Vim Configuration

I’m one of those Vim people. People say its codebase is bloated but in use it always feels crisp and fast to me. Because of the way you manipulate text objects and leap around the page it somehow feels efficient on low-powered/slow devices, and works particularly well with e-ink devices.

Some people argue that they’re faster while using Vim than they are in other code editors. I’m not sure if I’m faster (I might be), but I’m from the school of thought that if your bottleneck when writing code is your typing speed, that might not be a good thing.

More importantly for me, I like the way Vim makes you think about text, as beautifully described in this classic stackoverflow answer ‘Your problem with Vim is that you don’t grok vi’.

I don’t like my coding environment to be too noisy or intrusive, but there are some powerful IDE features available nowadays and I don’t want to cut off my nose to spite my face.

Below is some info on some Vim plugins I use to help with JS development.

coc-nvim - Code of Conquer project repo

Although the ’nvim’ stands for Neovim (the popular Vim fork), COC also works very well in modern versions of normal Vim. This plugin allows VSCode-type IDE features, being based on the Language Server Protocol standard.

Very simply, a language server is a tool which understands a specific language, and can be used by your editor/IDE to analyse and give feedback on your code as you write it. It can do things like:

Syntax highlighting
Code completion
Draw attention to errors and give warnings about code
Help when refactoring code

COC isn’t a language server itself but it allows you to install and maintain whichever servers you need for the languages you use.

For example the language server I use for JS is coc-tsserver. The ’ts’ stands for TypeScript but it works very well with standard JS too. Along with the benefits mentioned above, this language server allows you to use type-checking, like TypeScript but without the compilation step. It also provides documentation and hints on using standard built-in JS methods.

You annotate your code by providing special comments in JSDoc syntax. When these comments/annotations are present, COC and the language server give you lots of hints while writing code, and warn you if you try to provide arguments of the wrong type.

Here is an example (from the official docs) showing what the JSDoc syntax looks like:

/** Class representing a point. */
class Point {
    /**
     * Create a point.
     * @param {number} x - The x value.
     * @param {number} y - The y value.
     */
    constructor(x, y) {
        // ...
    }

    /**
     * Get the x value.
     * @return {number} The x value.
     */
    getX() {
        // ...
    }

    /**
     * Get the y value.
     * @return {number} The y value.
     */
    getY() {
        // ...
    }

    /**
     * Convert a string containing two comma-separated numbers into a point.
     * @param {string} str - The string containing two comma-separated numbers.
     * @return {Point} A Point object.
     */
    static fromString(str) {
        // ...
    }
}

Sometimes the help is a little noisy though…

I love the functionality, but for me it’s a bit over-eager, eg if I’m half-way through typing a word I don’t need a popup telling me the word doesn’t exist.

coc-nvim does also provide a small symbol at the start of each line where it detects an error or wants to warn about something. So I like to leave the symbol visible, but hide the info popups by default. I add a keyboard shortcut so I can show/hide the explanation of the error/warning when I want to.

To stop the popups:

// In ~/.vim/coc-settings.json

{
  ...
  "diagnostic.enableMessage": "jump",
  ...
}

To add the show/hide keyboard shortcuts to Vim:

" In ~/.vimrc

nmap <silent> gh <Plug>(coc-float-hide)
nmap <silent> gl <Plug>(coc-diagnostic-info)

Prettier

This opinionated code formatter formats your code according to some fairly strict rules and has few configurable options (ie it’s their way or the highway).

I think most people find at least some of the rules disagreeable, but the point of this kind of tool is that it does a ‘decent, not perfect’ job, but provides a standard for formatting and removes the need for discussing, disagreement and wasted time.

I found it a little annoying at first but then got over myself and prefer now to use it whenever possible.

The Bad Parts

Fri, 05 Apr 2024 11:44:43 +0100

The Low Memory Killer Daemon - Official Docs.

This feature of modern Android versions kills tasks (/apps) when it thinks they are causing memory-related performance problems. It can feel pretty heavy-handed, with apps abruptly closing at seemingly random intervals.

In practice this can irritate but I’ve never lost any work because of it. The most noticeable occurrence is when Termux:X11 closes — my Linux desktop suddenly shuts down! Sounds terrible But actually my Vim buffers are constantly auto-saved, my browser tabs are stored… I just have to restart the session, then Tmux, then maybe a couple of other items. I’m back up and running in 1-2 minutes, and this problem might occur once or twice per week at the most.

There are various ways to tweak the daemon though, search for terms like ‘disable android oom killer’, ’tweak low memory killer daemon android’ etc.

Missing Tools for ARM Architecture

As the types of CPU in Android devices (usually ARM) are not the same as those used in laptops/desktops (usually X86), the standard desktop versions of software can’t be installed and compatible versions must be available (unless you want to compile them yourself).

This isn’t as bad as it sounds as the package manager for whichever distro you use will automatically pull in the correct version for your CPU architecture. Some software might be missing though.

A couple of specific gaps in the system affect me:

Garmin Monkey C
Garmin apps are written in their own language ‘Monkey C’, which has to use their own Java-based compiler and device simulator. As far as I can tell this isn’t happening on ARM. It was hard enough to get it working on Linux and Vim as they presume Windows/VSCode in their tutorials, so once I achieved that I didn’t take it any further.
Flutter cross-compilation
The situation with Flutter is strange. Flutter is used to make multi-platform apps, so the same code can output apps for Linux, Android, Windows etc. I can build a Linux (ARM64) binary on my Android Ubuntu… and I can build an Android binary on my (normal, X86 desktop) Linux. But I can’t build an Android binary from Android. Apparently people were able to do this on older versions of Flutter, so it seems like Google (who control Flutter) intentionally stopped supporting it.

Fingers crossed!

Mine is a niche use-case and I understand that I can’t expect organisations to support such a tiny minority as ARM Linux. Apple’s M1/M2/M3 ARM chips are doing very well, which is great if you’re into that but I don’t enjoy their ecosystem.

So for now I need to hang on to my X86 desktop. But I hope ARM continues to gain recognition as more than a mobile phone CPU and can be used for more development tasks in future.

Software

Fri, 05 Apr 2024 11:44:41 +0100

Although this is all in relation to working on Android, the software I use is generally the same as if I was on a Linux desktop. That’s kind of the point, nowadays I can use almost the same environment on Android as I can on my desktop.

Termux

Termux is a brilliant free terminal app and Linux environment for Android. It has most of the Linux commands I need and I can install git, Vim, Tmux, mutt, ImageMagick and the majority of the terminal software I like to use. For certain jobs such as scripting, SSH sessions and writing, this is all I need.

Termux can run non-terminal, graphical apps too, but you won’t be able to see them. That’s what the next app is for.

Termux:X11

This amazingly useful app describes itself as being in early development, but I’ve been daily-driving it for months without significant issues.

Termux:X11 is an X Server for Termux. This allows you to install graphical Linux apps, as long as they are available for your system architecture (ie AArch64 — or ARM64 as it’s known — for most modern Android devices).

This means I can run, directly on my tablet:

Desktop Firefox (so, full developer tools)
Desktop Chromium (this was quite hard to get installed and isn’t the absolute latest version but is still very useful for development)
Krita/Inkscape/GIMP for image processing
Thunar file manager
Audacity (using PulseAudio over the network to pipe the audio up to Android by using the devices own IP address… I’m sure this creates some delay/latency but I’ve not noticed it in practice and for my typical usage of minor adjustments to audio files it works well)

PRoot Distro

This is a ‘container environment manager’ and allows you to easily install, uninstall, backup and restore Linux distributions, alongside/on-top-of Android, sharing the kernel.

It’s really simple to use, eg installing Debian Linux is a matter of opening Termux and entering:

proot-distro install debian

Then for a shell login:

proot-distro login debian

PRoot Distro provides a nice choice of distros including:

Alpine Linux (edge)
Arch Linux ARM
Artix Linux (AArch64 only)
Debian (stable)
Fedora 38 (AArch64 only)
Manjaro (AArch64 only)
OpenSUSE (Tumbleweed)
Pardus (yirmibir)
Ubuntu (23.10)
Void Linux

You can install as many distros as you want and swap between them easily. Each distro just sits in a directory on the Android device. Using the backup tools the entire system can be bundled up as a single archive file and moved over to another Android device.

Window Management

I believe that the best user interface (actually, interface of any kind) is one that I don’t even notice exists. It doesn’t shout at me, it doesn’t wow me with animations and eye-candy. It allows me to get to the tools I need to get the job done and stay immersed in the task. It is at peace with itself enough to take a back seat and not demand attention. It is, conceptually, transparent.

A lot of people like the gestural UI featured in ‘Minority Report’. I like the film, and the scene with Tom Cruise manipulating the UI is well produced and fun to watch. But as a UI it’s terrible! Hugely inefficient with all those big sweeping hand gestures, it’s not quick or practical. I don’t want to have to perform interpretive dance to use my computer (nor watch my computer doing little performances when I’m trying to get stuff done).

I found similar sentiment in this Daring Fireball article about the Apple Vision Pro:

“To me the Macintosh has always felt more like a place than a thing. Not a place I go physically, but a place my mind goes intellectually. When I’m working or playing and in the flow, it has always felt like MacOS is where I am. I’m in the Mac.

“Interruptions — say, the doorbell or my phone ringing — are momentarily disorienting when I’m in the flow on the Mac, because I’m pulled out of that world and into the physical one.”

I am nearly exactly like this too, except I don’t want to feel like I’m “in the Linux” (and definitely not “in the Mac” or “in Windows”!). I don’t want to be aware of that layer of abstraction at all. I don’t mind feeling that I’m “in Vim”, but really I want to be “in the code” that I’m editing. I specifically don’t want to feel the manicured claw of a corporate behemoth like Apple or Microsoft on my shoulder… but that’s just me.

I’d rather forget that my OS(/window manager) exists. It’s a facilitator, not an end in itself.

DWM: Tiling Window Manager

I used to prefer larger screens and multiple monitors. I knew about the concept of workspaces but they sounded annoying and fiddly. It took me a long time to give them a chance and it happened accidentally, as a side-effect of trying a tiling window manager.

Very briefly, a tiling window manager organises your windows into tiles, meaning instead of having oddly-sized overlapping windows floating around making a confusing mess of your screen and using space inefficiently, your screen splits into fractions, each being filled by an app. If you have too many apps on the screen they can go onto other screens, or ‘workspaces’. Personally I tend to run most apps fullscreen most of the time, switching between them with keyboard shortcuts.

I started with i3 Window Manager, which is a really nice piece of software. After a few years of using it though I realised I didn’t use most of its features, so landed on the very minimal Suckless Tools DWM.

With workspaces, each app can have its own screen and can be accessed instantly via a keyboard shortcut. With familiarity this becomes a chord in muscle memory, a hand-shape to mash at keyboard. I have habits for where I keep each app, so eg my terminal is in workspace 1 so [Alt]+[1] puts me in the terminal. [Alt]+[9] puts me in FireFox and so on.

I’ve [Alt]+[Tab]bed through apps in the past. Workspaces are similar except you know the keyboard chord will take you directly to the app you want, you don’t have to play ‘stop the magical app carousel at the right time’… which may be a drawback for some I guess, as carousels can be fun.

We can only focus on one thing at a time, that’s how attention works. I can see some uses for multiple screens eg if you have to monitor lots of things, like real-time charts or meters. In that case your peripheral vision is good at detecting change/motion so you can make use of all the space. But personally I’m completely over large/multiple displays. In my circumstances, they waste power, waste space, and are just not necessary.

Android Bluetooth Keyboard Mapping

Thu, 08 Feb 2024 12:05:30 +0000

I use Bluetooth keyboards with Android a lot, and often want to remap keys (eg swapping CAPS_LOCK to CTRL_LEFT).

Maybe it’s a failing in my search technique or I’m doing something that nobody else does nowadays, but whenever I search for info about this I find really old articles with out-of-date info, or just links to apps. To be fair Key Mapper is good and I do use it, but sometimes I just want to be able to do it at a system level without requiring extra apps.

To remap keys in Android, you need to edit a particular keylayout *.kl file, specific to the device (eg Bluetooth keyboard) you want to re-map. The files have names like Vendor_04e8.kl. The name is derived from various properties of the device, such as vendor ID, product ID and other things as explained here in the Android docs.

The sticking point for me, for ages, was that I had trouble finding the vendor and product codes and, consequently, the correct file to edit. In Linux I’d use lsusb or bluetoothctl or something like that, but on Android even in the wonderful Termux, these things don’t work properly for reasons I haven’t completely wrapped my head around, related to security and/or the way Android and its underlying Linux kernel inter-relate.

Problem 1: Finding which keylayout file needs to be edited

In Termux or whichever shell/terminal you use, do su to log in as superuser
Do dumpsys and wait for it to finish. It can take a while (couple of minutes on some of my devices)…
If I don’t do this step, on some devices the next step doesn’t work… I hate this kind of situation as I have no idea why this should work and it feels nonsensical, but all I know is it fixed the problem when I encountered it.
Do dumpsys input to narrow the output
Even this input-specific command outputs quite a lot. I tend to just take my time and scroll through it, but you can make it easier by trying something like:
dumpsys input | grep -i -A12 bluetooth
- The -i means case-insensitive
- The -A12 means show the matching line plus the 12 lines After it
- You may want to tweak the number but for me 12 was good
In the output, look for a description for your device (mine is Device 45: Bluetooth 3.0 Keyboard), and soon after that you’ll find a line like KeyLayoutFile: /system/usr/keylayout/Vendor_04e8.kl — that’s the *.kl file you need to edit to remap your keyboard

Problem 2: Editing system files on modern Android

On modern versions of Android, even if you have root, it’s not possible to directly edit files in the /system/ directory (it is mounted as read-only).

The way around this is to make a Magisk module. This is simpler than it might sound. The basic idea is that you create a folder in /data/adb/modules, (eg in my case I use mm, so /data/adb/modules/mm/). Inside that folder, you can create files and folders which Magisk will overlay onto the system at boot time, eg:

/data/adb/modules/mm/system/usr
# becomes
/system/usr

/data/adb/modules/mm/system/usr/keylayout/Vendor_04e8.kl
# becomes
/system/usr/keylayout/Vendor_04e8.kl

Now you can edit the keylayout file to make your changes, eg to change CAPS_LOCK to CTRL_LEFT on my keyboard it’s:

# key 58    CAPS_LOCK
key 58    CTRL_LEFT

Other apps which might be useful

Depending on your Android device and software version, it can be difficult/confusing to find the right key codes. I find that one or another of the apps below usually gets me what I need: