Jonas HietalaModify

Why Fennel?

Lua has been a fantastic boon to Neovim and it’s a significant improvement over Vimscript, yet I can’t help but raise an eyebrow when I hear people describe Lua as a great language. It’s definitely great at being a simple and fast embeddable language but the language itself leaves me wanting more.

Fennel doesn’t solve all issues as some of Lua’s quirks bleeds through but it should make it a little bit nicer. I particularly like the destructuring; more functional programming constructs; macros for convenient DSL; and the amazing pipe operator.

But the biggest reason is that I’m simply a bit bored and trying out new programming languages is fun.

Why vim.pack?

I don’t rewrite my config often. But when I do, I do it properly.

Folke, maker of many popular plugins such as lazy.nvim and snacks.nvim, recently had a ~5 month break from working on his plugins. Of course, they continued to work and anyone working on open source projects can (and should) take a break whenever they want.

But it exemplifies that core Neovim features will likely be better maintained than standalone plugins and should probably be preferred (if they provide the features you need).

While Neovim’s built-in plugin manager is a work in progress and still a bit too simplistic for my needs I wanted to try it out.

Source pack specs from files

I want to be able to create a file under plugins/, have it return a vim.pack.Spec, and have it automatically added. This is similar to the structured plugins approach of lazy.nvim.

To build this I first list all files under plugins/ like so:

;; List all files, with absolute paths.


(local paths (-> (vim.fn.stdpath "config")

                 (.. "/fnl/plugins/*")

                 (vim.fn.glob)

                 (vim.split "\n")))

This uses Fennel’s -> threading macro, Fennel’s version of the pipe operator. It’s one of my favorite features of Elixir and was stoked to discover that Fennel has it too. (Fennel actually has even more power with the ->>, -?>, and -?>> operators!)

Now we need to loop through and transform the paths to relative paths and evaluate the files to get our specs. (I’m using accumulate to explicitly build a list instead of collect as we’ll soon expand on it):

;; Make the paths relative to plugins and remove extension, e.g. "plugins/snacks"

;; and require those packages to get our pack specs.


(local specs (accumulate [acc [] _ abs_path (ipairs paths)]

               (do

                 (local path (string.match abs_path "(plugins/[^./]+)%.fnl$"))

                 (if (and path (not= path "plugins/init"))

                     (do

                       (local mod_res (require path))

                       (table.insert acc mod_res))

                       acc)

                     acc))))

Now we can populate specs from files under plugins/, for example like this that returns a single spec:

{:src "https://github.com/romainl/vim-cool"}

But I also want to be able to return multiple specs:

[{:src "https://github.com/nvim-lua/popup.nvim"}

 {:src "https://github.com/nvim-lua/plenary.nvim"}]

To support this we can match on the return value to see if it’s a list, and then loop and insert each spec in the list, otherwise we do as before:

(local specs (accumulate [acc [] _ abs_path (ipairs paths)]

               (do

                 (local path (string.match abs_path "(plugins/[^./]+)%.fnl$"))

                 (if (and path (not= path "plugins/init"))

                     (do

                       (local mod_res (require path))

                       (case mod_res

                         ;; Flatten return if we return a list of specs.

                         [specs]

                         (each [_ spec (ipairs mod_res)]

                           (table.insert acc spec))

                         ;; Can return a string or a single spec.

                         _

                         (table.insert acc mod_res))

                       acc)

                     acc))))

Now all that’s left is to call vim.pack.add with our list of specs and our plugins are now automatically added from files under plugins/:

(vim.pack.add specs {:confirm false})

Lazy loading with lze

lze is a nice and simple plugin to add lazy-loading to vim.pack.

We’ve already added it as a dependency in our init.lua so all we need to do is modify the load parameter to vim.pack.add like so:

;; Override loader when adding to let lze handle lazy loading

;; when specified via the `data` attribute.


(vim.pack.add specs {:load (fn [p]

                             (local spec (or p.spec.data {}))

                             (set spec.name p.spec.name)

                             (local lze (require :lze))

                             (lze.load spec))

                     :confirm false})

Now we can specify lazy loading via the data parameter in our specs:

{:src "https://github.com/romainl/vim-cool"

 :data {:event ["BufReadPost" "BufNewFile"]}}

It relies on wrapping configuration under data but that’s annoying, so let’s simplify things a little.

Simplified specifications

The idea here is to transform the specs before we call vim.pack.add.

We can do it easily when we collect our specs by calling the transform_spec function:

(local specs (accumulate [acc [] _ abs_path (ipairs paths)]

               (do

                 (local path (string.match abs_path "(plugins/[^./]+)%.fnl$"))

                 (if (and path (not= path "plugins/init"))

                     (do

                       (local mod_res (require path))

                       (case mod_res

                         [specs]

                         (each [_ spec (ipairs mod_res)]

                           (table.insert acc (transform_spec spec)))

                         _

                         (table.insert acc (transform_spec mod_res)))

                       acc)

                     acc))))

I want transform_spec to transform this:

{:src "https://github.com/romainl/vim-cool"

 :event ["BufReadPost" "BufNewFile"]}

Into this:

{:src "https://github.com/romainl/vim-cool"

 :data {:event ["BufReadPost" "BufNewFile"]}}

By storing keys other than src, name, and version under a data table. This is what I came up with:

(λ transform_spec [spec]

  "Transform a vim.pack spec and move lze arguments into `data`"

  (case spec

    {}

    (do

      ;; Split keys to vim.pack and rest into `data`.

      (local pack_args {})

      (local data_args {})

      (each [k v (pairs spec)]

        (if (vim.list_contains [:src :name :version] k)

            (tset pack_args k v)

            (tset data_args k v)))

      (set pack_args.data data_args)

      pack_args)

    ;; Bare strings are valid vim.pack specs too.

    other

    other))

Another quality of life feature I’d like is to make it simpler to call setup functions. lazy.nvim again does this well and it’s pretty convenient.

For example, this is how it looks like with lze to add an after hook and call a setup function:

{:src "https://github.com/A7Lavinraj/fyler.nvim"

 :on_require :fyler

 :after (λ []

          (local fyler (require :fyler))

          (fyler.setup {:icon_provider "nvim_web_devicons"

                        :default_explorer true}))}

What if we could instead do:

{:src "https://github.com/A7Lavinraj/fyler.nvim"

 :on_require :fyler

 :setup {:icon_provider "nvim_web_devicons" :default_explorer true}}]

But this is just data and we can transform the second case to the first one fairly easily. In the transform_spec function:

(λ transform_spec [spec]

  "Transform a vim.pack spec and move lze arguments into `data`


   and create an `after` hook if `setup` is specified."

  (case spec

    {}

    (do

      ;; Split keys to vim.pack and rest into `data`.

      (local pack_args {})

      (local data_args {})

      (each [k v (pairs spec)]

        (if (vim.list_contains [:src :name :version] k)

            (tset pack_args k v)

            (tset data_args k v)))



      (λ after [args]

        ;; Call `setup()` functions if needed.

        (when spec.setup

          (local pkg (require spec.on_require))

          (pkg.setup spec.setup))

        ;; Load user specified `after` if it exists.

        (when spec.after

          (spec.after args)))



      (set data_args.after after)

      (set pack_args.data data_args)

      pack_args)

    ;; Bare strings are valid vim.pack specs too.

    other

    other))

How to figure out the package name to require (since it may differ from the path)? lazy.nvim has a bunch of rules to try to figure this out automatically but I chose to be explicit. lze uses the on_require argument so it can load on a require call (on (require :fyler) for example), which seems like a good idea to reuse.

And just to prevent me from making mistakes, I added a sanity check:

;; `:setup` needs to know what package to require,

;; therefore we use `:on_require`


(when (and spec.setup (not spec.on_require))

  (error (.. "`:setup` specified without `on_require`: "

             (vim.inspect spec))))




(λ after [args]

  ;; ...

Build scripts via PackChanged events

There’s one last feature I really want from lazy.nvim and that’s to automatically run build scripts after a package is installed or updated.

I basically want to specify this in my specs:

{:src "https://github.com/eraserhd/parinfer-rust"

 :build ["cargo" "build" "--release"]}

Again, we’ll rely on PackChanged for this:

;; Before `vim.pack.add` to capture changes.


(augroup! :plugin_init (au! :PackChanged pack_changed))

The above code uses macros from nvim-laurel to define an autocommand that calls the pack_changed function. That function will then run pack_changed when the package is updated or installed:

(λ pack_changed [event]

  (when (vim.list_contains [:update :install] event.data.kind)

    (execute_build event.data))

  ;; Return false to not remove the autocommand.

  false)




(λ execute_build [pack]

   ;; `?.` will prevent crashing if any field is nil.

  (local build (?. pack :spec :data :build))

  (when build

    (run_build_script build pack)))

To run the scripts I use vim.system with some simple printing:

(λ run_build_script [build pack]

  (local path pack.path)

  (vim.notify (.. "Run `" (vim.inspect build) "` for " pack.spec.name)

              vim.log.levels.INFO)

  (vim.system build {:cwd path}

              (λ [exit_obj]

                (when (not= exit_obj.code 0)

                  ;; If I use `vim.notify` it errors with:

                  ;; vim/_editor.lua:0: E5560: nvim_echo must not be called in a fast event context

                  ;; Simply printing is fine I guess, it doesn't have to be the prettiest solution.

                  (print (vim.inspect build) "failed in" path

                         (vim.inspect exit_obj))))))

This will now allow us to run build scripts like cargo build --release or make after a package is installed or updated. It’s a bit too basic as there’s no visible progress bar for long running builds (Rust, I’m looking at you!) and it doesn’t handle build errors that well but it works well enough I guess.

But what about user commands or requiring a package? For example with nvim-treesitter you’d want to run :TSUpdate after an update, something like this:

{:src "https://github.com/nvim-treesitter/nvim-treesitter"

 :version :main

 :build #(vim.cmd "TSUpdate")}

Let’s try it by allowing functions in the build parameter (and bare strings because why not):

(λ execute_build [pack]

  (local build (?. pack :spec :data :build))

  (when build

    (case (type build)

      ;; We can specify either "make" or ["make"]

      "string" (run_build_script [build] pack)

      "table" (run_build_script build pack)

      ;; Run a callback instead.

      "function" (call_build_cb build pack))))




(λ call_build_cb [build pack]

  (vim.notify (.. "Call build hook for " pack.spec.name) vim.log.levels.INFO)

  (build pack))

If we run this though it doesn’t work:

Error in /home/tree/code/nvim-conf/init.lua..PackChanged Autocommands for "*":

Lua callback: vim/_editor.lua:0: /home/tree/code/nvim-conf/init.lua..PackChanged Autocommands for "*"..script nvim_exec2() called

at PackChanged Autocommands for "*":0, line 1: Vim:E492: Not an editor command: TSUpdate

The problem is that PackChanged is run before the pack is loaded. Maybe we could work around this by calling packadd ourselves but that would shortcut lazy loading. In this instance we’d like to run TSUpdate after the pack is loaded but only if it’s been updated or installed so we don’t run it after every restart.

What I did was introduce an after_build parameter to the spec that’s run after load if a PackChanged event was seen before:

{:src "https://github.com/nvim-treesitter/nvim-treesitter"

 :version :main

 :after_build #(vim.cmd "TSUpdate")}

Then in plugins/init.fnl I use a local variable packs_changed that’s updated on PackChanged like so:

;; Capture packs that are updated or installed.


(g! :packs_changed {})




(λ set_pack_changed [name event]

  ;; Maybe there's an easier way of updating a table global...?

  (var packs vim.g.packs_changed)

  (tset packs name event)

  (g! :packs_changed packs))




(λ pack_changed [event]

  (when (vim.list_contains [:update :install] event.data.kind)

    (local pack event.data)

    (set_pack_changed pack.spec.name event)

    (execute_build pack))

  false)

Then we’ll call after_build from the after hook we setup before:

(λ transform_spec [spec]

  (case spec

    {}

    (do

      ;; Split keys to vim.pack and rest into `data`.

      ;; ...



      (λ after [args]

        (local pack_changed_event (. vim.g.packs_changed args.name))

        (set_pack_changed args.name false)

        (when spec.setup

          (local pkg (require spec.on_require))

          (pkg.setup spec.setup))

        ;; Run `after_build` scripts if a `PackChanged` event

        ;; was run with `install` or `update`.

        (when (and spec.after_build pack_changed_event)

          (spec.after_build args))

        (when spec.after

          (spec.after args)))



      (set data_args.after after)

      (set pack_args.data data_args)

      pack_args)

    other

    other))

With this we can finally specify build actions such as these:

{:build "make"

 :build ["cargo" "build" "--release"]

 :after_build #(vim.cmd "TSUpdate")}

Options

set relativenumber


set clipboard^=unnamed,unnamedplus


set backupdir=~/.config/nvim/backup


let mapleader=" "

vim.opt.relativenumber = true

vim.opt.clipboard:append({ "unnamed", "unnamedplus" })

vim.opt.backupdir = vim.fn.expand("~/.config/nvim/backup")

vim.g.mapleader = [[ ]]

(set! :relativenumber true)


(set! :clipboard + ["unnamed" "unnamedplus"])


(set! :backupdir (vim.fn.expand "~/.config/nvim/backup"))


(g! :mapleader " ")

With nvim-laurel macros I think Fennel is decent. Slightly better than Lua but not as convenient as Vimscript.

Keymaps

local map = vim.keymap.set

map("n", "<localleader>D", vim.lsp.buf.declaration,

  { silent = true, buffer = buffer, desc = "Declaration" }

)

map("n", "<leader>ep", function() find_org_file("projects") end,

  { desc = "Org projects" }

)

map("n", "]t", function()

  require("trouble").next({ skip_groups = true, jump = true })


end, {

  desc = "Trouble next",

  silent = true,

})

(bmap! :n "<localleader>D" vim.lsp.buf.declaration

       {:silent true :desc "Declaration"})


(map! :n "<leader>ep" #(find_org_file "projects")

      {:desc "Org projects"})


(map! :n "]t" #(do

                 (local trouble (require :trouble))

                 (trouble.next {:skip_groups true :jump true}))

      {:silent true :desc "Trouble next"})

Not a huge difference to be honest. I like the #(do_the_thing) shorthand for anonymous functions fennel has. Having to (sometimes) split up require and method calls on separate lines in Fennel is annoying.

Overriding highlight groups

One example that was a big step up with Fennel is overriding highlight groups. I’m using melange which is a fantastic and underrated color scheme but I’ve collected a fair bit of overrides for it.

In Lua you use nvim_set_hl to add an override, for example like this:

vim.api.nvim_set_hl(0, "@symbol.elixir", { link = "@label" })

When you do this 100 times this is annoying so I made an override table to accomplish the job:

local overrides = {

  { name = "@symbol.elixir", val = { link = "@label" } },

  { name = "@string.special.symbol.elixir", val = { link = "@label" } },

  { name = "@constant.elixir", val = { link = "Constant" } },

  -- And around 100 other overrides...


}




for _, v in pairs(overrides) do

  vim.api.nvim_set_hl(0, v.name, v.val)


end

In Fennel with the hi! macro this all becomes as simple as:

(hi! "@symbol.elixir" {:link "@label"})


(hi! "@string.special.symbol.elixir" {:link "@label"})


(hi! "@constant.elixir" {:link "Constant"})

Autocommands

Here are some autocommands to enable cursorline only in the currently active window (while skipping buffers such as the dashboard):

local group = augroup("my-autocmds", { clear = true })

autocmd({ "VimEnter", "WinEnter", "BufWinEnter" }, {

  group = group,

  callback = function(x)

    if string.len(x.file) > 0 then

      vim.opt_local.cursorline = true

    end

  end,

})

autocmd("WinLeave", {

  group = group,

  callback = function()

    vim.opt_local.cursorline = false

  end,

})

(augroup! :my-autocmds

          (au! [:VimEnter :WinEnter :BufWinEnter]

               #(do

                  (when (> (string.len $1.file) 0)

                    (let! :opt_local :cursorline true))

                  false))

          (au! :WinLeave #(do

                            (let! :opt_local :cursorline false)

                            false)))

Plugin specs

One thing I like more in Lua compared to Fennel is how readable tables are. The Fennel formatter fnlfmt might be partly to blame as it has a tendency to use very little whitespace. Regardless, I prefer this Lua code:

return {

  "https://github.com/stevearc/conform.nvim",

  { src = "https://github.com/mason-org/mason.nvim", dep_of = "mason-lspconfig.nvim" },

  { src = "https://github.com/neovim/nvim-lspconfig", dep_of = "mason-lspconfig.nvim" },

  "https://github.com/mason-org/mason-lspconfig.nvim",

  {

    src = "https://github.com/nvim-treesitter/nvim-treesitter",

    version = "main",

    after = function()

      vim.cmd("TSUpdate")

    end,

  },

}

Over this corresponding Fennel code:

["https://github.com/stevearc/conform.nvim"

 {:src "https://github.com/mason-org/mason.nvim" :dep_of :mason-lspconfig.nvim}

 {:src "https://github.com/neovim/nvim-lspconfig" :dep_of :mason-lspconfig.nvim}

 "https://github.com/mason-org/mason-lspconfig.nvim"

 {:src "https://github.com/nvim-treesitter/nvim-treesitter"

  :version :main

  :after #(vim.cmd "TSUpdate")}

To me the Lua code is for some reason easier to read.

Similarly I don’t have a problem with this lazy.nvim spec:

return {

  "folke/snacks.nvim",

  priority = 1000,

  lazy = false,

  opts = {

    indent = {

      indent = {

        enabled = true,

        char = "┆",

      },

      scope = {

        enabled = true,

        only_current = true,

      },

    },

    scroll = {

      animate = {

        duration = { step = 15, total = 150 },

      },

    },

    explorer = {},

  },

}

But with this new Fennel spec I use—even though it’s simpler in some ways—it’s harder for me to quickly see what table the keys belong to:

{:src "https://github.com/folke/snacks.nvim"

 :on_require :snacks

 :lazy false

 :setup {:indent {:indent {:enabled true :char "┆"}

                  :scope {:enabled true :only_current true}}

         :scroll {:animate {:duration {:step 15 :total 150}}}

         :explorer {}}}

Maybe it’s something you’ll get used to?

Native undotree

I’ve been using undotree a long time and it’s excellent. This feature was recently merged into Neovim:

;; It's optional so we need to use packadd to activate the plugin:


(vim.cmd "packadd nvim.undotree")

;; Then we can add a keymap to open it:


(map! :n "<leader>u" #(: (require :undotree) :open {:command "topleft 30vnew"}))

Simplified LSP config

Neovim routinely gets shit on for LSPs being so hard to setup. Yes, it could probably be easier but Neovim has recently made some changes to streamline LSP configuration and it’s not nearly as involved as it used to be.

Here’s how my base config looks like:

(require-macros :macros)



;; Convenient way of installing LSPs and other tools.


(local mason (require :mason))


(mason.setup)



;; Convenient way of automatically enabling LSPs installed via Mason.


(local mason-lspconfig (require :mason-lspconfig))


(mason-lspconfig.setup {:automatic_enable true})



;; Show diagnostics as virtual lines on the current line.

;; It's pretty cool actually, you should try it out.


(vim.diagnostic.config {:virtual_text false

                        :severity_sort true

                        :virtual_lines { :current_line true })



;; I like inlay hints.


(vim.lsp.inlay_hint.enable true)




(augroup! :my-lsps

          (au! :LspAttach

               (λ [_]

                 (local snacks (require :snacks))

                 (bmap! :n "<localleader>D" snacks.picker.lsp_declarations

                        {:silent true :desc "Declaration"})

                 (bmap! :n "<localleader>l"

                        #(vim.diagnostic.open_float {:focusable false})

                        {:silent true :desc "Diagnostics"})

                ;; etc

I also use nvim-lspconfig but it doesn’t do anything magical (anymore). It’s basically a collection of LSP configs, so I don’t have to fill my config with things like this:

(vim.lsp.config "expert"

                {:cmd ["expert"]

                 :root_markers ["mix.exs" ".git"]

                 :filetypes ["elixir" "eelixir" "heex"]})




(vim.lsp.enable "expert")

If you don’t want to change the keymaps (Neovim comes with defaults that I personally dislike) or customize specific LSPs then there’s not that much left. Mason is also totally optional and if you want to manage your LSPs outside of Neovim you can totally do that. The only thing missing is autocomplete, which blink.cmp provides out of the box.

Automatically install and enable treesitter grammars

Another thing that has changed since my last config overhaul is nvim-treesitter being rewritten and is now a much simpler plugin. The new version lives on the main branch and the old archived one on master and it contains a bunch of breaking changes.

For example, it no longer supports installing and activating grammars automatically. I think I saw a plugin for that somewhere but here’s some Fennel code that sets it up:

(require-macros :macros)




(local nvim-treesitter (require :nvim-treesitter))



;; Ignore auto install for these filetypes:


(local ignored_ft [])




(augroup! :treesitter

          (au! :FileType

               (λ [args]

                 (local bufnr args.buf)

                 (local ft args.match)

                 ;; Auto install grammars unless explicitly ignored.

                 (when (not (vim.list_contains ignored_ft ft))

                   (: (nvim-treesitter.install ft) :await

                      (λ []

                        ;; Enable highlight only if there's an installed grammar.

                        (local installed (nvim-treesitter.get_installed))

                        (when (and (vim.api.nvim_buf_is_loaded bufnr)

                                   (vim.list_contains installed ft))

                          (vim.treesitter.start bufnr))))))))

If you use nvim-treesitter-textobjects (which you should) remember to migrate to the main branch there too.

Some new fun plugins

1. fyler.nvim, edit a file explorer like a buffer

   oil.nvim is a great plugin that allows you to manage files by simply editing text. fyler.nvim takes it to the next level by combining it with a tree-style file explorer.

2. blink.cmp, faster autocomplete

   I’ve been using nvim-cmp as my completion plugin but I migrated to blink.cmp as it’s faster and more actively maintained. It’s too bad that it broke my custom nvim-cmp source for my blog but it wasn’t too hard to migrate.

3. snacks.nvim, a better picker

   telescope.nvim has been a solid picker but it’s no longer actively developed and the snacks.nvim is the replacement I settled on.

   I tried fff.nvim for file picking but surprisingly it felt really slow compared to snacks.nvim. fzf-lua is another great alternative that I haven’t given enough attention to.

4. grug-far.nvim, global query replace

   I’ve been happy with Neovim’s regular %s/foo/bar for single files (aided by search-replace.nvim for easy population). But query replace in multiple files has always felt lacking. I used to use telescope.nvim to populate the quickfix window and then use replacer.nvim to make it editable, updating multiple files.

   It worked but was a bit annoying so now I’m trying grug-far.nvim as a more “over engineered” solution. I haven’t used it that long to say for sure but I’m hopeful.

Connecting and subscribing to changes

I found the tortoise311 library that implements an MQTT client and it was quite pleasant to use.

First we’ll start a Tortoise311.Connection in our main Supervisor tree:

{Tortoise311.Connection,

 [

   # Remember to generate a unique id if you want to connect multiple clients

   # to the same MQTT service.

   client_id: :my_unique_client_id,

   # They don't have to be on the same server.

   server: {Tortoise311.Transport.Tcp, host: "localhost", port: 1883},

   # Messages will be sent to `Haex.MqttHandler`.

   handler: {Haex.MqttHandler, []},

   # Subscribe to all events under `zigbee2mqtt`.

   subscriptions: [{"zigbee2mqtt/+", 0}]

 ]},

I’ll also add Phoenix PubSub to the Supervisor, which we’ll use to propagate MQTT messages to our automation:

{Phoenix.PubSub, name: Haex.PubSub},

When starting Tortoise311.Connection above we configured it to call the Haex.MqttHandler whenever an MQTT message we’re subscribing to is received. Here we’ll simply forward any message to our PubSub, making it easy for anyone to subscribe to any message, wherever they are:

defmodule Haex.MqttHandler do

  use Tortoise311.Handler

  alias Phoenix.PubSub




  def handle_message(topic, payload, state) do

    payload = Jason.decode!(payload)

    PubSub.broadcast!(Haex.PubSub, Enum.join(topic, "/"), {:mqtt, topic, payload})

    {:ok, state}

  end


end

Then in our automation (which in my automation system is a regular GenServer) we can subscribe to the events the Tap Dial creates:

defmodule Automation.TapDialExample do

  use GenServer

  alias Phoenix.PubSub



  @impl true


  def init(_opts) do

    # `My tap dial` is the name of the tap dial in zigbee2mqtt.

    PubSub.subscribe(Haex.PubSub, "zigbee2mqtt/My tap dial")

    {:ok, %{}}

  end



  @impl true


  def handle_info({:mqtt, _topic, payload}, state) do

    dbg(payload)

    {:noreply, state}

  end


end

If everything is setup correctly we should see messages like these when we operate the Tap Dial:

payload #=> %{

  "action" => "button_1_press_release",

  ...

}



payload #=> %{

  "action" => "dial_rotate_right_step",

  "action_direction" => "right",

  "action_time" => 15,

  "action_type" => "step",

  ...

}

Controlling devices

To change the state of a device we should send a json payload to the “set” topic. For example, to turn off a light named My hue light we should send the payload {"state": "OFF"} to zigbee2mqtt/My hue light/set.

Here’s a function to send payloads to our light:

def set(payload) do

  Tortoise311.publish(

    # Important that this id matches the `client_id`

    # we gave to Tortoise311.Connection.

    :my_unique_client_id,

    "zigbee2mqtt/My hue light/set",

    Jason.encode!(payload)

  )


end

Normal press

Here’s how we can toggle the light on/off when we click the first button on the dial in our GenServer:

def handle_info({:mqtt, _topic, %{"action" => "button_1_press_release"}}, state) do

  set(%{state: "TOGGLE"})

  {:noreply, state}


end

(Remember that we subscribed to the "zigbee2mqtt/My tap dial" topic during init.)

Hold

You can also hold a button, which generates a hold and a hold_release event. Here’s how to use them to start moving through the hues of a light when you hold down a button and stop when you release it.

def handle_info({:mqtt, _topic, %{"action" => "button_3_hold"}}, state) do

  set(%{hue_move: 40, color: %{saturation: 100}})

  {:noreply, state}


end




def handle_info({:mqtt, _topic, %{"action" => "button_3_hold_release"}}, state) do

  set(%{hue_move: 0})

  {:noreply, state}


end

Double clicking

How about double clicking?
You could track the timestamp of the presses in the GenServer state and check the duration between them to determine if it’s a double click or not; maybe something like this:

def handle_info({:mqtt, _topic, %{"action" => "button_2_press_release"}}, state) do

  double_click_limit = 350



  now = DateTime.utc_now()



  if state[:last_press] &&

       DateTime.diff(now, state[:last_press], :millisecond) < double_click_limit do

    # If we double clicked.

    set(%{color: %{hue: 60}})

    {:noreply, Map.delete(state, :last_press)}

  else

    # If we single clicked.

    set(%{color: %{hue: 180}})

    {:noreply, Map.put(state, :last_press, now)}

  end


end

This however executes an action on the first and second click. To get around that we could add a timeout for the first press by sending ourselves a delayed message, with the downside of introducing a small delay for single clicks:

def handle_info({:mqtt, _topic, %{"action" => "button_2_press_release"}}, state) do

  double_click_limit = 350



  now = DateTime.utc_now()



  if state[:last_press] &&

       DateTime.diff(now, state[:last_press], :millisecond) < double_click_limit do

    set(%{color: %{hue: 180}})



    # The double click clause is the same as before except we also remove `click_ref`

    # to signify that we've handled the interaction as a double click.

    state =

      state

      |> Map.delete(:last_press)

      |> Map.delete(:click_ref)



    {:noreply, state}

  else

    # When we first press a key we shouldn't execute it directly,

    # instead we send ourself a message to handle it later.

    # Use `make_ref` signify which press we should handle.

    ref = make_ref()

    Process.send_after(self(), {:execute_single_press, ref}, double_click_limit)



    state =

      state

      |> Map.put(:last_press, now)

      |> Map.put(:click_ref, ref)



    {:noreply, state}

  end


end



# This is the delayed handling of a single button press.


def handle_info({:execute_single_press, ref}, state) do

  # If the stored reference doesn't exist we've handled it as a double click.

  # If we press the button many times (completely mash the button) then

  # we might enter a new interaction and `click_ref` has been replaced by a new one.

  # This equality check prevents such a case, allowing us to only act on the very

  # last press.

  # This is also useful if we in the future want to add double clicks to other buttons.

  if state[:click_ref] == ref do

    set(%{color: %{hue: 60}})

    {:noreply, Map.delete(state, :click_ref)}

  else

    {:noreply, state}

  end


end

You can generalize this concept to triple presses and beyond by keeping a list of timestamps instead of the singular one we use in :last_press, but I personally haven’t found a good use-case for them.

Other types of transitions

One of the reasons I wanted a custom implementation was to be able to do other things with the rotary dial.

For example, maybe I’d like to alter the hue of light by rotating? All we have to do is set the hue instead of the brightness:

set(%{hue_step: step * 0.75, transition: 0.4})

(This produces a very cool effect!)

Another idea is to change the volume by rotating. Here’s the code that I use to control the volume of our kitchen speakers (via Home Assistant, not MQTT):

rotate: fn step ->

  # A step of 8 translates to a volume increase of 3%

  volume_step = step / 8 * 3 / 100



  # Clamp volume to not accidentally set a very loud or silent volume.

  volume =

    # HAStates stores the current states in memory whenever a state is changed.

    (HAStates.get_attribute(kitchen_player_ha(), :volume_level, 0.2) + volume_step)

    |> Math.clamp(0.05, 0.6)



  # Calls the `media_player.volume_set` action.

  MediaPlayer.set_volume(kitchen_player_ha(), volume)

  # Prevents a possible race condition where we use the old volume level

  # stored in memory for the next rotation.

  HAStates.override_attribute(kitchen_player_ha(), :volume_level, volume)


end

Lights to control

These are the lights we can control in the room:

• A ceiling light with color ambiance
• A window light with white ambiance
• A floor lamp with white ambiance
• Night lights for both Loke and Isidor, with color ambiance
• A lava lamp for Loke, connected to a smart plug

(Yes, I need to get a lava light for Isidor too. They’re awesome!)

The window light isn’t controlled by the tap dial and there are other automations that controls circadian lighting for most of the lights.

Use Zigbee direct binding

I’m opting to use direct binding because of two reasons:

1. Direct binding allows us to dim the light even if the smart home server is down.
2. Despite my efforts, the dimming automation has some latency issues.

Even though it overrides the hold functionality I think direct binding for lights is the way to go.

The functionality

These are the functions for the tap dial in the boys room:

• Rotate: Dim brightness of ceiling light (direct binding)
• Hold any: Turns off the ceiling light (direct binding)
• Click 1: Toggle ceiling light on/off
• Double click 1: Toggle max brightness for ceiling light on/off
• Click 2: Each click goes through different colors for the ceiling light
• Click 3: Toggle floor lamp on/off
• Double click 3: Toggle Isidor’s night light on/off
• Hold 3: Loop through the hue of Isidor’s night light
• Click 4: Toggle Loke’s lava lamp on/off
• Double click 4: Toggle Loke’s night light on/off
• Hold 4: Loop through the hue of Loke’s night light

There’s many different ways you can design the interactions and I may switch it up in the future, but for now this works well.

Installation

The installation was straightforward and worked without a hitch:

1. Plug in the phone or tablet via an USB
2. Launch Chrome (or chromium)
3. Follow the instructions and click some buttons

App compatibility

Apps are also just as easy to install as on stock Android. I’ve installed most of the apps from the Play Store just as I would’ve on stock and they work just fine.

So far I’ve had a grand total of two issues with any apps I’ve tried:

1. At first I couldn’t copy BankID over from my old phone.

   I had to tweak some permissions and disable some of the location privacy features of GrapheneOS before the phones recognized each other. Presumably there’s some security measure there so that you can only copy it if the phones are nearby.

   

2. The AI detection feature of MacroFactor refused to work.

   MacroFactor is a food tracking app where you can take a photo and it’ll try to infer the food from the picture but uploading the photo simply failed. It’s a pretty cool feature and I instead use the app Cronometer that has the same functionality.

   MacroFactor uses Play Integrity which may in some cases break certain apps. I’ve got a few other apps that also uses Play Integrity but they don’t have any issues.

I was worried that I’d run into issues with the banking apps as I know there are some banking apps that have issues, but all the Swedish banking apps I’ve tried work well.

You can of course install apps from other sources such as F-Droid, Accrescent, Obtanium, or manually as well.

No bloatware

Finally, I love that there’s no bloatware bullshit on GrapheneOS. There are no shitty vendor specific apps that you cannot uninstall and it isn’t trying to trick you into installing stupid games via dark patterns. The downside is that GrapheneOS doesn’t come with a lot of customization and lets you install apps for that yourself.

In short, it feels like with GrapheneOS you’re in control, not some mega corporation.

How to set it up

1. Install Music Assistant

   I host my homelab things using docker compose and it was as simple as:

   
   services:
   
     music-assistant-server:
   
       image: ghcr.io/music-assistant/server:latest
   
       container_name: music-assistant-server
   
       restart: unless-stopped
   
       # Network mode must be set to host for MA to work correctly
   
   
       network_mode: host
   
       volumes:
   
         - ./music-assistant-server/data:/data/
   
       # privileged caps (and security-opt) needed to mount
   
   
       # smb folders within the container
   
   
       cap_add:
   
         - SYS_ADMIN
   
         - DAC_READ_SEARCH
   
       security_opt:
   
         - apparmor:unconfined
   
       environment:
   
         - LOG_LEVEL=info
   
   
   
     # And home assistant and other things.
   
   
   
   

2. Add providers

   A provider is a source of music. There’s a bunch of them but at the moment I only use a few:

   

   The Spotify provider for example should automatically sync all Spotify playlists into Music Assistant and allows you to search and play any song on Spotify.

3. Add players

   We need players to play our music, here’s what I currently use:

   

   The players should be automatically added as long as they have a matching provider enabled.

Open-source music management—particularly on Linux—has a reputation of being notoriously troublesome. But I’ve gotta say, Music Assistant was simple to setup and it works well (except issues with some players that I’ll get to shortly).

Sonos

It’s pretty funny, the Sonos speaker works better with Music Assistant than with the Sonos app. The radio completely stopped working via the Sonos app, while I can use Music Assistant to play the radio on the Sonos speaker.

The speaker might still disconnect or stop playback at odd times but it’s good enough to raise the mood in the washing room.

Squeezelite on Linux

I hate modern smart TVs with a passion so to stream we use a computer running Linux, connected to a dumb amplifier with some speakers. It works well but it makes it a bit more cumbersome to play music.

By installing Squeezelite the computer acts as a squeezebox client, effectively transforming it into a smart player for Music Assistant and Home Assistant to stream music to.

A HifiBerry player

As I had a Raspberry Pi lying around it made sense to try the HifiBerry AMP2 that transforms the Pi into a surprisingly capable amplifier and smart music controller. For simplicity I decided to start with their operating system HifiBerryOS that hopefully should “just work” and be open enough for me to manually fix things if needed.

While it works very well as a Spotify Connect device or to play over Bluetooth I had issues with it acting as a Squeezelite or Airplay client as the volume was super low when I tried to stream to it from Music Assistant.

It might be a software issue with HifiBerryOS but I haven’t had the energy to debug it or try other OS versions. Maybe I’ll revisit it when I want to outfit another room with speakers.

Arylic A50+

I was planning to use the HifiBerry to power two new speakers in the ceiling in the kitchen to replace the Sonos. But I got stressed out by the HifiBerry not working properly and the kitchen renovation was looming ever closer so I bought the Arylic A50+, hoping that it would work better with Music Assistant.

The device supports both Airplay and Squeezebox like the HifiBerry but again there were some issue with the volume being very low. I don’t know if both devices just happen to have similar bugs, if there’s a bug with Music Assistant, or if it’s some weird compatibility issue.

Sigh.

But there’s another way to make it work; the Arylic has an excellent Home Assistant integration and if you go that way then Music Assistant can use the Arylic as a player properly. You need the LinkPlay integration in Home Assistant and then enable the Home Assistant integration in Music Assistant.

(I tried the same with the HifiBerry integration but for some reason the HifiBerry media player exposed from Home Assistant didn’t show up in Music Assistant.)

In Home Assistant how do you…

• Play the P4 Norrbotten radio station?

  
  action: music_assistant.play_media
  
  
  target:
  
    entity_id: media_player.kitchen
  
  
  data:
  
    media_id: P4 Norrbotten
  

• Play the To Hell and Back track?

  
  action: music_assistant.play_media
  
  
  target:
  
    entity_id: media_player.kitchen
  
  
  data:
  
    media_id: To Hell and Back
  
    media_type: track
  

• Play the kpop playlist?

  
  action: music_assistant.play_media
  
  
  target:
  
    entity_id: media_player.kitchen
  
  
  data:
  
    media_id: kpop
  
    media_type: playlist
  

• Play the kpop playlist randomized?

  
  action: media_player.shuffle_set
  
  
  target:
  
    entity_id:
  
      - media_player.kitchen
  
  
  data:
  
    shuffle: true
  
  
  
  
  action: music_assistant.play_media
  
  
  target:
  
    entity_id: media_player.kitchen
  
  
  data:
  
    media_id: kpop
  
    media_type: playlist
  
    enqueue: replace
  

You get the new music_assistant.play_media action but otherwise you control the media players just as the other media player entities in Home Assistant.

What made me change my mind?

When I quit Whoop I was in a pretty good place in my life; I was feeling good, I was training consistently, and I didn’t need external stimuli to keep going.

But things have been different recently. I’ve been struggling with depression, haven’t been able to get back to a regular training routine, and I feel that I need all help I can get to get back on track.

During the 10 months without Whoop I relied on my Garmin watch and I realized that the watch simply wasn’t helping me to improve my fitness or health the way I wanted from a smart device.

I looked at alternatives but in the end I couldn’t find an alternative that matched Whoop’s feature set, so here I am.

Battery life is excellent

Some people don’t mind charging their devices a few times a week but I personally loath it. With the new Whoop and its battery life of ~2 weeks I can finally leave the battery pack when I go on my one week work trips and it’s super nice.

The app UI is great

Maybe it’s “just” pretty UI but I find Whoop’s presentation much more helpful than the rawer presentation that Garmin has.

For example I’ve always liked to look at sleep information ever since I started wearing an Oura ring and Whoop’s (recently redesigned) presentation is very good as it focuses on actionable metrics:

I also think Whoop is immediately more useful than Garmin’s when you open it up:

I have some gripes with the UI—I would like to be able to customize the home screen more for example—but overall I think Whoop’s app is a lot better designed and more useful than any alternative I’ve tried.

Whoop in the boxers

As I mentioned before, Whoop still thinks that I’m sleeping when I sit with Whoop in the boxers. Either before I go to bed, after I go to bed, and sometimes it thinks I’m napping when I’m sitting during the day.

Very annoying.

But they’re still a killer feature for me because it’s the best way to track Submission Grappling. Whoop is less accurate than the Polar bicep/chest straps but a biceps band or a chest strap sometimes gets in the way during training, while I’ve never noticed the Whoop in my boxers.

The Whoop 5.0 / MG also fits the 4.0 boxers well—lucky me as I have a bunch of the older 4.0 boxers. I’ve gotta admit, I like their boxers and I kept using them even after I canceled my subscription.

I was skeptical to their new design with the “pods” but I think they’re an improvement as it’s a lot easier to add/remove the device while wearing the boxers, and if you remove the pod they’re just like regular boxers. The holders on my old 4.0 boxers have started to peel away and started to chafe but I think there’s less of a risk with the new design.

ECG

The ECG feature is why I went with Whoop MG instead of Whoop 5.0.

I’ve had a couple of episodes with chest pain where I’ve hurried to the hospital to get a check-up. They never showed anything out of the ordinary (it was ruled as “something muscular”) but the experience has left me worried.

If the ECG and background “Irregular Heart Rhytm” detection makes me relax a little (or if they do detect something) then I figured it’s worth the price jump from the 5.0 to the MG.

Whoop age

I’ve seen Reddit warriors call the Whoop age metric a “gimmick” and dismiss it as just a combination of metrics available elsewhere.

But for me—a middle-aged programmer dad who’s struggling to exercise consistently—the Whoop age metric is a fantastic addition to Whoop as it gives me a pretty and actionable way to improve my overall health.

I already know that I need to lose weight and exercise more, but sometimes you need someone else to hit you with the reality before you internalize the problem and start doing something about it.

It’s too early to tell how useful the Whoop age metric ultimately ends up being but my first impression is very positive.

More measures

Whoop also comes with more “raw” measures that I glance at from time to time:

• Body weight and body composition

  Whoop now syncs with my Withings smart scale and I do need to lose weight; it’s nice to have it in the Whoop app so I don’t have to open up Withing’s app just to see my weight trend.

• Steps

  Again, I have (more accurate) step tracking in Garmin but it’s nice to have it all in one app. I try to hit a reasonable step goal and step counting—even if inaccurate—has made me try to move around more.

• VO₂ MAX

  It seems like an important metric but I’m not sure about it’s accuracy and it hasn’t caused me to change any behaviours. (I also need to calibrate it with a 15 minutes run. I haven’t been on a run the last decade…)

• Blood pressure

  I calibrated the blood pressure insights but I’m not sure how useful it is for me. Maybe it can act as an early indicator that my health is deteriorating but so far it’s just a pretty statistic.

Journaling and insights

The journal that correlates your behaviours with your recovery scores is an amazing feature and in theory this feature alone could carry a device such as Whoop by itself…

If it wasn’t for the fact that the usefulness tapers off hard as soon as you learn how the various behaviours affect your body. At the end of my last subscription I even turned off the journal as it didn’t provide any new information and filling it in only became a chore.

Right now I use it but I only track a handful of behaviours that I want to remind myself of. However, I’ll probably turn off the journal in a few months when the newness wears off again.

Bands

It was a really shitty move by Whoop to break backwards compatibility with the old bands. Granted, I don’t care that much about appearance so I never bought a lot of bands but I still want to switch between bands when they get wet after shower.

Where corporations fails the 3D printer community steps up to save the day. You can 3D print a whoop 5.0 / MG adapter to continue using your existing bands:

It’s not as secure as a proper 5.0/MG band and the old 4.0 battery doesn’t fit but it works well enough for regular usage (I don’t dare to swim with it).

I’ll probably buy another band anyway so I have another ECG-compatible clasp; possibly a colorful SportFlex band for when I swim with the kids.

Pricing

Whoop outdid themselves and the MG took something already expensive and made it even more expensive. While I don’t regret going with the more expensive option, the “Peak” plan (without Blood pressure and ECG) probably makes more sense for most people.

Still, if Whoop can help me improve my fitness or health—even in a small way—the cost is worth it for me.