My personal environment sync

[2021-10-30 Sat] on Yann Esposito's blog

I have a quite specific system that I improved along the years to manage my local environment. Think about, binaries I expect to have in my shell, as well as configuration files for various utilities, and a few personal scripts.

The notion of what is exactly my local environment is not perfectly defined. I expect every of my computers to behave slightly differently. Some are for work-only, some personal use only.

For the things I want everywhere, I have a peculiar personal system.

I use a personal script that depends on yadm and home-manager.

My script try to check if some files where updated and react accordingly:

  1. I download the dot-files changes via yadm.
  2. If my home-manager files changes, it will run home-manager switch ; if it fails, try to update nix channels then try again.
  3. If my doom emacs packages changed, it will run doom sync
  4. If the script itself changed, it re-run the script after updating itself.

If the script detect that I changed my emacs configuration, it runs doom sync or doom sync -u.

Here it is:

#!/bin/bash

### logs fn helpers

## colors for tput
# black=0
red=1
green=2
yellow=3
blue=4
# magenta=5
# cyan=6
# white=7
highpr() {
    printf "$(tput setaf $green)→$(tput sgr0) $(tput bold)%-60s$(tput sgr0)" "$*"
}
ok() {
    local txt="OK"
    echo -e " [$(tput bold)$(tput setaf $green)${txt}$(tput sgr0)]" >&2
}
info() {
    echo -e " [$(tput bold)$(tput setaf $blue)$*$(tput sgr0)]" >&2
}
warn() {
    echo -e "$(tput bold)$(tput setaf $yellow)$*$(tput sgr0)" >&2
}
err() {
    echo -e "$(tput bold)$(tput setaf $red)$*$(tput sgr0)" >&2
}
fail() {
    err -e "\n[ERR] $*"
    exit 1
}

highpr "check nix"
if ! [ -x "$(command -v nix)" ]; then
    echo
    err "nix does not seem to be installed."
    err "Install it from: https://nixos.org/nix/"
    exit 1
fi
ok

highpr "yadm fetch"
yadm fetch --quiet  || fail "yadm fetch failed"
ok

# check the hash of a few files before doing yadm pull
OLD_SYNC_ENV_ID=$(yadm rev-parse HEAD:bin/sync-env.sh)
OLD_HOME_MANAGER_ID=$(yadm rev-parse HEAD:.config/nixpkgs/home.nix)
OLD_DOOM_PACKAGES=$(yadm rev-parse HEAD:.doom.d/packages.el)
OLD_DOOM_INIT=$(yadm rev-parse HEAD:.doom.d/init.el)

highpr "yadm pull"
yadm pull --quiet || fail "yadm pull failed"
ok

# check the hash of a few files after doing yadm pull
NEW_SYNC_ENV_ID=$(yadm rev-parse HEAD:bin/sync-env.sh)
NEW_HOME_MANAGER_ID=$(yadm rev-parse HEAD:.config/nixpkgs/home.nix)
NEW_DOOM_PACKAGES=$(yadm rev-parse HEAD:.doom.d/packages.el)
NEW_DOOM_INIT=$(yadm rev-parse HEAD:.doom.d/init.el)

highpr "check sync-env diff"
if ! [ "$OLD_SYNC_ENV_ID" = "$NEW_SYNC_ENV_ID" ]; then
    warn " changed"
    warn " Starting ~/bin/sync-env.sh again"
    echo
    ~/bin/sync-env.sh
    exit $?
fi
ok

if [ -f "$HOME/.yadm/files.gpg" ]; then
  highpr "yadm decrypt"
  yadm decrypt || fail "yadm decrypt failed"
  ok
fi

highpr "home-manager"
USERNAME_NIX_FILE="$HOME/.config/nixpkgs/username.nix"
if [ ! -f  "$USERNAME_NIX_FILE" ]; then
  echo "\"$USER\"" >> "$USERNAME_NIX_FILE"
fi
if ! [ "$OLD_HOME_MANAGER_ID" = "$NEW_HOME_MANAGER_ID" ]; then
    echo
    highpr "home-manager switch"
    home-manager switch || \
      ( nix-channel --update && home-manager switch ) || \
      fail "home-manager switch failed"
    ok
else
    info "skipped"
fi

highpr "doom-emacs"
doompath="$HOME/.emacs.d/bin/doom"
if ! [ "$OLD_DOOM_PACKAGES" = "$NEW_DOOM_PACKAGES" ] || \
   ! [ "$OLD_DOOM_INIT"     = "$NEW_DOOM_INIT"  ]; then
  if [[ -x $doompath ]]; then
      echo
      highpr "doom sync"
      $doompath sync || fail "doom failed to sync"
      ok
  else
     fail "Cannot find doom executable at $doompath";
  fi
else
   info "skipped"
fi

Bootstrapping

Bootstrapping this system is always a nice problem to think about. It is smooth when everything is set but to bootstrap it I need binaries installed by this system… So… How to handle the dependency cycle correctly?

To minimize the pain, I removed more and more bootstrapping dependencies. Now my almost single dependence for bootstrapping my environment is nix. I haven't initialized any machine for a long time now. The following should work.

  1. Use fish1 chsh /bin/fish

  2. Install nix curl -L https://nixos.org/nix/install | sh

  3. Install home-manager

    nix-channel --add https://github.com/nix-community/home-manager/archive/release-21.05.tar.gz home-manager
nix-channel --update
export NIX_PATH=$HOME/.nix-defexpr/channels${NIX_PATH:+:}$NIX_PATH
nix-shell '<home-manager>' -A install

  4. Install and use yadm

    nix-shell -p yadm
yadm boostrap
yadm remote set-url origin <url-to-my-dot-files-repo>
yadm pull

  5. Still in the nix-shell with yadm run ~/bin/sync-env.sh

There is a risk that step 3 fail because I pin most of my packages in home-manager configuration, and it will try to install yadm. This can conflict with the yadm installed in the current nix-shell. So sometime I need to:

  1. Remove the line installing yadm in my home-manager configuration first
  2. run home-manager sync
  3. get out of the nix-shell,
  4. add yadm back in the home-manager config
  5. run home-manager sync again, but this time out of the nix-shell.
  6. Finally I can run my ~/bin/sync-env.sh command.

So this post will probably be useful as a personal note in the future. Because bootstrapping is generally not trivial. I will probably update this post if something is missing.

  1. I use fish for interactive shell. I use zsh for quick dirty scripts (a lot better than bash), and I switch to turtle if I need to be serious about the script.↩︎