Migrating my systems from org-babel to chezmoi

In the GNU/Linux world, it is common to store configuration files in a repository which can be shared among different systems. This technique becomes a lot more difficult when throwing a Windows system into the mix as Windows is quite different and less configurable.

I’ve been using Windows since about 1995 (MS-DOS before this) and GNU/Linux since 1999/2000, and most my problems with sharing configuration stem from Windows being extremely unfriendly towards developers and tinkerers. You’re expected to use Windows the way Microsoft has decided, not tailor it to work like you want – you’re not the owner of your computing, Microsoft is.

I still haven’t found a valid reason for using Windows other than “it’s forced upon me by contractors”, but that’s a different discussion – well, it does have more (and often better) consumer hardware support, so I guess there can be other reasons you’re forced to use it…

Windows has a long history of trying to have better “Unix support” (Microsoft POSIX Subsystem, Microsoft Services for Unix, Windows Subsystem for Linux), but only WSL exists today, and there are just too many seams and broken windows. Cygwin and MSYS2 should also be mentioned, and I use MSYS2 every day. Using tools written for Unix-like operating systems is necessary for my own productivity and well-being, but is neither simple nor easy.

WSL1 isn’t using a Linux Kernel, so not everything works as expected, and it also means spawning processes is as slow as on Windows. WSL2 works a lot better in this regard, but communicating with the host filesystem becomes very slow as it copies back and forth.

Symbolic Links requires administrator privileges (or Developer Mode). And for some reason, various tools doesn’t seem to work well with them.

NTFS has absolute abysmal performance for workflows containing many small files, and tools written for Unix-like operating systems often use a lot of files because of the “everything is a file” philosophy.

This makes some tools and workflows created for Unix-like operating systems unbearably slow. Operations which take 0.01 second on GNU/Linux might take many seconds on Windows. Operations which just flows without any noticeable delay on GNU/Linux becomes a huge bottleneck on Windows. Working effectively with git is a good example of operations which is a lot slower on Windows even though there has been put a lot of effort into making this faster.

Microsoft announced Dev Drive (built on top of the new Resilient File System (ReFS)) around a year ago, but I just recently started using it. As is customary in the Windows world, it’s really difficult to do something like replacing your home folder with a Dev Drive. Together with symlink issues, I gave up trying to integrate Dev Drive into my home folder and rather started hacking my setup to point to d:/ on Windows and ~ on GNU/Linux.

Corporations also injects the malware called Windows Defender, which Microsoft helpfully puts in a new “performance mode”. (But if possible, kill it with fire instead!)

The performance is a lot better, and I can highly recommend it! It is still only a fraction of the performance of GNU/Linux, but a whole lot better.

After having to hack on my system setup, I became frustrated on the poor tooling. Because of aforementioned problems with Windows, I settled on using org-mode and org-babel some years back. This gave me a lot of flexibility on what I wrote to each system and where I wrote it. The changes required were minimal; replacing ~ with a custom function home instead which expands to my “dev home” folder, i.e. d:/ on Windows.

But operating on a 7000+ line org-mode file with many blocks isn’t the best experience. org-edit-special helps, but it’s modal, requiring me to jump in and out of the context all the time. org-babel-detangle is great for propagating changes back from the tangled files, but I haven’t been able to make it work with noweb content (i.e. writing changes back to the original named black).

I am still tangling 56 blocks from my system setup file, so I’m not trying to migrate everything, and I will continue to use org-babel for notebooks and other work, just not everything in my system.

I am sure people are able to make org-babel work perfectly, but my setup just gives too much friction here. … and just opening the file is way too slow on Windows – as always.

I would have loved to migrate to Guix Home. I use Guix System on my laptop, my server, and WSL, but because WSL does have some integration issues with Windows, it’s not really an option. I’m also pretty sure it would be an uphill battle trying to make it work towards the Windows host. There’s probably wslu utilities which could help with the integration, but just the thought of it makes me shiver.

GNU Stow is one of the best known solutions for managing dotfiles, but it’s heavily built around symlinks, so I don’t dare to use it anywhere near NTFS gives my experience with symlinks on NTFS.

Which finally leads me to chezmoi:

Manage your dotfiles across multiple diverse machines, securely.

chezmoi helps you manage your personal configuration files (dotfiles, like ~/.gitconfig) across multiple machines.

It uses copy/paste rather than symlinks, and works natively on Windows too. It’s way too bloated for my needs, and I don’t like the configuration, but it seems to work as announced.

I cloned my old dotfiles repository into ~/.local/share/chezmoi, copied over many of the tangled files, deleted the sections from my system org file, added the files and pushed.

Back on Windows I cloned the repository, and applied it both on the regular home folder using chezmoi apply, and into my Dev Drive, chezmoi apply --destination d: (or -D for short). This means I have to juggle two destination directories, but I don’t think that will pose any real issue.

After changing files, I run chezmoi diff -D d: which gives me changed files. chezmoi add the-file -D d: copies the changes back to the chezmoi git folder. Some small emacs functions can make this an OK workflow.

The migration took an evening, and it worked fine on the first try on Windows. I doubt this is the final destination, but I hope pulling out larger parts of my setup helps a bit with development ergonomics.

I would have liked some software even better suited my needs, but I liked that it worked out-of-the-box.

Don’t use Windows.