Pull commentary into separate documentation

HACKING.md

@@ -0,0 +1,51 @@
+# Internals/design notes
+
+## Why [profiles](https://github.com/doomemacs/doomemacs/tree/master/profiles)?
+
+Because the profile loader runs early enough we can set `doom-profile-data-dir`
+(where the generated profile is stored and loaded from) outside `DOOMLOCALDIR`
+relatively cleanly. Not using the "global" profile is a largely unintended side
+effect, but the changes to other paths made seem largely reasonable so for now
+I'm sticking with it.
+
+After the profile loader, the next point we would get control is `doom-start.el`
+loading `init.el` from `doom-user-dir`. We currently point `doom-user-dir` into
+the store (see below): setting `doom-profile-data-dir` and `doom-profile-dir`
+from there (by prepending to the user's `init.el`) would probably also work.
+This seems a bit questionable because `doom-profile-dir` is set with `defconst`:
+its const-ness is not enforced but I'd prefer not to take advantage of that. I
+also have not checked if writing the profile would work out of the box with this
+approach.
+
+## Why put `doom-user-dir`/`DOOMDIR` in the Nix store?
+
+Doom forces my hand. I would prefer for just `packages.el` and possibly
+`init.el` to live in the store, but splitting out where those are loaded from
+looks non-trivial.
+
+Doom uses `doom-user-dir` as the path to a special module (`:user`).
+
+At profile generation time, this is how `packages.el` gets loaded. We want our
+generated `packages.el` to take effect, so we want the `:user` module's path to
+be a store path when generating the profile.
+
+Paths to all modules are embedded in the generated profile and used to
+initialize `doom-modules` at runtime. Among other things, this controls where
+the user's `config.el` is loaded from (it's loaded along with module `config.el`
+files). So (without further hacks) the path to `packages.el` at build time and
+`config.el` at runtime are the same.
+
+(And even if that wasn't the case, functions like `doom/help-packages` load
+`packages.el`, and I currently expect less overall confusion if that loads our
+generated `packages.el`, not the original one. So I do think we want the `:user`
+module loaded from the store.)
+
+In several places, Doom assumes (at runtime) that `doom-user-dir` and the path
+to the `:user` module are the same. This is mostly in functions like
+`doom/help-packages` and `doom-module-from-path` that map paths back to modules.
+
+Combine all that and I think consistently having `doom-user-dir` and the `:user`
+module live in the Nix store is the least bad option.
+
+This does break things that write to DOOMDIR at runtime. `custom-file` is an
+obvious example, but there are probably a few more.

doom.nix

@@ -41,10 +41,6 @@ let
     ++ optional (lib.pathExists doomInitFile) { name = "init.el"; path = doomInitFile; }
     ++ optional (lib.pathExists doomPrivateModule) { name = "packages.el"; path = doomPrivateModule; }
   );
-  # Set DOOMLOCALDIR somewhere harmless to stop Doom from trying to create it
-  # somewhere read-only.
-
-  # (If this step breaks, add DEBUG=1 to make Doom more verbose.)
 
   # XXX this may need to be runCommandLocal just in case conditionals an init.el
   # / packages.el evaluate differently on build systems.
@@ -53,7 +49,11 @@ let
       env = {
         EMACS = lib.getExe emacs;
         DOOMDIR = stage1DoomDir;
+        # Enable this to troubleshoot failures at this step.
+        #DEBUG = "1";
       };
+      # We set DOOMLOCALDIR somewhere harmless below to stop Doom from trying to
+      # create it somewhere read-only.
     } ''
     mkdir $out
     export DOOMLOCALDIR=$(mktemp -d)
@@ -218,17 +218,7 @@ let
   # the set from step 2.
   emacsWithPackages = doomEmacsPackages.emacsWithPackages (epkgs: (map (p: epkgs.${p}) (builtins.attrNames doomPackageSet)));
 
-  # Step 4: build a final DOOMDIR with packages.el from Step 1.
-  #
-  # This is used in three contexts:
-  # - To build the Doom profile. So we need our cli commands.
-  # - When loading that profile, as the path to the :user module.
-  #   This path is hardcoded into the profile.
-  # - As doom-user-dir at runtime.
-  #
-  # I would prefer to avoid that last one, but Doom uses doom-user-dir and the
-  # path to the :user module interchangeably in several places. Attempting to
-  # split them looks not just confusing for the user but error-prone.
+  # Step 4: build a final DOOMDIR with packages.el from step 1.
   #
   # TODO: symlink farm instead of copy?
   finalDoomDir = runCommand "doom-dir" {} ''
@@ -296,7 +286,7 @@ let
   # But during normal startup it suppresses packages.el's auto-activation,
   # which means elpa/*/*-autoloads.el don't load.
 
-
+  # Step 6: write wrappers to start the whole thing.
   pkg = runCommand "doom" {
     nativeBuildInputs = [ makeBinaryWrapper ];
   }