Why do some packages behave differently when installed using home.nix vs. running in nix-shell

Help
1

There are a few packages that behave differently, but I’ll take the home-manager itself as an example. When it’s installed as a package, by setting home.packages = [ pkgs.home-manager ] in home.nix, I get the following error:

$ home-manager switch
/nix/store/uifh238f9wf97mw7ef7as88g8g23f-home-manager-generation
Starting Home Manager activation
Activating checkLinkTargets
...
Activating remove_the_nix_channel_executable

/tmp/home-manager-build.Bof923hoasf/generation/activate:
line 340: out: unbound variable

But when I run exactly the same commannd in nix-shell -p home-manager, everything works as expected, there’s no error.

I mean, home-manager can be configured as a program, by setting programs.home-manager.enable = true, but a lot of packages can’t, and I’d like to have them in home.nix, but they only work in nix-shell.

2

Your channels must be in a very weird state, or you’re trying inexpert ways of pinning stuff. Something must be quite broken to get different behavior here.

I mean, frankly, something must be quite broken to get that output. Looks like you’re running a snippet from a derivation build script at switch time?!?

if you check where each home-manager comes from with readlink $(command -v home-manager) inside and outside the shell we might have a clue here. If they differ, share your channels and configuration.

3

Yeah, I have a weird set up, but that’s not relevant to the question. I observe the same behavior regardless.

But since you asked, I killed channels and even removed the nix-channel executable to prevent a possibility of them rising from the dead. I get nixpkgs from npins.

4

I mean, there are reports about other packages as well:

https://www.reddit.com/r/NixOS/comments/10ae4rm/why_does_nixshell_p_behave_differently_than/

5

nix-shell is for development, HM config is not. So differences in behavior due to differing envvars and linked paths is somewhat expected.

home-manager itself should work fine though; in fact, the only thing that programs.home-manager.enable = true; would do is add hm to home.packages:

So the only difference really is that hm is getting pulled from different sources (nixpkgs when you use pkgs.home-manager vs h-m upstream itself when you use the .enable option).

If you need more assistance, we need more details on how you retrieve nixpkgs.

6

Ah, of course, home-manager is an edge case o\

Nonetheless, other packages should almost always be exactly the same. While there are small differences in the environment, for most things this does not make a difference.

The thread you link has other folks chiming in that show the person who made it is the only one with that issue; my educated guess is that they likely have a very broken shell configuration and nix-shell accidentally fixes it by writing to PATH after the shell config finished overwriting PATH. Both nix-env and NixOS/home-manager can’t do much about a broken shell config, since their PATH is set before the shell configuration runs (or, more precisely, during the shell configuration, after which whatever that OP did probably overrides it).

The difference in behavior observed in your case goes well beyond environment issues like this. If you want help figuring out what’s wrong, please share your configuration, including how your npins work.

7

Ok, here’s the wrapper-home-manager.sh I run instead of of home-manager itself, it takes nixpkgs from npins:

#!/usr/bin/env bash

npins_channel="nixpkgs-unstable"
npins_channel_in_quotes="\"$npins_channel\""      # just in case there's a dot in the name

nixpkgsPath=$(nix-instantiate \
                  --read-write-mode \
                  --eval --expr "(import ./npins).$npins_channel_in_quotes.outPath" \
                  | \
                  tr -d \"      # should be jq -r, but it's an extra dependency
           )      # an easy hack: nix-instantiate --eval has no raw mode yet

# The home.nix is almost unchanged, it's imported in the wrapper.
# We only move imports of overlays and config from it.
homeWrapperPath=$(realpath -- "$(dirname -- "${BASH_SOURCE[0]}")/wrapper-home.hix")

exec home-manager "$@" -I nixpkgs="$nixpkgsPath" -f "$homeWrapperPath"

And here’s the wrapper-home.nix used above, which propagates nixpkgs further to home.nix:

{ lib, pkgs, config, ... }:
let
  nixPath_lst =
    [
      "nixpkgs=${pkgs.path}"
      "nixpkgs-overlays=${toString ./nixpkgs-overlays.nix}"
      # "nixos-config=${toString ./wrapper-configuration.nix}"      # Commented out, because we don't use nixos yet.
    ];
  nixPath = lib.concatStringsSep ":" nixPath_lst;
in
{
  # If it were nixos, we would set nix.nixPath, but it doesn't exist in home-manager.
  # But nix.nixPath is only used for setting $NIX_PATH anyway, so we can do this manually, see https://github.com/nix-community/home-manager/blob/8cf9cb2ee78aa129e5b8220135a511a2be254c0c/modules/misc/nix.nix#L280
  home.sessionVariables.NIX_PATH = "${nixPath}";

  home.sessionVariables.NIXPKGS_CONFIG = lib.mkForce (toString ./nixpkgs-config.nix);

  # The only parts that were moved from the main home.nix, because we have to set these vars here.
  nixpkgs = {
    config =   import ./nixpkgs-config.nix;
    overlays = import ./nixpkgs-overlays.nix;
  };              # Documentation for nixpkgs.config and nixpkgs.overlays: https://github.com/nix-community/home-manager/pull/230/commits/5ab57f4a873ba333579eddf62f9002d4825b2e35

  home.activation.remove_the_nix_channel_executable = lib.hm.dag.entryAfter ["writeBoundary"] ''
    run rm --force $out/bin/nix-channel
  '';

  # Our main configuraton file, almost unchanged, we only moved overlays and nixpkgs.config here.
  imports = [
    ./home.nix
  ];
}
8

Yeah, so my question is: how to make a package behave the same way in home-manager? I personally didn’t expect this difference.

Any kind of manual configuration for some packages required in home.nix? What should I look for and where?

9

If you’re talking about dev packages (libraries, compilers, interpreters) they simply do not go in home.packages. They only go in dev shells or package expressions.

https://nix.dev/tutorials/first-steps/declarative-shell#declarative-reproducible-envs

Other packages should be the same overall assuming the same nixpkgs input.

10

I understand this. But the line is blurry. What if I want an interpreter, which I use daily for my personal things, installed by home-manager?

11

You’re doing a couple of pretty crazy things there, I’m not surprised it’s borked.

Firstly, that wrapper script is inadvisable. home-manager explicitly supports changing the pkgs arg, you should be doing this:

# home.nix
let
  sources = import ./npins;
  pkgs = import sources.nixpkgs {};
in
{
  _module.args = {
    inherit pkgs;
  };
}

Secondly, using the activation script to remove a binary from $PATH isn’t very clean. It means that that deletion happens after nix finishes evaluating, as part of a big bash script. It’s unfortunate when you have to rely on this, because it means that home-manager cannot abort before the switch if you’ve made a mistake, and you potentially end up with a partial activation (consider this foreshadowing).

It’d be better to build a custom nix package with buildEnv or symlinkJoin and remove the nix-channel in the postBuild script of that, then set nix.package to that.

Consider your error message:

line 340: out: unbound variable

This snippet is almost certainly also the source of the different behavior you see. Presumably $out isn’t set in the activation environment of whatever version of home-manager you’re using. I believe it should not be set in fact, I’m curious which versions of home-manager would have it in there - it seems like a bug that your configuration doesn’t always fail.

Perhaps it does always fail and the fact that there is no change between generations just means the activation script isn’t run every other time? Or maybe using nix-shell ends up with a spuriously set $out, which results in a silently ignored failure to delete a file. You could confirm the latter by running echo $out in your nix-shell, or by using unset $out before running home-manager in there.

And finally, nix.nixPath does exist - you link to it yourself in that comment. Presumably you’re using an ancient home-manager module set since you haven’t been able to update since trying to delete nix-channel. Even if it didn’t, nix.settings.nix-path is likely nicer.

2 Likes
12

Then you add it to home.packages, and it will be in a directory somewhere in $PATH.

The only difference in behavior is that you will not be able to link against it if it also provides shared libraries as part of the package - this should not affect daily use as an interpreter, though.

Unless you’ve also made home-manager write a .bashrc which overrides $PATH or written a broken activation script which makes it never get installed, of course.

13

Exactly what nix.channel.enable = false; does btw (in NixOS config).

1 Like
14

Sadly not: nixpkgs/nixos/modules/config/nix-channel.nix at 5630cf13cceac06cefe9fc607e8dfa8fb342dde3 · NixOS/nixpkgs · GitHub

This might be where the misunderstanding of home-manager’s activation script containing $out ultimately comes from.

Maybe that --force should be removed from the command and replaced with an appropriate chmod or such, clearly it swallows errors.

1 Like
15

Thanks, this would simplify my configuration.

Yeah, when I do echo $out outside of nix-shell, it’s empty, when I do this in nix-shell -p home-manager, the $out var is /nix/store/3b6syck08n2lw7fyqa1i5ln6rfwdzgm0-shell.

Update: I’m wrong here, please see my next comment.
I believe this has nothing to do with the file removal, because it gets removed with rm -f.

It’s the oldest 25.05-pre inside and outside of the nix-shell.

An interpreter is an example. The better example would be the rr debugger. I can’t use it when it’s installed by home-manager, but it works fine within nix-shell. I understand that it’s probably better to have it in the latter, but I’m curious at this point how to make it work in hm.

16

Oh. Ok, my bad. $out is empty in home-manager, so it tries to remove /bin/nix-channel. I commented out this line and switch works fine. Thanks for pointing this out.

17

That’s my point :slight_smile: In the activation script using an unset variable results in an error, but deleting a nonexistent file does not. There is the source of the difference in behavior.

Why does nix-shell set $out? Well, if you compare the env in and out of nix-shell, you will see quite a lot of differences.

nix-shell drops you into the environment nix would use to build something. This is useful and in fact the real point of nix-shell. Setting $out is part of that, though mostly incidental.

But if you look at the difference in those environments, you’ll probably understand why we say it should not affect behavior - none of those variables should reasonably affect runtime behavior of any reasonable application beyond the effects that are intended by nix-shell (ability to link against libraries, binaries being added to $PATH, etc.). It would be impossible to create a nix-shell without setting some env variables, and it being a generic tool means it needs to be a bit more liberal.

This is also probably why rr doesn’t work for you. It most likely relies on being able to introspect some shared libraries which are not installed by home.packages, since they are only used for development purposes. nix-shell however adds appropriate environment variables to make them discoverable.

3 Likes
18

Thanks for fixing my set up. I couldn’t connect the dots here.

1 Like
19

Could you take a glance at this? This is rr not working, that is installed by home-manager:

$ rr record ./target/debug/guessing_game 
rr: Saving execution to trace directory `/home/alexander/.local/share/rr/guessing_game-26'.
optimization.
./target/debug/guessing_game: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.36' not found (required by /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libpthread.so.0)
./target/debug/guessing_game: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /nix/store/rmy663w9p7xb202rcln4jjzmvivznmz8-glibc-2.40-66/lib/libpthread.so.0)

It works fine in nix-shell.

Update: the guessing_game itself works fine in both.

20

Yep, it’s trying to introspect the glibc library used by one of the libraries in the application you’re trying to debug.

Since you’re not on NixOS, it then tries to pick up glibc from the global interpreter. This interpreter then goes and finds a glibc available on your system - unfortunately it’s older than what your nix-built guessing_game was built with, and you get an error.

nix-shell adds some variables to make the correct glibc available for discovery, so rr works correctly.