Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move devicePkgs to the pkgs.nvidia-jetpack package-set and clear up... #206

Merged
merged 1 commit into from
Jun 17, 2024

Conversation

jmbaur
Copy link
Contributor

@jmbaur jmbaur commented Apr 17, 2024

...ambiguities with systems compatible for flash/fuse scripts

Description of changes

Instead of maintaining a separate package-set location (outside of pkgs), we can just use regular overlays to apply our changes that are device-specific (stored under the pkgs.nvidia-jetpack scope). An alias is added to the old locations (config.system.build.jetsonDevicePkgs, config.hardware.nvidia-jetpack.devicePkgs, etc.) with a warning to indicate that users should just use pkgs.nvidia-jetpack.

Also included is clearing up the ambiguities of what systems are compatible with flash/fuse scripts. NVIDIA makes the decision for us as to what platforms we can run these tools on (x86_64-linux only), so we shouldn't allow for any flash/fuse derivations to be built for aarch64-linux.

The rundown:

  • a jetson device nixos config's hostPlatform must be aarch64-linux
  • a flash/fuse script's hostPlatform must be x86_64-linux

What we were doing before was mixing package-sets willy-nilly without much control over which hostPlatform we were dealing with (leading to lots of usage of hardcoded pkgsAarch64 to force a package-set to aarch64). This change makes a logical separation of what needs to be built with an aarch64 hostPlatform package-set vs an x86_64 hostPlatform package-set.

Testing

Tested building and running flash script and initrd flash script for an orin-agx-devkit

@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch from 227f421 to a005de7 Compare April 17, 2024 21:42
@danielfullmer
Copy link
Collaborator

Can you try building a NixOS system with autoUpdate enabled? That would at least run through a lot of the weird codepaths that do crazy stuff like running the flash script under qemu to get the signedFirmware on aarch64-linux.

Also, do you expect this PR to not change flash/fuse script and NixOS system hashes?

@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch 2 times, most recently from e755577 to 607503a Compare April 22, 2024 21:55
@jmbaur
Copy link
Contributor Author

jmbaur commented Apr 22, 2024

Just pushed a change that does allow for autoUpdate to be enabled without doing any reimport of nixpkgs.

@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch from 607503a to c7a4a4c Compare April 23, 2024 01:27
@jmbaur
Copy link
Contributor Author

jmbaur commented Apr 23, 2024

I do not expect the output hashes to be the same, although the nix-diff between the two do not show many differences. For example, here's the nix-diff for the flake output legacyPackages.x86_64-linux.flash-orin-agx-devkit on master vs this PR (the actual diff comes from the fact that we are now doing the fetches of sources like l4t, atf, etc with the devices pkgs, not the x86_64-linux pkgs that we use to construct the flash script):

$ nix-diff ./result-{master,pr}
- /nix/store/km0a4dvbmm8549v4z2z52yyw54w63l8w-flash-orin-agx-devkit:{out}
+ /nix/store/5j53lkavvh7sqp9w083yq9lwnzlpwmc0-flash-orin-agx-devkit:{out}
• The input derivation named `flash-tools-35.4.1` differs
  - /nix/store/k9zbpmsz3flfakby59wa7mcp6kj48qb0-flash-tools-35.4.1.drv:{out}
  + /nix/store/inhr4shiq8x546afs636gf39f1vgr76m-flash-tools-35.4.1.drv:{out}
  • The input derivation named `l4t-unpacked` differs
    - /nix/store/dc5chkiib0v7ndmiid42jfmvp9yk1lvg-l4t-unpacked.drv:{out}
    + /nix/store/d39cds5nkxqwh1ikhjfcgm64d2jnqh84-l4t-unpacked.drv:{out}
    • The input derivation named `Jetson_Linux_R35.4.1_aarch64.tbz2` differs
      - /nix/store/s5lhz8j4fm3rp4azx7v342wdiwfdkkwd-Jetson_Linux_R35.4.1_aarch64.tbz2.drv:{out}
      + /nix/store/mrnvnd8h8g5nsh7h1i1xmzihrww7bbv7-Jetson_Linux_R35.4.1_aarch64.tbz2.drv:{out}
      • The environments do not match:
          - outputHashAlgo=sha256
    • Skipping environment comparison
  • Skipping environment comparison
• The input derivation named `l4t-unpacked` differs
  - /nix/store/dc5chkiib0v7ndmiid42jfmvp9yk1lvg-l4t-unpacked.drv:{out}
  + /nix/store/fm0q0frjgbw7jayb0wb2dhpggixdl66y-l4t-unpacked.drv:{out}
  • The set of input source names do not match:
      + cross-file.conf
  • The input derivation named `Jetson_Linux_R35.4.1_aarch64.tbz2` differs
    • These two derivations have already been compared
  • The input derivation named `stdenv-linux` differs
    - /nix/store/mb9hk9cqwgrgl7gyipypn2h1wfz49h4s-stdenv-linux.drv:{out}
    + /nix/store/dspwjmjllwv7br3b9p6qpkhqxph43pwg-stdenv-linux.drv:{out}
    • The set of input derivation names do not match:
        - acl-2.3.1
        - attr-2.5.1
        - binutils-2.40
        - binutils-wrapper-2.40
        - ed-1.19
        - expand-response-params
        - gcc-12.3.0
        - gcc-wrapper-12.3.0
        - glibc-2.38-27
        - gmp-6.3.0
        - gmp-with-cxx-6.3.0
        - gnu-config-2023-09-19
        - isl-0.20
        - libidn2-2.3.4
        - libmpc-1.3.1
        - libunistring-1.1
        - linux-headers-6.5
        - mpfr-4.2.1
        - pcre2-10.42
        - xgcc-12.3.0
        - zlib-1.3
    • The set of input derivations named `bzip2-1.0.8` do not match
    • The set of input derivations named `update-autotools-gnu-config-scripts-hook` do not match
        + /nix/store/91q2h9x3vckwyivbvmqmzd6qhwnrli9l-update-autotools-gnu-config-scripts-hook.drv:{out}
    • The set of input derivations named `xz-5.4.4` do not match
    • The environments do not match:
        - allowedRequisites=/nix/store/202iqv4bd7lh6f7fpy48p7q4d96lqdp7-gzip-1.13 /nix/store/202iqv4bd7lh6f7fpy48p7q4d96lqdp7-gzip-1.13 /nix/store/ik7jardq92dxw3fnz3vmlcgi9c8dwwdq-bzip2-1.0.8-bin /nix/store/155qsyx1mv11fsi48nz4dlc0vh1a3drx-bzip2-1.0.8 /nix/store/76z4cjs7jj45ixk12yy6k5z2q2djk2jb-xz-5.4.4-bin /nix/store/wzqs107xkwdxls33skg2ma9rl7j1i68f-xz-5.4.4 /nix/store/q1c2flcykgr4wwg5a6h450hxbk4ch589-bash-5.2-p15 /nix/store/q1c2flcykgr4wwg5a6h450hxbk4ch589-bash-5.2-p15 /nix/store/1fn92b0783crypjcxvdv6ycmvi27by0j-binutils-2.40 /nix/store/mnlqpncq82lw07iy6yzbb7gxz78l10cy-binutils-2.40-lib /nix/store/bblyj5b3ii8n6v4ra0nb37cmi3lf8rz9-coreutils-9.3 /nix/store/bblyj5b3ii8n6v4ra0nb37cmi3lf8rz9-coreutils-9.3 /nix/store/8q25nyfirzsng6p57yp8hsaldqqbc7dg-diffutils-3.10 /nix/store/8q25nyfirzsng6p57yp8hsaldqqbc7dg-diffutils-3.10 /nix/store/l974pi8a5yqjrjlzmg6apk0jwjv81yqw-findutils-4.9.0 /nix/store/l974pi8a5yqjrjlzmg6apk0jwjv81yqw-findutils-4.9.0 /nix/store/7wfya2k95zib8jl0jk5hnbn856sqcgfk-gawk-5.2.2 /nix/store/7wfya2k95zib8jl0jk5hnbn856sqcgfk-gawk-5.2.2 /nix/store/injawg30ws0a6ydwbsvaaa1p0ky82pjr-gmp-with-cxx-6.3.0 /nix/store/injawg30ws0a6ydwbsvaaa1p0ky82pjr-gmp-with-cxx-6.3.0 /nix/store/v4iswb5kwj33l46dyh2zqh0nkxxlr3mz-gnumake-4.4.1 /nix/store/v4iswb5kwj33l46dyh2zqh0nkxxlr3mz-gnumake-4.4.1 /nix/store/9c5qm297qnvwcf7j0gm01qrslbiqz8rs-gnused-4.9 /nix/store/9c5qm297qnvwcf7j0gm01qrslbiqz8rs-gnused-4.9 /nix/store/xpidksbd07in3nd4sjx79ybwwy81b338-gnutar-1.35 /nix/store/xpidksbd07in3nd4sjx79ybwwy81b338-gnutar-1.35 /nix/store/rx2wig5yhpbwhnqxdy4z7qivj9ln7fab-gnugrep-3.11 /nix/store/rx2wig5yhpbwhnqxdy4z7qivj9ln7fab-gnugrep-3.11 /nix/store/cbj1ph7zi009m53hxs90idl1f5i9i941-patch-2.7.6 /nix/store/cbj1ph7zi009m53hxs90idl1f5i9i941-patch-2.7.6 /nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0 /nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0 /nix/store/2ii0b9dq7gn6m54aix8kyr1kv8bqy9p9-ed-1.19 /nix/store/2ii0b9dq7gn6m54aix8kyr1kv8bqy9p9-ed-1.19 /nix/store/qmfxld7qhk8qxlkx1cm4bkplg1gh6jgj-file-5.45 /nix/store/qmfxld7qhk8qxlkx1cm4bkplg1gh6jgj-file-5.45 /nix/store/vbb1l2krfp75m5w3y2h52l0pbq6fydz8-attr-2.5.1 /nix/store/mmy4fc1arb2ig642chgprgms27vk9hgn-acl-2.3.1 /nix/store/8xgb8phqmfn9h971q7dg369h647i1aa0-zlib-1.3 /nix/store/4px7vzc531y10r7mmxps1s32cbshbjr2-pcre2-10.42 /nix/store/08n25j4vxyjidjf93fyc15icxwrxm2p8-libidn2-2.3.4 /nix/store/s2f1sqfsdi4pmh23nfnrh42v17zsvi5y-libunistring-1.1 /nix/store/qn3ggz5sf3hkjs2c797xf7nan3amdxmp-glibc-2.38-27 /nix/store/mrgib0s2ayr81xv1q84xsjg8ijybalq3-glibc-2.38-27-dev /nix/store/cx01qk0qyylvkgisbwc7d3pk8sliccgh-glibc-2.38-27-bin /nix/store/am68gj0i5hdyfhagrilimq9zylf8wdwp-linux-headers-6.5 /nix/store/1alqjnr40dsk7cl15l5sn5y2zdxidc1v-binutils-wrapper-2.40 /nix/store/90h6k8ylkgn81k10190v5c9ldyjpzgl9-gcc-wrapper-12.3.0 /nix/store/hf2gy3km07d5m0p1lwmja0rg9wlnmyr7-gcc-12.3.0 /nix/store/myw67gkgayf3s2mniij7zwd79lxy8v0k-gcc-12.3.0-lib /nix/store/4dr83a4wn25zkb5pvsq2y3q3v586jzs9-expand-response-params /nix/store/7vq9zymw7zk2i0k1i4nfkl6hmlx3gxji-gcc-12.3.0-libgcc /nix/store/lmidwx4id2q87f4z9aj79xwb03gsmq5j-xgcc-12.3.0-libgcc /nix/store/9vk5fqaik9zx14zpfxipg7lxkdk8c5g9-update-autotools-gnu-config-scripts-hook /nix/store/c54kyc3vn0jbq5hznimvx6wz15jxw8jq-gnu-config-2023-09-19 /nix/store/7fsdrvpwxnfb8wrb9c6wa1hzw71gn4yz-gmp-6.3.0 /nix/store/3ysf58a4zshpi654ya4hsfbhvmnnjrqf-libmpc-1.3.1 /nix/store/w09nf1k5cwkdcs13317nl3bbmdcxgz9m-mpfr-4.2.1 /nix/store/v1nmbk9bzlxdgfa4cjlsxrpxqabn0bjb-isl-0.20 /nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0 /nix/store/9vk5fqaik9zx14zpfxipg7lxkdk8c5g9-update-autotools-gnu-config-scripts-hook /nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh /nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh /nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh /nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh /nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh /nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh /nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh /nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh /nix/store/wzdsbnv2ba3nj91aql8jjdddfmkkdh7h-patch-shebangs.sh /nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh /nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh /nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh /nix/store/wmknncrif06fqxa16hpdldhixk95nds0-strip.sh
        defaultNativeBuildInputs=''
        /nix/store/85jldj870vzcl72yz03labc93bwvqayx-patchelf-0.15.0 /nix/store/9vk5fqaik9zx14zpfxipg7lxkdk8c5g9-update-autotools-gnu-config-scripts-hook /nix/store/d719idzsh62kclpbjkzmfq8jigrq1dgz-update-autotools-gnu-config-scripts-hook /nix/store/h9lc1dpi14z7is86ffhl3ld569138595-audit-tmpdir.sh /nix/store/m54bmrhj6fqz8nds5zcj97w9s9bckc9v-compress-man-pages.sh /nix/store/wgrbkkaldkrlrni33ccvm3b6vbxzb656-make-symlinks-relative.sh /nix/store/5yzw0vhkyszf2d179m0qfkgxmp5wjjx4-move-docs.sh /nix/store/fyaryjvghbkpfnsyw97hb3lyb37s1pd6-move-lib64.sh /nix/store/kd4xwxjpjxi71jkm6ka0np72if9rm3y0-move-sbin.sh /nix/store/pag6l61paj1dc9sv15l7bm5c17xn5kyk-move-systemd-user-units.sh /nix/store/jivxp510zxakaaic7qkrb7v1dd2rdbw9-multiple-outputs.sh /nix/store/wzdsbnv2ba3nj91aql8jjdddfmkkdh7h-patch-shebangs.sh /nix/store/cickvswrvann041nqxb0rxilc46svw1n-prune-libtool-files.sh /nix/store/xyff06pkhki3qy1ls77w10s0v79c9il0-reproducible-builds.sh /nix/store/ngg1cv31c8c7bcm2n8ww4g06nq7s4zhm-set-source-date-epoch-to-latest.sh /nix/store/wmknncrif06fqxa16hpdldhixk95nds0-strip.sh
    ''
  • Skipping environment comparison
• The input derivation named `linux-aarch64-unknown-linux-gnu-5.10.120` differs
  - /nix/store/h280wn9p6pppadnbhzj7zaxj5zl6g3rc-linux-aarch64-unknown-linux-gnu-5.10.120.drv:{out}
  + /nix/store/cqlmbvwzdc040yzzvw7pmsr6fxy3n711-linux-aarch64-unknown-linux-gnu-5.10.120.drv:{out}
  • The input derivation named `linux-config-aarch64-unknown-linux-gnu-5.10.120` differs
    - /nix/store/l0vcb0svrakiqb2rznnrr5g2l9sh2q47-linux-config-aarch64-unknown-linux-gnu-5.10.120.drv:{out}
    + /nix/store/w3dkhsmz2z038yk59ms56avxdqgpn1bn-linux-config-aarch64-unknown-linux-gnu-5.10.120.drv:{out}
    • The input derivation named `nvidia-l4t-xusb-firmware-35.4.1-20230801124926` differs
      - /nix/store/h0w27pmr2q09f469b62p60wpxbbyw585-nvidia-l4t-xusb-firmware-35.4.1-20230801124926.drv:{out}
      + /nix/store/10sswdzdzl0zma85iwsyvkw1gfldgmls-nvidia-l4t-xusb-firmware-35.4.1-20230801124926.drv:{out}
      • The set of input source names do not match:
          + cross-file.conf
      • The input derivation named `stdenv-linux` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • The input derivation named `source-patched` differs
      - /nix/store/0wydwxhzynljcr5fbpzcm7dn2cm1gmn6-source-patched.drv:{out}
      + /nix/store/h5hc50fwxm4jcddma4sg37mix7s7sp7i-source-patched.drv:{out}
      • The set of input source names do not match:
          + cross-file.conf
      • The input derivation named `stdenv-linux` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • Skipping environment comparison
  • The input derivation named `source-patched` differs
    • These two derivations have already been compared
  • Skipping environment comparison
• The input derivation named `tos.img` differs
  - /nix/store/5b73hpz54gak2a092n6h62fjdbzz0lz1-tos.img.drv:{out}
  + /nix/store/r2i327x9x2gkax0p4apz7wg5ba81spvc-tos.img.drv:{out}
  • The input derivation named `arm-trusted-firmware-aarch64-unknown-linux-gnu-35.4.1` differs
    - /nix/store/gj749sy6wklpiihvf85vggn6rq5h8mil-arm-trusted-firmware-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
    + /nix/store/f646gxgxsxg8x6gfkh4fck051v0xa6pl-arm-trusted-firmware-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
    • The input derivation named `atf-794affd` differs
      - /nix/store/akkv8qa9bfqh374m7qiza1w0pqaazhdp-atf-794affd.drv:{out}
      + /nix/store/22hbcgcbk28ap930qnrj4hfg5k8fvd91-atf-794affd.drv:{out}
      • The set of input source names do not match:
          + cross-file.conf
      • The input derivation named `stdenv-linux` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • Skipping environment comparison
  • The input derivation named `l4t-unpacked` differs
    • These two derivations have already been compared
  • The input derivation named `optee-os-aarch64-unknown-linux-gnu-35.4.1` differs
    - /nix/store/m5ssj7a0acvdlfjak1ldwxw3in3biy4f-optee-os-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
    + /nix/store/28yj6ax5wjsj5avabnrg0230zk1cakw2-optee-os-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
    • The input derivation named `hwkey-agent-aarch64-unknown-linux-gnu-35.4.1` differs
      - /nix/store/82v0f7hh11p88ffmbclf3givqim0cbxz-hwkey-agent-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
      + /nix/store/c56f8xq561j74sizhsa10kcdmrdhv8wq-hwkey-agent-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
      • The input derivation named `nv-optee-3002010` differs
        - /nix/store/y4j92m4ykn0fsafb0y6f5zmarjrqfsnz-nv-optee-3002010.drv:{out}
        + /nix/store/svlfqzcwq6i7xz4b6fi1as7sysn4znki-nv-optee-3002010.drv:{out}
        • The set of input source names do not match:
            + cross-file.conf
        • The input derivation named `stdenv-linux` differs
          • These two derivations have already been compared
        • Skipping environment comparison
      • The input derivation named `optee-ta-dev-kit-aarch64-unknown-linux-gnu-35.4.1` differs
        - /nix/store/iz22868dsgmbmw14xxgmjs5dsszyr0sm-optee-ta-dev-kit-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
        + /nix/store/gmb323gggvnfxx6xayj02rs2spnfb5l2-optee-ta-dev-kit-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
        • The input derivation named `l4t-unpacked` differs
          • These two derivations have already been compared
        • The input derivation named `nv-optee-3002010` differs
          • These two derivations have already been compared
        • Skipping environment comparison
      • The input derivation named `optee_client-aarch64-unknown-linux-gnu-35.4.1` differs
        - /nix/store/289ndw83lflnccfviag6x1qh76xg3xdp-optee_client-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
        + /nix/store/hmi8nxg2425khkq2svma3yhj3w0mlq14-optee_client-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
        • The input derivation named `nv-optee-3002010` differs
          • These two derivations have already been compared
        • Skipping environment comparison
      • Skipping environment comparison
    • The input derivation named `l4t-unpacked` differs
      • These two derivations have already been compared
    • The input derivation named `nv-optee-3002010` differs
      • These two derivations have already been compared
    • The input derivation named `nvluks-srv-aarch64-unknown-linux-gnu-35.4.1` differs
      - /nix/store/5zwb1bfssvdisiqsbnp0ics7rpm2is1w-nvluks-srv-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
      + /nix/store/hz0r1pl0isxpa7msyz62lym20p09dy2m-nvluks-srv-aarch64-unknown-linux-gnu-35.4.1.drv:{out}
      • The input derivation named `nv-optee-3002010` differs
        • These two derivations have already been compared
      • The input derivation named `optee-ta-dev-kit-aarch64-unknown-linux-gnu-35.4.1` differs
        • These two derivations have already been compared
      • The input derivation named `optee_client-aarch64-unknown-linux-gnu-35.4.1` differs
        • These two derivations have already been compared
      • Skipping environment comparison
    • Skipping environment comparison
  • The input derivation named `tegra-234-optee.dtb` differs
    - /nix/store/kwd4g709s18d5yv0i4saj06gf2y8xhxl-tegra-234-optee.dtb.drv:{out}
    + /nix/store/kwjkalzjm0rmhv0dsi9kcgra6svwr8ac-tegra-234-optee.dtb.drv:{out}
    • The input derivation named `nv-optee-3002010` differs
      • These two derivations have already been compared
    • Skipping environment comparison
  • Skipping environment comparison
• Skipping environment comparison

@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch 8 times, most recently from 236b427 to 882ff14 Compare April 29, 2024 23:07
@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch from 882ff14 to 9f54678 Compare May 7, 2024 23:00
@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch 2 times, most recently from ebd2266 to f1ed411 Compare May 15, 2024 17:47
device-pkgs/default.nix Show resolved Hide resolved
flake.nix Show resolved Hide resolved
modules/default.nix Show resolved Hide resolved
modules/default.nix Show resolved Hide resolved
Copy link
Contributor

@jonringer jonringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I don't oppose anything in here. But I am largely glancing over the jetson specifics as that's outside my wheelhouse.

@danielfullmer
Copy link
Collaborator

danielfullmer commented Jun 15, 2024

There are two major components to this PR that by being combined make this difficult to both review and want to merge simultaneously.

Instead of maintaining a separate package-set location (outside of pkgs), we can just use regular overlays to apply our changes that are device-specific

I'm still skeptical that this is an approach we want to take. I'd be happy to provide an option to apply an overlay to the device packages set, if that helps, but it doesn't seem right to have the overall package set depend on the NixOS configuration to that extent. If anything, I'd like to have the flashing scripts, firmware, etc be able to be built without evaling an entire NixOS system. Given the goals of UEFI platform firmware, I'd like to allow people to build and flash platform firmware from this repo and then install not-NixOS on it. (e.g. Ubuntu/Redhat, etc)

clearing up the ambiguities of what systems are compatible with flash/fuse scripts

This part is uncontroversial and I'd be happy to merge, assuming the weird stuff we do with qemu on aarch64 to make signedFirmware still work. If you open a PR that does just this part I think it'll be easier for me to merge.

modules/default.nix Outdated Show resolved Hide resolved
ambiguities with systems compatible for flash/fuse scripts

Instead of maintaining a separate package-set location (outside of
`pkgs`), we can just use regular overlays to apply our changes that are
device-specific (stored under the `pkgs.nvidia-jetpack` scope). An alias
is added to the old locations (`config.system.build.jetsonDevicePkgs`,
`config.hardware.nvidia-jetpack.devicePkgs`, etc.) with a warning to
indicate that users should just use `pkgs.nvidia-jetpack`.

Also included is clearing up the ambiguities of what systems are
compatible with flash/fuse scripts. NVIDIA makes the decision for us as
to what platforms we can run these tools on (x86_64-linux only), so we
shouldn't allow for any flash/fuse derivations to be built for
aarch64-linux.

The rundown:
- a jetson device nixos config's `hostPlatform` _must_ be `aarch64-linux`
- a flash/fuse script's `hostPlatform` _must_ be `x86_64-linux`

What we were doing before was mixing package-sets willy-nilly without
much control over which hostPlatform we were dealing with (leading to
lots of usage of hardcoded `pkgsAarch64` to force a package-set to
aarch64). This change makes a logical separation of what needs to be
built with an aarch64 hostPlatform package-set vs an x86_64 hostPlatform
package-set.
@jmbaur jmbaur force-pushed the device-pkgs-to-overlay branch from f1ed411 to 29e6cea Compare June 16, 2024 03:12
@jmbaur
Copy link
Contributor Author

jmbaur commented Jun 16, 2024

I'm still skeptical that this is an approach we want to take. I'd be happy to provide an option to apply an overlay to the device packages set, if that helps, but it doesn't seem right to have the overall package set depend on the NixOS configuration to that extent. If anything, I'd like to have the flashing scripts, firmware, etc be able to be built without evaling an entire NixOS system. Given the goals of UEFI platform firmware, I'd like to allow people to build and flash platform firmware from this repo and then install not-NixOS on it. (e.g. Ubuntu/Redhat, etc)

Why would we not want to reuse the option that already exists for such behavior (nixpkgs.overlays)? The firmware is able to be built independent of the nixos system: nix build .#nixosConfigurations.<machine>._module.args.pkgs.nvidia-jetpack.uefi-firmware

edit: In the example of building the EDK2 firmware, doing it anyway that isn't similar to the way I just mentioned ends up being troublesome for a few different reasons:

  1. You don't get all the ways you've configured EDK2 (in the nixos config) applied to something you build in some other fashion
  2. The package-set used to build the firmware might be different than the package set the nixos config uses (e.g. cross compiling for a standalone build vs native compiling for the firmware built with the toplevel closure...that's probably the most common issue given how it's done right now)
  3. You will likely end up with different builds of EDK2, meaning that the machine may behave differently over the lifecycle of an installation (e.g. a user flashes some firmware they built out-of-band of their config that behaves one way, then the machine receives a capsule update that was built with the system config that behaves another way)

Of course the way I mentioned above is not convenient, the attribute path is quite long, but this can of course be made more convenient by dropping it under config.system.build or some other conventional place.

@danielfullmer
Copy link
Collaborator

While I'd still like to not apply nixpkgs overlays dependent on NixOS config, and not require evaling NixOS to create flash scripts, I did a prototype implementation over the weekend of restructuring this in the way I would prefer, and at least on initial benchmarks, my proposed implementation did not noticeably improve eval time compared to this PR. With that in mind, I think we can proceed with this in the meantime, as it is a correctness improvement over the current Nix code since it uses a pkgs import for the flasher machine which should always match the correct architecture.

@danielfullmer danielfullmer merged commit 1a54d38 into anduril:master Jun 17, 2024
1 check passed
@jmbaur jmbaur deleted the device-pkgs-to-overlay branch June 17, 2024 21:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants