Arch Linux User Repository

@rushaur: yes I guess you're right. i rarely install from git pkgs, and forgot that's how it goes.

EDIT: turns out my system was woefully out of date. upgrading solved everything

@callmejoe: That is the thing with git packages, they can break anytime. I would suggest to check the grub source repo for commits that deal with fixing build errors and rebuild when new stuff is committed.

having trouble building. getting a lot of configure.ac warnings&errors.

https://imgur.com/HpTcFta.png

any ideas? thanks

Hi, I modified air-g4p's script and it works great with btrfs and background image. Thanks.

Also give attention, if you change your Passphrase that it will have pbkdf2 again sudo cryptsetup luksChangeKey --hash sha512 --pbkdf=pbkdf2 --pbkdf-force-iterations=500000 /dev/sdb2

The Background lays next to the BOOTX64.EFI file (unencrypted)

#!/bin/bash

CONFIG=$(mktemp /tmp/grub-config.XXXXX) 
cat >"$CONFIG" <<EOF

insmod all_video
set gfxmode=auto
terminal_input console
terminal_output gfxterm

background_image $root/EFI/BOOT/background.jpg

cryptomount -u a7b02c3563e14a60bed8bf8f934ed89a 

set root=crypto0
set prefix=(crypto0)/@/boot/grub

insmod normal
normal
EOF

grub-mkimage \
    -p '/boot/grub' \
    -O x86_64-efi \
    -c "$CONFIG" \
    -o /tmp/image \
    luks2 lvm fat all_video png jpeg gfxterm gfxmenu gfxterm_background btrfs part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha256 gcry_sha512
rm "$CONFIG"

cp /tmp/image /efi/EFI/BOOT/BOOTX64.efi

Does anyone know how to include a keyfile unlocking on the early boot passphrase? Since the cryptomount -u UUID doesn't include a keyfile option, I'm stuck.....

I am trying to setup lvm on luks2 with boot inside lvm.

NAME                 FSTYPE       FSVER      FSAVAIL    FSUSE%  MOUNTPOINT
nvme0n1
├─nvme0n1p1          vfat         FAT32      510.7M     0%      /mnt/efi
└─nvme0n1p2          crypto_LUKS  2
  └─cryptlvm         LVM2_member  LVM2 001
    ├─ArchNVMe-swap  swap         1                             [SWAP]
    ├─ArchNVMe-root  ext4         1.0        27G        8%      /mnt
    └─ArchNVMe-home  ext4         1.0        395.5G     0%      /mnt/home

cryptomount works, and ls in grub rescue shows all the volumes, but it can't identify their filesystem (error: unknown filesystem), including ArchNVMe-root and nvme0n1p2. On wiki it says that it can happen if BIOS boot partition outside of the first 2TiB. But I didn't create BIOS boot partition because it also says that UEFI systems don't need one. Anyone has seen errors like that? Thanks in advance.

EDIT: I have tried with BIOS boot partition, it didn't change anything, still getting that error.

EDIT2: The issue was that I didn't install ext2 module

@rushaur - You're welcome, but thank you for testing the with and without 'crypto modules' cases, both without the modified grub-mkimage script!

No surprises here, but now we know for a for a fact that having the correct grub-install --modules="...." statement AND a correct grub-mkimage script (both adapted for each user's system) are mandatory for successful LUKS2 /boot unlocking!

Hopefully, this will save others wasting time speculating and to immediately begin efficiently implementing the correct grub 2.06 LUKS2 encrypted /boot upgrade procedure as documented below.

Cheers!

Anybody used this with ZFS (ZFS on LUKS), with /boot in the pool or outside the pool?

@air-g4p: I tested in a VM and here my results so far:
grub-install without --modules=".." and without auto unlock script --> grub rescue.

grub-install with --modules=".." and without auto unlock script --> grub rescue.

grub-install with --modules=".." and with auto unlock script --> success!

So, seems both are required for grub 2.06-rc to successfully unlock a LUKS2 container. Thanks for your reply and your effort!

@rushaur: I am not sure, because I have not tested grub-install under grub 2.06 without adding the modules.

You are correct that 2.06 does not yet support Argon2. In fact, a grub developer told me, today, he is actively working on this problem, but that Argon2 support will not become available until a subsequent version is released.

I do KNOW the modules="...." were required under grub-git - and the cryptographic modules I listed (specific for my system) were very likely also required for successful grub 2.06 installation, thereby enabling support for grub's subsequent encrypted LUKS2 /boot (Keyslot 1) unlocking.

As you may know, once grub unlocks Keyslot 1 (encrypted /boot), initramfs and the kernel then unlock Keyslot 0 (your LUKS2 encrypted / and any underlying LVs).

If you want to answer your own query, please have a go and document your results for the benefit of others.

Cheers

@air-g4p: With grub 2.06 is it really required to include/specify the modules to unlock a LUKS2 container? If grub 2.06 "supports" LUKS2, doesn't this "support" include auto detection of the modules? I thought, the only thing that is not yet supported is argon2. I might be confusing something :-)

As a heads up to all who are interested in native grub LUKS2 automated encrypted /boot, /, and swap unlocking:

grub 2:2.06rc1-1 - is now available from the Arch TESTING repo - and 2.06 DOES support native LUKS2 unlocking. I know that because I am using it to boot from both my LUKS2 laptops.

If that is the package you want, this is the correct upgrade process:

A. Replace grub-git with grub (2.06). This will overwrite your existing /etc/default/grub, so you might want to make a backup, first.

B. Reinstall grub, depending on your cryptsetup options and / filesystem choice, with something like:

grub-install --target=x86_64-efi --efi-directory=/efi --modules="luks2 part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512 btrfs" --bootloader-id=<some-ID>

C. For those desiring to automate their LUKS2 GRUB encrypted /boot unlocking process, Patrick Steinhardt (of grub-dev) was kind enough to develop and share with me a generic grub-mkimage unlocking script, which obviously needs to be modified in accordance with the specifics of your system.

The following script includes the modifications I made to unlock my system with grub 2.06, while remaining consistent with my prior system setup comments, which are now a few pages back within these grub-git comments.

#!/bin/bash

CONFIG=$(mktemp /tmp/grub-config.XXXXX) 
cat >"$CONFIG" <<EOF
cryptomount -u XYZ 

#(Where XYZ=the UUID of your Arch encrypted / partition, in my case:  /dev/nvmen0n1p21).#  

#Also note, unlike the previous iteration of grub-git, this UUID string must NOT contain ANY hyphens ('-')!!#

set prefix=(lvm/ArchNVMe-root)/boot/grub
set root=lvm/ArchNVMe-root

insmod normal
normal
EOF

grub-mkimage \
    -p '(lvm/ArchNVMe-root)/boot/grub' \
    -O x86_64-efi \
    -c "$CONFIG" \
    -o /tmp/image \
    luks2 lvm btrfs part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512

rm "$CONFIG"

D. Save your correctly modified script to a file. I call mine luks2.sh.

E. Run:

./luks2.sh

F. Ensure your /etc/default/grub is correct.

G. We need to overwrite our existing grubx64.efi payload with the image created by our luks2.sh script. Run something like:

cp /tmp/image /efi/EFI/<your bootloader-id>/grubx64.efi

H. Generate and write your final grub configuration with:

grub-mkconfig -o /boot/grub/grub.cfg

I. Finally, run:

reboot

Cheers, and enjoy native grub LUKS2 automated encrypted /boot, /, and swap unlocking!!!

The section of util/grub-mkconfig.in that the add-GRUB_COLOR_variables.patch references has shifted up a few lines in recent commits. It now starts at line 214 instead of 218. This is causing the build to fail.

Edit: Fixed patch here: https://github.com/Dylan1496/aur-pkgbuilds/blob/master/add-GRUB_COLOR_variables.patch Note, it appears by default os-prober is disabled. Another patch will probably be needed to fix that.

For the other space cadets out there...

cryptomount will only detect your luks2 partition UUID if there is no dashes in the identifier otherwise you will be greeted with no such cryptodisk found.

Thanks for making this accessible everybody.

@ceri This is due to grub-install not correctly configuring your grub EFI-Image. It seems to be one of the limitations of upstream LUKS2 support. I ran into the same issue and then followed air-g4p's comment to add the right early config. I created a file /boot/grub/init.cfg with the following contents:

cryptomount -u <uuid-of-luks2-partition>
set root='lvmid/<vg-uuid>/<lv-uuid>'
set prefix=($root)/grub

insmod normal
normal

Where <vg-uuid> and <vl-uuid> point to the lvm volume that contains your /boot directory (use vgdisplay and lvdisplay for lookup). Then i ran grub-mkimage with all the required modules and added the config with -c /boot/grub/init.cfg.

I'm using luks2 with pbkdf2 keys for encrypted /boot and I'm having difficultly with the grub EFI.

It complains/lies "no such cryptodisk found" with the UUID of my boot partition. If I manually unlock it, it shows the same UUID (with hyphens), and the main grub loader starts:

cryptomount (hd2,gpt2)

insmod normal

normal

This is the command I'm using to build my EFI stub

sudo grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub_new --recheck

Any ideas?

Here is my patch to use 4096-byte sectors with LUKS2

diff --git a/grub-core/disk/cryptodisk.c b/grub-core/disk/cryptodisk.c
index 473c93976..4016ed7ba 100644
--- a/grub-core/disk/cryptodisk.c
+++ b/grub-core/disk/cryptodisk.c
@@ -237,6 +237,8 @@ grub_cryptodisk_endecrypt (struct grub_cryptodisk *dev,
     return (do_encrypt ? grub_crypto_ecb_encrypt (dev->cipher, data, data, len)
        : grub_crypto_ecb_decrypt (dev->cipher, data, data, len));

+  sector <<= (dev->log_sector_size - 9);
+
   for (i = 0; i < len; i += (1U << dev->log_sector_size))
     {
       grub_size_t sz = ((dev->cipher->cipher->blocksize
@@ -391,7 +393,7 @@ grub_cryptodisk_endecrypt (struct grub_cryptodisk *dev,
    default:
      return GPG_ERR_NOT_IMPLEMENTED;
    }
-      sector++;
+      sector += 1 << (dev->log_sector_size - 9);
     }
   return GPG_ERR_NO_ERROR;
 }
diff --git a/grub-core/disk/luks2.c b/grub-core/disk/luks2.c
index d96764a02..8ec4ed9f5 100644
--- a/grub-core/disk/luks2.c
+++ b/grub-core/disk/luks2.c
@@ -498,7 +498,10 @@ luks2_decrypt_key (grub_uint8_t *out_key,
       goto err;
     }

+  int original_log_sector_size = crypt->log_sector_size;
+  crypt->log_sector_size = 9;
   gcry_ret = grub_cryptodisk_decrypt (crypt, split_key, k->area.size, 0);
+  crypt->log_sector_size = original_log_sector_size;
   if (gcry_ret)
     {
       ret = grub_crypto_gcry_error (gcry_ret);

I finally could boot from a LUKS2 encrypted root (not converted). If someone is interested, here the steps With help from @air-g4p comments: By the way thank you!

1- As usual, partition/format the disk. I had only two partitions:
sda1 ------> /boot/efi
sda2 ------> /

2- Encryption:

modprobe dm-crypt
cryptsetup -y --verbose --cipher aes-xts-plain64 --key-size 512 --hash sha512 --pbkdf pbkdf2 --type luks2 luksFormat /dev/sda2
__mapper__=/dev/mapper/luks-"$(cryptsetup luksUUID /dev/sda2)"

AFAIK --allow-discards --persistent are LUKS2 only:

cryptsetup --allow-discards --persistent open /dev/sda2 luks-"$(cryptsetup luksUUID /dev/sda2)"

3- Make the filesystem and mount it to /mnt:

mkfs.btrfs "${__mapper__}"
mount -o noatime,compress=zstd,ssd,space_cache=v2 "${__mapper__}" /mnt

Steps to create/mount subvolumes skipped here to keep things short

mkfs.fat -F 32 /dev/sda1
mount /dev/sda1 /mnt/boot/efi

4- Install the new system: the usual pacstrap /mnt pkg1 pkg2 pkgn...

pacman -r /mnt -U /path/to/grub-git.pkg.tar.zst

5- Tweaks: Replace xxxxx with the uuid of your root: You can get it by running:

cryptsetup luksUUID /dev/YOUR_ROOT_PARTITION

Edit (/mnt)/etc/default/grub to reflect this:

GRUB_CMDLINE_LINUX="cryptdevice=UUID=xxxxxxx-xxxxxx-xxxxx-xxxx:mapper-name"
GRUB_ENABLE_CRYPTODISK=y

If you plan to unlock with a keyfile, luksAddKey accepts --pbkdf pbkdf2 cmdline parameter

For later reuse, let's make the script provided by @air-g4p locally available. Modified for own use case. Because I had created a subvolume @grub to be later mounted at /boot/grub; you will see the @grub notation:

cat <<'__END_SCRIPT__' >/usr/local/bin/grub-mkluks2-image
#!/bin/bash
set -e
read -rp "Enter the partition containing /, without '/dev/': " __ROOT__
cr_root_uuid="$(cryptsetup luksUUID /dev/${__ROOT__})"
CONFIG=$(mktemp /tmp/grub-config.XXXXX)
cat >"$CONFIG" <<EOF
cryptomount -u ${cr_root_uuid//-/}

set prefix='(crypto0)/@grub'
set root='(crypto0)'

insmod normal
normal
EOF

grub-mkimage \
    -p '(crypto0)/@grub' \
    -O x86_64-efi \
    -c "$CONFIG" \
    -o /boot/efi/EFI/archlinux/grubx64.efi \
    luks2 btrfs part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512

rm "$CONFIG"
__END_SCRIPT__

chmod 755 /usr/local/bin/grub-mkluks2-image

It might be useless; but at least it will create the directory structure :)

grub-install --target=x86_64-efi --modules="luks2 part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512" --efi-directory=/boot/efi --bootloader-id="archlinux" --recheck

Create the "new" bootloader which will include the modules/instructions needed to unlock our luks2 container:

grub-mkluks2-image

If all goes well, you will be hopefully greeted by grub asking for enc. passphrase :)

In case someone like me who do not know how to modify an existing LUKS2 partition with argon2 keys to make it work, here is how I did it. You may need to remove every argon2 keys to make GRUB unlocking work. (For me, those argon2 keys that uses a keyfile instead of a passphrase can be kept)

# my encrypted device is /dev/nvme0n1p3, replace with your own device
DEVICE='/dev/nvme0n1p3'
# show current LUKS header, and observe which are existing key slots
sudo cryptsetup luksDump "$DEVICE"
# add a key using pbkdf2
sudo cryptsetup luksAddKey --pbkdf pbkdf2 "$DEVICE"
# remove old keys that use argon2/argon2i, replace the 0 with your key slot
sudo cryptsetup luksKillSlot "$DEVICE" 0

@drgr33n - Noted, however, if you parse through the grub-devel archives you will see there are numerous argon2 threads.

https://lists.gnu.org/archive/html/grub-devel/2020-03/msg00106.html is but one example.

From what I can discern, the devs believe that future support for argon2 will not be all that difficult to incorporate. That being said, it appears that the window for including argon2 support in 2.06 has already closed. So, we'll just have to be patient and continue our LUKS2 grub /boot unlocking with pbkdf2 until then.

Personally, I remain thankful that we now have a documented, working LUKS2 grub /boot unlocking procedure well ahead of schedule! I boot from LUKS2 /boot on both of my laptops without issue.

Cheers

luks2 works fine as long as you use pbkdf2. I have just set it up on my new laptop and it works great. Pitty about the argon2 support though. Hopefully that will come soon enough.

If anyone desires to automate their LUKS2 GRUB encrypted /boot unlocking process, Patrick Steinhardt (of grub-dev) was kind enough to develop and share with me a generic grub-mkimage unlocking script, which obviously needs to be modified in accordance with the specifics of your system.

The following script includes the modifications I made to unlock my system, remaining consistent with my prior system setup comments.

#!/bin/bash

CONFIG=$(mktemp /tmp/grub-config.XXXXX)
cat >"$CONFIG" <<EOF
cryptomount -u XXX #(where XXX=the UUID of your Arch encrypted / partition, in my case:  /dev/sda7)# 

set prefix=(lvm/ArchSSD-root)/boot/grub
set root=lvm/ArchSSD-root

insmod normal
normal
EOF

grub-mkimage \
    -p '(lvm/ArchSSD-root)/boot/grub' \
    -O x86_64-efi \
    -c "$CONFIG" \
    -o /tmp/image \
    luks2 lvm btrfs part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512

rm "$CONFIG"

That script when executed, generates a GRUB executable image file stored at: /tmp/image.

Finally, you need to overwrite your existing grubx64.efi file. Run something similar to:

cp /tmp/image /efi/EFI/Luks2Testing/grubx64.efi

Reboot, and enjoy automated unlocking of your LUKS2 encrypted /boot, / and swap!

FINALLY! The correct procedure to unlock a LUKS2 encrypted /boot:

I have been working with the fine folks on the grub-devel mailing list. Following MANY hours of testing, I have identified a process to successfully unlock a LUKS2 encrypted /boot.

This process still requires manual intervention following reboot, but the important part is that IT WORKS!

Carefully Note: I originally encrypted my partition with: cryptsetup -c aes-xts-plain64 -h sha512 -s 512 --use-random --type luks1 luksFormat /dev/sdXZ

You may have run cryptsetup luksFormat with different options - and how you set up encryption will become important in the grub-install (Step 5, below). Adapt to your requirements.

Then I set up two LVs: swap (512M) and / (remaining partition space). That encrypted swap LV is assigned as dm-1 and encrypted / is assigned as dm-2. I happen to run dm-2 with BTRFS, but any sane filesystem should also work.

GRUB has always booted my LUKS1 encrypted: /boot, / and swap system without issue.

The process I used to successfully unlock my LUKS2 encrypted /boot:

1. UEFI boot from any reasonably recent arch iso, and run: cryptsetup convert --type luks2 /dev/sdXZ. That command will succeed, and luksDump will show PBKDF: pbkdf2 for both Keyslot 0 and 1.

2. Run cryptsetup open /dev/sdXY <something>

3. Mount everything and arch-chroot into /

4. Run mkinitcpio -P linux

5. Run grub-install --target=x86_64-efi --efi-directory=/efi --modules="luks2 part_gpt cryptodisk gcry_rijndael pbkdf2 gcry_sha512" --bootloader-id=<some-id>. That installation command completes without error, ASSUMING you are actually running grub-git! If you are running Arch GRUB (version 2.04) from the mainline repos, you WILL GET a luks2.mod not found error! Also note my use of gcry_sha512 given my cryptsetup luksFormat options, shown above.

6. Run grub-mkconfig -o /boot/grub/grub.cfg

7. Exit, umount and reboot.

8. Immediately following power on: you are greeted by the dreaded: error: disk 'lvmid/some-lengthy-UUID' not found. Entering rescue mode. That lengthy UUID is the exact UUID of my dm-2 which is my encrypted / LV.

9. At the grub rescue> prompt: type ls. There I see (proc) (hd0) and (hd0,gpt1)...(hd0,gpt7) where gpt7 is my last partition and where my encrypted / and /boot reside.

10. Still at grub rescue> type: cryptomount (hd0,gpt7) which then requires my passphrase. After CORRECT passphrase entry, and hitting Enter: You should see 'Slot 0 opened' and then you are immediately returned to the grub rescue> prompt.

11. From grub rescue> type: ls. Unlike before, you will now see something similar to: (proc) (hd0) and (hd0,gpt1)...(hd0,gpt7) where gpt7 is my last partition and where my encrypted / resides. ADDITIONALLY, you should now also see your LVs similar to: (/lvm/ArchSDD-root) and (lvm/ArchSSD-swap) depending upon your local LV naming convention decisions. This is important PROGRESS as it demonstrates that GRUB has successfully decrypted your LUKS2 encrypted /boot using your LUKS2 key from Keyslot 1!!!

12. From grub rescue> type: insmod normal

13. From grub rescue> type: normal

That should launch your typical/welcome Arch Linux and Advanced options for Arch Linux screen as controlled by /etc/default/grub and by X.

After you select the kernel to boot, GRUB hands over control to your initramfs and the corresponding kernel which uses your LUKS2 key from Keyslot 0 to decrypt your encrypted swap (dm-1) and your encrypted / (dm-2).

My launcher (with multiple Arch kernels, and several multi-booting OSes) works perfectly...hope yours does also!

Cheers!

i have not gotten this to work with luks2, however once in grub rescue you might try:

set debug=all

ls #find your hd and partition

cryptomount hd0,gpt2

I've noticed that using sha256 as the hash results in a failed to parse digest error, switching to sha1 results in an invalid passphrase error. Hope this helps... if you make a working setup please post here...

I'm having the same problem as @air-g4p

@eschwartz - grub-git still fails to boot from a LUKS2 disk.

Steps to reproduce the LUKS2 boot failure from a currently, properly booting --type luks1 encrypted /boot Arch system:

1. UEFI boot from any reasonably recent arch iso, and run: cryptsetup convert /dev/sdXY --type luks2 --pbkdf pbkdf2 That command will succeed, and luksDump will show PBKDF: pbkdf2 for both keyslot 0 and 1. However, carefully note, you get the exact same pbkdf2 luksDump results by ONLY running: cryptsetup convert /dev/sdXY --type luks2 so it may be that cryptsetup is ignoring the appended --pbkdf pbkdf2 option. Additionally, man cryptsetup does not mention the --pbkdf option under the convert command. That same --pbdkf string is a valid option under the luksConvertKey command, but that command can only modify the pbkdf parameters for an existing LUKS2 keyslot.

2. Run cryptsetup open /dev/sdXY <something>

3. mount everything and arch-chroot into /

4. Run mkinitcpio -P linux

5. Run grub-install --target=x86_64-efi --efi-directory=/efi --bootloader-id=<some-id>

6. Run grub-mkconfig -o /boot/grub/grub.cfg

7. exit, umount and reboot.

8. Then you are greeted by the dreaded: error: disk 'lvmid/some-lengthy-UUID' not found. Entering rescue mode.

Again, if anyone discovers the correct grub-git LUKS2 encrypted /boot procedure, please carefully document it here.

@eschwartz apologies for not replying sooner - I got busy at work after compiling this and never switched to Luks2. I'll make a copy of my /boot onto my NFS share this weekend and try converting the existing partition to Luks2 using pbkdf2.

And did you use cryptsetup convert with the default pbkdf which is argon2i, or did you remember to pass --pbkdf pbkdf2 ?

The former is not yet implemented in grub and obviously cannot work. The latter is a non-default option to cryptsetup, which you may have forgotten...

I still have yet to see anyone actually confirm they tried using pbkdf2 and still got failures. I still have yet to see anyone actually confirm which one they used, in fact.

@WoefulDerelict - I built and installed grub-git without issue after your latest bump in February. However, after converting my / to LUKS2, reinstalling grub and re-running grub-mkconfig, my system refused to boot due to my LUKS2 conversion. Reverting back to LUKS1, of course, fixed everything.

Builds are currently succeeding on my test machine; however, I don't use LUKS or have the time spin up and test the LUKS2 support. If any brave users are willing to experiment please report back.

I'm currently in the process of reading through the recent commits as a some of these changes appear to fix issues the PKGBUILD has been working around. Once I've got all the unnecessary workarounds trimmed out and tested the resulting build I'll push an update.

eschwartz: Thanks for all the updates.

@drossbox,

How did it turn out?

@eschwartz thanks for letting me know. Built successfully, so I'll test Luks2 support over the weekend.

As of 17 hours ago, various build fixes have landed in upstream master.

(The Argon2 KDF feature is still being reviewed, though.)

So if anyone is interested in testing out luks2 support, now might be a good time. Especially since the code freeze for grub 2.06 is being called in less than a week.

The grub issues with --enable-mm-debug should hopefully be fixed by https://lists.gnu.org/archive/html/grub-devel/2020-03/msg00090.html

@WoefulDerelict

Thank you for taking the time to provide your superb backgrounder! Your thoughts really helped me to understand grub-git's PKGBUILD and your strategy on a much deeper level.

All the best....

air-g4p: This is an odd PKGBUILD. It attempts to balance human readability and ease of maintenance while simultaneously mirroring the package in [Core]. Each PKGBUILD contains instructions to build GRUB for up to four targets before carefully packaging the result together. This allows a single package to support both BIOS and UEFI systems. The PKGBUILD used to construct the binary package in the [Core] repository builds for three targets by default, BIOS systems, 64-bit UEFI and 32-bit UEFI. The [Core] PKGBUILD doesn't enable the fourth target, GRUB EMU, by default; however, the PKGBUILD here does.

This PKGBUILD also sources two other master branches in order to build. grub-extras and gnulib are included in the build and also have the possibility to change in a way that might cause unexpected breakage. This software is also sourced by the release package in [Core]; however, the utility to change vbemodes on Intel integrated graphics is the only part of grub-extras actually included in the build performed by the PKGBUILD for the [Core] repository. This PKGBUILD only excludes the lua module from grub-extras as it has not been updated to cope with recent changes to grub and currently fails to build. When it has been patched to cope with the offending change I'll enable it again.

As the primary audience one has targeted with this PKGBUILD is Arch Linux users who are curious about testing and contributing to GRUB's development I've erred on the side of code coverage, including extra features in the build like GRUB EMU and the other modules from grub-extras.

@WoefulDerelict

I think your pinning idea makes good sense if your schedule constraints at some point allow you sufficient time.

Thank you for the link. After reading Patrick's comments re: disk: Implement support for LUKS2 @ http://git.savannah.gnu.org/cgit/grub.git/commit/?id=365e0cc3e7e44151c14dd29514c2f870b49f9755 ...

I am confident that really smart people are thinking deeply about this currently intractable problem.

WoefulDerelict, I do realize you understand and are watching upstream like a hawk, but please allow me to acknowledge the superior contributions you have made as an AUR Maintainer...@rarely found (3+ standard deviations) EXCELLENCE!

I should probably add a pin here with a generalised warning that GRUB's master branch is on the raw side and that bleeding edge features are liable to live up to their name. I'll add that to my to do list and try to include some helpful resources for things like reporting issues upstream and browsing the commits to master from the web.

air-g4p: This may be one of those features that is best to wait until it hits a release before deploying it on an important system. The next time I'm able to build a package with this PKGBUILD I'll push the updated pkgver to the web. Check out http://git.savannah.gnu.org/cgit/grub.git/log/ for current and detailed information about commits to grub's source.

@RushAur: Thanks, your breakage, like mine, should serve as a heads up to everyone else interested in Grub's future LUKS2 support capabilities that we need to wait.

@air-g4p: Couldn't answer, my spare machine broke after installing the packages. I had to revert. Good to know that you have had backups; that is was saved me.

@WoefulDerelict : You are right. TIL successfully building doesn't necessarily mean it's going a work.

@WoefulDerelict

Thank you for your very clear follow-up, which I found instructive and helpful.

I will be patient on the LUKS2 front and will keep an eye out for any future comments you may make here. Cheers....

GRUB's master branch may at any time include breaking commits. There does not appear to be a continuous integration toolchain testing the build integrity of the master branch nor does there appear to be a requirement that the master branch must still compile after a commit is included. Users should not assume bleeding edge features are even usable let alone reliable enough to deploy.

RushAur: Ham fistedly editing the instructions to the build system is not the way to solve this issue.

air-g4p: I can reproduce the failure in a clean chroot; however, I have not had a chance to dig into the recent changes and find the exact nature of the breakage. Given GRUB's history of pushing incomplete feature commits to master which have broken the build in the past I suspect the new LUKS2 feature is incomplete and not presently something users can build and test.

drossbox: This is a -git PKDBUILD. Maintenance is performed when actually necessary and not every time an irresponsible commit upstream breaks things.

@RushAur

Thank you and good work done here, Sir!

Using your suggestions, I have successfully compiled AND INSTALLED grub-git 2.04.rc1.r19.g4e7b5bb3b-1 on two separate rigs.

One minor clarification: ./linguas.sh actually appears in prepare(), not under configure().

That challenge being solved, I tried to convert my existing luks1 encrypted /boot, / and swap to luks2 using:

cryptsetup convert --type luks2 /dev/sda23

from the Arch Installation ISO, as the encrypted device cannot be mounted.

Also assuming /dev/sda23 is where your existing /boot, / and swap reside, ofc.

That command SUCCEEDS, and cryptsetup luksDump /dev/sda23 shows ALL the Keyslots having been converted to luks2.

Then I chrooted back into my install, re-ran mkinitcpio -P linux, then reinstalled grub with grub-install --target=..... then re-edited /etc/default/grub, then re-ran grub-mkconfig -o /boot/grub/grub.cfg

Following reboot, grub immediately faults into error mode: reporting grub CANNOT find my encrypted disk.

ONLY previously saving my LUKSHeaderBackup on an external device saved my ass from disaster!!

My question for you, @RushAur, (or anyone else following this thread), is it possible to compile and install grub-git 2.04.rc1.r19.g4e7b5bb3b-1, then convert the existing luks1 /boot, / and swap to luks2 - and then successfully reboot?

If anyone knows how to complete this transition, please feel free to contact me with the correct procedure at: saltedcipher AT protonmail DOT com.

Cheers from a feeling deprecated luks1 --type of guy....

Hi guys, Facing same problems as in last 2 comments, I finally managed to compile grub: Here is the "working" pkgbuild: https://gitlab.com/snippets/1940462

Basically builds exclusively for 64bit efi, (errors within grub-pc), to workaround the luks2 debug thing, comment out or delete the line --enable-mm-debug argument to ../configure. To workaround errors in the gettext/Po files, comment out or delete the call to ./linguas.sh in configure(). This step would make grub english only I suppose,since no language files are made.

@WoefulDerelict

I also would like to convert my existing encrypted /boot, / and swap from Luks 1 to Luks 2.

However, the build of grub-git 2.04.rc1.r19.g4e7b5bb3b-1 failed this way using makepkg -sric:

grub_debug_free in luks2 is not defined grub_debug_malloc in luks2 is not defined make[3]: [Makefile:49709: moddep.lst] Error 1 make[3]: Leaving directory '/home/user/.builds/grub-git/src/grub/build_i386-pc/grub-core' make[2]: [Makefile:27533: all] Error 2 make[2]: Leaving directory '/home/user/.builds/grub-git/src/grub/build_i386-pc/grub-core' make[1]: [Makefile:11755: all-recursive] Error 1 make[1]: Leaving directory '/home/user/.builds/grub-git/src/grub/build_i386-pc' make: [Makefile:3745: all] Error 2 ==> ERROR: A failure occurred in build()

This build failure appears quite similar to the failure drossbox described on 6 Feb 2020.

BTW: grub-git also failed to build in a clean chroot.

Any ideas? Thanks!

Is this package no longer being maintained? I see it hasn't been updated for a while and it fails to build. It complains about grub_debug_free and grub_debug_malloc in luks2 not being defined, then fails. I'd hoped to get this working now that grub can support a luks2 encrypted boot.

Peter_Littmann: rc1 was indeed to indicate release candidate 1 which was the tag present on the repository the last time this PKGBUILD required changes to cope with issues introduced upstream. As I stated earlier the -git suffix is intended to communicate that this PKGBUILD will always attempt to construct the software from the current git trunk and that pkgver will be generated at build time from data provided by the VCS. As upstream has already moved on past the 2.04 release this PKGBUILD presently generates a pkgver of 2.04.r9.g8f6843ce6-1. Users are expected to have read the relevant documentation BEFORE they visit the AUR and should be familiar with VCS packaging conventions. The ONLY supported method for building and managing packages from PKGBUILDs hosted in the AUR is makepkg. Users are expected to have read and understood what the PKGBUILD does before they download and attempt to use it. The burden of knowing when changes have occurred upstream and when to rebuild a VCS package pulling from a project's git master branch is placed entirely on the end user.

The logic in pgkver() exists to reduce the maintenance burden of VCS packages. Cluttering the AUR with unnecessary updates every time there has been a commit upstream just to keep the pkgver current is generally frowned upon.

But is the name grub-git 2.04.rc1.r19.g4e7b5bb3b-1 not misleading? I thought rc1 stands for release candidate 1 and now there is a real release published as I understand. May be makepkg is called by hand and fetches the latest version, but how does pamac or yay get informed that there is a new version to start makepkg for it when alreaady installed???

Peter_Littmann: This is a VCS package. The -git suffix implies that it will fetch the latest (trunk) version from the upstream git server when makepkg is called. pkgver() automatically updates the package version when it is built.

It looks like grub 2.04 is released on 05. July 2019, please check this: https://ftp.gnu.org/gnu/grub/

Fixed by completely re-syncing all source.

Errors still for me: http://ix.io/1No7

scurrvy2020: You are correct, it is an issue with the upstream source. The problem arises from a failure to properly fold in CFLAGS consistently.

I called in some help and we managed to work out what the problem was and generate a minimally invasive hack to work around it.

Breakage is tracked by https://savannah.gnu.org/bugs/?56144

Workaround: roll back to Grub commit f8f35acb5b05d40e3707a9d2db9ede60023e4cac Patch at http://sprunge.us/La2tKW

I'm getting this error... looks like it is probably in the source and not your packaging, however.

In file included from ../../../../grub-core/lib/gnulib/argp-fmtstream.c:26: ./wchar.h:708:1: error: unknown type name ‘mbstate_t’ _GL_FUNCDECL_RPL (mbrtowc, size_t, ^~~~~~~~~~~~~~~~ ../../../../grub-core/lib/gnulib/argp-fmtstream.c: In function ‘add_width’: ../../../../grub-core/lib/gnulib/argp-fmtstream.c:125:3: error: unknown type name ‘mbstate_t’ mbstate_t ps; ^~~~~~~~~ ../../../../grub-core/lib/gnulib/argp-fmtstream.c:155:11: warning: implicit declaration of function ‘wcwidth’; did you mean ‘mbswidth’? [-Wimplicit-function-declaration] k = wcwidth (wc); ^~~~~~~ mbswidth

For me, I'm hitting an error just using makepkg by itself.

http://ix.io/1GTv

BTW, building grub-git from pikaur or pacaur may fail with:

./bootstrap: done. Now you can run './configure'.

mkdir: cannot create directory ‘/var/build/grub-git/src/grub/build_i386-pc’: File exists

==> ERROR: A failure occurred in build().

Instead, with pacaur, first remove the build directory. With pikaur, first remove both the build dir and ~/.cache/pikaur/build/grub-git/ before running the build for a second time.

EndlessEden: According to the updated INSTALL file bootstrap calls autogen.sh so the PKGBUILD would simply need to replace the call to autogen.sh with a call to bootstrap.

Ranguvar: Even after making the relevant changes to the PKGBUILD the build fails with unknown type name errors. This seems like an error caused by the changes upstream.

needs a bootstrap first, before autogen.

Not building for me as of https://git.savannah.gnu.org/cgit/grub.git/commit/?id=35b909062e7b334eb4af372be3260d0f62d85375

I'm using git branch pre-bootstrap f8f35acb after cd grub to get the commit right before it.

Hello everyone,

FYI, if anyone runs into this:

grub.po: warning: Charset "CHARSET" is not a portable encoding name.                                                                                                                                               
                  Message conversion to user's charset might not work.                                                                                                                                             
sed: couldn't open file grub.d.sed: No such file or directory                                                                                                                                                      
make[3]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                  
test ! -f ../../po/grub.pot || \                                                                                                                                                                                   
  test -z "../../po/de_CH.gmo ../../po/de@hebrew.gmo ../../po/en@arabic.gmo ../../po/en@cyrillic.gmo ../../po/en@greek.gmo ../../po/en@hebrew.gmo ../../po/en@piglatin.gmo ../../po/en@quot.gmo" || make ../../po/d
e_CH.gmo ../../po/de@hebrew.gmo ../../po/en@arabic.gmo ../../po/en@cyrillic.gmo ../../po/en@greek.gmo ../../po/en@hebrew.gmo ../../po/en@piglatin.gmo ../../po/en@quot.gmo                                         
make[3]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                 
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                 
make[4]: *** No rule to make target 'de.po', needed by 'de@hebrew.po-create'.  Stop.                                                                                                                               
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                  
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                 
make[4]: *** No rule to make target 'de.po', needed by 'de_CH.po-create'.  Stop.                                                                                                                                   
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                  
make[3]: *** [Makefile:1647: ../../po/de@hebrew.po] Error 2                                                                                                                                                        
make[3]: *** Waiting for unfinished jobs....                                                                                                                                                                       
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                 
LC_ALL=C msginit -i ../../po/grub.pot --no-translator -l en@hebrew -o - 2>/dev/null | LC_ALL=C msgconv -t UTF-8 | LC_ALL=C msgfilter -o ../../po/en@hebrew.po -i - sed -f ../../po/hebrew.sed                      
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'                                                                                                                                 
LC_ALL=C msginit -i ../../po/grub.pot --no-translator -l en@cyrillic -o - 2>/dev/null | LC_ALL=C msgconv -t UTF-8 | LC_ALL=C msgfilter -o ../../po/en@cyrillic.po -i - sed -f ../../po/cyrillic.sed                
make[3]: *** [Makefile:1647: ../../po/de_CH.po] Error 2
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
LC_ALL=C msginit -i ../../po/grub.pot --no-translator -l en@greek -o - 2>/dev/null | LC_ALL=C msgconv -t UTF-8 | LC_ALL=C msgfilter -o ../../po/en@greek.po -i - sed -f ../../po/greek.sed                        
make[4]: Entering directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
LC_ALL=C msginit -i ../../po/grub.pot --no-translator -l en@arabic -o - 2>/dev/null | LC_ALL=C msgconv -t UTF-8 | LC_ALL=C msgfilter -o ../../po/en@arabic.po -i - sed -f ../../po/arabic.sed                     
msgfilter: warning: Locale charset "ANSI_X3.4-1968" is different from
                    input file charset "UTF-8".
                    Output of 'msgfilter' might be incorrect.
                    Possible workarounds are:
                    - Set LC_ALL to a locale with encoding UTF-8.
                    - Convert the translation catalog to ASCII using 'msgconv',
                      then apply 'msgfilter',
                      then convert back to UTF-8 using 'msgconv'.
msgfilter: warning: Locale charset "ANSI_X3.4-1968" is different from
                    input file charset "UTF-8".
                    Output of 'msgfilter' might be incorrect.
                    Possible workarounds are:
                    - Set LC_ALL to a locale with encoding UTF-8.
                    - Convert the translation catalog to ASCII using 'msgconv',
                      then apply 'msgfilter',
                      then convert back to UTF-8 using 'msgconv'.
msgfilter: warning: Locale charset "ANSI_X3.4-1968" is different from
                    input file charset "UTF-8".
                    Output of 'msgfilter' might be incorrect.
                    Possible workarounds are:
                    - Set LC_ALL to a locale with encoding UTF-8.
                    - Convert the translation catalog to ASCII using 'msgconv',
                      then apply 'msgfilter',
                      then convert back to UTF-8 using 'msgconv'.
msgfilter: warning: Locale charset "ANSI_X3.4-1968" is different from
                    input file charset "UTF-8".
                    Output of 'msgfilter' might be incorrect.
                    Possible workarounds are:
                    - Set LC_ALL to a locale with encoding UTF-8.
                    - Convert the translation catalog to ASCII using 'msgconv',
                      then apply 'msgfilter',
                      then convert back to UTF-8 using 'msgconv'.
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[4]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[3]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[2]: *** [Makefile:1556: stamp-po] Error 2
make[2]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc/po'
make[1]: *** [Makefile:11522: all-recursive] Error 1
make[1]: Leaving directory '/home/myuser/.src/grub-git/src/grub/build_i386-pc'
make: *** [Makefile:3555: all] Error 2
==> ERROR: A failure occurred in build().
    Aborting...

More than likely, rsync failed silently connecting to the upstream rsync server retrieving necessary language files.

In this case, you would need to allow outgoing TCP port 873. Cheers

The PKGBUILD has undergone a substantial rewrite and some of the behaviour has changed. grub-emu is now enabled by default. grub-extras is now included in all builds instead of cherry picking the 915resolution module for the i386-pc build.

The lua module in grub-extras currently fails to build as it has not caught up with changes to grub_file_open() [http://git.savannah.gnu.org/cgit/grub.git/commit/include/grub/file.h?id=ca0a4f689a02c2c5a5e385f874aaaa38e151564e] so I have temporarily disabled it until the problem is rectified.

The duplicate symbol issue introduced by commit 0ba90a7f017889d32a47897d9107ef45cc50a049 has been resolved upstream in commit c79ebcd18cf3e208e9dda5e2ae008f76c92fe451. The work around has been removed from the PKGBUILD. The i386-efi build completes without issues on my test machines.

This has been reported back upstream.

The issue introduced by commit 0ba90a7f017889d32a47897d9107ef45cc50a049 is that i386-efi builds now contain a pair of conflicting fuctions named grub_reboot. It appears as tho i386-efi builds may have never been supported properly. i386-efi builds likely should have been pulling the instructions contained in grub-core/lib/efi/reboot.c prior to commit 0ba90a7f017889d32a47897d9107ef45cc50a049; however, they had instead been pointed to instructions for legacy systems contained in grub-core/lib/i386/reboot.c and grub-core/lib/i386/reboot_trampoline.S. Commit 0ba90a7f017889d32a47897d9107ef45cc50a049 moved the instructions contained in grub-core/lib/efi/reboot.c to grub-core/grub-core/kern/efi/efi.c. After commit 0ba90a7f017889d32a47897d9107ef45cc50a049 i386-efi builds include the function grub_reboot from both grub-core/grub-core/kern/efi/efi.c and grub-core/lib/i386/reboot.c resulting in a conflict that halts the build and displays the following message grub_reboot in reboot is duplicated in kernel.

I have introduced a conditional statement in the PKGBUILD’s _build_grub-efi function which should provide a work around for i386-efi builds until the issue is corrected upstream.

That may well be because there's no immediately obvious way to open a bug report... it's hidden in a drop-down under the Bug navigation bar item... -.-

Reported: https://savannah.gnu.org/bugs/?53517

jonathon: No, unfortunately, I don't think anyone has reported the issue upstream.

Has anyone reported the issue upstream? This page is the only one found via a web search which references the issue.

The build failure resulting in "grub_reboot in reboot is duplicated in kernel" only happens if the EFI version is built. If you don't need the EFI version (e.g. you're using BIOS) then you can disable it by setting _GRUB_IA32_EFI_ARCH_X64="0" in the PKGBUILD.

@WoefulDerelict - I would say this is probably something that the PKGBUILD will need to address at some point because the change looks like it was made on purpose rather than as a bug. I note that the core grub package doesn't address this either, but that's because it builds a version prior to this change being introduced.

I have tried to work out how to make it build, without success. For me, disabling EFI is an acceptible workaround until the correct fix is known.

This appears to be an issue introduced by upstream commit 0ba90a7f017889d32a47897d9107ef45cc50a049. One is able to build the previous commit b4d709b6ee789cdaf3fa7a80fd90c721a16f48c2 just fine.

timofonic: -git packages generally pull from the upstream 'master' branch. Upstream developers can introduce breaking changes at any time without warning. This is not an indication the PKGBUILD is out of date.

Please report your issues upstream where they can be addressed properly.

@timofonic It doesn't work for me too! Same error message "grub_reboot in reboot is duplicated in kernel" with up to date packages.