Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
grub2 + lvm2-raid1 - initrd = [just too soon - solved]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Thu Sep 17, 2020 3:10 pm    Post subject: grub2 + lvm2-raid1 - initrd = [just too soon - solved] Reply with quote

TL;DR: Now that I understand how to do it, Neddy has pointed out that early init of lvm raid volumes really isn't ready for use yet... ;)

Hi guys,

This is a new install and I'm trying something new.
I'm trying to avoid using an initial ramdisk.
I'm pretty sure I'm missing a kernel option... or?

GPT disks have a bios partition and a lvm-raid1 partition.
The lvm-raid1 partition has boot/swap/root volumes.

Grub launches fine and from its command line I can list and access all lvm partitions.
("grub> ls -l (lvm/raid1-root)/" lists the root file system contents.)

The problem is the kernel panics on boot with:
"not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

The system chroots and works fine from a gentoo-minimal disk.

Grub-mkconfig generated grub.cfg currently has:
"linux /vmlinuz-5.8.8 root=/dev/mapper/raid1-root ro init=/lib/systemd/systemd"

I've also tried these with the same result:
"linux /vmlinuz-5.8.8 root=lvm/raid1-root ro init=/lib/systemd/systemd"
"linux /vmlinuz-5.8.8 root=UUID=<correct root uuid> ro init=/lib/systemd/systemd"

I've read through various how-tos on grub2/lvm2/lvm-raid1 and have rebuilt the kernel several times, adding in various "recommended" kernel options. Maybe that isn't my issue? My reading suggests this should work. Any pointers would be greatly appreciated...


Last edited by drumgod on Mon Sep 21, 2020 10:52 pm; edited 1 time in total
Back to top
View user's profile Send private message
pietinger
Guru
Guru


Joined: 17 Oct 2006
Posts: 350
Location: Bavaria

PostPosted: Thu Sep 17, 2020 5:16 pm    Post subject: Reply with quote

Maybe you need the kernel(-module) parameter "dm-mod.create=" (dm_init) ? With this you can set needed parameters for all dm-modules .

(an example for this - but with encrypting instead of a raid - you will find here: https://forums.gentoo.org/viewtopic-t-1110764-highlight-.html
see also "raid_tests()" in https://gitlab.collabora.com/koike/dm-cmdline-test/-/blob/master/dm-cmd-test.sh)
Back to top
View user's profile Send private message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Thu Sep 17, 2020 8:09 pm    Post subject: Reply with quote

Thanks @pietinger

I do have dm_init compiled into the kernel but figured grub would do some magic and pass whatever it needed... I guess not. So I checked out your links and looked through several dm-mod LVM examples.

The kernel docs say, "The dm line should be equivalent to the one used by the dmsetup tool with the –concise argument." I added the relevant volume from "dmsetup --concise table" into /etc/defaults/grub and ended up with:

linux /vmlinuz-5.8.8 root=/dev/mapper/raid1-root ro dm-mod.create="raid1-root,<correct lv uuid>,14,rw,0 577339392 raid raid1 3 0 region_size 1024 2 253:10 253:11 253:12 253:13" init=/lib/systemd/systemd

Unfortunately there is no change in boot behavior.

Since the docs also say to use the "UUID" and the dmsetup command returns "LV UUID"s, I tried replacing the lv uuid with the uuid from blkid... Also no change in boot behavior.

No uuid at all, as shown in most examples, does not boot either.

I also tried including the entire (rather large) string returned by dmsetup thinking maybe it needed it all... Again, no go.

I did verify the changes were making it into grub.cfg before each attempt to boot.

Still poking around and trying different things...
Back to top
View user's profile Send private message
pietinger
Guru
Guru


Joined: 17 Oct 2006
Posts: 350
Location: Bavaria

PostPosted: Thu Sep 17, 2020 8:57 pm    Post subject: Reply with quote

I think the error is here:
drumgod wrote:
root=/dev/mapper/raid1-root

At this time (=kernel-boot) you have no link to /dev/mapper; you have only native /dev/dm-0 ...
Back to top
View user's profile Send private message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Sat Sep 19, 2020 7:33 pm    Post subject: Reply with quote

Hmm. I thought that was the point of dm-mod.create? From the device mapper doc about Early creation of mapped devices: (https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-init.html)

Quote:
It is possible to configure a device-mapper device to act as the root device for your system in two ways.

The first is to build an initial ramdisk which boots to a minimal userspace which configures the device, then pivot_root( 8 ) in to it.

The second is to create one or more device-mappers using the module parameter “dm-mod.create=” through the kernel boot command line argument.


Maybe I need to lose the /dev/mapper/ prefix? You might be right though. Also from that doc there's this example for mounting root from a linear array:

Code:
dm-mod.create="lroot,,,rw, 0 4096 linear 98:16 0, 4096 4096 linear 98:32 0" root=/dev/dm-0


I would have expected that to be root=/dev/mapper/lroot if it worked like I've been thinking... I'll try a quick edit to grub.cfg and set root to the dm-* device.

Thanks again!
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46352
Location: 56N 3W

PostPosted: Sat Sep 19, 2020 7:46 pm    Post subject: Reply with quote

drumgod,

The table at https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-init.html says raid is not allowed.
Its going to be very fragile too. If you grow lv-root to need to fix the kernel command line.

I was trying to do an initrd free root in lvm on top of kernel raid5 but dm-init tries to start the logical volume before the raid set is assembled.
My notes are at https://wiki.gentoo.org/wiki/User:NeddySeagoon/Root_Inside_LVM_On_Raid

raid on its own works. dm-mod.create= on its own works.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Sat Sep 19, 2020 10:43 pm    Post subject: Reply with quote

NeddySeagoon wrote:
drumgod,

The table at https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-init.html says raid is not allowed.
Its going to be very fragile too. If you grow lv-root to need to fix the kernel command line.

I was trying to do an initrd free root in lvm on top of kernel raid5 but dm-init tries to start the logical volume before the raid set is assembled.
My notes are at https://wiki.gentoo.org/wiki/User:NeddySeagoon/Root_Inside_LVM_On_Raid

raid on its own works. dm-mod.create= on its own works.

Thank you for taking the time to write that up for everyone! I did come across your notes in my searches but apparently didn't "grok" enough of it and may still be misunderstanding some. lol. For instance I missed that you are, in fact, using root=/dev/md-0. I understand the issue of needing to maintain the option line on any changes, but don't really plan to modify the lv. If I do change my mind, it doesn't seem to be too onerous a task... Mostly I'm just trying out new ways of doing stuff. This seemed like it might be a way to get a somewhat complicated configuration (mdadm raid1 with an inird, then lvm on top) accomplished with fewer elements.

I took "raid: constrained, userspace should verify metadata device" to mean that not all functionality is present, but some is; and I figured that normal bring-up of the system after that would verify the metadata. Perhaps not? You did get that configuration up and running though? If you have a chance... What did you find "fragile" about it? What is my biggest issue, as you see it, that I am missing with my current approach? Thanks again.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46352
Location: 56N 3W

PostPosted: Sun Sep 20, 2020 10:48 am    Post subject: Reply with quote

drumgod,

There are two sorts of raid which we may be confusing.
mdadm raid, which is entirely separate to LVM.
and LVM built in raid.

When you create a PV spread across several spindles, you can chose how the several spindles are used.
linear, is the space is allocated end to end, like one large partition.
raid means that space is allocated in some raid like way, I have not tried different levels) and lvm does the raid work, not mdadm.
Its this raid that is not permitted with dm-mod.create.

I was trying mdadm raid auto assemble with raid5 and dm-mod.create on top.
As i said, both bits work on their own but the kernel performs the dm-mod.create before the raid set is assembled, so the layers fail.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Sun Sep 20, 2020 7:40 pm    Post subject: Reply with quote

Thanks Neddy,

NeddySeagoon wrote:
There are two sorts of raid which we may be confusing.
mdadm raid, which is entirely separate to LVM.
and LVM built in raid.

Understood.

NeddySeagoon wrote:
When you create a PV spread across several spindles, you can chose how the several spindles are used.
linear, is the space is allocated end to end, like one large partition.
raid means that space is allocated in some raid like way, I have not tried different levels) and lvm does the raid work, not mdadm as well as the initrd.

This. I have a raid1 PV across two disk partitions. That PV has three LVs: boot, swap and root. If I'm understanding correctly, LVM2 now uses the same kernel libraries as mdadm... I'm hoping to cut out the "mdadmiddle" man.

EDIT: Pretty much this configuration but with three LVs + a bios boot partition: https://wiki.gentoo.org/wiki/Raid1_with_LVM_from_scratch

NeddySeagoon wrote:
Its this raid that is not permitted with dm-mod.create.

I was trying mdadm raid auto assemble with raid5 and dm-mod.create on top.
As i said, both bits work on their own but the kernel performs the dm-mod.create before the raid set is assembled, so the layers fail.


I appreciate your experience. So you are saying that the way I'm approaching this will not work at all? (See my interpretation of the support table in my previous post.) I was thinking your raid5 was an LVM PV. Basically the same thing I've been trying to do but raid5 instead of raid1... Seems ironic that grub accesses the LVs fine but I can't get the kernel to. I understand I'm playing with new stuff, but I keep finding little comments that seem to indicate it should work now that LVM2 is using the mdadm libraries for raid.

Thanks again guys.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 46352
Location: 56N 3W

PostPosted: Sun Sep 20, 2020 10:59 pm    Post subject: Reply with quote

drumgod,

I would not go so far as to say, will not work. The Target types table says.
raid constrained, userspace should verify metadata device.

That only says that horrible things might happen. :)

I read that before I put my experimental system together. Calling it raid5 is a bit of an exaggeration.
Its two partitions on the same spindle with the third partition missing :)

Lets look at the kernel code. Read the source, Luke. :)

drivers/md/dm-init.c:
const char * const dm_allowed_targets[] __initconst = {
        "crypt",
        "delay",
        "linear",
        "snapshot-origin",
        "striped",
        "verity",
};
raid is not on that list.

Further down the same file in
Code:
static char __init *dm_parse_table_entry(struct dm_device *dev, char *str)

the kernel shouts at you if your target type is not allowed.
Code:
        /* target_type */
        strscpy(sp->target_type, field[2], sizeof(sp->target_type));
        if (dm_verify_target_type(sp->target_type)) {
                DMERR("invalid type \"%s\"", sp->target_type);
                return ERR_PTR(-EINVAL);


If you add raid to the allowed target types, you get to keep all the pieces. :)
It would pass that check though.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
drumgod
n00b
n00b


Joined: 12 Dec 2003
Posts: 43

PostPosted: Mon Sep 21, 2020 10:40 pm    Post subject: Reply with quote

lol! Well that takes away any question as to whether this configuration will work. As the code stands, it will not. Thank you for patiently pointing it out. :) Just a little too close to the edge I guess.

Hmmm. Lets see if I can get it to work with an initrd... I appreciate the help guys!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum