Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
PXE booting disk less Gentoo OS
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Tue Jun 30, 2020 7:31 pm    Post subject: PXE booting disk less Gentoo OS Reply with quote

So me and my friend have been working on an disk auditing software and it is nearing completion. We are in the

process of adding PXE ability to the software to have client machines boot into the software. We just hit a bit of a wall..... and we arnt

sure how to pass it, we are currently looking for a jack hammer to break

down the wall and get though to the other side.....

So far what we do have working is the PXE boot itself.... When a client starts booting to the PXE. It loads grub and the initramfs files.

This is where we are stuck. Once initramfs loads it is asking for us to define a root filesystem......

How do we go about telling initramfs where the rootfs is? So far the PXE is booting the Gentoo boot files just fine... But where and

how do we tell initramfs to mount a file sytem?

We need initramfs to mount / over the network and boot the diskless kernel onto the client machine to execute our script......

Where do we put the stage 3 files for this and where do we tell initramfs where / is located at.

This is very easy when your doing a basic Gentoo install on a bare metal machine.... But how do you do it on the PXE.

I know there is an area where we put the stage 3 files and the OS it self for the initramfs to boot to............

The thing is initramfs is asking for a UUID.... can we make a folder under media and give that folder a UUID so that initramfs can boot

into the folder and load the OS on the client machine?

I know you guys have been seeing me talk about this disk auditing software for a while and we coming to the end of our project here

and would love to give some people some credit :)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45826
Location: 56N 3W

PostPosted: Tue Jun 30, 2020 8:03 pm    Post subject: Reply with quote

finalturismo,

Are you trying to do root over nfs or root over nfs when root is in a file rather than a directory on the nfs server?

root over nfs is well documented. Probably as 'thin client'.
The tricky bit is ensuring the IP address of the client does not change when userspace takes over or your root filesystem goes away.

You will not use a UUID to find root. It will be an IP address maybe with a path, depending on how the nfs serser /etc/exports is set up.

As you have an initrd, why bother with root over nfs?
Put everything in the initrd. I guess that makes logging difficult though.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 227
Location: Toowoomba/Australia

PostPosted: Wed Jul 01, 2020 5:24 am    Post subject: Reply with quote

When I have run diskless machines I have done it 1 of 2 ways.

1. Grab the kernel via pxe/tftp and have it mount / via nfs. When mounting / via nfs there are arguments you put in the append line in your pxe config that tell the kernel there root is (ie server ip and mount point). When I was doing this never used grub or an initrd/initramfs.

2. Grab the kernel via pxe/tftp and have everything I need in the initramfs and run the entire system from a ram disk. No local storage and no network file system needed. I have never used grub for this. This is a custom initramfs where the fs was created with debootstrap with all the programs I want.

These days option 2 is what I use for rescue and booting strapping a new system. The initramfs contains enough to be self contained. The exception to that is when I have copied the contents of a mint iso to the tftp server and it has booted a mix of pxe, nfs, extract the initrd from the iso to a ram disk.
_________________
Beware the grue.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Wed Jul 08, 2020 6:08 pm    Post subject: Reply with quote

Aiken wrote:
When I have run diskless machines I have done it 1 of 2 ways.

1. Grab the kernel via pxe/tftp and have it mount / via nfs. When mounting / via nfs there are arguments you put in the append line in your pxe config that tell the kernel there root is (ie server ip and mount point). When I was doing this never used grub or an initrd/initramfs.

2. Grab the kernel via pxe/tftp and have everything I need in the initramfs and run the entire system from a ram disk. No local storage and no network file system needed. I have never used grub for this. This is a custom initramfs where the fs was created with debootstrap with all the programs I want.

These days option 2 is what I use for rescue and booting strapping a new system. The initramfs contains enough to be self contained. The exception to that is when I have copied the contents of a mint iso to the tftp server and it has booted a mix of pxe, nfs, extract the initrd from the iso to a ram disk.


Ok it looks like i will be going with option 2 here. Do you have anymore information on making the rootfs with debootstrap and than setting the root with initramfs?
Like a little bit of a guide i guess iam asking for.... i just need a little nudge in the right direction and i will be able to figure out the rest by my self.
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 227
Location: Toowoomba/Australia

PostPosted: Thu Jul 09, 2020 12:48 am    Post subject: Reply with quote

My documentation is 3 scripts I threw together for something to do. Nothing awe inspiring but does what I want when I want. For awhile this was how I booted my router (old pc) so the only moving parts were cpu & power supply fans and a fresh fs every time it booted. Has been very handy for rescue when the host fs has a problem. This is how I boot strap a new gentoo install.

I also disable network interface renaming. Nice predictable names of eth0 and wlan0 are far more useful to me than not have a clue what name I will be presented with.

The build can be done with gentoo using

Code:

ROOT=/var/tmp/images/ram/ PORTAGE_CONFIGROOT=/var/tmp/images/ram/ emerge whatever-packages-you-want


but I found the fs size is a lot larger and one of the computers I was initially booting this way the fs size was too large for the amount of ram while using debian worked. The gentoo image would have had to be built then stripped of any files not needed compared to the debian image which is quick to make and small enough with no extra effort.

Need CONFIG_BLK_DEV_INITRD=y to use initramfs. I do this in /var/tmp/images and have the modules location hard coded based on that location when building the kernel.

Code:

#!/bin/bash

make -j8 && {
   make INSTALL_MOD_PATH=/var/tmp/images/modules/ modules_install
   cp arch/x86/boot/bzImage ../villa
}


Use debootstrap to create an image in a directory called ram.

Code:

#!/bin/bash

time debootstrap --include=openssh-client,openssh-server,vim,bc,hdparm,joe,iptables,iw,lftp,ntp,pciutils,rfkill,usbutils,wireless-tools,locales,less,bzip2,lsof,rsync,nfs-common,ntfs-3g,net-tools stable ram http://ftp.iinet.net.au/debian/debian

echo LANG=\"en_AU.UTF-8\" > ram/etc/default/locale
echo LANGUAGE=\"en_AU:en\" >> ram/etc/default/locale
echo en_AU.UTF-8 UTF-8 > ram/etc/locale.gen

echo \# stop the stupid interface renaming > ram/etc/udev/rules.d/80-net-setup-link.rules

echo set http_proxy=http://PROXY-SERVER:3128 > ram/root/.bashrc
echo Acquire::http::Proxy \"http://PROXY-SERVER:3128\"\; > ram/etc/apt/apt.conf
echo pxe > ram/etc/hostname

echo >> ram/etc/network/interfaces
echo auto eth0 >> ram/etc/network/interfaces
echo iface eth0 inet dhcp >> ram/etc/network/interfaces

cat > ram/root/post-inst << EOF
locale-gen
sed -i -e 's/main/main contrib non-free/' /etc/apt/sources.list
apt-get update
apt-get -y install firmware-realtek firmware-ralink
apt-get clean
rm -r /var/lib/apt/lists
cp /sbin/init /
EOF

cp /etc/localtime ram/etc/localtime
cp /etc/timezone ram/etc/timezone

cat > ram/etc/ntp.conf << EOF
driftfile /var/lib/ntp/ntp.drift

server TIME-SERVER iburst
EOF

chmod u+x ram/root/post-inst
chroot ram root/post-inst

echo
echo do not forget to set any user/passwords before deploying
echo


Pack the files and send them kernel and new initramfs to the tftp/pxe server

Code:

#!/bin/bash

chroot ram/ groupadd -g 22 sshd
chroot ram/ useradd -u 22 -g 22 -s /sbin/nologin -d /var/empty sshd
chroot ram/ useradd -r dhcp

cp modules/lib/* ram/lib/ -av
cp villa ram/
cp villa ram/boot/vmlinuz
cp /boot/intel-uc.img ram/boot/

cd ram
find | cpio -H newc -ov | gzip > ../initramfs.gz
cd ..

echo
echo Image is built, hit Enter when ready to send
read

scp initramfs.gz villa USER@SERVER:/tftpboot/

_________________
Beware the grue.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Thu Jul 09, 2020 7:20 pm    Post subject: Reply with quote

Aiken wrote:
My documentation is 3 scripts I threw together for something to do. Nothing awe inspiring but does what I want when I want. For awhile this was how I booted my router (old pc) so the only moving parts were cpu & power supply fans and a fresh fs every time it booted. Has been very handy for rescue when the host fs has a problem. This is how I boot strap a new gentoo install.

I also disable network interface renaming. Nice predictable names of eth0 and wlan0 are far more useful to me than not have a clue what name I will be presented with.

The build can be done with gentoo using

Code:

ROOT=/var/tmp/images/ram/ PORTAGE_CONFIGROOT=/var/tmp/images/ram/ emerge whatever-packages-you-want


but I found the fs size is a lot larger and one of the computers I was initially booting this way the fs size was too large for the amount of ram while using debian worked. The gentoo image would have had to be built then stripped of any files not needed compared to the debian image which is quick to make and small enough with no extra effort.

Need CONFIG_BLK_DEV_INITRD=y to use initramfs. I do this in /var/tmp/images and have the modules location hard coded based on that location when building the kernel.

Code:

#!/bin/bash

make -j8 && {
   make INSTALL_MOD_PATH=/var/tmp/images/modules/ modules_install
   cp arch/x86/boot/bzImage ../villa
}


Use debootstrap to create an image in a directory called ram.

Code:

#!/bin/bash

time debootstrap --include=openssh-client,openssh-server,vim,bc,hdparm,joe,iptables,iw,lftp,ntp,pciutils,rfkill,usbutils,wireless-tools,locales,less,bzip2,lsof,rsync,nfs-common,ntfs-3g,net-tools stable ram http://ftp.iinet.net.au/debian/debian

echo LANG=\"en_AU.UTF-8\" > ram/etc/default/locale
echo LANGUAGE=\"en_AU:en\" >> ram/etc/default/locale
echo en_AU.UTF-8 UTF-8 > ram/etc/locale.gen

echo \# stop the stupid interface renaming > ram/etc/udev/rules.d/80-net-setup-link.rules

echo set http_proxy=http://PROXY-SERVER:3128 > ram/root/.bashrc
echo Acquire::http::Proxy \"http://PROXY-SERVER:3128\"\; > ram/etc/apt/apt.conf
echo pxe > ram/etc/hostname

echo >> ram/etc/network/interfaces
echo auto eth0 >> ram/etc/network/interfaces
echo iface eth0 inet dhcp >> ram/etc/network/interfaces

cat > ram/root/post-inst << EOF
locale-gen
sed -i -e 's/main/main contrib non-free/' /etc/apt/sources.list
apt-get update
apt-get -y install firmware-realtek firmware-ralink
apt-get clean
rm -r /var/lib/apt/lists
cp /sbin/init /
EOF

cp /etc/localtime ram/etc/localtime
cp /etc/timezone ram/etc/timezone

cat > ram/etc/ntp.conf << EOF
driftfile /var/lib/ntp/ntp.drift

server TIME-SERVER iburst
EOF

chmod u+x ram/root/post-inst
chroot ram root/post-inst

echo
echo do not forget to set any user/passwords before deploying
echo


Pack the files and send them kernel and new initramfs to the tftp/pxe server

Code:

#!/bin/bash

chroot ram/ groupadd -g 22 sshd
chroot ram/ useradd -u 22 -g 22 -s /sbin/nologin -d /var/empty sshd
chroot ram/ useradd -r dhcp

cp modules/lib/* ram/lib/ -av
cp villa ram/
cp villa ram/boot/vmlinuz
cp /boot/intel-uc.img ram/boot/

cd ram
find | cpio -H newc -ov | gzip > ../initramfs.gz
cd ..

echo
echo Image is built, hit Enter when ready to send
read

scp initramfs.gz villa USER@SERVER:/tftpboot/



So iam having a bit of a hard time understand debootstrap and iam not sure why........... So i already have the image of the OS i want to use for the initramfs.... iam confused as to why i would need to use the following line
Code:
 --include=openssh-client,openssh-server,vim,bc,hdparm,joe,iptables,iw,lftp,ntp,pciutils,rfkill,usbutils,wireless-tools,locales,less,bzip2,lsof,rsync,nfs-common,ntfs-3g,


Why couldn't i just copy and past the current OS into the ram folder for initramfs? I was trying to stay on a strictly Gentoo based system here.


At this point i would be willing to pay for a quick phone call and or a remote session, so i can ask a few questions. Do you happen to have Skype or Discord.
Back to top
View user's profile Send private message
Aiken
Apprentice
Apprentice


Joined: 22 Jan 2003
Posts: 227
Location: Toowoomba/Australia

PostPosted: Mon Jul 13, 2020 2:23 am    Post subject: Reply with quote

Use what ever you want to prepare the image. If you have a already prepared image then that step is done and ignore any mention of debootstrap or ROOT=/some/dir emerge whatever. I don't have a prepared system so that is how I quickly prepare a new image with a new things I do before hand to prep the image for booting as well as the initramfs prep.

After creating the image there is not much more than creating a few needed groups, setting locale, making sure eth0 comes up correcly configured, copying images /sbin/init to the appropriate place and setting user/passwords as needed. It does get old creating a new image and after deploying to realise there is no way to log in.

Those scripts are not going to tell you exactly what to type to get your image running but they are an example of getting something pxe/tftp booting from scratch. Some details will have to be adapted to your needs. Might not be the best but for my needs I can have a new image created and deployed in a couple of minutes.

I do not have skype or discord.
_________________
Beware the grue.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Sun Aug 02, 2020 11:22 pm    Post subject: Reply with quote

NeddySeagoon wrote:
finalturismo,

Are you trying to do root over nfs or root over nfs when root is in a file rather than a directory on the nfs server?

root over nfs is well documented. Probably as 'thin client'.
The tricky bit is ensuring the IP address of the client does not change when userspace takes over or your root filesystem goes away.

You will not use a UUID to find root. It will be an IP address maybe with a path, depending on how the nfs serser /etc/exports is set up.

As you have an initrd, why bother with root over nfs?
Put everything in the initrd. I guess that makes logging difficult though.



I need to load Gentoo over IPXE by using filesystem.squarefs. I need the image to boot fully from memory.
ipxe script:
Code:
#!ipxe

menu THRONG OF BAZ
item --gap LAN
item gentoo Gentoo x86_64 minimal

:gentoo
set g tftp://{ipAddress}
kernel ${g}/gentoo boot=live root=/dev/ram0 init=/linuxrc aufs looptype=squashfs loop=${g}/filesystem.squashfs cdroot vconsole.keymap=us locale=en_US console=tty1 --
initrd ${g}/gentoo.xz
boot


Problems

when gentoo.xz loading / loaded.....

Pretty much i need to know how to get root filesystem going while using squashfs

filesystem.squashfs not loaded and it says media not found
https://i.ibb.co/CQmGDSZ/image.png

Please help we are so close to finishing our project.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Mon Aug 03, 2020 9:42 pm    Post subject: Reply with quote

bump
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1898

PostPosted: Tue Aug 04, 2020 6:13 pm    Post subject: Reply with quote

This is a line from my working setup. Haven't used it in a long time, but I'm pretty sure it would still boot.
I'm using isolinux, but it does not affect kernel params.
Code:
APPEND ip=dhcp root=/dev/ram0 cdroot=1 real_root=/dev/nfs nfsroot=10.0.1.254:/mnt/linux.images/tftp/gentoo-amd64 initrd=gentoo-amd64/initramfs loop=2018-07-07.sqfs looptype=squashfs net.ifnames=0 console=ttyS1,57600

You can surely figure out which params are important for your case. (Hint: redirecting system console to serial probably is not :lol: )

Doing it this way, you stay connected to the NFS at all times.
I've learned a few tricks since then and I now prefer to copy everything into RAM if possible, so I can detach from NFS releasing the network bandwidth, cutting down IO lags, removing remote SPOF, and making my diskless machine strong and independent :lol:

And yes, one way to do that is putting everything into your initramfs - if you can compress it under ~100MB (tftp can't transfer bigger files, so it won't boot if you exceed that).
Another way is to boot into initramfs, download that image, and then switch root just like any regular boot process would. In this case your squashfs size is not limited by tftp anymore.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Wed Aug 05, 2020 12:48 am    Post subject: Reply with quote

szatox wrote:
This is a line from my working setup. Haven't used it in a long time, but I'm pretty sure it would still boot.
I'm using isolinux, but it does not affect kernel params.
Code:
APPEND ip=dhcp root=/dev/ram0 cdroot=1 real_root=/dev/nfs nfsroot=10.0.1.254:/mnt/linux.images/tftp/gentoo-amd64 initrd=gentoo-amd64/initramfs loop=2018-07-07.sqfs looptype=squashfs net.ifnames=0 console=ttyS1,57600

You can surely figure out which params are important for your case. (Hint: redirecting system console to serial probably is not :lol: )

Doing it this way, you stay connected to the NFS at all times.
I've learned a few tricks since then and I now prefer to copy everything into RAM if possible, so I can detach from NFS releasing the network bandwidth, cutting down IO lags, removing remote SPOF, and making my diskless machine strong and independent :lol:

And yes, one way to do that is putting everything into your initramfs - if you can compress it under ~100MB (tftp can't transfer bigger files, so it won't boot if you exceed that).
Another way is to boot into initramfs, download that image, and then switch root just like any regular boot process would. In this case your squashfs size is not limited by tftp anymore.



We need to boot fully from ramfs, how can we get this solved?
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1898

PostPosted: Fri Aug 07, 2020 10:33 pm    Post subject: Reply with quote

Well, you can send your requirements along with some btc.
Or use "search" function and ask specific questions. We tend to be quite happy educating newcomers and picking challenges.
You already know the general direction, so get going and let us know what bumps you hit on your way.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45826
Location: 56N 3W

PostPosted: Sat Aug 08, 2020 8:10 am    Post subject: Reply with quote

finalturismo,

Make it work with an nfs root first.
That will teach you a lot and you will need that knowledge to move on.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
finalturismo
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2020
Posts: 99

PostPosted: Tue Aug 11, 2020 1:29 pm    Post subject: Reply with quote

NeddySeagoon wrote:
finalturismo,

Make it work with an nfs root first.
That will teach you a lot and you will need that knowledge to move on.


Iam at the point where i will pay someone to do a remote session with me to help me on this part, i really need help on it and it is driving me up a wall.

We are literally at the very end of the process, we already did the squashFS. The PXE starts posting the OS correctly and everything.

It just stops on initramfs message because unable to mount root.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 45826
Location: 56N 3W

PostPosted: Tue Aug 11, 2020 4:02 pm    Post subject: Reply with quote

finalturismo,

Pastebin your kernel .config
and your bootloader config.

I used to use syslinux by others should work too.

For root over nfs, you must use nfs version 3.

Pepper the initscript with debug to see why it can't mount root.

Does it work if root is not over the network?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum