Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Using Rock Pi 4 as a gentoo router.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM
View previous topic :: View next topic  
Author Message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Fri Feb 07, 2020 12:28 am    Post subject: Using Rock Pi 4 as a gentoo router. Reply with quote

I currently use a MikroTik wireless router. It is quite flexible, but it is not as flexible as a full-fledged linux operating system.

I'm thinking about turning the MikroTik router into a switch and a wireless access point and using Rock Pi 4 as a router by adding a USB 3 gigabit ethernet adaptor to Rock Pi 4.

Internet <--> USB3 gigabit ethernet adaptor <--> Rock Pi 4A with 256GB NVMe SSD and 4GB RAM <--> Onboard gigabit ethernet adaptor <--> MikroTik switch & wireless access point <--> wireless clients and wired clients

Rock Pi 4 can serve as its own build machine with 4GB RAM and 256GB NVMe SSD and zswap.

I have some doubts.

  • Upgrading gentoo on a router is not a good idea. If the router becomes unavailable due to maintenance, I cannot use the internet until it is fixed.
    • I haven't yet made atomic upgrades possible on gentoo.
    • While I can build binary packages in a ZFS clone, things can still break during binary package upgrades.
    • A binary package upgrade can take anywhere between 30 minutes and 1.5 hours.
  • Building gentoo packages saps a lot of CPU resource from routing and firewalling.
Back to top
View user's profile Send private message
pa4wdh
Guru
Guru


Joined: 16 Dec 2005
Posts: 420

PostPosted: Fri Feb 07, 2020 7:05 am    Post subject: Reply with quote

I'm not using a Rock Pi 4 but a Soekris Net5501-70 (http://www.soekris.com/products/net5501-1.html), so i can't say anything about the Rock Pi, but i can share my experience with Gentoo on this low-power hardware.

Quote:

Upgrading gentoo on a router is not a good idea. If the router becomes unavailable due to maintenance, I cannot use the internet until it is fixed.

In my case I run gentoo on the router itself, including compiling (a recent gcc takes 4 days 8O). By setting the PORTAGE_NICENESS the system is still responsive, and this is a 500 MHz single core AMD Geode, i'd expect a recent CPU to do better. The Rock Pi 4 has a quad core processor, setting MAKEOPTS to -j3 for example will make sure you'll always have enough CPU power left to do some routing work. I don't know what you WAN throughput is, but my setup easily handles my 55 Mbit down/11 Mbit up link including two VPN's and some services.
The only downtime i have are reboots, for the rest it has been running 24/7 since 2008.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

Free as in Freedom is not limited to software only:
Music: http://www.jamendo.com
Recipes: http://www.opensourcefood.com
Back to top
View user's profile Send private message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Fri Feb 07, 2020 10:46 am    Post subject: Reply with quote

pa4wdh wrote:
In my case I run gentoo on the router itself, including compiling (a recent gcc takes 4 days 8O).


Do you build binary packages in a ZFS clone? Or, do you just upgrade the main system? Do you use swap?


Last edited by crocket on Sat Feb 08, 2020 12:43 am; edited 1 time in total
Back to top
View user's profile Send private message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Fri Feb 07, 2020 1:56 pm    Post subject: Reply with quote

I'm also thinking about going cheap and hooking an HDD to a USB 2.0 port of NanoPi R2S.
NanoPi R2S has 1GB RAM and two gigabit ethernet ports.

I'm worried that routing and firewalling might slow down to a crawl while compilation uses swap on HDD because routing and firewalling will also be done on swap.
VPN might also use swap during compilation.
Back to top
View user's profile Send private message
pa4wdh
Guru
Guru


Joined: 16 Dec 2005
Posts: 420

PostPosted: Sat Feb 08, 2020 9:46 am    Post subject: Reply with quote

crocket wrote:

Do you build binary packages in a ZFS clone? Or, do you just upgrade the main system? Do you use swap?

Nope. It's a complete system on it's own, building everything it needs. It follows the regular method for updates just as any other gentoo box, (sync, update, etc.).
Swap is required in my case, 512MB is not enough to compile bigger packages, even with MAKETOPS="-j1". For quite some time 512MB of swap was enough, i've just added anther 2 GB to make sure GCC fits, since it's really annoying when it stops after 3 days of work because of the lack of memory.
The 4 GB of the Rock Pi might be enough, but having a bit of swap available during compiling won't hurt it.

To give you the complete overview, this is how I use it:
The system has a built-in CompactFlash adapter which i use to boot. It contains grub and /boot. The normal system is placed on an external USB2.0 HDD which contains / and LVM for some additional storage.
I update this system in the same way as you would update any gentoo system and has ext3 for / (ext4 didn't exist when i started this :-) ).
When the updates are ready i have a script that creates a ramdisk image by picking the files it needs from the filesystem and placing it in the image. When that is done I reboot the system to run the newly created ramdisk image and disconnect the external HDD.
The extra advantage is that when something goes really wrong (like being hacked or whatever), a reboot always gives a clean image, and the integrity of the system on the HDD is never compromised.

Quote:

Im also thinking about going cheap and hooking an HDD to a USB 2.0 port of NanoPi R2S.
NanoPi R2S has 1GB RAM and two gigabit ethernet ports.

Sounds good to me, although more ram usually results in faster compiles (building small packages in tmpfs, less swap usage for big packages). Also, be aware that one of the ethernet ports is also connected through USB, so they might impact each others performance.

Quote:

I'm worried that routing and firewalling might slow down to a crawl while compilation uses swap on HDD because routing and firewalling will also be done on swap.

I've never experienced any routing/firewalling delays when the system is on full load and swapping, for example when compiling the kernel or gcc.
Your firewall rules live in the kernel space and as far as I know it will never be swapped out, also, the amount of memory freed by that would not be worth the effort. I'm using ip(6)tables extensively and never experienced any slowdowns, even on this slow CPU. To give you an indication for the size of my ruleset: The export for iptables is 616 lines, for ip6tables it's 589 lines.

Quote:

VPN might also use swap during compilation.

You VPN doesn't use more memory when there's more traffic, so it's memory usage it quite stable. If the VPN is in use it would be a foolish decision for the kernel to swap that out.

Since you seem to be worried about the load: During normal use (now for example) the load (as reported by uptime or top) is less than 0.1. During comping just make sure to set NICE and IONICE to the highest values (basically giving it lowest priority) and from the network you'll barely notice it's happening. In that way your compile jobs will just take longer if you use the system for other things (like routing, vpn, whatever)

I hope this helps, feel free to ask additional questions.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

Free as in Freedom is not limited to software only:
Music: http://www.jamendo.com
Recipes: http://www.opensourcefood.com
Back to top
View user's profile Send private message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Sat Feb 08, 2020 10:28 am    Post subject: Reply with quote

NanoPi R2S has an onboard gigabit ethernet port and a USB 3 gigabit ethernet port. Since USB 3 and USB 2 live separately, they will not interfere with each other.

It seems a live upgrade that takes days don't really break programs running in RAM on your system although I couldn't re-attach to existing tmux sessions if tmux was upgraded during the system upgrade.
The upgrade was running in a tmux session.
I solved this problem partially by building binary packages in a ZFS clone and installing binary packages. A binary upgrade takes anywhere between 30 minutes to 1.5 hours.

I recommend considering zswap or zram for reducing swap usage.
Back to top
View user's profile Send private message
pa4wdh
Guru
Guru


Joined: 16 Dec 2005
Posts: 420

PostPosted: Sat Feb 08, 2020 11:09 am    Post subject: Reply with quote

crocket wrote:
It seems a live upgrade that takes days don't really break programs running in RAM on your system although I couldn't re-attach to existing tmux sessions if tmux was upgraded during the system upgrade.
The upgrade was running in a tmux session.

I had the same situation once with screen. The update was still running but i couldn't attach to it. I just waited for the update to finish (visible in /var/log/emerge.log) and killed screen after that. Since that only happened once in 12 years now i didn't bother to do anything to prevent it from happening again.

Quote:

I recommend considering zswap or zram for reducing swap usage.

Well, the CPU isn't a powerhouse to start with, i think the slowdown of using the HDD is actually less than compressing/decompressing stuff ram/swap. Are there any benchmarks on zram/zswap?
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world

Free as in Freedom is not limited to software only:
Music: http://www.jamendo.com
Recipes: http://www.opensourcefood.com
Back to top
View user's profile Send private message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Sat Feb 08, 2020 11:39 am    Post subject: Reply with quote

Even if there were benchmarks on zswap or zram, they wouldn't be helpful to you because benchmark results would depend on HDDs and CPUs.
Back to top
View user's profile Send private message
erm67
l33t
l33t


Joined: 01 Nov 2005
Posts: 647
Location: EU

PostPosted: Sat Feb 08, 2020 1:28 pm    Post subject: Reply with quote

Zram was developed by Google to be used on devices without a swap, it is proven beyond doubt that using zram and a swap will lead to starvation since the zram will be full of old unused pages that will be never evicted, and all transient memory needs will use the swap. In practice the result Is less memory available since it will be occupied by useless pages never used, and no reduction in swapping.

Zswap instead will correctly send to the disk old unused pages and use the RAM for short term needs, this assumes the swap Is functional of course.

Swapping on USB2 won't work because Is half duplex probably zram and no swap is better.
_________________
Ok boomer
True ignorance is not the absence of knowledge, but the refusal to acquire it.
Ab esse ad posse valet, a posse ad esse non valet consequentia

My fediverse account: @erm67@erm67.dynu.net
Back to top
View user's profile Send private message
crocket
Guru
Guru


Joined: 29 Apr 2017
Posts: 454

PostPosted: Sat Feb 08, 2020 1:32 pm    Post subject: Reply with quote

So, adding a USB 3.0 gigabit ethernet adaptor to rock pi 4 or nanopi m4 seems to be the only sane option for now.

What about using Rock Pi 4A 1GB, Rock Pi 4A 2GB, or NanoPi M4 2GB with zswap on an NVMe SSD?

Do you know cheaper but still decent options for a gentoo router? It's hard to beat Rock Pi 4A 4GB and 256GB NVMe SSD with cheaper options. If I really wanted to go cheaper, I would probably choose Rock Pi 4A 1GB and 128GB NVMe SSD.

I'm still not sure that using a USB 3 gigabit ethernet adaptor in a router is a good idea. I'm still not convinced of reliability of USB devices. Some USB microphones are known to die after months of usage.

I'm also thinking about using a binary linux distribution on NanoPi R2S and cross-compiling linux kernel on my AMD64 desktop machine although I would lose the flexibility of portage and the time investments I made into my overlay. Gentoo cannot be a friend with low-end machines.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on ARM All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum