Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
How can I avoid always rebuilding qtwebengine?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Mon Jun 29, 2020 7:37 pm    Post subject: How can I avoid always rebuilding qtwebengine? Reply with quote

Every few weeks I have to rebuild qtwebengine. It takes between 7 and 9 hours on my little i7 with 16GB RAM. Here's the history:
Code:
   Fri Aug 23 20:19:02 2019 >>> dev-qt/qtwebengine-5.12.3
       merge time: 9 hours, 8 minutes and 28 seconds.

     Wed Aug 28 20:34:47 2019 >>> dev-qt/qtwebengine-5.12.3
       merge time: 9 hours, 14 minutes and 21 seconds.

     Tue Oct 22 21:12:01 2019 >>> dev-qt/qtwebengine-5.12.5
       merge time: 9 hours, 17 minutes and 46 seconds.

     Fri Jan 24 08:09:15 2020 >>> dev-qt/qtwebengine-5.12.5
       merge time: 5 hours, 45 minutes and 8 seconds.

     Fri Feb  7 07:36:02 2020 >>> dev-qt/qtwebengine-5.13.2
       merge time: 6 hours, 27 minutes and 32 seconds.

     Wed Feb 12 06:07:55 2020 >>> dev-qt/qtwebengine-5.13.2
       merge time: 6 hours, 32 minutes and 33 seconds.

     Wed Mar 25 04:02:50 2020 >>> dev-qt/qtwebengine-5.14.1
       merge time: 7 hours, 17 minutes and 57 seconds.

     Thu May 21 06:35:16 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 7 hours, 9 minutes and 27 seconds.

     Sun May 24 03:32:17 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 7 hours, 16 minutes and 50 seconds.

     Thu May 28 09:57:30 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 8 hours, 40 minutes and 32 seconds.

     Fri Jun 26 22:56:23 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 8 hours, 8 minutes and 29 seconds.

And it's doing it again right now a mere 3 days since the last time. I tried masking it, but that messed up other things. In any case, even it it were masked I'd still have had to rebuild it 5 times in the last month.
Is there some way to avoid this? What I'd like is a "-bin" version, but that's not available (or is there one on an overlay somewhere?). If I can't solve this, I'm thinking of just unmerging the two packages that depend on this, but I'd rather keep them.
Any suggestions?
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4717
Location: Dallas area

PostPosted: Mon Jun 29, 2020 8:09 pm    Post subject: Reply with quote

--exclude ( emerge option, man emerge)
_________________
PRIME x570-pro, 3700x, RX 550 & 560 - 5.8 zen kernel
Acer E5-575 (laptop), i3-7100u - i965 - 5.5 zen kernel
---both---
gcc 9.3.0, profile 17.1 (no-pie & modified) amd64-no-multilib, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7874
Location: Austria

PostPosted: Mon Jun 29, 2020 8:20 pm    Post subject: Reply with quote

What is the reason for the latest rebuild?
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
toralf
Developer
Developer


Joined: 01 Feb 2004
Posts: 3797
Location: Hamburg

PostPosted: Mon Jun 29, 2020 8:30 pm    Post subject: Reply with quote

At an i5 it takes here 5 1/2 hour with "-j3" - do you use "-j1" ?
Back to top
View user's profile Send private message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Mon Jun 29, 2020 9:19 pm    Post subject: Reply with quote

Anon-E-moose wrote:
--exclude ( emerge option, man emerge)


Isn't this going to cause problems?
Back to top
View user's profile Send private message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Mon Jun 29, 2020 9:22 pm    Post subject: Reply with quote

toralf wrote:
At an i5 it takes here 5 1/2 hour with "-j3" - do you use "-j1" ?


Code:
MAKEOPTS="-j5"
Back to top
View user's profile Send private message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Mon Jun 29, 2020 9:38 pm    Post subject: Reply with quote

asturm wrote:
What is the reason for the latest rebuild?


I'm not exactly sure but I think it's that one or more of its dependencies were updated. Sometimes when I emerge -uDN @world I get the message saying "somethingorother is causing rebuilds in these packages" (or something to that effect).

Didn't notice what it said this time, but emerge -pvuDN @world output includes this line:

Code:
[ebuild  rR    ] dev-qt/qtwebengine-5.14.2:5/5.14::gentoo  USE="alsa system-ffmpeg system-icu widgets -bindist -debug -designer -geolocation -jumbo-build -kerberos -pulseaudio -test" 0 KiB


I believe that the rR indicates forced rebuilds. Isn't this how emerge is automatically doing what revdep-rebuild used to do (but is not necessary anymore)?
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 4717
Location: Dallas area

PostPosted: Mon Jun 29, 2020 10:50 pm    Post subject: Reply with quote

sl70 wrote:
Anon-E-moose wrote:
--exclude ( emerge option, man emerge)


Isn't this going to cause problems?


Well, yes and no :lol:

For multiple rebuilds of the same version, you could use exclude, when changing versions, then rebuild don't exclude.
_________________
PRIME x570-pro, 3700x, RX 550 & 560 - 5.8 zen kernel
Acer E5-575 (laptop), i3-7100u - i965 - 5.5 zen kernel
---both---
gcc 9.3.0, profile 17.1 (no-pie & modified) amd64-no-multilib, eudev, openrc, openbox, palemoon
Back to top
View user's profile Send private message
Tyrus
Apprentice
Apprentice


Joined: 03 Feb 2018
Posts: 293

PostPosted: Mon Jun 29, 2020 11:42 pm    Post subject: Reply with quote

System is here: x86_64 Intel(R) Core(TM) i3-6100 CPU @ 3.70GHz
Memory: 8 GB
MAKEOPTS="-j3"

For May I had 3 rebuilds (Version 5.14.2 was merged in April first time) and 1 update to Version 5.15.0
Code:

[...]

     Fri May  1 10:31:21 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 3 hours, 30 minutes and 54 seconds.

     Fri May 22 10:31:46 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 3 hours, 34 minutes and 32 seconds.

     Sun May 24 18:58:38 2020 >>> dev-qt/qtwebengine-5.14.2
       merge time: 3 hours, 48 minutes and 11 seconds.

     Wed May 27 21:39:30 2020 >>> dev-qt/qtwebengine-5.15.0
       merge time: 4 hours, 18 minutes and 10 seconds.


   Total builds: 56
   Global build time: 13 hours, 33 minutes and 53 seconds.
   Average merge time: 3 hours, 14 minutes and 32 seconds.

   Info about currently installed ebuild:

   * dev-qt/qtwebengine-5.15.0
   Install date: Wed May 27 21:39:30 2020
   USE="alsa designer jumbo-build pulseaudio widgets -bindist -geolocation -kerberos -+system-ffmpeg -+system-icu"
   CFLAGS="-march=native -O2 -pipe"   CXXFLAGS="-march=native -O2 -pipe"   LDFLAGS="-Wl,-O1 -Wl,--as-needed"


The merge is done at night in most cases. USE='jumbo-build' still helps to speed up the build process.

Most rebuilds are caused by USE='system-ffmpeg' and USE='system-icu' but thats ok for me. I want both USE-flags.

Edit:
BTW - "Global build time: 13 hours, 33 minutes and 53 seconds." - looks strange - checked it again with 'qlop'.

Code:

qlop -c qtwebengine

dev-qt/qtwebengine: 3:14:32 average for 56 merges
total: 181:33:53 for 56 merges
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2789
Location: Canada

PostPosted: Tue Jun 30, 2020 5:59 am    Post subject: Reply with quote

asturm wrote:
What is the reason for the latest rebuild?




upgrade to dev-libs/re2 . And that little thing tends to be updated relatively frequently. This ' efficient, principled regular expression library', probably already burnt a small forest in energy consumption.
Back to top
View user's profile Send private message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Tue Jun 30, 2020 6:03 am    Post subject: Reply with quote

dmpogo wrote:
asturm wrote:
What is the reason for the latest rebuild?




upgrade to dev-libs/re2 . And that little thing tends to be updated relatively frequently :)


The latest stable version is already installed.

Code:
 equery l dev-libs/re2
 * Searching for re2 in dev-libs ...
[IP-] [  ] dev-libs/re2-0.2020.05.01:0/7
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2789
Location: Canada

PostPosted: Tue Jun 30, 2020 6:14 am    Post subject: Reply with quote

sl70 wrote:
dmpogo wrote:
asturm wrote:
What is the reason for the latest rebuild?




upgrade to dev-libs/re2 . And that little thing tends to be updated relatively frequently :)


The latest stable version is already installed.

Code:
 equery l dev-libs/re2
 * Searching for re2 in dev-libs ...
[IP-] [  ] dev-libs/re2-0.2020.05.01:0/7



Ok, in my case it is not yet, and I am delaying its upgrade together with rebuild of qtwebengine
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7874
Location: Austria

PostPosted: Tue Jun 30, 2020 6:45 am    Post subject: Reply with quote

sl70 wrote:
dmpogo wrote:
upgrade to dev-libs/re2. And that little thing tends to be updated relatively frequently :)


The latest stable version is already installed.

Exactly, re2 was stabilised by the beginning of June, not just very recently. That's why I was asking sl70 because there should not have been any rebuild triggered for qtwebengine in weeks.
sl70 wrote:
Code:
[ebuild  rR    ] dev-qt/qtwebengine-5.14.2:5/5.14::gentoo  USE="alsa system-ffmpeg system-icu widgets -bindist -debug -designer -geolocation -jumbo-build -kerberos -pulseaudio -test" 0 KiB


I believe that the rR indicates forced rebuilds. Isn't this how emerge is automatically doing what revdep-rebuild used to do (but is not necessary anymore)?

Yes, and you look out for the package that is responsible for that [r] as mentioned by portage at the end of the package list. And *that* package - not qtwebengine - you can delay upgrading via --exclude until e.g. there are several upgrades like that.

Last qtwebengine stabilisation was actually coordinated with all its dependencies that usually trigger rebuilds, but still ended up a few days apart because of individual issues in those deps. For having to rebuild it three times is really bad luck and timing though, but hints at a daily update routine. If course, ~arch users get every single update and thus all the rebuilds.
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
toralf
Developer
Developer


Joined: 01 Feb 2004
Posts: 3797
Location: Hamburg

PostPosted: Tue Jun 30, 2020 7:11 am    Post subject: Reply with quote

sl70 wrote:
toralf wrote:
At an i5 it takes here 5 1/2 hour with "-j3" - do you use "-j1" ?


Code:
MAKEOPTS="-j5"
Hhm, do you run - by any chance - emerge with niceness?
Back to top
View user's profile Send private message
Goverp
l33t
l33t


Joined: 07 Mar 2007
Posts: 877

PostPosted: Tue Jun 30, 2020 8:24 am    Post subject: Reply with quote

AFAIR qtwebengine is one where using the compiler cache works, though of course it makes the first compilation of each new version even slower.
_________________
Greybeard
Back to top
View user's profile Send private message
dmpogo
Advocate
Advocate


Joined: 02 Sep 2004
Posts: 2789
Location: Canada

PostPosted: Tue Jun 30, 2020 6:23 pm    Post subject: Reply with quote

I made myself a rule not to update qtwebengine on the first day I see the update, but wait for a couple of weeks at least to see if dependencies stabilize.
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7874
Location: Austria

PostPosted: Wed Jul 01, 2020 9:13 pm    Post subject: Reply with quote

Actually fired up the stable box now and I am sorry but I can't find a reason why qtwebengine would be rebuilt *now*. In fact, it wasn't rebuilt at all since stabilisation of 5.14.2 plus a couple of days. Maybe you should check your package.accept_keywords.
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
sl70
Guru
Guru


Joined: 18 Jun 2002
Posts: 397
Location: Chicago, USA

PostPosted: Sat Jul 04, 2020 1:42 pm    Post subject: Reply with quote

Goverp wrote:
AFAIR qtwebengine is one where using the compiler cache works, though of course it makes the first compilation of each new version even slower.


This is a good idea, but I'm still trying to figure out how to build only qtwebengine with ccache. I don't want to use it for everything, mainly because I'm afraid I don't have enough disk space for the cache, and don't want to increase the build time for all packages.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 15623

PostPosted: Sat Jul 04, 2020 4:40 pm    Post subject: Reply with quote

As I read https://wiki.gentoo.org/wiki//etc/portage/package.env, you could use a per-package environment modification (scoped to qtwebengine) to enable FEATURES=ccache just for that package.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum