Something you might want to know about Calculate Utilities 3.5.1

As you might have noticed, Calculate Utilities have been updated recently. A small yet important update.

(Auto)magic dependencies

After so many betas, we were literally happy to finally release v3.5.1. Solving the so-called automagic dependencies, that are effective without being seen, is fully automated now. And solving means not avoiding, as it were the case in earlier beta versions!

That’s how it works:
At building time, a clean system is emulated and emerge is called for every package to be built. Dependencies are thus calculated for every package. Then the package manager checks for automagic dependencies - that is, how they match the libraries already in use. The binary will be recompiled accordingly. Or it may be the opposite way, when the package is linked to libraries it should not depend on: unnecessary dependencies will be disabled, the binary will be recompiled accordingly. For instance, digiKam can be recompiled without Akonadi.

Even though the package manager installs all new dependencies the Calculate tools suggest, the '-depclean’ action removes them all, as priority is given to the ebuilds provided by Portage. We turned around this problem by adding the @autodeps set, that lists all those automagic dependencies based on autodeps files (/var/db/pkg/*/*/autodeps) CU have put in binaries.

Try running for instance:

emerge -Op @autodeps

The list is very likely empty right now. This means that either your packages have no automagic dependencies, or the latter were removed, or else something went wrong. Now let’s install www-client/otter. Otter is a browser running on QtWebKit, but it is also linked with QtWebEngine and might so be quite operational when used with it.

emerge www-client/otter

Once the browser has been installed, the autodeps set will not be empty any more. Run the following to see for yourself:

emerge -pO @autodeps

[binary R ] dev-qt/qtwebengine-5.7.1-r1

The set is simply a configuration file (/usr/share/portage/config/sets/calculate.conf), containing a command for unpacking ‘autodeps’ files.

Better management of the binary repo

OK, you might have been ignorant about automagic dependencies. But there is a nuisance you must have noticed: the update procedure is far from beeing speedy, as emerge was not specifically created for binary support. While Portage is synchronized wth a single call, the Packages index is updated every time you run emerge. This behaviour is both good and bad. On the one hand, it allows to install packages without Portage update; on the other hand, it can generate binary conflicts.

To reduce update time, considerable effort was made by the maintainers. Bandwidth consumption was adapted both to HTTP and FTP servers, either by fetching the file header or by reading the first 8 Kb of the file to extract the modification time, thus avoiding complete download. On HTTP servers only, packing index files is also supported. If there are any emerge errors though (for example, if Internet connection is broken for some reason on a FTP server, or due to incorrect timezone detection when scanning the HTTP query), a 6 Mb file will be fetched. As the result, the update may be lagging.

Well, we fixed this with the TTL option. It sets the time for the Packages files to be considered valid. Cl-update performs a mirror validity checkup, fetches the Packages archive and puts TTL inside. Once this is complete, emerge will not update Packages. The file size is now about 680 Kb. Isn’t it nice? As the Utilities do the whole of the index update job, we respect reverse compatibility and do not have to modify Portage to add any new software. This, of course, was our plan all along!

Miscellaneous

The cl-update tool used to cache the calculated dependencies and had the ‘–nodeps’ option enabled by default to make package installation faster (when launched from the system tray). Well, it shouldn’t. Regular update checks are a strain on the processor and on your Internet bandwidth, while the cached data will not always be used. As the result, the list of dependencies was quite often calculated twice, first to display it and then to update. We fixed this in this new version. However, we are still working on a less bandwidth consuming update procedure. We also hope to take the load from the processor.

Last but not least: our nightly stages are more lightweight now, as we were able to get rid of some 110 packages that were only needed for compilation. You can do the same on your system if you wish so:

emerge -ac --with-bdeps=n

(freeing up around 300 MB).