Funtoo Metatools Project

Latest Status

Metatools 1.1.0 has been released, which includes a lot of new features and improvements.

19 August 2022

Subpages:

Metatools (git) (pypi) is the framework used by Funtoo Linux to create meta-repo and autogenerate ebuilds. It is a foundational tool for Funtoo Linux development.

The latest release of metatools is 1.3.5, released on 09 November 2023. It is a regular software release. View Release Notes.

What is an "Autogen"?

Quite simply, an "autogen" is kind of like an ebuild generator, using the metatools Python framework to dynamically generate ebuilds. A key functionality of an autogen is to allow ebuilds to be generated to reflect the current version of source code that is available from upstream.

Where can I Find Autogens?

You can find autogens by looking in the kit-fixups repository (git) for any files named:

autogen.py
autogen.yaml

Both YAML and Python autogens use our metatools framework -- the YAML variants are higher-level autogens that feed metatools "generators", kind of like a pipeline. As of Jan 29, 2022, all Funtoo Linux source overlays -- kit-fixups as well as others -- support the creation of autogens. What this means is that it's possible to write an autogen for anything in Funtoo to ensure that it is always kept up-to-date.

Installation

To install metatools, emerge it:

root # emerge metatools

Autogen Setup

The command in metatools that performs autogeneration locally is called doit. Prior to using this command locally, it's recommended to create an ~/.autogen file under your user account that contains a GitHub personal access token. This will allow doit to use it when querying GitHub and will prevent doit from hitting API limits:

~/.autogen

authentication:
  api.github.com:
    username: danielrobbins
    password: <insert access token string here>

For more information on setting up GitHub personal access tokens, see: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token

Autogen Usage

Funtoo's meta-repo is defined in the kit-fixups repository (git), which is the starting point for all autogeneration of ebuilds.

To perform your first doit run, enter into the appropriate kit-fixups directory and run the doit command:

user $ cd development/kit-fixups/core-kit/curated
user $ doit
[INFO 21:02:35] Fetched https://git-scm.com/download/linux 6042 bytes
[INFO 21:02:35] Created: dev-vcs/git/git-2.35.1.ebuild
[INFO 21:02:35] Fetched https://api.github.com/repos/hashicorp/vagrant_cloud/tags 9513 bytes
[INFO 21:02:36] Created: dev-ruby/vagrant_cloud/vagrant_cloud-2.0.3.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/distcc/distcc/releases 34549 bytes
[INFO 21:02:36] Created: sys-devel/distcc/distcc-3.4.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/dbrgn/tealdeer/releases 151336 bytes
[INFO 21:02:36] Fetched https://api.github.com/repos/tmux/tmux/releases 94211 bytes
[INFO 21:02:36] Created: app-misc/tmux/tmux-3.2a.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/lotabout/skim/releases 225892 bytes
[INFO 21:02:36] Fetched https://ftp.debian.org/debian/pool/main/c/ca-certificates/ 7174 bytes
[INFO 21:02:36] Created: app-misc/ca-certificates/ca-certificates-20211016.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/jarun/nnn/releases 564842 bytes
[INFO 21:02:36] Fetched https://api.github.com/repos/netwide-assembler/nasm/tags 13895 bytes
[INFO 21:02:36] Created: app-misc/nnn/nnn-4.4.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/doxygen/doxygen/tags?per_page=100 46121 bytes

What you are seeing above is output from the doit command, which will:

Find all autogens (autogen.yaml and autogen.py files) in the current directory tree.
Execute them using multiple threads, during which doit will:
1. Query upstream Web sites and APIs (like GitHub) for current versions of source code,
2. Fetch all source code artifacts (distfiles) by using an efficient spider engine, and
3. Auto-create ebuilds for these packages, along with Manifests.

Funtoo contributors will use the doit command locally to test their autogens and ensure their proper operation before submitting a PR.

Where Can I Learn More about Metatools?

The best place to learn is to join us in the #metatools and #dev-help channels on Discord.

There is "in progress" documentation for metatools at our to-be-launched projects site: https://projects.funtoo.org/metatools/docs/install.html SSL Cert is not yet valid and the documentation is a work in progress. Refer to this wiki page for current setup documentation until the documentation has caught up with the current release of metatools.