Funtoo:Metatools

   Summary

This project is focused on the development of Funtoo's metatools and autogeneration technology.

View Scope...

   Subpages

• Releases

   People

Leads

• drobbins

Contributors

• alex2101
• invakid404

   Latest Status

Metatools 1.1.0 has been released, which includes a lot of new features and improvements.

19 August 2022

Metatools (git) (pypi) is the framework used by Funtoo Linux to create meta-repo and autogenerate ebuilds. It is a foundational tool for Funtoo Linux development.

The latest release of metatools is 1.3.5, released on 09 November 2023. It is a regular software release. View Release Notes.

What is an "Autogen"?

Quite simply, an "autogen" is kind of like an ebuild generator, using the metatools Python framework to dynamically generate ebuilds. A key functionality of an autogen is to allow ebuilds to be generated to reflect the current version of source code that is available from upstream.

Where can I Find Autogens?

You can find autogens by looking in the kit-fixups repository (git) for any files named:

1. autogen.py
2. autogen.yaml

Both YAML and Python autogens use our metatools framework -- the YAML variants are higher-level autogens that feed metatools "generators", kind of like a pipeline. As of Jan 29, 2022, all Funtoo Linux source overlays -- kit-fixups as well as others -- support the creation of autogens. What this means is that it's possible to write an autogen for anything in Funtoo to ensure that it is always kept up-to-date.

Installation

To install metatools, emerge it:

root # emerge metatools

Autogen Setup

The command in metatools that performs autogeneration locally is called doit. Prior to using this command locally, it's recommended to create an ~/.autogen file under your user account that contains a GitHub personal access token. This will allow doit to use it when querying GitHub and will prevent doit from hitting API limits:

   ~/.autogen

authentication:
  api.github.com:
    username: danielrobbins
    password: <insert access token string here>

For more information on setting up GitHub personal access tokens, see: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token

Autogen Usage

Funtoo's meta-repo is defined in the kit-fixups repository (git), which is the starting point for all autogeneration of ebuilds.

To perform your first doit run, enter into the appropriate kit-fixups directory and run the doit command:

user $ cd development/kit-fixups/core-kit/curated
user $ doit
[INFO 21:02:35] Fetched https://git-scm.com/download/linux 6042 bytes
[INFO 21:02:35] Created: dev-vcs/git/git-2.35.1.ebuild
[INFO 21:02:35] Fetched https://api.github.com/repos/hashicorp/vagrant_cloud/tags 9513 bytes
[INFO 21:02:36] Created: dev-ruby/vagrant_cloud/vagrant_cloud-2.0.3.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/distcc/distcc/releases 34549 bytes
[INFO 21:02:36] Created: sys-devel/distcc/distcc-3.4.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/dbrgn/tealdeer/releases 151336 bytes
[INFO 21:02:36] Fetched https://api.github.com/repos/tmux/tmux/releases 94211 bytes
[INFO 21:02:36] Created: app-misc/tmux/tmux-3.2a.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/lotabout/skim/releases 225892 bytes
[INFO 21:02:36] Fetched https://ftp.debian.org/debian/pool/main/c/ca-certificates/ 7174 bytes
[INFO 21:02:36] Created: app-misc/ca-certificates/ca-certificates-20211016.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/jarun/nnn/releases 564842 bytes
[INFO 21:02:36] Fetched https://api.github.com/repos/netwide-assembler/nasm/tags 13895 bytes
[INFO 21:02:36] Created: app-misc/nnn/nnn-4.4.ebuild
[INFO 21:02:36] Fetched https://api.github.com/repos/doxygen/doxygen/tags?per_page=100 46121 bytes

What you are seeing above is output from the doit command, which will:

1. Find all autogens (autogen.yaml and autogen.py files) in the current directory tree.
2. Execute them using multiple threads, during which doit will:
   1. Query upstream Web sites and APIs (like GitHub) for current versions of source code,
   2. Fetch all source code artifacts (distfiles) by using an efficient spider engine, and
   3. Auto-create ebuilds for these packages, along with Manifests.

Funtoo contributors will use the doit command locally to test their autogens and ensure their proper operation before submitting a PR.

Where Can I Learn More about Metatools?

The best place to learn is to join us in the #metatools and #dev-help channels on Discord.

There is "in progress" documentation for metatools at our to-be-launched projects site: https://projects.funtoo.org/metatools/docs/install.html SSL Cert is not yet valid and the documentation is a work in progress. Refer to this wiki page for current setup documentation until the documentation has caught up with the current release of metatools.

Status Updates

   Funtoo:Harvester

harvester/2023-11 has been merged into official Funtoo, and harvester/2024-02 has been created. Please hold off on PRs to harvester. We are going to define a more structured approach to harvester development very soon so we can move forward strategic initiatives.

01 February 2024

   Funtoo:Harvester

harvester/2023-10 is merged into Funtoo and we now have harvester/2023-11 ready for PR's!

01 November 2023

   Funtoo:Harvester

Harvester/2023-10 is launched. See the #gumball3000 channel on Discord for details.

07 October 2023

   Funtoo:Harvester

harvester/2023-08 is available, and is primarily targeted right now towards 6.4 kernel and core toolchain updates.

27 July 2023

   Funtoo:Harvester

Harvester/2023-06, along with quite a few meson updates from geaaru/macaroniOS, have been merged into master. We will be starting our next harvester branch a bit later due to a slower-paced July. Enjoy your summer.

13 July 2023

   Funtoo:Harvester

The harvester/2023-05 branch was merged into Funtoo.

30 May 2023