The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.
Difference between revisions of "Funtoo:Evolved Bootstrap"
| Line 66: | Line 66: | ||
Join us in the #bootstrap channel in [https://discord.gg/BNUSpUU Funtoo Discord] and also see below for status updates and instructions on how to join our collaborative effort. | Join us in the #bootstrap channel in [https://discord.gg/BNUSpUU Funtoo Discord] and also see below for status updates and instructions on how to join our collaborative effort. | ||
{{Important|The most detailed documentation is viewable at https://code.funtoo.org/bitbucket/projects/CORE/repos/ffs/browse/README.rst. This README file documents how to build Funtoo from Scratch inside an LXD container, which is now the recommended way to use Funtoo from Scratch.}} | |||
== Basic Use == | |||
The following steps demonstrate the basic workflow for Funtoo from Scratch, by walking through the simplified build process, without using LXD to | |||
host the build. Funtoo from Scratch can build MUSL micro-environments as well as perform a Funtoo stage1 bootstrap. Below, we look at the MUSL | |||
option. | |||
This will build a complete native toolchain and initial environment with a MUSL libc on {{c|arm-64bit}}, and you can also specify {{c|x86-64bit}} (for a fake-cross on a PC), {{c|riscv-64bit}}, or {{c|powerpc-64bit}}. Once the native toolchain and initial environment is built, you should be able to {{c|fchroot}} into this environment using QEMU: | |||
{{console|body= | {{console|body= | ||
Revision as of 21:52, October 10, 2022
The Evolved Bootstrap stage1 has been validated to successfully build a stage3 using Metro. Here's full documentation that explains how to do this. This allows Funtoo to be built from scratch (from a non-Funtoo/Gentoo system) and "feed" Metro to build official stages.
10 October 2022
The 'evolved bootstrap' is a new project to create a technology to build Funtoo Linux completely from scratch. This project includes the Fchroot tool and the "Funtoo from Scratch" repository (click the "View scope..." link to the right for more information.)
'From scratch' means that it should be able to build Funtoo Linux for architecture 'X' using any Linux-like environment running on architecture 'Y'.
It will function by building an initial local toolchain which will be used to cross-build an initial Funtoo environment for any architecture.
Active Community Efforts
You can view the current community activities below, and click on the tile title for a detail page. You can also view Activities for a more detailed view of all activities:
Evolved Bootstrap is now successfully building a Funtoo Stage 1. We have successfully integrated this stage1 into Metro. The documentation for how to do this can be found at Funtoo:Metro/Evolved Bootstrap.
What are the Benefits?
Right now, it can be challenging to bootstrap Funtoo Linux on unsupported architectures. Many Gentoo and Funtoo packages assume a Gentoo environment, creating a chicken-and-egg problem. In addition, Portage dependencies are not always the best option for bootstrapping, as things need to be built in a very specific order and in a very controlled environment. Gentoo dependencies are designed to make sense in an already-existing Gentoo environment, but options are limited in a bootstrap environment.
The goals of the evolved bootstrap is to make the Funtoo bootstrap:
- Reliable -- will work consistently and predictably using a controlled process.
- Flexible -- will run in any Linux-like environment, and allow bootstrapping of any architecture.
- Hackable -- provide an easy mechanism for extending or expanding functionality with new languages and compilers.
- Optimized -- support the ability to build up specific parts of the toolchain using different optimized compilers.
- Extendable -- When desired, allow the bootstrap to be used to build hyper-customized Funtoo environments.
What are we Building?
I will give you a brief overview of the vision, and then talk more specifically about what this project will become. As far as the vision, here's the idea. Imagine you have access to a computer. It's not running Funtoo or even Gentoo, and it may even not truly be a Linux system. But there's a C compiler on the system. Now, imagine there was an easy way to build Funtoo entirely from source code -- even for a completely different CPU architecture (ARM, PowerPC) that you are currently running. No need to download a stage3 -- everything is fully bootstrapped, entirely from source code. The Funtoo system literally emerges from nothing before your eyes, rather than relying on any pre-built download from Funtoo.
That is the general vision of the bootstrap process. Specifically, what we are building is a new framework which gives us the benefits listed above (see "Benefits"), which will give us the ability to innovate much faster with the core system, and use technologies like different libc (MUSL is already a target) as well as different compilers. It just gives us a ton of low-level control and allows us to play with the core build process and associated tools independently from an already 'fully baked' Funtoo system. This may also evolve into a new way to build up the 'core system' of Funtoo in a much more controlled way, or be leveraged to create minimal container images for arbitrary architectures. The possibilities as vast, and we want this technology because it opens so many doors for the project.
Then there is a further question: what is exactly the final end product? What we will ultimately end up with is an enhancement to metatools which will autogenerate a bootstrap script for any architecture. The contents of the bootstrap script -- what it builds, and what versions -- will be defined in YAML for easy maintenance. The script will be likely written in bash or Bourne shell for maximum compatibility. So we will have an autogenned bootstrap script that will be widely compatible with a lot of different types of systems. If we want to have a new bootstrap with a newer gcc, we can update the YAML, regen the script, and then test it. So we will have a very easy-to-maintain framework to create different variations (for MUSL, different arches, different compilers) depending on what we want to do. This will be a very useful tool.
How Can I Get Involved?
Join us in the #bootstrap channel in Funtoo Discord and also see below for status updates and instructions on how to join our collaborative effort.
The most detailed documentation is viewable at https://code.funtoo.org/bitbucket/projects/CORE/repos/ffs/browse/README.rst. This README file documents how to build Funtoo from Scratch inside an LXD container, which is now the recommended way to use Funtoo from Scratch.
Basic Use
The following steps demonstrate the basic workflow for Funtoo from Scratch, by walking through the simplified build process, without using LXD to host the build. Funtoo from Scratch can build MUSL micro-environments as well as perform a Funtoo stage1 bootstrap. Below, we look at the MUSL option.
This will build a complete native toolchain and initial environment with a MUSL libc on arm-64bit, and you can also specify x86-64bit (for a fake-cross on a PC), riscv-64bit, or powerpc-64bit. Once the native toolchain and initial environment is built, you should be able to fchroot into this environment using QEMU:
user $ sudo emerge -v pyyaml jinja user $ git clone https://code.funtoo.org/bitbucket/scm/core/ffs.git user $ cd ffs user $ bin/sourcer musl fetch user $ bin/builder musl arm-64bit cross_tools user $ bin/builder musl arm-64bit tools user $ sudo emerge fchroot user $ bin/builder musl arm-64bit fchroot user $ sudo fchroot . /bin/bash --login Funtoo fchroot 0.2 ("Darth Elmo"); Copyright 2020-2022 Funtoo Solutions, Inc. Licensed under the Apache License, Version 2.0 >>> Entering arm-64bit (cortex-a53 CPU) fchroot... bash-5.1# nano <nano starts, executing arm64-bit instructions via QEMU>
Note that previous versions of fchroot required setting a CLFS environment variable to the current directory. This will now get automatically set by builder so it is no longer necessary.
Also note that we are currently working on adding a gnu glibc bootstrap. As of May 10, 2022, currently cross_tools is supported with work ongoing on tools and further steps.
x86-64bit "fake" fchroot is not working yet because fchroot does not yet support just doing a "regular" chroot. This will be added to a future fchroot release. For now, you will need to skip the fchroot step above for the x86-64bit arch. However, all non-x86 arches should work correctly.
Status Updates
Please note that when you follow the CLFS steps, it is not necessary to create a separate partition for your work as indicated in the CLFS docs -- just create a directory. This works fine!
- 2022-10-10
- The Evolved Bootstrap stage1 has been validated to successfully build a stage3 using Metro. Here's full documentation that explains how to do this. This allows Funtoo to be built from scratch (from a non-Funtoo/Gentoo system) and "feed" Metro to build official stages.
- 2022-04-07
- The "Funtoo From Scratch" ffs repository now has the ability to build arm-64bit and powerpc-64bit initial native environments, which can then be fchrooted into. See How Can I Get Involved for easy steps you can follow.
- 2022-02-22
- Fchroot 0.2 prerelease is being tested on our shared alkaline-dev-1 server. It allows us to chroot into alternate architectures using QEMU. It now has PowerPC support as well as several other improvements.
- 2022-02-10
- We are currently working on a community-oriented warm-up activity to get familiar with the cross-compile process using CLFS. Pnoecker is pushing through the CLFS build process while Drobbins is working on automating the build streamlining it as much as possible.
- 2022-02-01
- Official project launch. Gekman and drobbins are leading the effort. What we are suggesting people do, who want to be involved, is to simply go to "Cross Linux From Scratch", pick a build, and follow their documentation to bootstrap by hand. As you do this, take notes! Note what little changes you need to make to get the instructions to work. Linux From Scratch is a great project. Cross Linux From Scratch is a sub-project which is not always as current as LFS but it has a cross-build toolchain as its foundation. As our own 'from scratch' project evolves, we want to also reflect the positive qualities of the LFS projects by having each bootstrap step documented in a guide. The different with the 'evolved bootstrap' project is that this effort will eventually build a Funtoo image, and it will be automated - but, we will start by just doing the manual upstream CLFS process, observing, and then I will build out a portion of metatools to autogenerate bootstrap scripts to do everything automatically. So start here: https://trac.clfs.org/wiki/read -- pick a book, and start whittling away. The last time I did a CLFS build, a few packages needed patches so they would compile with the newer version of glibc on funtoo systems. CLFS tends to trail LFS by quite a lot. If you desire to try more updated versions of packages in your CLFS build, please do, and take notes on any tweaks if any that were needed to get the new version to work.
CLFS Build Notes
CLFS build notes can now be added to "Related Pages" by clicking "Edit with Form" and entering the name of your page.
We are going to encourage people to document their CLFS build notes on the wiki. It's recommended you create a "userpage" at Your User Page/CLFS Notes, and then add a link to your notes in the list above this line.