The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.
Difference between revisions of "Funtoo:Metro/Initial Setup"
| (21 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
This page will walk you through setting up metro for the first time. | |||
You | == Cloning and configuring Metro == | ||
Metro is run from its own git tree, and does not have its own ebuild. You can get the latest version of Metro from the git repository using git. | |||
{{console|body= | {{console|body= | ||
# ##i## | # ##i##cd /root | ||
# ##i## | # ##i##git clone https://code.funtoo.org/bitbucket/scm/core/metro.git | ||
}} | }} | ||
== | == Installing dependencies == | ||
Because the git repo is cloned directly, you will need to install some Python modules used by the scripts. Please make sure you have the following packages installed in your system: | |||
{{console|body= | {{console|body= | ||
# ##i## | # ##i##emerge -av dev-python/lxml dev-python/requests | ||
}} | }} | ||
{{note|metro is running as root and while not strictly necessary but best practice would be using dedicated environment such as [[LXD]] container or similar where you will run your builds.}} With this setup, before installing dependencies of metro, run {{c|ego sync}} in your container. | |||
=== Configuring Metro === | |||
The default options for metro are usually fine for most cases. If you are happy with the defaults, just copy the default configuration file to /root/.metro: | The default options for metro are usually fine for most cases. If you are happy with the defaults, just copy the default configuration file to /root/.metro: | ||
| Line 25: | Line 26: | ||
}} | }} | ||
If you wish to tweak the settings then you should edit | Two things to note -- the {{c|/root/.metro}} file specifies ''where'' the metro git repository is installed -- and this must match where you put it on your system. If you are not using the location of {{c|/root/metro}}, then you will need to fix {{c|[section path] / install}} to reflect this. | ||
Also, metro will set up what is called a "mirror" -- this is an organized repository of stages that metro manages, and it uses the directory structure you will find at https://build.funtoo.org. By default, the path {{c|/home/mirror/funtoo}} is used as the base of this repository. This path is also defined in {{f|.metro}} and should be changed if you are using the non-default location. | |||
If you wish to further tweak the settings then you should edit {{f|.metro}} accordingly (see [[Funtoo:Metro/Configuring_Metro|Configuring_Metro]]) to learn how to do that. | |||
Now your system is ready to run Metro's automatic setup tool, which will take care of the rest of the preparation. | Now your system is ready to run Metro's automatic setup tool, which will take care of the rest of the preparation. | ||
| Line 31: | Line 36: | ||
== Setting up your build == | == Setting up your build == | ||
The Python modules you installed earlier allows {{c|setup}} to run, which generally will take care of everything else. When you run {{c|setup}}, the following tasks will be performed: | |||
These steps are done automatically by the {{c|setup}} script: | # Independent {{c|ego}} cloned to {{f|/root/ego}} (which is used by metro to create snapshots); | ||
# If [[Package:fchroot]] is not installed, it will be emerged; | |||
# A "seed" stage3 will be downloaded so that you can use it to build a new stage3; | |||
# The "mirror" repository will be set up to build an up-to-date version of this stage3. | |||
These steps are done automatically by the {{c|setup}} script, which only needs to be run once. | |||
When this is done, you will be able to use metro to perform a build on your system. Our simplified setup script will get you up and running with a basic metro configuration which we recommend for new users. Specifically, it will allow you to take one of our official Funtoo stage3's of a particular arch and subarch, and build an updated version of the same arch and subarch. If you want to set up another arch and subarch to build locally, it is safe to run setup a second time. | |||
Let's launch setup: | |||
{{console|body= | {{console|body= | ||
| Line 40: | Line 54: | ||
=== Selecting a release === | === Selecting a release === | ||
You can build a stage tarball for any of the available Funtoo releases. Currently, these are the only releases available: | You can build a stage tarball for any of the available Funtoo releases. Currently, these are the only releases available: | ||
{{console|body= | {{console|body= | ||
\##i##www.funtoo.org / funtoo | |||
##i##1. 1.4-release-std | ##i##1. 1.4-release-std | ||
| Line 71: | Line 86: | ||
=== Selecting an Architecture === | === Selecting an Architecture === | ||
Choose a general architecture from the menu. You'll have a chance to choose a more specific sub-architecture in the next step. | Choose a general architecture from the menu. You'll have a chance to choose a more specific sub-architecture in the next step. | ||
{{console|body= | {{console|body= | ||
\##i##www.funtoo.org / funtoo / next | |||
| Line 104: | Line 120: | ||
Choose the specific sub-architecture you wish to build your stage tarball for. | Choose the specific sub-architecture you wish to build your stage tarball for. | ||
{{console|body= | {{console|body= | ||
\##i##www.funtoo.org / funtoo / next / x86-64bit | |||
##i##1. amd64-bulldozer | ##i##1. amd64-bulldozer | ||
| Line 128: | Line 144: | ||
. | . | ||
}} | }} | ||
Note that thanks to [[Package:fchroot]], you will be able to update an arm-64bit stage on an x86-64bit system if you want. | |||
{{Important|More advanced uses of Metro allow for a new subarch to be "bootstrapped" from an existing subarch, but that is not provided as an option as it's a more sophisticated use of the tool and not ideal for your initial local build. Let's keep things simple to get started :)}} | |||
=== Wait for the Setup to Complete === | === Wait for the Setup to Complete === | ||
The setup script will go through several steps to set up your build. If you are running it for the first time, | The setup script will go through several steps to set up your build. If you are running it for the first time, this can include installing supporting software and downloading a tarball, so it may take some time to complete. | ||
Here is a sample output: | Here is a sample output: | ||
| Line 161: | Line 181: | ||
To build a new release, use ezbuild (or set up buildbot): | To build a new release, use ezbuild (or set up buildbot): | ||
\##g##scripts/ezbuild.sh next x86-64bit generic_64 freshen+gnome | |||
}} | }} | ||
== Building a new release == | == Building a new release == | ||
In the example above, a "next x86-64bit generic_64" build has been set up to build a new Gnome | In the example above, a "next x86-64bit generic_64" build has been set up to build a new Gnome stage3 tarball after updating the existing stage3 tarball downloaded from https://build.funtoo.org. If you're satisfied with those settings, you can just go ahead and run the command as suggested: | ||
{{console|body= | {{console|body= | ||
| Line 172: | Line 192: | ||
}} | }} | ||
Otherwise, continue reading to learn how to create a stage3 | Otherwise, continue reading to learn how to create a stage3 tarball with different specs. | ||
The first 3 arguments in the example above: 'next', 'x86-64bit' and ' | The first 3 arguments in the example above: 'next', 'x86-64bit' and 'generic_64' correspond to the options you chose when you ran the setup script. If you'd like to run {{c|ezbuild.sh}} for a different release, architecture or sub-architecture, you need to run the setup script again with the appropriate options. You only need to do this ''once'' per subarch -- just to make sure you have a local copy of the stage3 for that subarch. After that, {{c|ezbuild.sh}} can keep the stage3 updated using the most recent successfully-built stage3. | ||
The 4th argument corresponds to a combination of the stages to build + the [[#Choosing a Target|target specs]] for your stage tarball. | The 4th argument corresponds to a combination of the [[#Stages to Build|stages to build]] + the [[#Choosing a Target|target specs]] for your stage tarball. | ||
== Checking the logs == | == Checking the logs == | ||
The {{c|ezbuild.sh}} script is very quiet. If you were expecting to get a lot of output like a usual {{c|emerge}} session, it may look like your computer froze. You can check what {{c|ezbuild.sh}} is doing by watching the logs, which can be found in a path with the following structure: {{f|/home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/log/ | |||
The {{c|ezbuild.sh}} script is very quiet. If you were expecting to get a lot of output like a usual {{c|emerge}} session, it may look like your computer froze. You can check what {{c|ezbuild.sh}} is doing by watching the logs, which can be found in a path with the following structure: {{f|/home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/log/<target>.txt}}. | |||
{{console|body= | {{console|body= | ||
# ##i##tail -f /home/mirror/funtoo/next/x86-64bit/ | # ##i##tail -f /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/log/stage3-freshen.txt | ||
}} | }} | ||
| Line 190: | Line 211: | ||
{{console|body= | {{console|body= | ||
# ##i##cat /home/mirror/funtoo/next/x86-64bit/ | # ##i##cat /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/.targets | ||
}} | }} | ||
| Line 197: | Line 218: | ||
=== Stages to Build === | === Stages to Build === | ||
There are 2 options for this argument: | There are 2 options for this argument: “freshen” and “full”. | ||
“Freshen” will basically run a @world update on an existing stage3 tarball to generate a ''fresh'', updated version of the same tarball. | “Freshen” will basically run a @world update on an existing stage3 tarball to generate a ''fresh'', updated version of the same tarball. | ||
| Line 205: | Line 226: | ||
=== Build Targets === | === Build Targets === | ||
You can create different flavors of a stage3 | You can create different flavors of a stage3, with the software and settings necessary for different purposes. For example, a GNOME Desktop stage3 tarball, an LXD container image, etc. The available targets can be found at {{f|/root/metro/targets/gentoo}}. You can also create your own custom targets. See [[Funtoo:Metro/Data Model|Metro Data Model]] to learn how to do that. | ||
== Using Gnu Screen == | == Using Gnu Screen == | ||
| Line 225: | Line 246: | ||
{{console|body= | {{console|body= | ||
# ##i##/home/root/metro/scripts/ezbuild.sh next x86-64bit | # ##i##/home/root/metro/scripts/ezbuild.sh next x86-64bit generic_64 freshen+gnome | ||
}} | }} | ||
| Line 237: | Line 258: | ||
== Where is my Stage Tarball? == | == Where is my Stage Tarball? == | ||
When the build completes, the stage tarball can be found in a path with the following structure: {{f|/home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/}}. In our example, it will be found at: | When the build completes, by default the stage tarball can be found in a path with the following structure: {{f|/home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/}}. In our example, it will be found at: | ||
{{console|body= | {{console|body= | ||
# ##i##ls /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/ | |||
}} | }} | ||
If you'd like your tarballs to go elsewhere, you can change your configuration in {{f|/root/.metro}} (see [[Funtoo:Metro/Configuring Metro|Configuring Metro]]). | |||
== Where to Go From Here == | == Where to Go From Here == | ||
=== Additional Architectures === | === Additional Architectures === | ||
The automatic setup script doesn't offer all the possible architectures you could be building a | The automatic setup script doesn't offer all the possible architectures you could be building a stage tarball for. Read the instructions on [[Funtoo:Metro/Manual_Setup|Manual Setup]] to learn how to set up a build for any of the supported platforms. In theory, you can actually set up Metro to build a stage tarball for any platform supported by the GCC (and QEMU, if it's the target platform is different from the host platform). | ||
=== | === Custom Targets === | ||
Study | Study [[Funtoo:Metro/Data_Model|Metro Data Model]] to learn how to create new custom targets, which you can then use for your own customized builds. | ||
=== Automation (Buildbots) === | === Automation (Buildbots) === | ||
Latest revision as of 19:26, March 26, 2024
This page will walk you through setting up metro for the first time.
Cloning and configuring Metro
Metro is run from its own git tree, and does not have its own ebuild. You can get the latest version of Metro from the git repository using git.
root # cd /root root # git clone https://code.funtoo.org/bitbucket/scm/core/metro.git
Installing dependencies
Because the git repo is cloned directly, you will need to install some Python modules used by the scripts. Please make sure you have the following packages installed in your system:
root # emerge -av dev-python/lxml dev-python/requests
metro is running as root and while not strictly necessary but best practice would be using dedicated environment such as LXD container or similar where you will run your builds.
With this setup, before installing dependencies of metro, run ego sync in your container.
Configuring Metro
The default options for metro are usually fine for most cases. If you are happy with the defaults, just copy the default configuration file to /root/.metro:
root # cp /root/metro/metro.conf /root/.metro
Two things to note -- the /root/.metro file specifies where the metro git repository is installed -- and this must match where you put it on your system. If you are not using the location of /root/metro, then you will need to fix [section path] / install to reflect this.
Also, metro will set up what is called a "mirror" -- this is an organized repository of stages that metro manages, and it uses the directory structure you will find at https://build.funtoo.org. By default, the path /home/mirror/funtoo is used as the base of this repository. This path is also defined in .metro and should be changed if you are using the non-default location.
If you wish to further tweak the settings then you should edit .metro accordingly (see Configuring_Metro) to learn how to do that.
Now your system is ready to run Metro's automatic setup tool, which will take care of the rest of the preparation.
Setting up your build
The Python modules you installed earlier allows setup to run, which generally will take care of everything else. When you run setup, the following tasks will be performed:
- Independent
egocloned to/root/ego(which is used by metro to create snapshots); - If Package:fchroot is not installed, it will be emerged;
- A "seed" stage3 will be downloaded so that you can use it to build a new stage3;
- The "mirror" repository will be set up to build an up-to-date version of this stage3.
These steps are done automatically by the setup script, which only needs to be run once.
When this is done, you will be able to use metro to perform a build on your system. Our simplified setup script will get you up and running with a basic metro configuration which we recommend for new users. Specifically, it will allow you to take one of our official Funtoo stage3's of a particular arch and subarch, and build an updated version of the same arch and subarch. If you want to set up another arch and subarch to build locally, it is safe to run setup a second time.
Let's launch setup:
root # /root/metro/scripts/setup
Selecting a release
You can build a stage tarball for any of the available Funtoo releases. Currently, these are the only releases available:
www.funtoo.org / funtoo 1. 1.4-release-std 2. next .
Selecting an Architecture
Choose a general architecture from the menu. You'll have a chance to choose a more specific sub-architecture in the next step.
www.funtoo.org / funtoo / next 1. arm-64bit 2. riscv-64bit 3. x86-64bit .
Selecting a Sub-Architecture
Choose the specific sub-architecture you wish to build your stage tarball for.
www.funtoo.org / funtoo / next / x86-64bit 1. amd64-bulldozer 2. amd64-excavator 3. amd64-k10 4. amd64-piledriver 5. amd64-steamroller 6. amd64-zen 7. amd64-zen2 8. amd64-zen3 9. core2_64 10. generic_64 11. intel64-broadwell 12. intel64-haswell 13. intel64-ivybridge 14. intel64-nehalem 15. intel64-sandybridge 16. intel64-silvermont 17. intel64-skylake 18. intel64-westmere .
Note that thanks to Package:fchroot, you will be able to update an arm-64bit stage on an x86-64bit system if you want.
More advanced uses of Metro allow for a new subarch to be "bootstrapped" from an existing subarch, but that is not provided as an option as it's a more sophisticated use of the tool and not ideal for your initial local build. Let's keep things simple to get started :)
Wait for the Setup to Complete
The setup script will go through several steps to set up your build. If you are running it for the first time, this can include installing supporting software and downloading a tarball, so it may take some time to complete.
Here is a sample output:
Created file /home/mirror/funtoo/next/x86-64bit/generic_64/.control/strategy/build with contents 'local'
Created file /home/mirror/funtoo/next/x86-64bit/generic_64/.control/strategy/seed with contents 'stage3'
Created file /home/mirror/funtoo/next/x86-64bit/generic_64/.control/version/stage3 with contents '2022-09-13'
--2022-10-09 08:06:24-- https://build.funtoo.org/next/x86-64bit/generic_64/2022-09-13/stage3-generic_64-next-2022-09-13.tar.xz
Resolving build.funtoo.org... 89.187.187.14, 89.187.187.12, 89.187.187.19, ...
Connecting to build.funtoo.org[89.187.187.14]:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 701542432 (669M) [application/octet-stream]
Saving to: ‘/home/mirror/funtoo/next/x86-64bit/generic_64/2022-09-13/stage3-generic_64-next-2022-09-13.tar.xz’
funtoo/next/x86-64bit/g 100%[==================================================>] 669.04M 95.5MB/s in 7.3s
2022-10-09 08:06:32 (92.0 MB/s) - ‘/home/mirror/funtoo/next/x86-64bit/generic_64/2022-09-13/stage3-generic_64-next-2022-09-13.tar.xz’ saved [701542432/701542432]
Cloning into '/root/ego'...
remote: Enumerating objects: 3385, done.
remote: Counting objects: 100% (3385/3385), done.
remote: Compressing objects: 100% (1382/1382), done.
remote: Total 3385 (delta 2093), reused 2902 (delta 1727), pack-reused 0
Receiving objects: 100% (3385/3385), 760.96 KiB [ 27.18 MiB/s, done.
Resolving deltas: 100% (2093/2093), done.
Metro build for release next, x86-64bit, generic_64 has been initialized.
To build a new release, use ezbuild (or set up buildbot):
scripts/ezbuild.sh next x86-64bit generic_64 freshen+gnome
Building a new release
In the example above, a "next x86-64bit generic_64" build has been set up to build a new Gnome stage3 tarball after updating the existing stage3 tarball downloaded from https://build.funtoo.org. If you're satisfied with those settings, you can just go ahead and run the command as suggested:
root # /root/metro/scripts/ezbuild.sh next x86-64bit generic_64 freshen+gnome
Otherwise, continue reading to learn how to create a stage3 tarball with different specs.
The first 3 arguments in the example above: 'next', 'x86-64bit' and 'generic_64' correspond to the options you chose when you ran the setup script. If you'd like to run ezbuild.sh for a different release, architecture or sub-architecture, you need to run the setup script again with the appropriate options. You only need to do this once per subarch -- just to make sure you have a local copy of the stage3 for that subarch. After that, ezbuild.sh can keep the stage3 updated using the most recent successfully-built stage3.
The 4th argument corresponds to a combination of the stages to build + the target specs for your stage tarball.
Checking the logs
The ezbuild.sh script is very quiet. If you were expecting to get a lot of output like a usual emerge session, it may look like your computer froze. You can check what ezbuild.sh is doing by watching the logs, which can be found in a path with the following structure: /home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/log/<target>.txt.
root # tail -f /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/log/stage3-freshen.txt
Under the same directory, you will find a log file for every spec file involved in the build you chose. In the case of the example above, we had chosen “freshen+gnome”, so we can expect to find stage3-freshen.txt and gnome.txt.
If you'd like to know all the targets that were selected for your build, check the file .targets at (in our example):
root # cat /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/.targets
Choosing a Target
Stages to Build
There are 2 options for this argument: “freshen” and “full”.
“Freshen” will basically run a @world update on an existing stage3 tarball to generate a fresh, updated version of the same tarball.
“Full” will build a stage3 from scratch, starting by building a stage1, then a stage2 and finally, from that, an entirely new stage3 tarball, completely built from source.
Build Targets
You can create different flavors of a stage3, with the software and settings necessary for different purposes. For example, a GNOME Desktop stage3 tarball, an LXD container image, etc. The available targets can be found at /root/metro/targets/gentoo. You can also create your own custom targets. See Metro Data Model to learn how to do that.
Using Gnu Screen
Depending on the arguments you passed to ezbuild.sh, the build process may take several hours to complete.
If you are running ezbuild.sh from another machine through an ssh session, you might want to use a terminal multiplexer like app-misc/tmux or app-misc/screen, which will allow you to close the terminal and even log out from the remote machine machine. Here is how you can do it using Gnu Screen:
Emerging app-misc/screen:
root # emerge -av app-misc/screen
Running app-misc/screen:
root # screen -T ${TERM} -S Metro
This will open a screen session where you can run ezbuild.sh:
root # /home/root/metro/scripts/ezbuild.sh next x86-64bit generic_64 freshen+gnome
Now, if you'd like to leave the terminal, just press Ctrl+A and then Ctrl+D to detach the screen session. Once you've detached, you are free to leave you ssh session and/or close your terminal. The script will keep running inside the detached session.
When you want to go back, log back in if you are working from a remote host and just restore your session:
root # screen -T ${TERM} -r Metro
Where is my Stage Tarball?
When the build completes, by default the stage tarball can be found in a path with the following structure: /home/mirror/funtoo/<release>/<arch>/<subarch>/<date>/. In our example, it will be found at:
root # ls /home/mirror/funtoo/next/x86-64bit/generic_64/2022-10-08/
If you'd like your tarballs to go elsewhere, you can change your configuration in /root/.metro (see Configuring Metro).
Where to Go From Here
Additional Architectures
The automatic setup script doesn't offer all the possible architectures you could be building a stage tarball for. Read the instructions on Manual Setup to learn how to set up a build for any of the supported platforms. In theory, you can actually set up Metro to build a stage tarball for any platform supported by the GCC (and QEMU, if it's the target platform is different from the host platform).
Custom Targets
Study Metro Data Model to learn how to create new custom targets, which you can then use for your own customized builds.
Automation (Buildbots)
You can set up multiple builds and have Metro automatically build each one of them or just refresh the ones that are outdated. Read the instructions at Automation to learn how to do that.