Note

The Funtoo Linux project has transitioned to "Hobby Mode" and this wiki is now read-only.

Core Developer Reference

From Funtoo
Revision as of 18:48, April 29, 2011 by Golodhrim (talk | contribs) (→‎Funtoo portage tree: adding link to Git Guide)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page is a reference for Funtoo core developers.

Core developers

A Funtoo core developer is a member of the Core Team who help in the maintenance/evolution of Funtoo Linux.

Funtoo portage tree

A frequently asked question is: "What your portage tree is made of?". The answer lies in a Python script named merge.py present in the directory funtoo/scripts of the Funtoo overlay. Basically, this script:

  • Takes a snapshot of the Gentoo portage tree (imported every 12 hours)
  • Merges with the Funtoo overlay: the Funtoo historical development overlay (still maintained, the Funtoo overlay contains for example the scripts used to regenerate the Funtoo portage tree)
  • Merges with the Multimedia overlay (only one package here -> media-video/mplayer-uau)
  • Merges with the overlays maintained by several Funtoo developpers:
    • Tarsius overlay (tarsius/Funtoo Core Team)
    • Foo overlay (slashbeast)
    • Bar overlay (404 error)
    • Golodhrim overlay (golodhrim)

A short guide on how to go working with github is available at Git Guide, so everyone new to git, github and CoreTeam should have a look there to get the basics.

When a new core developer or contributor want to share his ebuild repository, he must:

  1. Ask for having a read-write access on the Funtoo overlay (With great privileges come great responsibilities, do not change something you are sure about, you will be very likely to break things so please ask before act).
  2. Clone the Funtoo overlay (use git@github.com:funtoo/funtoo-overlay.git, not git://github.com/funtoo/funtoo-overlay.git that later is a public read-only reference)
  3. Edit funtoo/scripts/merge.py:
bar_overlay = Tree("bar-overlay","/root/git/bar-funtoo-overlay", pull=True)
multimedia_overlay = Tree("multimedia-overlay", "/root/git/multimedia-overlay",pull=True)
your_overlay = Tree("name-of-your-overlay", "/root/git/name-of-your-overlay",pull=True)

steps = [
	SyncTree(gentoo_src,exclude=["/metadata/cache/**","sys-kernel/openvz-sources"]),
	ApplyPatchSeries("%s/funtoo/patches" % funtoo_overlay.root ),
	SyncDir(funtoo_overlay.root,"profiles","profiles", exclude=["repo_name","categories"]),
	(.... some lines removed ....)
	InsertEbuilds(bar_overlay, select="all", skip=None, replace=False),
	InsertEbuilds(multimedia_overlay, select=["media-video/mplayer-uau"], skip=None, replace=False),
	InsertEbuilds(your_overlay, select="all", skip=None, replace=False),
	GenCache()
]

Once the modifications are in place, just push merge.py back to github (and please, again, do that with a core team member on IRC) 

# git push origin

Basically you define an ebuild tree object which points to your repository and tell the script:"Select all the content of my package tree, skip no packages and do not replace duplicates if any".

Masking or unmasking your packages

The rule is: "If you update a package which is either a part of the system core or can have a severe impact on the system, please mask it".

This is the case for example of GCC, Glibc, Binutils, autoconf, automake, Python, Perl, DHCPD, NTP, OpenRC. By definition, every package that could make your system unusable after an update must be masked. If you add an optional package that have minor or no impact at all (e.g. an IDE, a Pascal compiler, a multimedia file player...) there is no need to mask the package first.

There are 3 files used in the Funtoo overlay to achive that (see funtoo-overlay/profile/package.mask). By convention:

  • funtoo-staging : when a core package is added, its masking must go there. Once the package has been tested enough by the community, its masking statement is removed from that file.
  • funtoo-cautionary : permanent maskings should go there. This file is also updated if a package is masked prior to its removal from the Funtoo tree
  • sunrise : updated on very rare occasions this file is related to the Gentoo sunrise overlay.

How to edit and commit your changes in funtoo-overlay/profile/package.mask

The way to go

A good practice is to do a git pull before changing anything in the funtoo-overlay/profile/package.mask directory (please do not add any other files there, use only existing ones). This helps to get a clearer commit history and make your and our life a bit more happy. Once this refresh is done you can bring your changes in the adequate files in accordance to the following rules:

  1. Always put your changes at the top of the list
  2. Always mention in the first line who you are, your e-mail and the date of the day
  3. Unless you have a good reason to not do it, please always mask like: >=category/package-revision. E.g. >=dev-lang/perl-5.12.3
  4. Do not mask the same package several times (i.e. never put more that one line mentioning the same category/package).

# John Doe <john.doe@funtoo.org> (31 Dec 2010)
# Masked for testing
>=dev-lang/python-4.0

# John Doe <john.doe@funtoo.org> (30 Dec 2010)
# Masked for testing
>=net-dialup/ppp-7.0.1_beta

...

Stashing and merging your local changes

If you have changed some files before doing a git pull, you will be likely to encounter a conflict when pulling from the GitHub repository: 

# git pull
Enter passphrase for key '/root/.ssh/id_rsa':
remote: Counting objects: 145, done.
remote: Compressing objects: 100% (115/115), done.
remote: Total 117 (delta 66), reused 0 (delta 0)
Receiving objects: 100% (117/117), 37.20 KiB, done.
(...)
error: Your local changes to the following files would be overwritten by merge:
        profiles/package.mask/funtoo-staging
Please, commit your changes or stash them before you can merge.

To sort this situation out:

  1. Stash your changes by issuing git stash
  2. Issue git pull
  3. Merge back your stashed changes with the current HEAD by issuing: git stash pop

Example: 

# git stash
Saved working directory and index state WIP on master: 7990315 gptfdisk unmask
HEAD is now at 7990315 gptfdisk unmask
(...)
# git pull
Enter passphrase for key '/root/.ssh/id_rsa':
Updating 7990315..efecb0d
Fast-forward
 app-arch/tar/Manifest                              |    2 +-
 app-arch/tar/tar-1.25-r1.ebuild                    |    2 +-
 profiles/funtoo/1.0/linux-gnu/flavor/core/virtuals |    2 -
(...)
# git stash pop
Auto-merging profiles/package.mask/funtoo-staging
(...)

At this point, it is highly probable that Git will report a conflict if another Funtoo developer had also brought and pushed on GitHub some changes: 

CONFLICT (content): Merge conflict in profiles/package.mask/funtoo-staging

Just correct the conflict in the file (see the marks put by Git in the file). Once everything is in order, just commit and push your changes (remember to add a short explicative note in the commit to track down the reason of the change).

Committing your changes on funtoo-overlay/profile/package.mask

A highly recommended way to commit your changes is to proceed in the following order:

  1. Commit and push the changes brought in profiles/packages.mask
  2. Commit and push the changes brought in the package tree (in your overlay and/or the funtoo-overlay in itself).

Why? Simple: remember that the Funtoo portage tree is automatically refreshed every 12 hours and this refresh of merging several Funtoo developers repositories on GitHub, the Funtoo overlay master copy (also located on GitHub) and a snapshot of the Gentoo portage tree. At the time you commit, it is not impossible that the refreshment process and you are acting in a concurrent way and the following scenario could occur:

  1. You have updated some Funtoo core packages (or, worse, a Funtoo critical package) and you commit and push the changes brought either in the Funtoo overlay either in your local repository on GitHub
  2. The Funtoo tree refreshment process starts and grab the place where your updated package resides and the profiles/packages.mask directory in the Funtoo overlay
  3. You commit and push your changes in profiles/packages.mask to mask

Not respecting this order could lead to catastrophic consequences for Funtoo users the next time they will resync their portage tree if your package breaks for various reasons or if the deployed software has severe issues that you had not been able to detect on your side.

Things to do after pushing a package (update or new) - IMPORTANT

Advise the Funtoo community

The following is not optional so pay attention: once your bits have been pushed on GitHub, you have to post a notice about what you have updated/added to the Funtoo tree on all of the following Funtoo communication channels:

  • the funtoo-dev mailing list on Google Groups
  • the funtoo.org Forums

The reason is simple: some of the Funtoo users use the mailing list and some others prefer to use the Funtoo forums and both categories must be reached.

Update the additions/forks pages in the wiki

Depending on what you did, please update the following pages:

  • Forked_Ebuilds if you have forked an existing Gentoo package (giving a new revision)
  • Specific_Funtoo_Ebuilds if you have introduced a totally new package in Funtoo (not a revision of an existing Gentoo/Funtoo package)
Retrieved from "https://www.funtoo.org/index.php?title=Core_Developer_Reference&oldid=1186"