User:Pytony/Minimize Python2 Usage

In this document I describe how I moved to python 3.5 with less python 2.7 as possible.

First thing to do was to unmask python-3.5 and install it :

root # echo '=dev-lang/python-3.5*' >> /etc/portage/package.unmask
root # emerge '=dev-lang/python3.5*'

Then I set PYTHON_SINGLE_TARGET and PYTHON_TARGETS to python3_5 in my make.conf:

   /etc/portage/make.conf

PYTHON_SINGLE_TARGET="python3_5"
PYTHON_TARGETS="${PYTHON_SINGLE_TARGET}"

Now let's try to emerge to update @world. It is very likely that it will fail because of packages requiring python but not supporting python 3.5.

In most cases you should be able to just disable python support. For instance, here is how I handled sci-libs/geos:

➜  ~ emerge -pvuDN @world

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! The ebuild selected to satisfy ">=sci-libs/geos-3.5.0" has unmet requirements.
- sci-libs/geos-3.6.1::gentoo USE="python -doc -ruby -static-libs" PYTHON_TARGETS="-python2_7"

  The following REQUIRED_USE flag constraints are unsatisfied:
    python? ( python_targets_python2_7 )

(dependency required by "dev-db/postgis-2.3.2-r1::gentoo" [installed])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
➜  ~ sudo chuse geos -python --because it requires python2.7

In some cases, python 2.7 will be mandatory:

➜  ~ emerge -pvuDN @world

These are the packages that would be merged, in order:

Calculating dependencies... done!

!!! The ebuild selected to satisfy ">=dev-libs/gobject-introspection-1.46.0:=" has unmet requirements.
- dev-libs/gobject-introspection-1.48.0::gentoo USE="-cairo -doctool -test" PYTHON_TARGETS="-python2_7"

  The following REQUIRED_USE flag constraints are unsatisfied:
    python_targets_python2_7

  The above constraints are a subset of the following complete expression:
    python_targets_python2_7 test? ( cairo )

(dependency required by "dev-python/pygobject-3.20.1::gentoo" [ebuild])
(dependency required by "x11-wm/qtile-0.10.7::gentoo[dbus]" [ebuild])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
➜  ~ sudo chuse gobject-introspection python_targets_python2_7 --because only python2.7 available

In some other cases, python 2.7 support will be required by another package:

The following USE changes are necessary to proceed:
 (see "package.use" in the portage(5) man page for more details)
root # required by dev-util/itstool-2.0.2::gentoo
root # required by app-text/yelp-tools-3.18.0::gentoo
root # required by app-text/evince-3.20.1::gentoo
root # required by @selected
root # required by @world (argument)
>=dev-libs/libxml2-2.9.4-r1 python_targets_python2_7
➜  ~ sudo chuse libxml2 python_targets_python2_7 --because required by evince

Finally once you fixed every issue raised by emerge -pvuDN @world, you can run it without pretend option.

Still some packages may fail to build.

➜  ~ sudo emerge -uDN @world
Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Running pre-merge checks for dev-python/pycparser-2.17
>>> Emerging (1 of 53) sci-libs/geos-3.6.1::gentoo
>>> Emerging (2 of 53) virtual/jpeg-62::gentoo
>>> Emerging (3 of 53) dev-python/six-1.10.0::gentoo
>>> Installing (2 of 53) virtual/jpeg-62::gentoo
>>> Installing (1 of 53) sci-libs/geos-3.6.1::gentoo
>>> Installing (3 of 53) dev-python/six-1.10.0::gentoo
>>> Emerging (4 of 53) dev-python/appdirs-1.4.3::gentoo
>>> Installing (4 of 53) dev-python/appdirs-1.4.3::gentoo
>>> Emerging (5 of 53) dev-python/pyparsing-2.2.0::gentoo
>>> Installing (5 of 53) dev-python/pyparsing-2.2.0::gentoo
>>> Emerging (6 of 53) dev-python/pyelftools-0.23-r1::gentoo
>>> Emerging (7 of 53) dev-python/virtualenv-clone-0.2.6::gentoo
>>> Emerging (8 of 53) dev-python/packaging-16.8::gentoo
>>> Failed to emerge dev-python/virtualenv-clone-0.2.6, Log file:
>>>  '/var/tmp/portage/dev-python/virtualenv-clone-0.2.6/temp/build.log'
>>> Installing (6 of 53) dev-python/pyelftools-0.23-r1::gentoo
>>> Installing (8 of 53) dev-python/packaging-16.8::gentoo
>>> Jobs: 7 of 53 complete, 1 failed                Load avg: 2.12, 1.25, 0.75
*** Resuming merge...

In the above case, I noticed in /var/tmp/portage/dev-python/virtualenv-clone-0.2.6/temp/build.log that the issue was that setuptools was not yet built with python3.5 support. Probably a missing dependency in dev-python/virtualenv-clone. In such case, just rebuild setuptools before updating world:

root # emerge -1 setuptools
root # emerge -uDN @world

I also faced a more specific error with sys-libs/cracklib which was that it was not explicitly set in REQUIRED_USE that python useflags required python_targets_python2_7. So it failed at build time. The fix was just to remove python useflag.

Once you have successfully updated @world, it is not entirely over. Indeed, there might still be packages that were not updated though python targets changed. I can't explain this behavior. Yet I could easily find these packages using emerge -pve @world and fix issues (which are basically the same as with former emerge -pvuDN @world) until there is no more error raised.

Once emerge -pve @world gives you a clean output, it would be a shame to loose time recompiling every single package with emerge -e @world. So, I just listed packages which had a useflag change, and emerged these packages only:

➜  ~ emerge -pve @world | grep '*'
[ebuild   R    ] dev-libs/libxml2-2.9.4-r1:2::gentoo  USE="icu ipv6 python readline -debug -examples -lzma -static-libs {-test}" ABI_X86="32 (64) (-x32)" PYTHON_TARGETS="python2_7* python3_5 -python3_4" 0 KiB
[ebuild   R    ] dev-lang/yasm-1.3.0::gentoo  USE="nls -python*" PYTHON_TARGETS="-python2_7*" 0 KiB
[ebuild   R    ] dev-python/pygments-2.2.0::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python3_5* -pypy -pypy3 -python2_7* -python3_4* -python3_6" 0 KiB
[ebuild   R    ] dev-python/docutils-0.13.1::gentoo  PYTHON_TARGETS="python3_5* -pypy -pypy3 -python2_7* -python3_4* -python3_6" 0 KiB
[ebuild   R    ] dev-util/gdbus-codegen-2.48.2::gentoo  PYTHON_TARGETS="python3_5* -python2_7* -python3_4*" 0 KiB
➜  ~ sudo emerge -1 libxml2 yasm pygments docutils gdbus-codegen
Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Emerging (1 of 5) dev-python/pygments-2.2.0::gentoo
>>> Emerging (2 of 5) dev-libs/libxml2-2.9.4-r1::gentoo
>>> Installing (1 of 5) dev-python/pygments-2.2.0::gentoo
>>> Emerging (3 of 5) dev-python/docutils-0.13.1::gentoo
>>> Installing (3 of 5) dev-python/docutils-0.13.1::gentoo
>>> Installing (2 of 5) dev-libs/libxml2-2.9.4-r1::gentoo
>>> Emerging (4 of 5) dev-lang/yasm-1.3.0::gentoo
>>> Emerging (5 of 5) dev-util/gdbus-codegen-2.48.2::gentoo
>>> Installing (5 of 5) dev-util/gdbus-codegen-2.48.2::gentoo
>>> Installing (4 of 5) dev-lang/yasm-1.3.0::gentoo
>>> Jobs: 5 of 5 complete                           Load avg: 2.67, 1.50, 0.79
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

Finally, you can remove packages that became unneeded:

➜  ~ sudo emerge -c

 * Always study the list of packages to be cleaned for any obvious
 * mistakes. Packages that are part of the world set will always
 * be kept.  They can be manually added to this set with
 * `emerge --noreplace <atom>`.  Packages that are listed in
 * package.provided (see portage(5)) will be removed by
 * depclean, even if they are part of the world set.
 * 
 * As a safety measure, depclean will not remove any packages
 * unless *all* required dependencies have been resolved.  As a
 * consequence of this, it often becomes necessary to run
 * `emerge --update --newuse --deep @world` prior to depclean.

Calculating dependencies... done!
>>> Calculating removal order...

 dev-python/pyxattr
    selected: 0.6.0 
   protected: none
     omitted: none

 dev-python/pygtk
    selected: 2.24.0-r4 
   protected: none
     omitted: none

 dev-python/trollius
    selected: 2.1 
   protected: none
     omitted: none

 gnome-base/libglade
    selected: 2.6.4-r2 
   protected: none
     omitted: none

 dev-python/pygobject
    selected: 2.28.6-r55 
   protected: none
     omitted: 3.20.1 

 virtual/python-futures
    selected: 0 
   protected: none
     omitted: none

 dev-python/futures
    selected: 3.1.1 
   protected: none
     omitted: none

 dev-lang/python
    selected: 3.4.6 
   protected: none
     omitted: 2.7.13 3.5.3 

All selected packages: =dev-python/pyxattr-0.6.0 =dev-lang/python-3.4.6 =gnome-base/libglade-2.6.4-r2 =dev-python/pygtk-2.24.0-r4 =dev-python/trollius-2.1 =virtual/python-futures-0 =dev-python/pygobject-2.28.6-r55 =dev-python/futures-3.1.1

>>> 'Selected' packages are slated for removal.
>>> 'Protected' and 'omitted' packages will not be removed.

>>> Waiting 5 seconds before starting...
>>> (Control-C to abort)...
>>> Unmerging in: 5 4 3 2 1
>>> Unmerging (1 of 8) dev-python/pyxattr-0.6.0...
>>> Unmerging (2 of 8) dev-python/pygtk-2.24.0-r4...
>>> Unmerging (3 of 8) dev-python/trollius-2.1...
>>> Unmerging (4 of 8) gnome-base/libglade-2.6.4-r2...
>>> Unmerging (5 of 8) dev-python/pygobject-2.28.6-r55...
>>> Unmerging (6 of 8) virtual/python-futures-0...
>>> Unmerging (7 of 8) dev-python/futures-3.1.1...
>>> Unmerging (8 of 8) dev-lang/python-3.4.6...               
Packages installed:   579                           
Packages in world:    30                                          
Packages in system:   78                                             
Required packages:    579            
Number removed:       8

In the end, here is what my package.use looks like:

➜  ~ sudo chuse --dump | grep -B1 '^[^#].*python'
root # only python2.7 available
sys-devel/llvm python_targets_python2_7
--
root # it requires python2.7
media-libs/alsa-lib -python
--
root # only python2.7 available
sys-boot/boot-update python_targets_python2_7
--
root # it requires python2.7
net-libs/libproxy -python
--
root # it requires python2.7
sci-libs/geos -python
--
root # it requires python2.7
dev-libs/libxslt -python
--
root # required by evince
dev-libs/libxml2 python_targets_python2_7
--
root # only python2.7 available
dev-libs/gobject-introspection python_targets_python2_7
--
root # it requires python2.7
sys-libs/cracklib -python
--
root # it requires python2.7
dev-util/boost-build -python
--
root # only python2.7 available
dev-util/itstool python_targets_python2_7
--
root # it requires python2.7
media-gfx/gimp -python
--
root # it requires python2.7
dev-lang/yasm -python
--
root # it requires python2.7
dev-vcs/git -python
--
root # it requires python2.7
sys-apps/usbutils -python

Please ping me on IRC, if you have any question about it or wish to suggest me other issue examples relevant to python2 removal.

PS : If not already done, you might want to set python 3.5 as default interpreter as well :

root # eselect python set python3.5