Python 3.6 and Matplotlib 2 on macOS Big Sur

2021-02-05

After upgrading to macOS Big Sur (11.2), I had to work with Python 3.6 and Matplotlib 2.2.5. As of February 2021, Python 3.6 is no longer available using Homebrew. So I use Pyenv to manage multiple Python versions.

I had an old Python 3.6 installation from before the upgrade. The following error occured when using Matplotlib:

.../lib/python3.6/site-packages/matplotlib/backends/backend_macosx.py", line 17, in <module>
    from matplotlib.backends import _macosx
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function 
correctly if Python is not installed as a framework. See the Python documentation for more information 
on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try 
one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 
'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

There is an explanation on the problem on Matplotlib’s website.

Check if Python was compiled as a framework

As I use a Python version installed using Pyenv, I did not know if it was compiled as a framework. Let’s have a look using the PYTHONFRAMEWORK config var:

import sysconfig
sysconfig.get_config_var("PYTHONFRAMEWORK")

It returns '' when checking the version installed with Pyenv and 'Python3' for the system version (/usr/bin/python3).

Reinstall Python 3.6 as a framework

As the error message says, I tried reinstalling Python 3.6 as a framework (instead of a regular unix install):

$ PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install 3.6.12
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.6.12.tar.xz...
-> https://www.python.org/ftp/python/3.6.12/Python-3.6.12.tar.xz
Installing Python-3.6.12...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.2 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/gt/cz8lhhpx62x1w06s1kvmt4000000gn/T/python-build.20210205224833.35736
Results logged to /var/folders/gt/cz8lhhpx62x1w06s1kvmt4000000gn/T/python-build.20210205224833.35736.log

Last 10 log lines:
./Modules/posixmodule.c:8210:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(in, out, offset, &sbytes, &sf, flags);
              ^
./Modules/posixmodule.c:10432:5: warning: code will never be executed [-Wunreachable-code]
    Py_FatalError("abort() called from Python code didn't abort!");
    ^~~~~~~~~~~~~
1 warning and 1 error generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated.

… Ok, so it doesn’t compile anymore. The cause is due to older CPython versions not supporting macOS Big Sur. Fortunately someone already made it work.

The new install command is:

PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install --patch 3.6.12 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

Checking PYTHONFRAMEWORK afterwards gives 'Python'.
I can then install Matplotlib (pip3 install matplotlib==2.2.5), and run it:

$ python3
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2], [1,2])
[<matplotlib.lines.Line2D object at 0x12876ce48>]
>>> plt.show()
[1]    61755 segmentation fault  python3

The Matplotlib macosx backend does not work very well… The issue was fixed in Matplotlib 3.2.2 (at least). I found out afterwards that using that version fixes the problem entirely and Python 3.6 does not even have to be compiled as a framework.
But I am stuck with an older version 2.2.5 due to environment constraints.

Use Tk as a backend

Matplotlib supports different backends, so I decided to use TkAgg as a backend since it is also available for macOS:

>>> import matplotlib
>>> matplotlib.use("TKAgg")
>>> import matplotlib.pyplot as plt
...stack trace
.../lib/python3.6/tkinter/__init__.py", line 36, in <module>
    import _tkinter # If this fails your Python may not be configured for Tk
ModuleNotFoundError: No module named '_tkinter'

I have to reinstall Python again linked to tk:

First install tcl-tk from Homebrew: brew install tcl-tk
Install with Pyenv (remove the old version if needed) and the patch (enable all necessary libs for pandas, …):

PYTHON_CONFIGURE_OPTS="--with-tcltk-includes='-I$(brew --prefix tcl-tk)/include' --with-tcltk-libs='-L$(brew --prefix tcl-tk)/lib -ltcl8.6 -ltk8.6'" CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix xz)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib -L$(brew --prefix xz)/lib" pyenv install --patch 3.6.10 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

I can finally display an interactive window:

>>> import matplotlib
>>> matplotlib.use("TKAgg")
>>> import matplotlib.pyplot as plt
>>> plt.plot([1,2], [1,2])
[<matplotlib.lines.Line2D object at 0x14593d2b0>]
>>> plt.show()