Compile Python3.7 on Solaris 10

#python #opensource

Today I've spent some hours fighting with Solaris 10 trying to compile Python3.7 from sources with openSSL support.

The process is quite simple so I'll write it here so it can help someone with the same problem.

pkgadd -d http://get.opencsw.org/now
/opt/csw/bin/pkgutil -U
/opt/csw/bin/pkgutil -y -i libssl_dev
/opt/csw/bin/pkgutil -y -i libssl1_0_0
/opt/csw/bin/pkgutil -y -i libreadline6 
/opt/csw/bin/pkgutil -y -i libreadline_dev 
/opt/csw/bin/pkgutil -y -i libncurses_dev 
/opt/csw/bin/pkgutil -y -i libssl_dev 
/opt/csw/bin/pkgutil -y -i libsqlite3_dev 
/opt/csw/bin/pkgutil -y -i tk_dev 
/opt/csw/bin/pkgutil -y -i libgdbm_dev 
/opt/csw/bin/pkgutil -y -i libbz2_dev 
/opt/csw/bin/pkgutil -y -i libffi_dev 
/opt/csw/bin/pkgutil -y -i tk 
/opt/csw/bin/pkgutil -y -i pkgconfig 
/opt/csw/bin/pkgutil -y -i libsodium_dev # for paramiko
wget https://www.python.org/ftp/python/3.7.6/Python-3.7.6.tgz
gunzip -c Python-3.7.6.tgz | tar xvf -
cd Python-3.7.6

tip: edit Modules/socketmodule.c

$ diff -u ../socketmodule.c Modules/socketmodule.c
--- ../socketmodule.c   Wed May 15 16:36:32 2019
+++ Modules/socketmodule.c      Wed May 15 15:34:50 2019
@@ -5212,6 +5212,10 @@
 extern int sethostname(const char *, size_t);
 #endif

+#if (defined(__sun) && defined(__SVR4))
+extern int sethostname(const char *, size_t);
+#endif
+
     if (!PyArg_ParseTuple(args, "S:sethostname", &hnobj)) {
         PyErr_Clear();
         if (!PyArg_ParseTuple(args, "O&:sethostname",

To avoid an error related to sethostname.

Also, yo avoid problems with ctypes module you need to add the PKG_CONFIG_PATH option in the configure statement as follow:

  ./configure --prefix=/opt/python3 --with-openssl=/opt/csw/ --enable-optimizations LDFLAGS='-L/opt/local/lib -I/opt/csw/include -L/opt/csw/lib  -R/opt/local/lib' PKG_CONFIG_PATH=/opt/csw/lib/amd64/pkgconfig/ CPPFLAGS='-L/opt/local/lib -I/opt/csw/include -L/opt/csw/lib  -R/opt/local/lib'

 gmake
 gmake install
 PATH=/opt/python3/bin/:$PATH

You won't be able to use pip3 without adding such openssl flag to the configure step. And you need to install exactly the version 1.0.0 of libssl from CSW.

With all of this we will have python3 installed and pip3 working!

If you need to install pandas or numpy or somilar software I would recommend:

/opt/csw/bin/pkgutil -y -i gcc5g++ libffi_dev libssl_dev libbabl_dev cython automake autoconf libsodium_dev jq pkgconfig cswpki libatlas_c++ libatk_dev
pip3 install --upgrade cython
pip3 install pandas

Best regards.

Top comments (2)

rsandoz • May 21 '21

Hi I am attempting this on Solaris 11.4 x86 on an esxi VM in hopes of using letsencrypt with certbot to get free SSL certs. I am building both Python 3.7.10 and 3.9.5 from source (latest stable from python.org). My config and needs are probably different from yours, but if you happen to have any hints, much appreciated. BTW, your writeup is much more straightforward than other stuff I found. Thank you!

I get the following issues:

(1) diff -u ../socketmodule.c Modules/socketmodule.c
What is your current working directory when you start? Mainly because on your step where you call to edit socketmodule.c, you enter the Python subdir and compare your Python sources copy with a copy that you have in the .. directory.

From your diff output, I just assumed that I needed to replace the ifdef in my copy:
#ifdef AIX
extern int sethostname(const char *, size_t);
#endif
with the if referenced in your instructions:
#if (defined(sun) && defined(_SVR4))
extern int sethostname(const char *, size_t);
#endif

(2) Python3.7.10
$ gmake
...
0:10:37 load avg: 1.32 [ 25/416] test_asyncio
/scratch/Python-3.7.10/Lib/asyncio/unix_events.py:878: RuntimeWarning: A loop is being detached from a child watcher with pending handlers
RuntimeWarning)
test test_asyncio failed
0:17:16 load avg: 1.26 [ 26/416] test_asyncore -- test_asyncio failed in 6 min 38 sec
0:17:17 load avg: 1.26 [ 27/416] test_atexit -- test_asyncore skipped
...
0:31:09 load avg: 1.33 [ 55/416] test_cmd_line
test test_cmd_line failed
0:46:10 load avg: 2.45 [ 56/416] test_cmd_line_script -- test_cmd_line failed in 15 min
1:00:40 load avg: 2.37 [ 57/416] test_code -- test_cmd_line_script passed in 14 min 30 sec
...
1:56:16 load avg: 2.22 [ 84/416] test_contextlib_async
Task was destroyed but it is pending!
task: ()>>
Task was destroyed but it is pending!
task: ()>>
Task was destroyed but it is pending!
task: ()>>
...
1:57:17 load avg: 1.67 [ 92/416] test_ctypes
1:57:17 load avg: 1.67 [ 93/416] test_curses -- test_ctypes skipped
1:57:18 load avg: 1.67 [ 94/416] test_dataclasses -- test_curses skipped (resource denied)
...
1:57:24 load avg: 1.62 [ 97/416] test_dbm_dumb
test test_dbm_dumb failed
1:57:25 load avg: 1.62 [ 98/416] test_dbm_gnu -- test_dbm_dumb failed
test test_dbm_gnu failed
1:57:26 load avg: 1.62 [ 99/416] test_dbm_ndbm -- test_dbm_gnu failed
test test_dbm_ndbm failed
1:57:26 load avg: 1.61 [100/416] test_decimal -- test_dbm_ndbm failed
...
1:58:25 load avg: 1.42 [114/416] test_dtrace
test test_dtrace failed
1:58:38 load avg: 1.40 [115/416] test_dummy_thread -- test_dtrace failed
...
2:11:50 load avg: 1.34 [139/416] test_float
test test_float failed
2:11:51 load avg: 1.34 [140/416] test_flufl -- test_float failed
...
2:24:39 load avg: 1.37 [206/416] test_locale
test test_locale failed
2:24:40 load avg: 1.38 [207/416] test_logging -- test_locale failed
Exception in thread Thread-415:
Traceback (most recent call last):
File "/scratch/Python-3.7.10/Lib/threading.py", line 926, in _bootstrap_inner
self.run()
File "/scratch/Python-3.7.10/Lib/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "/scratch/Python-3.7.10/Lib/test/test_logging.py", line 875, in serve_forever
asyncore.loop(poll_interval, map=self._map)
File "/scratch/Python-3.7.10/Lib/asyncore.py", line 203, in loop
poll_fun(timeout, map)
File "/scratch/Python-3.7.10/Lib/asyncore.py", line 144, in poll
r, w, e = select.select(r, w, e, timeout)
OSError: [Errno 9] Bad file number
...
2:26:29 load avg: 1.32 [234/416] test_ntpath
test test_ntpath failed
2:26:38 load avg: 1.32 [235/416] test_numeric_tower -- test_ntpath failed
...
2:33:56 load avg: 1.28 [259/416] test_posixpath
test test_posixpath failed
2:34:05 load avg: 1.29 [260/416] test_pow -- test_posixpath failed
...
2:36:09 load avg: 1.38 [278/416] test_re
test test_re failed
2:36:12 load avg: 1.38 [279/416] test_readline -- test_re failed
...
2:50:09 load avg: 2.79 [288/416] test_sax
test test_sax failed
2:50:10 load avg: 2.77 [289/416] test_sched -- test_sax failed
...
2:51:55 load avg: 1.35 [300/416] test_signal
test test_signal failed
2:55:41 load avg: 1.28 [301/416] test_site -- test_signal failed in 3 min 45 sec
...
2:57:18 load avg: 1.34 [307/416] test_socket
/scratch/Python-3.7.10/Lib/test/test_socket.py:2321: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg(bufsize, *args)
/scratch/Python-3.7.10/Lib/test/test_socket.py:2412: RuntimeWarning: received malformed or improperly-truncated ancillary data
result = sock.recvmsg_into([buf], *args)
test test_socket failed
2:57:45 load avg: 1.01 [308/416] test_socketserver -- test_socket failed
...
2:59:21 load avg: 1.27 [313/416] test_ssl
test test_ssl failed
2:59:27 load avg: 1.25 [314/416] test_startfile -- test_ssl failed
...
3:17:11 load avg: 1.43 [350/416] test_time
test test_time failed
3:17:14 load avg: 1.40 [351/416] test_timeit -- test_time failed
...
3:32:34 load avg: 1.34 [369/416] test_unicode_file
test test_unicode_file failed
3:32:35 load avg: 1.34 [370/416] test_unicode_file_functions -- test_unicode_file failed
...
Failed to build these modules:
_ctypes
...

(3) Python 3.9.5
$ gmake
...
0:19:58 load avg: 2.35 [17/43] test_float -- test_embed passed in 16 min 36 sec
test test_float failed
0:20:00 load avg: 2.35 [18/43] test_fstring -- test_float failed
...
0:26:38 load avg: 2.36 [34/43] test_re
test test_re failed
0:26:43 load avg: 2.37 [35/43] test_set -- test_re failed
...
0:29:11 load avg: 2.37 [40/43] test_time -- test_tabnanny passed in 1 min 48 sec
test test_time failed
0:29:14 load avg: 2.36 [41/43] test_unicode -- test_time failed
...
building '_ctypes' extension
gcc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -D_REENTRANT -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -fprofile-use -fprofile-correction -I./Include/internal -I./Include -I. -I/opt/csw/include -I/usr/local/include -I/scratch/Python-3.9.5/Include -I/scratch/Python-3.9.5 -c /scratch/Python-3.9.5/Modules/_ctypes/_ctypes.c -o build/temp.solaris-2.11-i86pc.64bit-3.9/scratch/Python-3.9.5/Modules/_ctypes/_ctypes.o -DPy_BUILD_CORE_MODULE
/scratch/Python-3.9.5/Modules/_ctypes/_ctypes.c:107:10: fatal error: ffi.h: No such file or directory
#include
^~~~~~~
compilation terminated.
...
Failed to build these modules:
_ctypes

$ find / | egrep "\/ffi.h"
/scratch/Python-3.7.10/Modules/_ctypes/libffi_osx/include/ffi.h
/scratch/Python-3.7.10/Modules/_ctypes/libffi_msvc/ffi.h
/scratch/Python-3.9.5/Modules/_ctypes/libffi_osx/include/ffi.h
/usr/lib/libffi-3.2.1/include/ffi.h
/usr/lib/amd64/libffi-3.2.1/include/ffi.h
/opt/csw/lib/amd64/libffi-3.2.1/include/ffi.h
/opt/csw/lib/libffi-3.2.1/include/ffi.h

(I tried running the gcc compile with manually adding the include -I with the above, but more errors)