Re: [IBMiOSS] Building cryptography for python2 -- OPENSOURCE

Yes, we're in the process of rebuilding cryptography against OpenSSL 1.1,
but need to modify our OpenSSL 1.1 build configuration first. Until then
you will only be able to build cryptography against 1.0.2
----
Kevin Adler
Software Development - Open Source, PASE, IBM i Access ODBC
IBM Systems, Dept 0NB
Phone: +1-507-253-5111
Email: kadler@xxxxxxxxxx

015-3 C117
2800 37th St NW
Rochester, MN 55901-4441
United States

----- Original message -----
From: Jeremy Meyer <JMeyer@xxxxxxxxxxxxx>
Sent by: "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx>
To: IBMi Open Source Roundtable <opensource@xxxxxxxxxxxxxxxxxx>
Cc:
Subject: [EXTERNAL] Re: [IBMiOSS] Building cryptography for python2
Date: Mon, Nov 4, 2019 8:21 AM

So Massimo gave me the below Make script. Another thing I ran into was
the OpenSSL version. I couldn't get CFFI to work with 1.1.1 I had to go
back to 1.0.2. If you search for "failure on installing cryptography"
you can find the full exchange. It was August 2018.

#!/QOpenSys/pkgs/bin/bash

OBJECT_MODE=64

export CONFIG_SHELL=/QOpenSys/pkgs/bin/bash
export CONFIG_ENV_ARGS=/QOpenSys/pkgs/bin/bash

export CC=gcc
export CFLAGS='-maix64 -fPIC -D_AIX -D_AIX32 -D_AIX41 -D_AIX43 -D_AIX51
-D_AIX52 -D_AIX53 -D_AIX61 -D_AIX71 -D_ALL_SOURCE -DFUNCPROTO=15'
export CXX="g++ -maix64"
export CXXFLAGS=$CFLAGS

export F77=xlf
#export FFLAGS="-O -I/QOpenSys/pkgs/include"

export AR="ar -X64"
export NM="nm -X64"

export LD="ld -b64"
# for 32-bit applications
#export LDFLAGS='-L/QOpenSys/pkgs/lib
-Wl,-blibpath:/QOpenSys/pkgs/lib:/usr/lib:/lib -Wl,-bmaxdata:0x80000000'
# for 64-bit applications
export LDFLAGS="-Wl,-brtl -Wl,-bexpall -Wl,-bbigtoc
-Wl,-blibpath:/QOpenSys/pkgs/lib:/usr/lib:/lib:./src/elfobjs
-Wl,-bmaxdata:0x80000000"

export PATH=/QOpenSys/pkgs/bin:/QOpenSys/pkgs/sbin:/usr/local/bin

./configure --prefix=/QOpenSys/pkgs --host=powerpc-ibm-aix6.1.0.0 make
clean make make install

-----Original Message-----
From: OpenSource <opensource-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of
Pete Helgren
Sent: Sunday, November 3, 2019 5:48 PM
To: IBMi Open Source Roundtable <opensource@xxxxxxxxxxxxxxxxxx>
Subject: Re: [IBMiOSS] Building cryptography for python2

************************************************
CAUTION: External email. Do not open attachments or click links.
Forward suspicious emails to Anti-Spam.
************************************************

Thanks. Still not making progress. The error message at the end of the
build attempt is:

ImportError: Could not load module
/tmp/pip-build-env-qs8sS8/overlay/lib/python2.7/site-packages/_cffi_backend.so.

And yet the location for_cffi_backend.so is here:

-rwxr-xr-x 1 pete 0 1004329 Nov 3 17:38
/QOpenSys/pkgs/lib/python2.7/site-packages/_cffi_backend.so

There is nothing, nada, zip in the /tmp folder. That may be normal but
I was hoping to find some logs or some details indicating what is going
on.

Pete Helgren
www.petesworkshop.com
GIAC Secure Software Programmer-Java
Twitter - Sys_i_Geek IBM_i_Geek

On 11/3/2019 12:18 PM, Jesse Gorzinski wrote:
> Yes, that does get sufficiently confusing.
> Yum puts on the language runtimes, and the language runtime gives you
> the ability to install language packages.
> The exception here is Python. For some packages that need "IBM i
> magic" or are needed as dependencies from other RPMs, IBM delivers
> those in RPM form. Generally, you want to install the RPM version
> whenever possible.
> Mixing RPM packages with pip packages can allow for things to get "out
> of sync" of you're not careful, meaning the package's "real" status
> can end up misaligned with what pip and/or yum thinks is installed.
>
>
> "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx> wrote on
> 11/03/2019
> 07:54:46 AM:
>
>> From: Pete Helgren <pete@xxxxxxxxxx>
>> To: IBMi Open Source Roundtable <opensource@xxxxxxxxxxxxxxxxxx>
>> Date: 11/03/2019 07:54 AM
>> Subject: [EXTERNAL] Re: [IBMiOSS] Building cryptography for python2
>> Sent by: "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx>
>>
>> The python3 packages gave me a clue to what was needed and yes, cffi
>> was installed :
>>
>> bash-4.4$ pip2 install cffi
>> DEPRECATION: Python 2.7 will reach the end of its life on January
>> 1st, 2020. Please upgrade your Python as Python 2.7 won't be
>> maintained after that date. A future version of pip will drop support
for Python 2.7.
>> More details about Python 2 support in pip, can be found at
>> [1]INVALID URI REMOVED
>>
> u=https-3A__pip.pypa.io_en_latest_development_release-2Dprocess_-23pyt
> hon-2D2-2Dsupport&d=DwIGaQ&c=jf_iaSHvJObTbx-
>> siA1ZOg&r=ZCVxF-
>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=KVk84tfgy9h5ALhWU-
>>
> pYQrwZVBDvRATk_FtjrnHzZ7M&s=1zLqIhfBBwnuBSKJpxSojl5wCaa7H-FMDeL-_F2L1T
> 8&e=
>
>> Requirement already satisfied: cffi in
>> /QOpenSys/pkgs/lib/python2.7/site-packages (1.13.1) Requirement
>> already satisfied: pycparser in
>> /QOpenSys/pkgs/lib/python2.7/site-packages (from cffi) (2.19)
>>
>> But, help me out here....yum vs pip is throwing me a bit. yum is
>> needed to install some aix/pase packages and then pip to install the
>> python dependencies? Maybe that is where I missed the boat: First
>> yum and then pip on the install?...I haven't had to do much with
>> python packages because they have generally been available without
>> the need to build so I may have missed a few steps or misread the
errors.
>>
>> I'll follow the link. Thanks
>>
>> Pete Helgren
>> www.petesworkshop.com
>> GIAC Secure Software Programmer-Java
>> Twitter - Sys_i_Geek IBM_i_Geek
>>
>> On 11/2/2019 8:58 PM, Jesse Gorzinski wrote:
>>> That particular error just means you need to "pip2 install cffi"
> first.
>>> If you need any hints on how IBM built any of their packages, you
>>> can
> find
>>> all answers to every package IBM has built here:
>>>
[2]ftp://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo/src/
>>>
>>> For instance, the spec file for the Python3 package shows the
> necessary
>>> runtime dependencies
>>> Requires: python3-cffi
>>> Requires: python3-asn1crypto
>>> Requires: python3-idna
>>> Requires: python3-six
>>>
>>> The BuildRequires shows you everything that was needed to build a
> package.
>>> For instance, if you chased down the spec for the python3-cffi
>>> module you'd see
>>> BuildRequires: python3-devel
>>> BuildRequires: python3-pip
>>> BuildRequires: libffi-devel
>>> BuildRequires: pkg-config
>>> BuildRequires: python3-pycparser
>>>
>>> So you'll have some time to spend building dependencies (probably
>>> 7/8
> in
>>> total) but overall this should be a pretty doable exercise.
>>> Start with "yum install libffi-devel pkg-config python2-six"
>>> Then "pip2 install pycparser cffi asn1crypto idna"
>>> ... and go from there!
>>>
>>>
>>>
>>>
>>> "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx> wrote on
> 11/02/2019
>>> 01:43:26 PM:
>>>
>>>> From: Pete Helgren <pete@xxxxxxxxxx>
>>>> To: IBMi Open Source Roundtable <opensource@xxxxxxxxxxxxxxxxxx>
>>>> Date: 11/02/2019 01:43 PM
>>>> Subject: [EXTERNAL] [IBMiOSS] Building cryptography for python2
>>>> Sent by: "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx>
>>>>
>>>> I see that python3 has the cryptography library but not python2 .
>>>> I
> am
>>>> on V7R4 and python 2.7.17 installed through the Open Source Package
>>>> manager (the only way OSS has been installed on this machine). I
>>>> get the following error:
>>>>
>>>> bash-4.4$ pip2 install cryptography
>>>> DEPRECATION: Python 2.7 will reach the end of its life on January
> 1st,
>>>> 2020. Please upgrade your Python as Python 2.7 won't be maintained
> after
>>>> that date. A future version of pip will drop support for Python
2.7.
>>>> More details about Python 2 support in pip, can be found at
>>>> [3]INVALID URI REMOVED
>>>>
> u=https-3A__pip.pypa.io_en_latest_development_release-2Dprocess_-23pyt
> hon-2D2-2Dsupport&d=DwIGaQ&c=jf_iaSHvJObTbx-
>>>> siA1ZOg&r=ZCVxF-
>>>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=qeuQ6fdglsaTQCDEAX2aNmBq-
>>>> PeFfrGYVzpLL9P3Vyo&s=3wH7Ck1Ej8m3L-wHynkTSk18wRVkVuuEX-_Arh2C_rM&e=
>>>> Collecting cryptography
>>>> Using cached
>>>> [4]INVALID URI REMOVED
>>>>
>
u=https-3A__files.pythonhosted.org_packages_be_60_da377e1bed002716fb2d5d1d1cab720f298cb33ecff7bf7adea72788e4e4_cryptography-2D2.
>>>> 8.tar.gz&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=ZCVxF-
>>>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=qeuQ6fdglsaTQCDEAX2aNmBq-
>>>> PeFfrGYVzpLL9P3Vyo&s=2CYauS3gLrXfzH-lhatf2FcPc2vmHf3dAiA5MXfW7Eo&e=
>>>> Installing build dependencies ... done
>>>> Getting requirements to build wheel ... done
>>>> Preparing wheel metadata ... error
>>>> ERROR: Command errored out with exit status 1:
>>>> command: /QOpenSys/pkgs/bin/python2
>>>>
> /QOpenSys/pkgs/lib/python2.7/site-packages/pip/_vendor/pep517/_in_proc
> ess.py
>>>> prepare_metadata_for_build_wheel /tmp/tmpV3U20Q
>>>> cwd: /tmp/pip-install-CW5LUD/cryptography
>>>> Complete output (41 lines):
>>>> Traceback (most recent call last):
>>>> File
>>>> "/QOpenSys/pkgs/lib/python2.7/site-packages/pip/_vendor/pep517/
>>>> _in_process.py",
>>>> line 257, in <module>
>>>> main()
>>>> File
>>>> "/QOpenSys/pkgs/lib/python2.7/site-packages/pip/_vendor/pep517/
>>>> _in_process.py",
>>>> line 240, in main
>>>> json_out['return_val'] = hook(**hook_input['kwargs'])
>>>> File
>>>> "/QOpenSys/pkgs/lib/python2.7/site-packages/pip/_vendor/pep517/
>>>> _in_process.py",
>>>> line 110, in prepare_metadata_for_build_wheel
>>>> return hook(metadata_directory, config_settings)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> setuptools/build_meta.py",
>>>> line 156, in prepare_metadata_for_build_wheel
>>>> self.run_setup()
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> setuptools/build_meta.py",
>>>> line 142, in run_setup
>>>> exec(compile(code, __file__, 'exec'), locals())
>>>> File "setup.py", line 275, in <module>
>>>> **keywords_with_side_effects(sys.argv)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> setuptools/__init__.py",
>>>> line 145, in setup
>>>> return distutils.core.setup(**attrs)
>>>> File "/QOpenSys/pkgs/lib/python2.7/distutils/core.py",
>>>> line
> 111,
>>>> in setup
>>>> _setup_distribution = dist = klass(attrs)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> setuptools/dist.py",
>>>> line 447, in __init__
>>>> k: v for k, v in attrs.items()
>>>> File "/QOpenSys/pkgs/lib/python2.7/distutils/dist.py",
>>>> line
> 287,
>>>> in __init__
>>>> self.finalize_options()
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> setuptools/dist.py",
>>>> line 735, in finalize_options
>>>> ep.load()(self, ep.name, value)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/cffi
>>>> /
>>>> setuptools_ext.py",
>>>> line 217, in cffi_modules
>>>> add_cffi_module(dist, cffi_module)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/cffi
>>>> /
>>>> setuptools_ext.py",
>>>> line 49, in add_cffi_module
>>>> execfile(build_file_name, mod_vars)
>>>> File
>>>> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/cffi
>>>> /
>>>> setuptools_ext.py",
>>>> line 25, in execfile
>>>> exec(code, glob, glob)
>>>> File "src/_cffi_src/build_openssl.py", line 106, in
<module>
>>>> extra_link_args=extra_link_args(compiler_type()),
>>>> File "src/_cffi_src/utils.py", line 57, in
> build_ffi_for_binding
>>>> extra_link_args=extra_link_args,
>>>> File "src/_cffi_src/utils.py", line 65, in build_ffi
>>>> ffi = FFI()
>>>> File
>>>>
> "/tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/cffi/ap
> i.py",
>>>> line 48, in __init__
>>>> import _cffi_backend as backend
>>>> ImportError: Could not load module
>>>> /tmp/pip-build-env-3ziLDd/overlay/lib/python2.7/site-packages/
>>>> _cffi_backend.so.
>>>> The local-exec model was used for thread-local
>>>> storage, but the module is not the main program.
>>>> Examine the .loader section header with the 'dump -Hv'
command.
>>>> ----------------------------------------
>>>> ERROR: Command errored out with exit status 1:
>>>> /QOpenSys/pkgs/bin/python2
>>>>
> /QOpenSys/pkgs/lib/python2.7/site-packages/pip/_vendor/pep517/_in_proc
> ess.py
>>>> prepare_metadata_for_build_wheel /tmp/tmpV3U20Q Check the logs for
> full
>>>> command output.
>>>>
>>>> Searches turned up nothing useful.cffi is installed: (with "pip2
> install
>>>> cffi") just can't quite sort out what the error is trying to tell
>>>> me except that _cffi_backend.so shared object isn't found.
>>>>
>>>> --
>>>> Pete Helgren
>>>> www.petesworkshop.com
>>>> GIAC Secure Software Programmer-Java Twitter - Sys_i_Geek
>>>> IBM_i_Geek
>>>>
>> --
>> This is the IBMi Open Source Roundtable (OpenSource) mailing list To
>> post a message email: OpenSource@xxxxxxxxxxxxxxxxxx To subscribe,
>> unsubscribe, or change list options,
>> visit: [5]INVALID URI REMOVED
>>
> u=https-3A__lists.midrange.com_mailman_listinfo_opensource&d=DwIGaQ&c=
> jf_iaSHvJObTbx-
>> siA1ZOg&r=ZCVxF-
>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=KVk84tfgy9h5ALhWU-
>>
> pYQrwZVBDvRATk_FtjrnHzZ7M&s=Rg4NuO9SRVsSsIFj0l_h79I5jNtCToduCmNtv54cMZ
> A&e=
>
>> or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
>> Before posting, please take a moment to review the archives at
>> [6]INVALID URI REMOVED
>> u=https-3A__archive.midrange.com_opensource&d=DwIGaQ&c=jf_iaSHvJObTbx
>> -
>> siA1ZOg&r=ZCVxF-
>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=KVk84tfgy9h5ALhWU-
>>
> pYQrwZVBDvRATk_FtjrnHzZ7M&s=IuwdE9m_iK1aCC77r-r2QBMJdUWG6B32ny2sTbfUX6
> g&e=
> .
>> Help support midrange.com by shopping at amazon.com with our
>> affiliate
>> link: [7]INVALID URI REMOVED
>> u=https-3A__amazon.midrange.com&d=DwIGaQ&c=jf_iaSHvJObTbx-
>> siA1ZOg&r=ZCVxF-
>> w6wAginVZ42dfeYbM5huvio_77jTCJXaiFgrE&m=KVk84tfgy9h5ALhWU-
>>
> pYQrwZVBDvRATk_FtjrnHzZ7M&s=7R0euJdtLmCIYaGHVNv3a3gauyOXg2qORQQ_N2zFXy
> g&e=
>
>
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list To
post a message email: OpenSource@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: [8]https://lists.midrange.com/mailman/listinfo/opensource ;
or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
[9]https://archive.midrange.com/opensource ;.

Help support midrange.com by shopping at amazon.com with our affiliate
link: [10]https://amazon.midrange.com ;
NOTICE: This electronic mail message and any files transmitted with it
are intended
exclusively for the individual or entity to which it is addressed. The
message,
together with any attachment, may contain confidential and/or privileged
information.
Any unauthorized review, use, printing, saving, copying, disclosure or
distribution
is strictly prohibited. If you have received this message in error,
please
immediately advise the sender by reply email and delete all copies.
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list
To post a message email: OpenSource@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: [11]https://lists.midrange.com/mailman/listinfo/opensource ;
or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at [12]https://archive.midrange.com/opensource ;.

Help support midrange.com by shopping at amazon.com with our affiliate
link: [13]https://amazon.midrange.com ;

References

Visible links
1. file:///tmp/INVALID URI REMOVED
2. ftp://public.dhe.ibm.com/software/ibmi/products/pase/rpms/repo/src/
3. file:///tmp/INVALID URI REMOVED
4. file:///tmp/INVALID URI REMOVED
5. file:///tmp/INVALID URI REMOVED
6. file:///tmp/INVALID URI REMOVED
7. file:///tmp/INVALID URI REMOVED
8. https://lists.midrange.com/mailman/listinfo/opensource
9. https://archive.midrange.com/opensource
10. https://amazon.midrange.com/
11. https://lists.midrange.com/mailman/listinfo/opensource
12. https://archive.midrange.com/opensource
13. https://amazon.midrange.com/