Re: [IBMiOSS] Error installing pandas

I found the reason for the HTTPSHandler error and am posting here because I
*believe* it effects everyone and not just Litmis Spaces. I ran the
following on two different systems, one that is up to date on PTFs and one
that isn't:

find /QOpenSys/QIBM/ProdData/SC1 -name libssl.a

The one that is more up to date does NOT have
/QOpenSys/QIBM/ProdData/SC1/OpenSSL/lib/libssl.a So now I will comb
through the updates I've applied to see if there are some openssl notes I
missed.


---Site Note---
I am not a Python expert but I thought others could learn from how I
pursued finding the issue. The error happens when I try to install a
simple Python package...

% pip install appier

Traceback (most recent call last):


File "/QOpenSys/usr/bin/pip", line 7, in <module>


from pip import main


File
"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/__init__.py",
line 10, in <module>
from pip.util import get_installed_distributions, get_prog


File
"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/util.py",
line 18, in <module>
from pip._vendor.distlib import version


File
"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/_vendor/distlib/version.py",
line 14, in <module>
from .compat import string_types
File
"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/_vendor/distlib/compat.py",
line 66, in <module>
from urllib.request import (urlopen, urlretrieve, Request,
url2pathname,

ImportError: cannot import name 'HTTPSHandler'


Not knowing where HTTPSHandler existed I did a 'find':

% find /QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4 -type f | xargs
grep 'class HTTPSHandler*'
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/_vendor/distlib/util.py:class
HTTPSHandler(BaseHTTPSHandler):
grep: can't open
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/setuptools/__pycache__/script
grep: can't open template
grep: can't open (dev).cpython-34.pyc
grep: can't open
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/setuptools/__pycache__/script
grep: can't open template.cpython-34.pyc
grep: can't open
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/setuptools/command/launcher
grep: can't open manifest.xml
grep: can't open
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/setuptools/script
grep: can't open template
grep: can't open (dev).py
grep: can't open
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/setuptools/script
grep: can't open template.py
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/urllib/request.py:
class HTTPSHandler(AbstractHTTPHandler):



At this point I have two files with HTTPSHandler declared as a class. I
then did a google search for how to import a fully qualified path so I
could manually do an import in a Python REPL to further narrow down the
error. Here is what I found:
http://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path

That SO link gave me enough to put together the following in a Python REPL:

% python

Python 3.4.2 (default, Jun 12 2015, 19:07:14) [C] on aix6


Type "help", "copyright", "credits" or "license" for more information.

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name",

"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/urllib/request.py").load_module()

foo = SourceFileLoader("module.name",

"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/_vendor/distlib/util.py").load_module()

Traceback (most recent call last):


File "<stdin>", line 1, in <module>


File "<frozen importlib._bootstrap>", line 539, in _check_name_wrapper


File "<frozen importlib._bootstrap>", line 1614, in load_module


File "<frozen importlib._bootstrap>", line 593, in _load_module_shim


File "<frozen importlib._bootstrap>", line 1153, in exec


File "<frozen importlib._bootstrap>", line 1129, in _exec


File "<frozen importlib._bootstrap>", line 1471, in exec_module


File "<frozen importlib._bootstrap>", line 321, in
_call_with_frames_removed

File
"/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages/pip/_vendor/distlib/util.py",
line 18, in <module>
import ssl


File "/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/ssl.py", line
97, in <module>

import _ssl # if we can't import it, let the error
propagate

ImportError: Could not load module
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/lib-dynload/_ssl.so.

Dependent module libssl.a(libssl.so.1) could not be loaded.


Could not load module libssl.a(libssl.so.1).


System error: No such file or directory


Could not load module
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/lib-dynload/_ssl.so.

Dependent module
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/lib-dynload/_ssl.so
could not be loaded.

This lead me to search for libssl.a, which takes us back to the beginning
of this thread.




Aaron Bartell
litmis.com - Services for open source on IBM i


On Mon, Jun 20, 2016 at 9:27 PM, Aaron Bartell <aaronbartell@xxxxxxxxx>
wrote:

Excellent feedback, John. To date Litmis Spaces has been used mostly for
Node.js and RubyOnRails. Python not so much. Today we follow the same
directory structure as IBM.

Concerning multiple versions; developers will want easy switching from one
version of language to another. And then in production there should
(hopefully) be only one. I accomplish the multiple version in Node.js
using a custom shell script and run it as follows:

% . node_litmis.sh node-0.12.9

Btw, here's the result of the site-package code for those that are lurking
(this is a Python REPL session)...

import site

site.getsitepackages()>>>


['/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages',
'/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/site-python']

And the contents of that directory...

% ls /QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages

README easy_install.py pkg_resources.py


__pycache__ pip setuptools


_markerlib pip-1.5.6.dist-info
setuptools-2.1.dist-info


Aaron Bartell
litmis.com - Services for open source on IBM i


On Mon, Jun 20, 2016 at 9:00 PM, John Yeung <gallium.arsenide@xxxxxxxxx>
wrote:

On Mon, Jun 20, 2016 at 1:53 PM, Aaron Bartell <aaronbartell@xxxxxxxxx>
wrote:

*>(A huge step would be to install ibm_db and itoolkit.)*

Both should be installed on Litmis Spaces. Run the following command to
see: ls -all /QOpenSys/QIBM/ProdData/OPS/Python-pkgs

I think you meant either -al or -alF, rather than -all (each letter is
its own switch; there's no advantage to repeating a switch).

I was expecting those packages to be installed in Litmis Spaces
already, and indeed they do show up in the directory you've shown. But
I had no idea that was where they were until you told me. Anyone
familiar with Python will be looking for a directory called
site-packages[1].

I guess for now the simplest way to import them is to manually add the
necessary directory to the search path from within Python:

import sys
sys.path.append('/QOpenSys/QIBM/ProdData/OPS/Python-pkgs')
import ibm_db

But really, this is something that should be fixed.

[1]Python comes with a module called site. It has functions that
affect how the interpreter searches for packages to import. You can
see where Python is expecting to find installed packages as follows
(from the Python prompt):

import site
site.getsitepackages()

If those directories don't exist, they don't make it into sys.path.

I think something probably has to be done about the directory
structure anyway, because you absolutely have to have different
site-packages for different versions of Python. Sure, a Litmis space
can probably be configured to only have one version of Python, but
that seems like an artificial limitation.

Also, we can expect some extra effort in these initial stages of

adopting

open source on IBM i**. The key is ~documenting~ our progress so the

next

person doesn't have to go through the same hoops. Documenting can

happen

in many locations (stackoverflow.com, github gists, blogs, wikis, etc).
Also, feel free to make a new Python page at the following link to

document

resolutions: https://bitbucket.org/ibmi/opensource/wiki/Home

Totally agree. I will see what I can do, though I can't promise
anything as my time is even more limited than usual.

John Y.
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list
To post a message email: OpenSource@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/opensource
or email: OpenSource-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/opensource.