Re: [IBMiOSS] [EXTERNAL] Re: beginning python -- OPENSOURCE

Hi Mike,

There are probably much better qualified people on this forum to explain
some of these things to you than myself, but I'll have a go and see if I
can shed any light.

On our system I believe we have 4 versions of python (not counting the
virtual environments I set up)

The 5733-OPS ones which are by default in your PATH.
I'm on a vpn and logged on from Windows 10 via SSH.

If I type: which python2
I see /QOpenSys/usr/bin/python2

If I type python2 -V
I see Python 2.7.16.

(Similarly which pip: shows me /QOpenSys/usr/bin/pip)

If I type: which python3
I see /QOpenSys/usr/bin/python3

If I type: python3 -V
I see 3.4.10.

(Similarly which pip3: shows me /QOpenSys/usr/bin/pip3)

These (above) are the versions that are by default in your PATH.

=================================================
When you install the opensource versions (which I did from the ACS menu,
but I guess Yum works the same) those versions get put into a different ifs
location which is not by default in your path:

They go into /QOpenSys/pkgs/bin

You can run these versions by modifying your path or providing the full
path on the command e.g:
/QOpenSys/pkgs/bin/python3 -V shows me 3.6.8.

I haven't really got time right now to explain all about virtual
environments, but that is how I've set my stuff up so that I can install
things and not worry about corrupting the base installation:

Suppose you've decided to put all of your python virtual environments in
the ifs under /home/pve and you decide to create a new one called test.

You could do it like this:
cd /home/pve
mkdir test
cd test
/QOpenSys/pkgs/bin/python3 -m venv ./venv --system-site-packages

That provides the full path to the opensource python3 and tells it to
create a virtual environment called venv (in my current directory)

To use that virtual environment you need to activate it. This is done as
follows:
cd /home/pve/test
source ./venv/bin/activate.

If you are in bash, you will see you shell change to show (venv) which
indicates that the virtual environment is active.
If you type which python now, you will see it shows
/home/pve/test/venv/bin/python - it is running the virtual environment
version of python and if you type: python -V it shows 3.6.8 e.g it is a
copy of the later opensource version.

Now, with the virtual environment activated, you would do your package
installation.
So, maybe first run: pip install --upgrade pip
Then install any other packages you want but make sure you have the virtual
environment activated so that you are installing the packages into that
environment.

To deactivate the virtual environment you just run: deactivate

You can see that this is achieved by manipulation of your path, for example
without the virtual environment activated if I type echo $PATH I see:
/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin

Notice that /QOpenSys/usr/bin is where the older versions of python are
installed (and therefore run by default)

If you activate your virtual environment again and look at you path you
will see something like:
/home/pve/test/venv/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin

So you can see that the virtual environment directory has been inserted at
the beginning of my path.

I don't know if this helps, and I apologise if I've described anything in a
slightly incorrect or simplified way, but this is my attempt at helping to
explain things :-)

It is particularly important to understand this to make sure you install
packages into your intended version of python.

Finally, I have some utility code I wrote that is passed the python code
and virtual environment you want to run it from.
Commands are pre-pended to path to and activate the virtual environment and
appended to deactivate the virtual environment.

As an example, suppose I wanted to run my python script
/home/py/duff_beer.py

If I just wanted to run this directly using the opensource version of
python you could run:
/QOpenSys/pkgs/bin/python3 /home/py/duff_beer.py

E.g simply provide the full paths.

If I ran this through my utility, I would simply pass: Python
/home/py/duff_beer.py and my virtual environment name (prod) to my utility.
The utility would build the following command string:

cd /home/pve/prod && source ./venv/bin/activate && Python
/home/py/duff_beer.py && deactivate

and pass that to QSH to run.

There is other stuff going on to capture output and make sure escape
messages get sent on errors etc but I don't have time to go into that at
the moment.

Anyway, I need to stop as it's getting late over here in the UK.
I hope this helps explain some stuff.
Apologies if I've made any typos along the way...

best regards,
Craig

On Wed, 11 Dec 2019 at 21:34, Smith, Mike <Mike_Smith@xxxxxxxxxxxxxxxx>
wrote:

John

Thanks for the answers.

Thanks for the reply. For an old RPG'r I can get lost in yums, rpms,
pips, etc.

I haven't had 5733-OPS on my system for quite a while, but I do see PTFGRP
SF99225 showing. Maybe I need to delete this PTF?

I did upgrade the OPENSSL and got rid of my initial error, but there is
definitely something amiss

User mike sees 3.4.6
User supermike sees 3.6.9

However after running pip3 install xlsxwriter. It showed installed
successfully, but tells me it ModuleNotFound Error: No module named
'xlswriter'

If I run pip3 freeze I see
appier==1.9.7
bottle==0.12.17
six==1.10.0
XlsxWriter==1.2.6

So then I tried to reinstall xlxswriter. I get
pip3 install xlsxwriter
[33mDEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be
the last one supporting it. Please upgrade your Python
as Python 3.4 won't be maintained after March 2019 (cf PEP 429). [0m
Requirement already satisfied: xlsxwriter in
/QOpenSys/QIBM/ProdData/OPS/Python3.4/lib/python3.4/site-packages (1.2.6)

So it does appear I need to get rid of 3.4, but I have no idea how to
accomplish this.

Subject: Re: [IBMiOSS] beginning python
From: John Yeung <gallium.arsenide@xxxxxxxxx>
Date: Mon, 9 Dec 2019 16:27:29 -0500

I have python 2.7 and and just upgraded python3 from 3.4.6 to 3.6.9 .

Python 3.4.6 is from 5733-OPS and should be removed if possible.
Everything is more straightforward if you exclusively use the stuff
which can be installed via yum. (If you use ACS to view and install
packages, then you're using yum.)

I will let someone else address the multiple profile issue. (Or maybe
a future version of myself.)

From python can I create spool files? Don't know if I want to just
wondering!

Yes. By default, the normal Python output (from the print function) as
well as Python's error messages are sent to QPRINT if invoked in batch
mode.

Trying out a sample I found to create a spreadsheet, I found I needed to
import xlsxwriter. I did not see that listed under the ACS Package manager,
but was able to do a PIP Install. Just trying to understand when and why I
should use ACS vs PIP.

pip is Python's package manager. It automatically finds, downloads,
and installs Python packages and their dependencies. It cannot handle
packages unrelated to Python. There are some packages *for* Python
which are implemented solely *in* Python, and virtually any of these
can be installed using pip. XlsxWriter is such a package.

However, there are some Python packages which are implemented at least
partly in some other language (typically C or C++) which needs to be
compiled to "machine" code. (I put that in quotes because IBM i of
course utilizes quite a bit of virtualization.) pip will attempt to
call the necessary compiler to build this code, but if the compiler
isn't present, or if the code is written in such a way that it's not
PASE-compatible (something may compile fine on Linux but not
necessarily on PASE), then pip will fail. The folks at IBM have been
trying to identify packages like this and make the necessary tweaks to
get them working on PASE, and once they do, they release something
called an RPM, which is a package that is installable by yum or ACS.

The relationship between pip and yum is this: both perform similar
functions (find, download, install packages), but pip is the package
manager for Python while yum is the package manager for the whole
system (which in this case is PASE).

Can I connect to sql server from python running on the I. I tried to
import pyodbc, but it is failing. Should I be using this?

You should, but you need to install the ODBC driver, as well as the
unixODBC package, as well as the pyodbc package.

Can someone point me to some slightly more advanced example of connecting
to db2 and creating a report.

I don't know of any. I know there are a few that involve connecting to
Db2. Have you done that part yet? There is very little to it, and once
you're connected, you use SQL to retrieve what you want.

Most people generate Excel output rather than spooled files. You can
find examples for that (sounds like you already did). If you can work
through both ends (input from database, output to Excel workbook),
then you are most (if not all) of the way there.

I think Python in PASE is new enough that there are very few published
examples of working in it. For now, I would guess it's more fruitful
to try what you can, then ask for specific help on this list or Ryver
when you get stuck.

John Y.

-----Original Message-----
From: OpenSource <opensource-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of
Kevin Adler
Sent: Monday, December 9, 2019 4:10 PM
To: opensource@xxxxxxxxxxxxxxxxxx
Subject: [EXTERNAL] Re: [IBMiOSS] beginning python

That's wrong, the openssl package does *not* ship any libraries so this
is
useless. Besides, the python3 package should be pulling in the correct
dependencies automatically.

If you ever see a load error due to an unresolved symbol or similar
error
coming from IBM RPMs, that's either a bug with setting LIBPATH and
picking
up the wrong library (don't do this) or it's a packaging bug that should
be reported to [1]
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbitbucket.org%2fibmi%2fopensource%2fissues&c=E,1,mB0w-M9xirrK54yR7LJh37IV7fYd8Y6GkPwqJKd9jXSIUsfkF9CBhW3QuvjpGmFk_FFrzWh2o72JbFFWircYhnPMtQT_lqa-mTp4bRG2aRKJLwpAhj1p4A,,&typo=1

----- Original message -----
From: Jack Woehr <jwoehr@xxxxxxxxxxxxxxxxxxxxxxxx>
Sent by: "OpenSource" <opensource-bounces@xxxxxxxxxxxxxxxxxx>
To: IBMi Open Source Roundtable <opensource@xxxxxxxxxxxxxxxxxx>
Cc:
Subject: [EXTERNAL] Re: [IBMiOSS] beginning python
Date: Mon, Dec 9, 2019 2:54 PM

need to update OpenSSL to that version or newer.

yum install openssl

--
Jack Woehr
Absolute Performance, Inc.
12303 Airport Way, Suite 100
Broomfield, CO 80021

NON-DISCLOSURE NOTICE: This communication including any and all
attachments is for the intended recipient(s) only and may contain
confidential and privileged information. If you are not the intended
recipient of this communication, any disclosure, copying further
distribution or use of this communication is prohibited. If you
received
this communication in error, please contact the sender and
delete/destroy
all copies of this communication immediately.
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list
To post a message email: OpenSource@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: [2]
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2flists.midrange.com%2fmailman%2flistinfo%2fopensource%c2%a0&c=E,1,mNM32Jq2CleRAkK3WCte55CRk-BJPS3PdBFLKry2iJ8twjl9xO7th5xn37GEObGpWWAnUS_NuS6eaZ5f5Z131BAV7N7Fe6XVHLvexgnjg0ym0-A,&typo=1
or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at [3]
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2farchive.midrange.com%2fopensource%c2%a0.&c=E,1,ZXGUzOksxQmPGNOT_S01cLEa7LUeQ-AASYA4qLq7z4MOiVZl4P6fQxPz6WvNo8zK_lGatrgmuDfpxkFJiU4aE9dAYoOMfb0cEZNA64Ehi9Y,&typo=1

Help support
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fmidrange.com&c=E,1,2BK7EU-8PJoKYPtk575vEkJ4WHt6Uylibd0aVyhCU7ejkz7Y05B4FEMNX3BOt-asarNVA_rGfSe7qwwaj1th386tzc4GoqGFr4nYEcnmTWUVwqjODbJ_bg,,&typo=1
by shopping at amazon.com with our affiliate
link: [4]https://amazon.midrange.com

References

Visible links
1.
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fbitbucket.org%2fibmi%2fopensource%2fissues&c=E,1,xMAJtel9dGa4lLXO3JNQayV04E0p_Jp1Chl3HBXcvrp0TDuBD3tEUxJToHBUYI2fQ3iCKDRRNJB_CAdKVokI3pDy1UEVI5QLpFzXGtl3lgBNdrA,&typo=1
2.
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2flists.midrange.com%2fmailman%2flistinfo%2fopensource&c=E,1,LWdph7MtCkdaX4uuwwuSQZ2fsCQKIezONYj7jlcpRJ71ApefNvqMpau0QmxlWkTUgi36yAgKKvZ3ZSE5FvZcubk8VcSTtDTCAJcZKuBfIiSDSfCU_3UP&typo=1
3.
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2farchive.midrange.com%2fopensource&c=E,1,CHmY0Q0zHRYmHG38bXgbGNDHMbW_5088EhgB9M3GakM8lE9nwmQkjo3bfUdGoc1u4Ff-jTmZykQmtKe91QYtvb1XMGqDupdZExwjq_EqGA,,&typo=1
4.
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2famazon.midrange.com%2f&c=E,1,3tPst-okaSP0u2iWyHNaqf5mqTZk4jws6RwyhnW3I4-Rxd9lH6gnv6Ua0o-BFMkFqJ0rxIpszCpUPGgFJvwAShYZf_LGbCJ7RtPtWM_k7IyJHw,,&typo=1
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list To post
a message email: OpenSource@xxxxxxxxxxxxxxxxxx To subscribe, unsubscribe,
or change list options,
visit:
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2flists.midrange.com%2fmailman%2flistinfo%2fopensource&c=E,1,TFhlrk-AqDaYVHhenMMwPBMrK-IoCqDKHHxEboHTKa03TZi0Z8Fpaj71N4QFH3JQ9YbRaAjBoalaaFkX55Kia6fT-3fEh_gvBrsB-QilrosNohuUadGPqldB6A,,&typo=1
or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2farchive.midrange.com%2fopensource.&c=E,1,09F6ZF-tjXC2SwnLrcxYy2SorBfe8PKPqw6Nxwjq7oRrGRjkvzQxPGlxtnhwyul0qc7S1HndvMbcxHbMGYxQO7bGvHzte-sX52O3xdkXb5vNBf3dUD9J&typo=1

Help support
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fmidrange.com&c=E,1,NS-N3MBvHwFWZvlrqC7tXpqH6TNnRxnF4VAsUFPCd4IEppb4EVLns-EmAzZa7NsIR8Xd4VAT8vMVJIsCqIEbGSHPUQnI5HT1CpnG3F2Axa2KYeI_-B_ijEc,&typo=1
by shopping at amazon.com with our affiliate
link:
https://linkprotect.cudasvc.com/url?a=https%3a%2f%2famazon.midrange.com&c=E,1,PS-89TaIuJLK_Wxxrq5Sng_A8ZWwZULuQ7Jbn4bAnfY-seCVLZg5zkQ8SD5QdfjhEYKG6x7nhOXYc_tzdQxh0uSxz2FNm_5GThdYCgGP0VG5vQ,,&typo=1
NOTICE: This message, including any attachment, is intended as a
confidential and privileged communication. If you have received this
message in error, or are not the named recipient(s), please immediately
notify the sender and delete this message.
--
This is the IBMi Open Source Roundtable (OpenSource) mailing list
To post a message email: OpenSource@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/opensource
or email: OpenSource-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/opensource.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com