Re: FTP into PF-SRC re-orders records?

On Thu, Jun 29, 2017 at 2:49 PM, Charles Wilt <charles.wilt@xxxxxxxxx> wrote:

On Thu, Jun 29, 2017 at 9:22 AM, John Yeung <gallium.arsenide@xxxxxxxxx>
wrote:

Some things to try:

- Use mode 'r' instead of 'rb' when opening the file. (Plain 'r' is
the default and can be omitted.) The significance of this being that
in some contexts, buffering is different when dealing with text mode
versus binary mode.

doesn't storlines() require 'rb'?
https://stackoverflow.com/questions/17777745/python3-ftplib-storlines-error

Not in Python 2. That Stack Overflow question was specifically about
Python 3. (And indeed, the asker even starts of the question by saying
that the regular 'r' mode was what they used successfully in Python
2.)

In Python 3, strings are Unicode, not bytes. (This is the fundamental
difference between Python 2 and 3, and it's why they are not quite
compatible.) FTP operates on bytes, even in ASCII mode. (Note that FTP
has "binary" and "ascii" modes, NOT "binary" and "character" modes!)
That is why you need to open a file in binary mode in Python 3 for use
with FTP.

- Rework the process so that you can use FTP's binary mode (via the
storbinary() method) instead. This is of course more involved, and
there are several very different approaches that fall under this
umbrella (for example, you could translate the data to EBCDIC in
Python and continue trying "direct" FTP, or you could zip up the data
and unzip on the i, etc.) but still not especially difficult, and
worth a try if the simpler measures fail.

Given that we're sending to a PF-SRC...I wouldn't expect FTP binary mode
to work...don't we need the ASCII<-->EBCDIC conversion done by the FTP
server on the i?

Well, you seem to be talking about what I called "direct" FTP. I don't
know if binary mode works where the destination is PF-SRC. If it
doesn't, it's due to restrictiveness on the IBM i side. Which would be
understandable. If the i is being restrictive, it ought to be because
it assumes the outside world doesn't know about the SRCSEQ and SRCDAT
fields, not because the outside world can't handle translation to
EBCDIC.

I mentioned in my post that you could do the translation in Python.
It's super easy. But I also mentioned that you could get the data (by
hook or by crook) to the i in a non-PF-SRC form, and from there, deal
with getting it into the PF-SRC. We're talking CPYFRMSTMF or
CPYFRMIMPF or Python which lives on the i or whatever you've got. Rexx
maybe. It's easy to preprocess the data on the PC side into whatever
form is going to be accepted most easily on the i side. For my own
use, I even maintain SRCDAT on the PC side, and you could easily add
SRCSEQ and SRCDAT on the fly on the PC before sending over the data.

Finally, in the unlikely event that none of the discussed approaches
work, you still have the option of non-FTP methods, such as ODBC (via
pyodbc). That is actually what I use most often, even for PF-SRC.
(This is in part because I like to have control over SRCDAT.)

John Y.