On Thu, Jun 29, 2017 at 10:09 AM, Charles Wilt <charles.wilt@xxxxxxxxx> wrote:
I updated from 2.7.12 to 2.7.13...
So far, the issue hasn't reoccurred..
I'm glad it seems to have been resolved. But...
I noticed in the changelog for 2.7.13:
- Issue #27211: Fix possible memory corruption in io.IOBase.readline().
That issue was actually fixed in 2.7.12. (The changelog goes back
quite a ways, well past just the latest point release. It's easy to
miss the version header separating the releases when you're scanning
the whole thing.) And I don't think it would manifest in a way that
reorders lines but is otherwise clean.
On Thu, Jun 29, 2017 at 9:41 AM, Charles Wilt <charles.wilt@xxxxxxxxx> wrote:
connection.storlines('STOR ' + ibmi_file, open(source_file.fullpath, 'rb'))
Based on your description of the symptoms, and looking at the above
line of Python code, my best guess is that you were seeing some kind
of I/O buffering effect.
I guess you don't need any workarounds at the moment, but it might be
handy to keep some ideas in your pocket in case this comes up again.
Some things to try:
- Use mode 'r' instead of 'rb' when opening the file. (Plain 'r' is
the default and can be omitted.) The significance of this being that
in some contexts, buffering is different when dealing with text mode
versus binary mode.
- Fiddle with the explicit buffering setting on the open(). You can
choose 0 for unbuffered, 1 for line-buffered, or a positive number to
specify a desired byte size.
- Try to eliminate operating-system-level buffering by loading all the
data into a StringIO (memory file) object first, and send from that.
- Rework the process so that you can use FTP's binary mode (via the
storbinary() method) instead. This is of course more involved, and
there are several very different approaches that fall under this
umbrella (for example, you could translate the data to EBCDIC in
Python and continue trying "direct" FTP, or you could zip up the data
and unzip on the i, etc.) but still not especially difficult, and
worth a try if the simpler measures fail.
John Y.
As an Amazon Associate we earn from qualifying purchases.