Re: FTP from unix/linux system causing EOL issues

Hi Scott,

Caught by the assumed :-)

Yes we are ftp'ing directly to a PF

Yes we are using the IBM provided ftp client.

Yes we are explicitly using ascii mode.

If I ftp from a windows machine the CRLF are removed

If I ftp from the linux system the CRLF is replace with a LF (0D)

Scott wrote: If you are transferring directly to a PF, the IBM FTP client
should
already know that in ASCII mode the records will end in CRLF, and remove
that for you.

I wish that was the case but that is not what is happening - see below
data from DSPPFM + F10 + F11

From the trail

After ftp'ing the file directly to a physical file we have a end of

line

character of hex 0D

Here is an example
,,,13/02/23,12:50,$20.55,$0.00,006793,V31142█
666FF6FF6FF6FF7FF65FF4FF65F4FF6FFFFFF6EFFFFF0444
BBB13102123B12A50BB20B55BB0B00B006793B531142D000

With files that we ftp from a windows server there is no end of line
character at all as per the following example

Pay Period,,2302,,Account Number ,,4770,,,,,,,,,,
D8A4D8989866FFFF66C889A9A4DA9889466FFFF666666666644444
7180759964BB2302BB133645305442590BB4770BBBBBBBBBB00000

Thank you for you responses

Don





From: "Scott Klement" <sk@xxxxxxxxxxxxxxxx>
To: midrange-l@xxxxxxxxxxxxxxxxxx
Date: 16/02/2023 08:39 AM
Subject: Re: FTP from unix/linux system causing EOL issues
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxxxxxxxx>



Hi Don,

I guess I assumed you were FTP into an IFS stream file. Are you loading
it into a physical file?

If you are transferring directly to a PF, the IBM FTP client should
already know that in ASCII mode the records will end in CRLF, and remove
that for you. If you are using a different FTP client than the one that
came with the OS, you should check with whomever is responsible for
supporting that client.

If you are FTPing to the IFS, then you can simply set the CCSID of the
object to whichever type of ASCII it's encoded with.

-SK

On 2/15/23 3:58 PM, Don Brown via MIDRANGE-L wrote:

Hi Scott,

Thanks for your input ...

The file being retrieved from the linux ftp server is in ascii.

If I use ascii mode on the transfer the file is converted to ebcidic.

If I use bin(ary) mode it is not converted. (I know as this was my first
thought and I tried it)

Now I am presuming ccsid's come into this somehow to convert from ascii

to

ebcidic but I do not know how.

Or do I need to ftp the file to the IFS and then copy it specifying the
from ccsid ?

Our current solution of checking and replacing the EOL character(s) is
working but is realy a brute force and ignorance solution.

As we are likely to encounter more unix/linux ftp servers in the future
having a more specific approach is preferable.

And I presume we will have exactly the same issues using sftp/scp
etcetera.

I am sure others have had this issue or will have so hopefully there is

a

simple process providing the solution..

Thanks

Don





From: "Scott Klement" <midrange-l@xxxxxxxxxxxxxxxx>
To: midrange-l@xxxxxxxxxxxxxxxxxx
Date: 16/02/2023 06:45 AM
Subject: Re: FTP from unix/linux system causing EOL issues
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxxxxxxxx>



Hi Don,

Indeed, that is the way ASCII mode is supposed to work. in ASCII mode,
the sending computer converts the end-of-lines to CRLF (hex 0d0a). The
receiving computer should, then, convert these to it's own native

format.

For example, if you are on a Windows PC and connecting to Unix, when you
switch to ASCII mode, it will send CRLF, and since it's the norm on
Windows, it'd keep it as-is when saving the file.

On the other hand, if you are on a Unix machine, the receiving FTP
client would convert the CRLF to just LF. Because that's the norm on
Unix.

This is the purpose of ASCII mode.

Since IBM i does not really have a standard between LF and CRLF, it will
just save the CRLF.

Maybe you should be receiving the data using binary mode so that it's
not converted?

-SK


On 2/15/23 1:30 AM, Don Brown via MIDRANGE-L wrote:

We are transferring files from a new ftp server hosted on a unix/linux
system.

The transfer is using ascii mode

If you look at the ascii file there are CRLF (0A0D) at the end of each
line

After ftp'ing the file directly to a physical file we have a end of

line

character of hex 0D

Here is an example
,,,13/02/23,12:50,$20.55,$0.00,006793,V31142█
666FF6FF6FF6FF7FF65FF4FF65F4FF6FFFFFF6EFFFFF0444
BBB13102123B12A50BB20B55BB0B00B006793B531142D000

Our process to then read the file wich is in csv format was then

failing

on the end of line character.

We have added some code to replace the hex 0D with a blank and this has
enabled the file to be processed but I have not been able to find any
other option.

So just asking the group if someone has hit this issue before and how

they

resolved the problem

With files that we ftp from a windows server there is no end of line
character at all as per the following example

Pay Period,,2302,,Account Number ,,4770,,,,,,,,,,
D8A4D8989866FFFF66C889A9A4DA9889466FFFF666666666644444
7180759964BB2302BB133645305442590BB4770BBBBBBBBBB00000


Any suggestions welcomed

Don



--
This email has been scanned for computer viruses. Although MSD has

taken

reasonable precautions to ensure no viruses are present in this email,

MSD

cannot accept responsibility for any loss or damage arising from the use
of this email or attachments.