RE: floundering with sftp

Thanks for your reply Paul
At the moment, I have had to shelve the sftp as a higher priority job needs to be worked on
However, I WILL be returning back to this sftp
Scott Klement, sent me an e-mail last night (and I just realized I didn't acknowledge that fact)
He also said to use QSH
I will give that a go

Alan Shore
E-mail : ASHORE@xxxxxxxx
Phone [O] : (631) 200-5019
Phone [C] : (631) 880-8640
'If you're going through hell, keep going.'
Winston Churchill

-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Steinmetz, Paul
Sent: Thursday, March 13, 2014 12:05 PM
To: 'Midrange Systems Technical Discussion'
Subject: RE: floundering with sftp

Alan,

We've developed a similar solution for SFTP logging and error handling.
We use QSH instead of "call qp2term", QSH offers some advantages QSH will create a spoolfile.
We copy that spoolfile to a PF in qtemp.
We then call a RPGLE utility program, (SFTPLOG) that scans the PF.
We search for "Uploading" or Fetching "
If found, we know the SFTP was successful.
If not found, we know it failed,
Message can be sent as needed to QSYSOPR, MPLUS will pickup the message and alert us.

0044.00 /free
0046.00 pos = %scan(searchfor: irec :1);
0048.00 /end-free

0034.00 DCL VAR(&CM31) TYPE(*CHAR) LEN(62) +
0035.00 VALUE('/QOpenSys/bin/sftp ') /* Latest +
0036.00 Version */
0037.00
0038.00 DCL VAR(&CM32) TYPE(*CHAR) LEN(6) VALUE('-v -b ')
0039.00
0040.00 DCL VAR(&CM33) TYPE(*CHAR) LEN(29) +
0041.00 VALUE('/home/PAULS/Paulssftptsti.txt')
0045.00
0046.00 DCL VAR(&CM34) TYPE(*CHAR) LEN(71) VALUE(' +
0047.00 PAULS@xxxxxxxxxxxxxxxxxxxxxxxxx') /* +
0048.00 Pencor Linux Test Server */
0049.00
0050.00 DCL VAR(&SYSNAME) TYPE(*CHAR) LEN(8)
0051.00 RTVNETA SYSNAME(&SYSNAME)
0052.00
0053.00 CHGVAR VAR(&CM3) VALUE(&CM31 *CAT &CM32 *CAT &CM33 +
0054.00 *CAT &CM34)
0056.00 QSH CMD(&CM3)
0057.00
0058.00 RTVJOBA JOB(&JOBNAME) USER(&JOBUSER) NBR(&JOBNUM)
0059.00
0060.00 CRTPF FILE(QTEMP/SFTPLOG) RCDLEN(136)
0061.00 MONMSG MSGID(CPF0000)
0062.00
0063.00 CPYSPLF FILE(QPRINT) TOFILE(QTEMP/SFTPLOG) +
0064.00 JOB(&JOBNUM/&JOBUSER/&JOBNAME) +
0065.00 SPLNBR(*ONLY) CTLCHAR(*PRTCTL)
0066.00
0067.00 CALL PGM(SFTPLOG) PARM(&OK)
0068.00
0069.00 IF COND(&OK *NE 'Y') THEN(DO)
0070.00 SNDMSG MSG('SFTP Test Transfer failed') +
0071.00 TOUSR(QSYSOPR) /* T */
0072.00 ENDDO
0073.00
0074.00 IF COND(&OK *EQ 'Y') THEN(DO)
0075.00 SNDMSG MSG('SFTP Test Transfer Successful') +
0076.00 TOUSR(QSYSOPR) /* T */
0077.00 ENDDO

Paul

-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: Wednesday, March 12, 2014 10:32 PM
To: Midrange Systems Technical Discussion
Subject: Re: floundering with sftp

Alan,

I would strongly recommend never using QP2SHELL from a CL program. The problem with QP2SHELL is that it expects the Unix standard I/O descriptors to be set up and working, already. Granted, you can set these up yourself (via spawn() and pipe() APIs) but hardly anyone ever
does that... So they call QP2SHELL and let the descriptors fall back
to the ILE stdio support, which works sometimes, but is often fraught with problems, especially when using software that uses multiple
threads... I never got a really clear answer from IBM as to why this
causes problems.

Same is true of OVRDBF FILE(STDOUT) -- this is using ILE's implementation of stdio. Same problem.

Instead, please use QShell for this. (the QSH or STRQSH commands, not
QP2SHELL!) It will set up the descriptors properly, etc.

Or, if for some reason, QShell isn't going to work, use a tool like my UNIXCMD, or Tony Cairn's popen() these are designed to set up the pipes, etc, properly.

In fact, your code seems designed to use Qshell, it's using some QIBM_QSH variables, which are for Qshell, and will do nothing all all when calling a PASE API like QP2SHELL. (This really makes me wonder?)

Anyway...

I can't tell if this is what's causing the problem or not, there doesn't seem to be much info about the failure, here. But see if it helps, or at least gives you more info. And try running the program (with the CL program controling it, etc) interactively to see if errors are going to the screen (since they'd be harder to spot in a batch subsystem/jobq.)




On 3/12/2014 7:28 AM, Alan Shore wrote:

Hi everyone
Before I forget, we are on V5r4
I have to use sftp to obtain files from another server I have already
posted a couple of questions here before, that people have been so
kind to answer Unfourtunately - I am floundering I googled quite a few
things and found this site that does part of what I need to do

http://forums.iprodeveloper.com/forums/aft/55771

titled "How to get results of LS command from SFTP" where someone
posted a question that easily could have been me Scott Klement
provided examples of which I used Here is what I have ended up with

PGM
/* To compile: */
/*> CRTPF QTEMP/LSOUTPUT RCDLEN(1000) <*/

/* FIXME: Replace the following 3 lines with the appropriate +
values for your environment */

DCL VAR(&USERID) TYPE(*CHAR) LEN(15) +
VALUE('j_nbty')
DCL VAR(&HOST) TYPE(*CHAR) LEN(100) +
VALUE('sftp ftpnonprodnbtysldc ')
DCL VAR(&DIR) TYPE(*CHAR) LEN(100) +
VALUE('DNBT2O/incoming/ATG/ESB/RequestCatalog/feed/')
DCL VAR(&SCRIPT) TYPE(*CHAR) LEN(100) +
VALUE('/tmp/sftpscript.txt')
DCL VAR(&CMD) TYPE(*CHAR) LEN(500)
DCL VAR(&EOF) TYPE(*LGL) VALUE('0')
DCL VAR(&NULL) TYPE(*CHAR) VALUE(X'00')
DCLF FILE(LSOUTPUT)

ADDENVVAR ENVVAR(SFTP_USER) VALUE(&USERID) REPLACE(*YES)
ADDENVVAR ENVVAR(SFTP_HOST) VALUE(&HOST) REPLACE(*YES)
ADDENVVAR ENVVAR(SFTP_SCRIPT) VALUE(&SCRIPT) +
REPLACE(*YES)
ADDENVVAR ENVVAR(SFTP_DIR) VALUE(&DIR) REPLACE(*YES)
ADDENVVAR ENVVAR(QIBM_QSH_CMD_OUTPUT) +
VALUE('FILEAPPEND=/tmp/sftplog.txt') +
REPLACE(*YES)
ADDENVVAR ENVVAR(QIBM_QSH_CMD_ESCAPE_MSG) VALUE(Y) +
REPLACE(*YES)

DLTF FILE(QTEMP/LSOUTPUT)
MONMSG MSGID(CPF2105)

CRTPF FILE(QTEMP/LSOUTPUT) RCDLEN(1000)

CHGVAR VAR(&CMD) VALUE('echo "cd $SFTP_DIR\nls" > +
$SFTP_SCRIPT && sftp -b $SFTP_SCRIPT +
$SFTP_USER@$SFTP_HOST | grep -Ev +
"^[.]|^sftp> /qtcptmm/SFTPlogoutput.txt +
2>&1"')

SNDPGMMSG MSGID(CPF9897) MSGF(QCPFMSG) MSGDTA(&CMD) +
MSGTYPE(*DIAG)

OVRDBF FILE(STDOUT) TOFILE(QTEMP/LSOUTPUT)
/* CHGVAR VAR(&CMD) VALUE(&CMD *TCAT '> +
/qtcptmm/SFTPlogoutput.txt 2>&1') */
CHGVAR VAR(&CMD) VALUE(&CMD *TCAT &NULL)
CALL PGM(QP2SHELL) PARM('/QOpenSys/usr/bin/sh' +
'-c' &CMD)
DLTOVR FILE(STDOUT)
OVRDBF FILE(LSOUTPUT) TOFILE(QTEMP/LSOUTPUT)

DOUNTIL COND(&EOF *EQ '1')
RCVF
MONMSG MSGID(CPF0864) EXEC(CHGVAR VAR(&EOF) +
VALUE('1'))
IF COND(&EOF *EQ '0') THEN(DO)

/* FIXME: The following line is for debugging purposes only. +
Replace it with one that uses &LSOUTPUT to do some +
work. */

SNDPGMMSG MSGID(CPF9897) MSGF(QCPFMSG) +
MSGDTA(&LSOUTPUT) MSGTYPE(*DIAG)
ENDDO
ENDDO

DLTOVR FILE(LSOUTPUT)

ENDPGM

I have compiled this and run it, but the file QTEMP/LSOUTPUT is empty
I have also run sftp interactively using the following commands
CALL PGM(QP2TERM)
And pressed Enter

On the
/QOpenSys/usr/bin/-sh'
Screen, I typed
sftp ftpnonprodnbtysldc
and pressed Enter

I then typed
cd DNBT2O/incoming/ATG/ESB/RequestCatalog/feed/
and pressed Enter

I then typed
Ls
And pressed Enter
And what was displayed is what I expected to see

The questions are - what is wrong with my CLP program and how can I
log what happens within sftp


As always - any and all responses gratefully accepted

Alan Shore
E-mail : ASHORE@xxxxxxxx
Phone [O] : (631) 200-5019
Phone [C] : (631) 880-8640
'If you're going through hell, keep going.'
Winston Churchill


Disclaimer: This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. Any views or opinions presented are solely those of the author and do not necessarily represent those of the company.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a moment to review the archives at http://archive.midrange.com/midrange-l.