Re: Job hangs after processing 220 records.

On 01-Jul-2014 11:45 -0500, Marie O'Rourke wrote:

I'm wondering if anyone could help.

If I submit the following job using the Job Description the job
hangs after processing 220 records.
SBMJOB CMD(CALL PGM(HQDOBJ/UMR_UPDATE)) JOB(PCODUMRUPQ)
JOBD(QGPL/CQAJOBD) INLLIBL(*JOBD)

From what was concluded "hangs"? Apparently more records [than the 220 processed] require(d) processing and\or some amount of time was given to allow the job to complete, but within that time-frame, the job never completed.? What is the status of the job [as appears on Work With Active Jobs (WRKACTJOB)? Minimally, what is the program stack? Ideally the entire Work With Job (WRKJOB) spooled output would be given to clarify a /hang/, and then that would be collected more than once, typically with a separation in time that shows or is verified to have shown no changes.

However if I manually set the library list and submit the job then
the job completes without hanging.
SBMJOB CMD(CALL PGM(HQDOBJ/UMR_UPDATE)) JOB(PCODUMRUPQ)

Presumably then, this request has INLLIBL(*CURRENT) JOBD(*USRPRF).

This looks like a problem with the Job Description , but I don't
have the slightest idea what to look for. The library list on the
Job description is correct, but I don't understand all the other
details.

What differs from the *JOBD of the User Profile and the Job Description specified in the former request are not detailed, with respect to each of the parameter specifications of the Submit Job (SBMJOB) request that end up being defaulted to the special value *JOBD. Even so, probably only the library list is consequential, and that would be impacted most by unqualified references; for the SQL, unqualified references using system-naming rules [NAMING(*SYS)].

This SQLRPGLE program does the following:
* SQL Deletes from a table
* SQL Deletes from a second table
* Reads a file and for every entry on that file
- SQL Inserts to a file
- SQL Selects details to work variables
- SQL Inserts into a second file
using details from file Read and work variables

Are the SQL statements using qualified references? The Print SQL Information (PRTSQLINF) could be at least somewhat informative.

Some of the errors visible on the Job log (This happens regardless
of the way the job is submitted):

All three errors occur in both the /hangs/ request and the successful requests? Or just the MCH3603 errors happen in both?

MCH3603 Escape 40 30/06/14 12:31:55.669663
From Program . : #cfochkr 000C2C
To Program . . : QSQSBAS QSYS *STMT
To module . . . : QSQHDCLS
To procedure . : SQHRDCLS
Statement . . . : 4007
Message . . . . : Pointer addresses object type not valid for this
operation.
Cause . . . . . : Pointer is X'000000000000000004817F77B1000200'.

msgMCH3603 F/#cfochkr x/0C2C
T/QSQSBAS TM/QSQHDCLS TP/SQHRDCLS stmt/4007
object type 0x0200 *PGM is referenced, but whatever was the Object Checker invocation required a different object type... Ouch! Very bad!

Unfortunately that is not manifest as a Function Check (*FC), so the specific error is not as easily caught. There may be a VLog produced implicitly however. If no VLog(s) for around that time, I believe the Start Watch (STRWCH) feature allows the MCH3603 message identifier to be watched and to have a [job dump] VLog forced.

MCH3603 Escape 40 30/06/14 12:31:55.676036
From Program . : QSQSBAS QSYS *STMT
To Program . . : QSQRUN3 QSYS *STMT
From module . . : QSQHDCLS
From procedure : SQHRDCLS
Statement . . . : 4546
To module . . . : QSQOPEN
To procedure . : CKLNGNAM
Statement . . . : 35571
Message . . . . : Pointer addresses object type not valid for this
operation.
Cause . . . . . : Pointer is X'000000000000000004817F77B1000200'.

msgMCH3603 F/QSQSBAS FM/QSQHDCLS FP/SQHRDCLS stmt/4546
T/QSQRUN3 TM/QSQOPEN TP/CKLNGNAM stmt/35571
object type 0x0200 *PGM
QSQOPEN is performing Check Long Name in Hard Close processing

SQL0901 Diagnostic 50 30/06/14 12:32:09.839561
From Program . : QSQRUN3 QSYS *STMT
To Program . . : QSQRUN3 QSYS *STMT
From module . . : QSQINS
From procedure : CLEANUP
Statement . . . : 31437
To module . . . : QSQINS
To procedure . : CLEANUP
Statement . . . : 31437
Message . . . . : SQL system error.
Cause . . . . . : An SQL system error has occurred. The current SQL
statement cannot be completed successfully. The error will not
prevent other SQL statements from being processed. Previous messages
may indicate that there is a problem with the SQL statement and SQL
did not correctly diagnose the error. The previous message identifier
was MCH3603. Internal error type 3109 has occurred. <<SNIP>>

msgSQL0901 sqlcode -901 et3109 rc3109 rcMCH3603
F/QSQRUN3 FM/QSQINS FP/CLEANUP stmt/31437
T/QSQRUN3 TM/QSQINS TP/CLEANUP stmt/31437
SQL INSERT effective function check failure occurs 14 seconds after the pointer type error that transpired in an SQL Open to implement that insert activity, that had initiated hard close processing

The -901 is indicative of a probable defect; so too, the errors that lead to the SQL generic function check error. That issue should be reported to your service provider. FWiW there are two HIPer APARs with the symptom msgMCH3603, that may or may not be related:
MA43140 <http://www.ibm.com/support/docview.wss?uid=nas24e3a53f12c951f0486257c0e003c6f10>
SE52953 <http://www.ibm.com/support/docview.wss?uid=nas2daba7b8258636d0c86257a5a003c8865>

With some additional information, the "hangs" could be better diagnosed [outside of assistance from a service provider; i.e. here on the list].