RE: RPGLE Converted into SQL having performance issue -- RPG400-L

Thanks so much all Charles/Frank/Vern and others, this code was given to me recently. I had worked with SQL and have always used sets or if at all needed multiple fetch and knew something was wrong there.

Yesterday was sitting with the client and saw SQL hogging the whole system, processing about 3000 records in 45 minutes :(

So this logical was picked up (I know it was Devs bad decision) to use LF, one record at a time fetch and this file has trigger as well as it contains PCI data.

I ran Visual Explain and it never used SQE rather used CQE because of existing triggers.

I can bring a block of data and remove cursor as that is really not required, but this trigger has raised another question should I use SQL at all ?

If I bring block of data how many rows are ideal to fetch in one shot!

As always appreciate you guys very much!

Thanks,
Ashwani
-----Original Message-----
From: RPG400-L <rpg400-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Justin Barrett
Sent: Tuesday, December 15, 2020 7:10 PM
To: rpg400-l@xxxxxxxxxxxxxxxxxx
Subject: Re: RPGLE Converted into SQL having performance issue

⚠ EXTERNAL MESSAGE – Think Before You Click

Cannot agree more with this. Cursors are expensive, row at a time cursors doubly so, scroll cursors row at a time and just forget about your program having any kind of performance.

If you absolutely must work row at a time, stick to native I/O.

If you do use cursors then always pull batches of records, never record at a time.

Only use scroll cursors if you must randomly move around the data backwards and forwards.

And finally, if you are pulling data from a table, try not to use * and instead list the columns that you actually need.

Without know more about what's going on, the best bet is to just stick to file I/O unless there's some way to do your operations in bulk.

I don't know what you've got going on but if you're using that :YY0008 as a predicate in your first query a lot, you may want to look at where the value going into that parameter is coming from and if it's already in a table in of itself, you can always do a inner join on that source with this FILEXXX with that YY0008 as the join key. But it would really depend on where that's coming from.

Cheers

On 12/15/20 09:20, Charles Wilt wrote:

Not really surprising, you've replaced one access method with another
that has more overhead.

You can't simply replace RPG 1 record-at-time with SQL 1 record-at-time.
Better off to stick with RPG.

Used properly, SQL is faster than RPG. But you have to use it
properly; which means working with SETS of records.

- unless there's no other option, don't use cursors. (Display &
printing they are required)
- if using a cursor, read in blocks of records (100+)

You might find this presentation I gave a few years ago helpful..
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.linkedin.com_
posts_charles-2Dwilt-2D2a4a303-5Fsql-2Dfor-2Dthe-2Drpg-2Ddeveloper-2Da
ctivity-2D6641756015354355712-2DwoZR&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw
&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8E
M3vnNvaifNTqw20-u-nPw4&s=DIOflfIQy7VAl7N6qAxoQ_HDmfjXlCsfEKpBdjO9h4I&e
=

Charles

On Tue, Dec 15, 2020 at 1:33 AM Singh, Ashwani (Lake Mary) via
RPG400-L < rpg400-l@xxxxxxxxxxxxxxxxxx> wrote:

Hi Guys,

We converted our one of the main module for transaction processing
from RPGLE to SQLRPGLE and we are getting a lot latency and low
performance issue.

We are thinking of going back to native IO, please let us know what's
wrong with below and is there anything we can do to improve the performance.

Here are the SQL that have been added , this one process about 40-50K
records daily.

C/EXEC SQL DECLARE DAT_FILEXX CURSOR FOR

C+ SELECT *

C+ FROM FILEXXX WHERE XX0055 = :YY0008

C+ ORDER BY XX0097

C+ FOR READ ONLY

C/END-EXEC

C/EXEC SQL OPEN DAT_FILEXX

C/END-EXEC

This one process about 100 - 500 records

C/EXEC SQL DECLARE DAT_FILEYY DYNAMIC SCROLL CURSOR FOR

C+ SELECT *

C+ FROM FILEYY

C+ ORDER BY HCNRPRTI

C+ FOR READ ONLY

C/END-EXEC

C/EXEC SQL OPEN DAT_FILEYY

C/END-EXEC

PRTSQLINF

CRTSQLRPGI

OBJ(QTEMP/XXF0170)

SRCFILE(LIBX/USSORC)

SRCMBR(XX0170)

COMMIT(*CHG)

OPTION(*SYS *NOEXTIND *SYSTIME *PERIOD)

TGTRLS(V7R2M0)

ALWCPYDTA(*OPTIMIZE)

CLOSQLCSR(*ENDACTGRP)

RDB(*LOCAL)

DATFMT(*MDY)

DATSEP('/')

TIMFMT(*HMS)

TIMSEP(':')

DFTRDBCOL(*NONE)

DYNDFTCOL(*NO)

SQLPKG(LIBYY/XX0170)

MONITOR(*USER)

SQLCURRULE(*DB2)

ALWBLK(*ALLREAD)

DLYPRP(*NO)

DYNUSRPRF(*USER)

USRPRF(*OWNER)

SRTSEQ(*HEX)

LANGID(ENU)

RDBCNNMTH(*DUW)

TEXT('Authorization

SQLPATH(*LIBL)

DECRESULT(31 31 0)

DECFLTRND(*HALFEVEN)

CONACC(*DFT)

STATEMENT TEXT CCSID(37)

DECLARE DAT_FILEXX CURSOR FOR SELECT * FROM FILEXX WHERE XX0055 = : H
ORDER BY

XX0097 FOR READ ONLY

SQL4021 Access plan last saved on 12/12/20 at 08:01:32.

SQL4020 Estimated query run time is 0 seconds.

SQL4017 Host variables implemented as reusable ODP.

SQL4002 Reusable ODP sort used.

SQL4007 Query implementation for join position 1 table 1.

SQL4008 Index FILEXXLE used for table 1.
--> This is an Index

SQL4026 Index only access used on table number 1.

DECLARE DAT_FILEYY DYNAMIC SCROLL CURSOR FOR SELECT * FROM FILEYY
ORDER BY

HCNRPRTI FOR READ ONLY

SQL4021 Access plan last saved on 12/12/20 at 08:01:32.

SQL4020 Estimated query run time is 0 seconds.

SQL4009 Index created for table 1.

SQL4026 Index only access used on table number 1.

SQL4008 Index MTI(16) used for table 1.

Thanks,

Ashwani
--
This is the RPG programming on IBM i (RPG400-L) mailing list To post
a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.midrange.c
om_mailman_listinfo_rpg400-2Dl&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=Ff
wKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vn
NvaifNTqw20-u-nPw4&s=63VfN2SkcL4YookUFxuyucrshpX4MH1c4gL5Z14i5_o&e=
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.midrange.com_rpg400-2Dl&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vnNvaifNTqw20-u-nPw4&s=3Daw3FyEwGDpTxC3zUFUM7Edr2b5FTOnWmGH-oMLONw&e= .

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription
related questions.

Help support midrange.com by shopping at amazon.com with our
affiliate
link:
https://urldefense.proofpoint.com/v2/url?u=https-3A__amazon.midrange.
com&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS
-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vnNvaifNTqw20-u-nPw4&s=VpEeXj
KcRrLYIKENJGsb4jozPFginIibVxMZhf3AO6U&e=

--
This is the RPG programming on IBM i (RPG400-L) mailing list To post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe, unsubscribe, or change list options,
visit: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.midrange.com_mailman_listinfo_rpg400-2Dl&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vnNvaifNTqw20-u-nPw4&s=63VfN2SkcL4YookUFxuyucrshpX4MH1c4gL5Z14i5_o&e=
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.midrange.com_rpg400-2Dl&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vnNvaifNTqw20-u-nPw4&s=3Daw3FyEwGDpTxC3zUFUM7Edr2b5FTOnWmGH-oMLONw&e= .

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link: https://urldefense.proofpoint.com/v2/url?u=https-3A__amazon.midrange.com&d=DwIGaQ&c=rE3mhBYFJfJGqQ7WI0-DPw&r=FfwKOXXYX1q9F5u-iJzZSViRPIoMS-xj5OEEpmrlGdQ&m=Q4TNJi0OHJ_YUBqh3ST8EM3vnNvaifNTqw20-u-nPw4&s=VpEeXjKcRrLYIKENJGsb4jozPFginIibVxMZhf3AO6U&e=