Slow Response on "grep" when called within RPG thru QSH Command

Hello,


I have an IFS folder with thousands of XML files.

And I am developing an application coded in RPG, that will list on a
subfile, each IFS filename that contains a specific search string.


My program is working as expected, but the "grep" took about 4 to 5 minutes
to return successfully with the search results.


Here are my steps, please let me know what I am doing Wrong OR what I can
do to increase the performance.

One thing I can think of is that there are 745 files in the folder that I
am searching.

The size of each file ranges from 18 KB up to 1 MB. Is that too many files
& sizes huge ?

So far I am only testing in a Test environment.

And the production system could have up to a half a million IFS files.

1. Change to Directory "/i3pl/XX/arc"
Cmd_Str = 'CD DIR(' + Quote + Source_Dir + Quote + ')';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

2. Create the QTEMP file that will have the file listing from from "grep"
Cmd_Str = 'CRTDUPOBJ OBJ(I3DIR_LIST) FROMLIB(*LIBL) OBJTYPE(*FILE)
TOLIB(QTEMP)';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

3 Direct the output of grep command from STDOUT to the QTEMP file
Cmd_Str = 'OVRDBF FILE(STDOUT) TOFILE(QTEMP/I3DIR_LIST) ' +
'OVRSCOPE(*JOB) SHARE(*YES) ' +
'OPNSCOPE(*JOB) SECURE(*YES) INHWRT(*NO)';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

4 Run the Shell command to search
* QSH CMD('grep -Rli "00353970985316338146" *.xml')
Cmd_Str = 'grep -Rli ' + D_Quote + %Trim(The_Search) + D_Quote
+ ' *.xml';

Cmd_Str = 'QSH CMD(' + Quote + %Trim(Cmd_Str) + Quote + ')';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

----
Regards,
Mohan Eashver
* Change to Directory "/i3pl/XX/arc"
Cmd_Str = 'CD DIR(' + Quote + Source_Dir + Quote + ')';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

* Create the QTEMP file that will have the file listing from from "grep"
Cmd_Str = 'CRTDUPOBJ OBJ(I3DIR_LIST) FROMLIB(*LIBL) ' +
'OBJTYPE(*FILE) TOLIB(QTEMP)';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

* Direct the output of grep command from STDOUT to the QTEMP file
Cmd_Str = 'OVRDBF FILE(STDOUT) TOFILE(QTEMP/I3DIR_LIST) ' +
'OVRSCOPE(*JOB) SHARE(*YES) ' +
'OPNSCOPE(*JOB) SECURE(*YES) INHWRT(*NO)';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));

* Run the Shell command to search
* QSH CMD('grep -Rli "00353970985316338146" *.xml')
Cmd_Str = 'grep -Rli ' + D_Quote + %Trim(The_Search) + D_Quote + ' *.xml';

Cmd_Str = 'QSH CMD(' + Quote + %Trim(Cmd_Str) + Quote + ')';
Exec_OS_Command (Cmd_Str : %size(Cmd_Str));