Re: Dealing with very large blocks of character data

It is so simple in plain RPGLE so why not stick to that ?

h actgrp('XXXXXXXX') dftactgrp(*no)
h bnddir('QC2LE')

d TS_Malloc pr * extproc('_C_TS_malloc')
d size 10u 0 value
d TS_Realloc pr * extproc('_C_TS_realloc')
d addr * value
d size 10u 0 value
d TS_Free pr extproc('_C_TS_free')
d addr * value

d heap s 10i 0 inz(100000)
d heapSize s 10i 0 inz(100000)

d i s 10i 0
d text s 65535a varying
d search s 65535a varying

d data s 65535a based(p)
d p s *

d addr s *
d size s 10i 0

/free
addr = TS_Malloc(heap);
text = 'my data my data';

// build a buffer from field values

for i = 1 to 10000;
if size + %len(text) > heapSize;
heapSize += heap;
p = addr;
addr = TS_Realloc(p:heapSize);
endif;
p = addr + size;
data = text;
size += %len(text);
endfor;

search = 'xxx';

// search the buffer
for i = 1 to size-%len(search);
p = addr + i - 1;
if search = %subst(data:1:%len(search));
dsply %char(i);
endif;
endfor;

TS_Free(addr);

*inlr = *on;
/end-free
On Thu, Dec 1, 2011 at 10:34 PM, CRPence <CRPbottle@xxxxxxxxx> wrote:

On 01-Dec-2011 12:28 , David Gibbs wrote:

On 12/1/2011 2:20 PM, CRPence wrote:

How about using some SQL procedures [and\or functions] to perform
the work using CLOB data type, since SQL has similar capability to
various RPG BIFs? For example a scan implemented as a procedure
might look like:

I'm still limited to 64k for the text variable ... since I would need
to pass it to the stored procedure.

I am not sure I understand that comment. What is the "text variable"
in that comment?

The example showed how to pass multiple "chunks" of 32K of data which
are then processed by the SQL as CLOB concatenated [32K] CHAR data; up
to the full length of all contiguous chunks combined by concatenation.
The CLOB could have been defined as 1M instead of 64K; I merely chose
64K and only two 32K chunks, because I was under the impression that 64K
in a newer release would have been satisfactory.? Thus the LOCATE in
place of %scan is operating on the full length of data [up to the length
of any number of combined 32K parameters, within the limit of a CLOB] in
the memory of the RPG program; the memory having been mapped as any
number of address-"based" 32K chunks. Note: In this case the 32K limit
[for the chunks] is due to the database, not the RPG, since the CHAR
declarations in the SQL stored procedure have that limit.

PLUS (and this is just a personal preference) I really dislike RPG
SQL ... I like the concept, but not the implementation (I dislike
code generators in general).

As negative as the original expressed distaste [and seconded with
comments about the need for null-terminating the data as strings] for
using the C approach? What about treating the SQLRPGLE as a separate
ILE language, like C would be, and limit any SQLRPGLE to those source
members built into modules of a service program rather than making the
main program SQLRPGLE?

Implementing just a scan procedure to replace some existing %scan
BIFs for a test version of the program could give an idea whether the
SQL approach is even feasible for performance requirements. The
performance of the program mapping chunks of its memory via based
max-length variables passed as parameters of the procedure should be
great, per passing by reference. However, that the SQL LOCATE scalar
then operates only after CASTing the data from those chunks into
concatenated CLOB data, the database will possibly have to copy the
data, so performance may not even meet expectations. If poorly
performing, the SQLRPGLE could be eliminated as an option. Or maybe the
ease of using the SQL as compared to C would outweigh some level of
performance impact.

Regards, Chuck
--
This is the RPG programming on the IBM i / System i (RPG400-L) mailing list
To post a message email: RPG400-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/rpg400-l.