RE: Database to Excel

Scott,

I have not had the opportunity to use the XML side yet, but I do have a
project that will use it for import. Seems I may have to use it for
export as well. The coding will be extremely more complex since I am
using a control file to say what files and how many fields get outputted
as well as an Excel baseline from which to start from. It is going to
be one "hairy" program to write. Since it is on my plate within the
next two months, I will let you know how it goes.


Sharon


-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx
[mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: Tuesday, November 18, 2008 4:42 PM
To: Midrange Systems Technical Discussion
Subject: Re: Database to Excel

Hi Sharon,

POI/HSSF has proven to struggle when spreadsheets get larger and larger.

(I don't know why that is.) In particular, adding more columns seems
to make things disproportionately slower.

The problem isn't because POI is written in Java. There's an inefficient

algorithm in it somewhere that's causing it to be slower. (Though, it'd

be perhaps twice as fast if it were pure RPG, since pure RPG executes
faster... but that's still not good. Excel could write your document
to disk in seconds, it should take 45 minutes *or* 23 minutes!)

One option you might consider, instead of generating standard XLS
documents, is to use the new format that's based on XML. The newer
versions of Excel use XML as their native format. Outputting XML can be

done in pure RPG and can be done efficiently. Of course, it requires you

to use a recent version of Excel, and may be more complex to code
(POI/HSSF does a lot of things for you that you might have to write
extra code to output in HSSF) but... it's something to look into, at
least.


Wintermute, Sharon wrote:

I have a question. I recently used Scott Klement's Excel articles to
create an Excel processor. It uses a control file that describes

what

file, what Excel worksheet to use as a base, then adds the data to a

new

copy of the worksheet. Once this is done, another job actually sends
the file from the IFS to the user. I had to separate the creation and
the sending since SNDDST only uses QDLS and QDLS cannot be used in a

job

where Java is involved (multi-threading).



The process itself works great. It runs every 10 minutes looking for
new requests to process. Now we have discovered that some of these
requests are huge files. Over 75K rows to be created. This one

request

ran for over 45 minutes on a usually lightning box.



This lpar has over 8 gig memory and its on a M525-7792 so its usually

a

speed demon.



I really need to decrease the time significantly. Any ideas? Is

there

another way to do this, perhaps without Java?





Sharon Wintermute