Re: Database to Excel

[Scott Klement <midrange-l@xxxxxxxxxxxxxxxx>]

Hi Sharon,

POI/HSSF has proven to struggle when spreadsheets get larger and larger. 
 (I don't know why that is.) In particular, adding more columns seems 
to make things disproportionately slower.

The problem isn't because POI is written in Java. There's an inefficient 
algorithm in it somewhere that's causing it to be slower.  (Though, it'd 
be perhaps twice as fast if it were pure RPG, since pure RPG executes 
faster...  but that's still not good.  Excel could write your document 
to disk in seconds, it should take 45 minutes *or* 23 minutes!)

One option you might consider, instead of generating standard XLS 
documents, is to use the new format that's based on XML. The newer 
versions of Excel use XML as their native format.  Outputting XML can be 
done in pure RPG and can be done efficiently. Of course, it requires you 
to use a recent version of Excel, and may be more complex to code 
(POI/HSSF does a lot of things for you that you might have to write 
extra code to output in HSSF) but...  it's something to look into, at least.


Wintermute, Sharon wrote:
> I have a question.   I recently used Scott Klement's Excel articles to
> create an Excel processor.   It uses a control file that describes what
> file, what Excel worksheet to use as a base, then adds the data to a new
> copy of the worksheet.   Once this is done, another job actually sends
> the file from the IFS to the user.  I had to separate the creation and
> the sending since SNDDST only uses QDLS and QDLS cannot be used in a job
> where Java is involved (multi-threading). 
>
>  
>
> The process itself works great.  It runs every 10 minutes looking for
> new requests to process.  Now we have discovered that some of these
> requests are huge files.  Over 75K rows to be created.  This one request
> ran for over 45 minutes on a usually lightning box.  
>
>  
>
> This lpar has over 8 gig memory and its on a M525-7792 so its usually a
> speed demon. 
>
>   
>
> I really need to decrease the time significantly.  Any ideas?  Is there
> another way to do this, perhaps without Java? 
>
>  
>
>  
>
> Sharon Wintermute
>
>