Re: Additional Journaling/Commitment Control Info From IBM

Joe,

On Freitag, 9. Juli 2004 00:31, Joe Pluta wrote:
> > From: Paul Holm
> >
> > Group....  FYI if it helps anyone.  Larry is considered a premier
>
> source
>
> > for HA, recovery, etc on the 400
>
> Paul, thanks for this.  One of the problems with experience is that it
> can go out of date.  It seems that my knowledge from seven or eight
> years ago is a bit stale.  I think Larry might be stretching the point a
> little on the "performance enhancement" issue - I'd like to see some
> real numbers on performance, 

just to give an impression to you, I have contacted a custumor of mine to get 
actual numbers:
for a data staging process of a Datawarehouse (ca. 0,6 Terrabyte) we have 
1000000 (in words one million) records per day to process; for each record we 
have a transaction under commit with > 50 database operations for each 
transaction (read records for versioning, update/insert aggregated data in 27 
aggregates, drawing keys etc.) the whole process (ILE RPG with embedded SQL) 
lasts about 60 minutes doing it in 15 to 20 parallel Jobs (parallelism 
requires Commit, one inconsistent transaction would be not recoverable)

breaking it down, we have 270 complex transactions/second on the load boundary 
of the system (multiprocessor with 7 cpus in the partition)

especially based on isolation levels - 

I've made some benchmarks for JDBC: the result was, that isolation level 
doesn't matter, as long as you don't use serializable. But this must be 
treated with care: not the isolation level, the contention of processes 
waiting of a record is crucial for performance!!! Minimizing contention was 
the most important step in optimizing the datawarehouse load process. (BTW: 
this problems were solved by changing order by clauses of processes, 
impossible with record level access, very easy with embedded SQL).

> but
> it's clear that the arguments FOR commitment control are at least
> gaining ground on the reasons against it, if indeed they haven't already
> surpassed them.
>
> At the same time, let's still be clear that it's a business decision.
> There are costs to be weighed against benefits.  As I pointed out, a
> system that is mostly static data (few though there are) would likely
> not benefit from commitment control.  

and the costs would be very low!!!

And as Marc points out, commitment
> control is not a slam dunk to implement - transaction boundaries get
> pretty important, and it's still a bit of an art.

no, its no art, its handcraft, but as well states of the art.

>
> It's not clear to me that you would actually combine every update done
> by an order (including inventory, A/R, sales, and so on) into a single
> commitment boundary, and if not, where you would draw the line, 

the line is the logical unit of work and nothing else; if its one record, you 
have a commit operation for each record. For the datawarehouse load process 
for instance,  one sales detail record is our unit and all database 
operations in the process for this record belong to one transaction.

> and if
> you did draw a line, what would happen in the case of a catastrophic
> failure and how that would be different from no commitment control at
> all.  

If you cancel the job with *immed, within a transaction, the system recognizes 
in the step of completion of the job, that an uncommited transaction was 
interrupted and does an automatic rollback, in other words: goes back to the 
last completed transaction.
If the endjob *immed ocurred by pulling off the power plug of the as400 (or 
another catastrophy) this will be done at next IPL.

But I'm willing to learn!  And since it's really important in the
> world of JDBC and especially EJB, I think it's important to this list.
>

Joe, sometimes I like to contradict, its important for rpg applications as 
well, why shouldn't they be (nearly <grin> ) as good as EJBs?

Dieter Bender  


> Joe
>
> --
> This is the Java Programming on and around the iSeries / AS400 (JAVA400-L)
> mailing list To post a message email: JAVA400-L@xxxxxxxxxxxx
> To subscribe, unsubscribe, or change list options,
> visit: http://lists.midrange.com/mailman/listinfo/java400-l
> or email: JAVA400-L-request@xxxxxxxxxxxx
> Before posting, please take a moment to review the archives
> at http://archive.midrange.com/java400-l.

-- 
mfG

Dieter Bender


DV-Beratung Dieter Bender
Wetzlarerstr. 25
35435 Wettenberg
Tel. +49 641 9805855
Fax +49 641 9805856
www.bender-dv.de
eMail dieter.bender@xxxxxxxxxxxx