V5R4 native xml processing - a user's critique -- RPG400-L

Hi,

I decided to spend a bit of time going through the xml docs in theinfocenter over the weekend and I've got to say that it looks very well puttogether. Here are my thoughts - you are free to contradict me on any point.:-)



XML-SAX

The use of a single handler procedure is extremely useful. I did wonder howthat would work, but passing the event type as the second parameter prettymuch solves that one. The nice things about having a single point of contactbetween the parser and the program are (1) simplicity, and (2) extensibility? it will be much easier to add new events without changing thearchitecture.

The use of XML-SAX and a handler procedure would work fine with our currentcode base (which uses the xml toolkit). We can simply refractor our code andput the calls to our existing event-handling procedures within this newprocedure ? probably in a simple select statement based on event type. Therewould be some adjustments for handling attributes, but nothing too nasty.

Regarding well-formedness checking ? I can?t see any difference in what wedo now and what we?d need to do with this native parsing method. At themoment we always parse the xml fully without any processing occurring ? weonly register the warning/error/fatal error event handling procedures on thefirst pass. This is to allow us to validate the document in its entirety. Ifthis straight parse does not return any errors we register the processingevent handlers (start of element, process characters, end of element, etc?)and parse again. It is during this second parse that we decompose the xmldata into database variables. Of course, the handler procedures could bedifferent in each pass. Funnily, this native method allows a nice way ofusing the same handler procedure for both passes. The first parameter passedto XML-SAX is custom-built and can be anything. As a result, you could havetwo almost-identical lines of code for both passes, using the same handlerprocedure in both cases. To inform the handler procedure that it is invalidation-only mode you could pass a special value in the first parameter.Within this procedure you could then simply return if the event is not anerror event. As no SAX parser would assume well-formedness ? onlyprogrammers - I would always advocate a two-pass approach to xmldecomposition if you can not guarantee the quality of the document. XML-SAXprovides all I need for production-level code.


I?d probably do something like:

// validate the xml?

xml-sax %handler(myHandler : ?*VALIDATE?) %xml(?/home/xped/file.xml??doc=file ccsid=job?);


// If no exceptions, decompose the xml?
if not XML_EXCEPTION;

xml-sax %handler(myHandler : ?*DECOMPOSE?) %xml(?/home/xped/file.xml??doc=file ccsid=job?);

endif;

This is very neat and tidy - I like it.


XML-INTO

Now I had to have a good think about this one. As I?m used to having eventhandler procedures to parse complex xml I wasn?t sure how much use thiswould be. I know XML-INTO can use an event handler procedure but I can?timagine myself using it this way unless I wrote a service program for doingbulk inserts into database tables using arrays and sql. (very nice ideathough)


For me the obvious use is?

Many of our processes run asynchronously using NEPs sitting on data queues.The idea is to keep our workflow modular and to allow us to isolatefunctions within the workflow. This gives us the ability to scale-up orscale-down according to the business needs. These data queue entries are 512bytes long strings mapped over a data-structure. This data-structure is usedextensively by our programs ? and is used as an entry parameter to manyfunctions. The problem is that some functions need some data and otherfunctions need different data, accommodating the needs of all of thefunctions was difficult using a fixed-format DS within a data queue entry.We constantly have issues about space within this DS. If we need a newvariable we can add it to the DS and recompile the programs. This isundesirable but easily achieved. But if we exceed the 512-byte limit we willneed to change all of the data queues to use the longer length. This is areal pain in the butt. Now, this is where XML-INTO will be a dream for us.We could simply create the structure in xml, and only put the populatedelements into the xml doc. This would then be passed to the data queue. Thereceiving NEP could then use xml-into to get the data from the passed xmland place it into the data structure it uses. As we would never use morethan 20% of the DS fields in any one application, we would have a lot ofroom for manoeuvre with this technique.


I?d probably do something like:

xml-into localDS %xml(dqstring :?doc=string ccsid=job allowmissing=yespath=datastructurename case=lower?);

My only question regarding xml-into is this: How deep can we fill a DS withxml data? If we have a DS which has a sub-field which is LIKE another DS,could I expect a 3-level xml doc segment to decompose into the wholestructure - including the fields in the nested DS? Does the process handlethis kind of structure?



Regarding entity references: I assume that, given the following xml:

<?xml version=?1.0? encoding=?ISO-8859-1? ?>
<root>
<var1>That&apos;s life!</var1>
<var2>Bodgitt &amp; Scarper electricians</var2>
</root>

I would expect the parser to call the handler in the following combinationof calls:


*XML_START_DOCUMENT (no data pointer passed)
*XML_VERSION_INFO (data=?1.0?)
*XML_ENCODING_DECL (data=?ISO-8859-1?)
*XML_START_ELEMENT (data=?root?)
*XML_START_ELEMENT (data=?var1?)
*XML_CHARS (data=?That?)
*XML_PREDEF_REF (data=?&apos;?)
*XML_CHARS (data=?s life?)
*XML_END_ELEMENT (data=?var1?)
*XML_START_ELEMENT (data=?var2?)
*XML_CHARS (data=?Bodgitt ?)
*XML_PREDEF_REF (data=?&amp;?)
*XML_CHARS (data=? Scarper electricians?)
*XML_END_ELEMENT (data=?var2?)
*XML_END_ELEMENT (data=?root?)
*XML_END_DOCUMENT (no data pointer passed)

(Although I would imagine that it is possible that *XML_CHARS may be calledseveral times for the same element if the data is very large.)

So it would appear that we will have to resolve escaped entity referenceswithin our code. This is not a big thing - and we have been provided theability to handle custom-references using *XML_UNKNOWN_REF, which is cool.

Anyway, this looks like a pretty comprehensive implementation and I cancertainly use it in my production code. I would still need the xml toolkitfor parsing with xml schemas, but that?s not too bad ? I only use schemas indevelopment and can treat the toolkit as part of my IDE and not part of theproduction code. Also, the way this code handles attributes is excellent. Inever really liked the way standard SAX parsers present attribute data tothe call-back procedures at the start of an element ? even in java it ismessy. This way is MUCH nicer.

Excellent work Barbara ? I will certainly have a good play with this once weget the upgrade!


Cheers

Larry Ducie