RE: Large XML file processing via XMLTABLE

Thanks Marco.
I will look into this as well.

Paul

-----Original Message-----
From: RPG400-L <rpg400-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Marco
Facchinetti
Sent: Saturday, February 23, 2019 4:42 AM
To: RPG programming on the IBM i (AS/400 and iSeries)
<rpg400-l@xxxxxxxxxxxxxxxxxx>
Subject: Re: Large XML file processing via XMLTABLE

Hi Paul,

our customers process many Xml files (average size 20/40MB) without
problems. We use Scott Klement port of Expat and it's really fast. Few
minutes on a POWER 7+ machine.

HTH
--
Marco Facchinetti

Mr S.r.l.

Tel. 035 962885
Cel. 393 9620498

Skype: facchinettimarco


Il giorno gio 21 feb 2019 alle ore 20:37 Therrien, Paul via RPG400-L <
rpg400-l@xxxxxxxxxxxxxxxxxx> ha scritto:

(Cross Posting from Midrange-L)

We are on IBM I V7R3.

The immediate question: Is XMLTABLE the best way to process a large
XML file?

We have XML files that contain over 40,000 individual data records
with file sizes over 7 megs and one even over 70 megs in one instance.
I am parsing the data from XML into an SQL table using the following
statement:

exec sql
insert into pjtlib/zmpsnxfl
select x.*
from pjtlib/xmltext,
XMLTABLE( '$doc/IMB_Personalization/MailingData/Personalization'
passing info as "doc"
columns
mailingNumber varchar(5) default ' ' path
'../../MailingNumber'
, imbNumber varchar(21) default ' ' path '../IMB_Number'
, psnValue varchar(100) default ' ' path 'PsnAlpha'
, psnCode varchar(3) default ' ' path 'PsnCde'
, MFentity varchar(3) default ' ' path 'MFentity'
, MFelement varchar(3) default ' ' path 'MFelement'
) X

This works great for a file with a few hundred transactions in it.
But when we have 40,000 transactions it takes a lot of time. (I have
a file that is 35meg in size, with over 50,000 records and it took 20
minutes to
complete.)

Is there a way to speed this up using XMLTABLE() or is there a more
efficient way other than XMLTABLE?

Thanks,

Paul

+++++ This email and related attachments may contain confidential
information intended exclusively for the addressee. Unauthorized use,
disclosure or distribution of this material is prohibited. If you
received this message in error, please advise the sender and delete
all copies of it. Content is provided by the individual sender and
does not necessarily reflect the views of the Company. Though sender
believes this transmission to be virus-free, it is the recipient's

responsibility to ensure that it is.

--
This is the RPG programming on the IBM i (AS/400 and iSeries)
(RPG400-L) mailing list To post a message email:
RPG400-L@xxxxxxxxxxxxxxxxxx To subscribe, unsubscribe, or change list
options,
visit: https://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list To post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related questions.

Help support midrange.com by shopping at amazon.com with our affiliate link:
https://amazon.midrange.com