× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



Why XML couldnt your data be in JSON? maybe is faster using JSON

On Thu, Feb 21, 2019, 8:18 PM Paul Therrien <paultherrien@xxxxxxxxxxxxxxxxxx
wrote:

Thanks Vern,
One thing that was slowing me down was I was working with an .xml file in
the IFS and passing that into XMLTABLE. - Very Slow.
I then copied the XML file to an SQL table to an XML column. And ran
XMLTABLE over that and it sped things up significantly.
It is still slow, but at least it finished processing the data in under a
half-hour.

Paul

-----Original Message-----
From: RPG400-L <rpg400-l-bounces@xxxxxxxxxxxxxxxxxx> On Behalf Of Vernon
Hamberg
Sent: Thursday, February 21, 2019 4:59 PM
To: RPG programming on the IBM i (AS/400 and iSeries)
<rpg400-l@xxxxxxxxxxxxxxxxxx>
Subject: Re: Large XML file processing via XMLTABLE

Hi Paul

I'll take a stab at this - I'm going to say that XMLTABLE is probably the
fastest - at least no slower, and, for me, pretty easy to set up - again,
for me, easier than XML-INTO.

Here's why -we receive from our field associates several XML files, one of
which is for the various components of the service they do.

It was being processed with XML-SAX, and at 7.1 we had trouble with
conversion of "interesting" characters into CCSID(37).

I learned about XMLTABLE, and we use that now - it's a dream, and it did
character substitution better than RPG did - and I have around half a dozen
evocations of XMLTABLE, despite which, the time to process things is no
worse than it was with XML-SAX - the latter would have a single pass
through
the XML, multiple uses of XMLTABLE would have a pass for each use - yet,
the
overall time was not longer with those multiple passes.

I know that Jon P. doesn't grok XMLTABLE, and I stumble over complex stuff
with XML-INTO, and all the code I have to use for XML-SAX, well, the SQL
solution is just cleaner all around for me.

So that's as much as I know and have seen here - HTH!

Vern

On 2/21/2019 1:37 PM, Therrien, Paul via RPG400-L wrote:
(Cross Posting from Midrange-L)

We are on IBM I V7R3.

The immediate question: Is XMLTABLE the best way to process a large XML
file?

We have XML files that contain over 40,000 individual data records with
file sizes over 7 megs and one even over 70 megs in one instance.
I am parsing the data from XML into an SQL table using the following
statement:

exec sql
insert into pjtlib/zmpsnxfl
select x.*
from pjtlib/xmltext,
XMLTABLE(
'$doc/IMB_Personalization/MailingData/Personalization'
passing info as "doc"
columns
mailingNumber varchar(5) default ' ' path
'../../MailingNumber'
, imbNumber varchar(21) default ' ' path
'../IMB_Number'
, psnValue varchar(100) default ' ' path 'PsnAlpha'
, psnCode varchar(3) default ' ' path 'PsnCde'
, MFentity varchar(3) default ' ' path 'MFentity'
, MFelement varchar(3) default ' ' path 'MFelement'
) X

This works great for a file with a few hundred transactions in it.
But when we have 40,000 transactions it takes a lot of time. (I have
a file that is 35meg in size, with over 50,000 records and it took 20
minutes to complete.)

Is there a way to speed this up using XMLTABLE() or is there a more
efficient way other than XMLTABLE?

Thanks,

Paul

+++++ This email and related attachments may contain confidential
information intended exclusively for the addressee. Unauthorized use,
disclosure or distribution of this material is prohibited. If you received
this message in error, please advise the sender and delete all copies of
it.
Content is provided by the individual sender and does not necessarily
reflect the views of the Company. Though sender believes this transmission
to be virus-free, it is the recipient's responsibility to ensure that it
is.

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list To post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives at
https://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link:
https://amazon.midrange.com

--
This is the RPG programming on the IBM i (AS/400 and iSeries) (RPG400-L)
mailing list
To post a message email: RPG400-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/rpg400-l
or email: RPG400-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/rpg400-l.

Please contact support@xxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com


As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.