Re: [EXTERNAL] Re: Rise of Node

Nathan,



It is correct that if you have to loop 10,000,000 times it takes the time
it does, however it is easy to activate the event loop while you loop so
you thereby split’s the loop into intervals of processing:

async function longRun(rid) {

for (let i = 0; i < 10000000; i++) {

cx = Date.now()

if(i % 1000000 == 0) {

console.log(`pid ${process.pid} loop-wait ${rid} -
${i}`)

await asyncEventTrigger(0)

}

}

}



function asyncEventTrigger(delay) {

return new Promise(resolve => {

setTimeout(() => {

resolve('event')

}, delay)

})

}



However (again) there is no need to do this if there is nobody else using
the node instance and thereby competes for resources.



An Await waits by transferring control to the event loop! How on earth
would node else continue to process other code waiting for resources. If
process A has transferred control to process B (that runs your 10,000,000
loop without interruptions) process A waits for the event that process B is
finished with the loop – if process A then fires an SQL command it will
expect to go into a I/O long wait but before it transfers the control to
the event handler so process B may continue. The above shown code will
cause that process A and B (if the run the same code above) will process it
in trunks of 1,000,000 and shift between them for each call to
asyncEventTrigger so the processing would be … A[0] > B[0] > A[1] > B[1] …
A[9] > B[9] … - this is cause by that setTimeout() is a async process and
that is why you can’t use setTimeout() even in normal javascript to pause
the processing – you just pause the processing within the setTimeout()
exactly like a async AJAX call.



Batch example

For each year in 30 years back get each 2 packages of documentation each
about 1.5 MB zipped.

1. Download a zip-file from UN’s homepage

2. Save it in a folder

3. Unzip it

4. Unzip 10 sub-folders in the main unzipped folder

5. Read through each documentation text document and create
metadata Javascript Objects for each document type that defines the whole
standard based on the content (reverse documentation)

by looping through the documentation byte for byte.

- Data-elements

- Composites

- Messages

- Segments

- Codelists 1 + 2

6. Stringify the javascript object into JSON and save the JSON into
files.



Wouldn’t you call that a batch process???

There are many others. E.g. you receive some XML files via FTP you have to
process – some may have a lot of data and may update several tables in you
DB or you have to transfer your daily invoices in trunks to each partner in
XML which requires reading of a queue, reading in tables, creating a XML
document pr. Invoice/partner saving the “package” in the IFS and sending
them with FTP etc.

Or you may want to create binary office documents that includes images or
a lot of other stuff so nodejs is used to many other things than just
serving web-pages and should imho be considered just like java – a VM that
runs javascript to whatever purpose (many that are covered by the 500,000
free npm’s.)

On Tue, Mar 20, 2018 at 1:24 AM, Nathan Andelin <nandelin@xxxxxxxxx>
wrote:

On Mon, Mar 19, 2018 at 10:11 AM, Kelly Cookson <KCookson@xxxxxxxxxxxx>
wrote:

Henrik,

Being a newbie, I am curious. Why would I want to use Node if I’m not
leveraging the event loop with asynchronous programming?

I'm not aware of any way to avoid the event loop in Node.js. It appears
that you can block the cycle from running by coding something like a tight
for-loop (i.e. for i = 1 to 100000000). That would be bad form, of course.
Something like that would block any other scripts from running in that
server. If you ran that in the master process that had forked a set of
child processes, then it would block any inter-process communication that
any applications may rely on.

Perhaps Henrik hadn't put much thought into his message. It wasn't clear
(to me at least) whether his reference to asynchronous programming was in
regard to the new "async" and "await" keywords which make functions appear
to wait for things like DB I/O to occur. They don't actually wait. It's
just a clever way to implement callbacks in Node.js without passing a
reference to callback functions. The actual callback is to the same
function code following the "await" keyword. The function appears to
resume
where it left off during an I/O operation, or the like. Several cycles of
the event loop could have actually occurred in between, while the I/O had
actually been passed off to another thread in the thread pool.

I understand in theory that I could have an instance of the event loop

running that processes a batch script instead of handing it off to

worker

threads. But that why would I want to do that? I’m genuinely curious

since

I’m still learning and this may be a failure of my imagination.

I'm curious too. Particularly if the "batch script" were performing any DB
I/O. Why would you do that in Node.js?
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.org <http://powerext.org/>