<< Do I get this correct? You had a socket listener running (as PJ?), and
dynamically spawn workers?>>
No, that is not how socket servers work. The socket server is just sitting
and listening for connection requests. When it gets one it forks a new
process. That process receives the request and then spawns a new job to do
the processing. That job is a pre-start job so it can start
instantaneously. The operating system automatically starts additional jobs
as needed.

I have also done this in RPG which works a bit differently. It receives the
connection request, then spawns a new job to do the work and send the
results back.

If you are interested, we might want to talk.

On Tue, Oct 26, 2021 at 5:27 AM Patrik Schindler <poc@xxxxxxxxxx> wrote:

Hello Alain,

Am 25.10.2021 um 23:50 schrieb Alan Campin <alan0307d@xxxxxxxxx>:

I have done all of this in C for an open source project that I was
working
on. I had pre-start jobs running in my own subsystem and I would spawn a
job to perform programming work that came from a socket program.

Do I get this correct? You had a socket listener running (as PJ?), and
dynamically spawn workers?

It would perform the work which could be of any length but what I ended
up
doing was using a User Queue to return a result which worked fine.

I don't really understand what the queue is meant to do. A deposit for
communication between the socket listener and the peer application?

Received the request, looked up to see if it had Job running and created
one if it did not and then sent that request to a pre-start job and got
back the response through a user queue.

Ah, now I get it. This is different than I'm used to from Linux/Unix.
There you usually have one (rarely more) "main" listener process(es). If a
TCP session request comes in, the listener will copy itself (do a fork
call) and finish the TCP connection. This one will do the work until the
socket is closed or whatever condition should make the child process end.
Meanwhile, the main listener is free to handle additional requests, because
the socket accept() call is done in the worker.

This is working very fine for a long time, but despite all kernel-side
tricks like copy on write and whatnot, a fork call is expensive. And the
slower the machine, the more time it takes to fork a new worker process.
(Inetd is doing likewise and was mainly thought as lessening a system's
memory usage when a lot of tcp listeners were running independently.)

Another example: The Apache Web Server on Linux had the "prefork" process
model available for years, to alleviate the time and CPU load needed for a
fork, trading speed for higher memory requirements. At system boot time it
preforks a configurable amount of worker children, handing incoming
requests over, opening files in the file system as needed, reading them,
and pushing data over the tcp connection. Apparently the old IBM http
server does likewise. (I'm ignoring CGIs for the sake of simplicity here.)

What I have in mind is to mimic this behavior, but not with truly running
workers, instead "having PJ jobs available", and triggering them active by
whatever means is needed by the socket listener "handing over" a connection
request. Since I also want to do the accept() in the worker, the tcp socket
listener is unaffected and continues to dispatch incoming request to
workers.

Terms I've put into quotes are still something I need to learn about.
Especially considering that OS/400 has no fork() but spawn(), I'm not sure
if I can duplicate the behavior of a fork with spawn. I will re-read the
UNIX C Applications Porting to AS/400 (
https://ibmdocs.pocnet.net/SG24-4438-00.pdf and
https://ibmdocs.pocnet.net/SG24-4938-00.pdf) to learn more about that
topic.

This "accept in child process"-approach doesn't require queues, because
the socket communication and file system access stuff is done completely
within one task/thread/process. Still, some details remain with question
marks.

In parallel, I'll re-read Chris Hird's "Let's C" blog posts to understand
how he dealt with that handover to a worker process.

Anyway I would be willing to share some code if you are interested.

I'm interested! In addition, if you don't mind, what about publishing your
OpenSource code on Github, so everybody interested can see and learn?

One more point I also did the same thing for CGI jobs. I wrote a single
program that was called for all CGI requests. That program then
dispatched
to one of pre-start job or started one. The result was sent back and
returned to the caller.

I see. But exactly that complete spawning of a new process is what inetd
currently does and what takes 2-3 seconds. This delay is what I want to
eliminate. And I'm curious how to exactly do this "dispatching to a PJ".

I should mention that when I said, "I have heard that data queues are
very

slow" I was referring to communicating from the outside world through
ACS.

Not relevant to me. :-)

Data queue inside the IBM i are fast although User Queues a lot quicker.

Probably also not relevant.

:wq! PoC

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://archive.midrange.com/midrange-l.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription related
questions.

Help support midrange.com by shopping at amazon.com with our affiliate
link: https://amazon.midrange.com


As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2022 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.