Thanks Scott this is top draw information, and your writing style is
excellent (you'd make a killing documenting systems!!)
I thought the post was about programs because of this line:
" Given some of the observations previously noted about normal ILE
calls--specifically that if PGM-A calls PGM-B passing 3 parameters and then
PGM-A calls PGM-B passing 2 parameters...."
That said, I have a much better understanding of how parms work now,
although I will probably have to read the email a good few times before it
all sinks in.
It does seem to me that this level of complexity is something that a managed
language shields you from.
However if I have followed this correctly, your example of the 3 parm
procedure only being passed 2 parms in an SQL UDF call the problem is that
the descriptor is not set?
I am guess there is a valid reason the descriptor is not set? Or has someone
been a bit lazy :-)
-----Original Message-----
From: midrange-l-bounces@xxxxxxxxxxxx
[mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of Scott Klement
Sent: 03 December 2009 00:00
To: Midrange Systems Technical Discussion
Subject: Re: AW: Overloaded UDF
Hi Neill,
I wasn't aware that programs could have optional parameters, I thought
that
this was only procedures?
Yes. This discussion is about procedures.
If you want to discuss programs instead of procedures, the major
difference is that %PARMS will be correct for a program call. %PARMS
always works on program calls, it's just on procedure calls where it
might not.
Also I always thought (and this may have been tainted by my c# programming
days...) that references to parameters were placed on the stack by the
caller and scooped up off the stack by the callee. If it wasn't on the
stack
then the callee would assume it was null?
Yes, I agree with what you're saying here except the last part. It
won't "assume" it's null.
Under the covers, an ILE procedure calls the _NPMPARMLISTADDR MI
function to get a pointer to the parameter area. I don't know whether
that parameter area is on a stack or not (but that doesn't really matter.)
The pointer points to a data structure defined like this (expressed in
RPG, despite that the OS isn't actually written in RPG):
D Npm_ParmList_t ds qualified
D based(the-pointer)
D desclist *
D workarea 16A
... plus the parameter values here ...
The first 16 bytes, as you can see, are a pointer that points to an
operational descriptor. If a descriptor was passed, this will correctly
point to a structure that contains the number of parameters passed to
the procedure, as well as any opdesc's for the parameters that were passed.
The trouble is, if there's no opdesc passed (not even a minimal one)
'desclist' is uninitialized memory. It will usually be *NULL,
indicating that the pointer isn't valid. But this is not guaranteed.
(Or, at least, I seem to remember someone telling me that.) If it does
happen to be *NULL, the %PARMS BIF will return -1. But if it happens to
be a valid pointer, the value of %PARMS is unpredictable!
If you're interested... if at least a minimal descriptor is passed, the
'desclist' pointer points to a structure that looks like this:
D Npm_DescList_t ds qualified
D based(desclist_ptr)
D argc 10i 0
D 28A
D desc * dim(varies)
where 'argc' is the parameter count. This is where %PARMS gets it's
value. The 'desc' array is an array of pointers to operational
descriptors, one for each descriptor passed. Everything about the
'desc' array is optional. the length of the array is variable, the
number of descriptors actually filled in is variable... keep in mind
that the parameter count (argc) is not necessarily the same as the
number of descriptors passed, because in some languages (ILE C is the
only one I know of, actually) you can control which parameters get
descriptors on a per-parameter basis.
If there is a descriptor passed, the pointer for that descriptor points
to a structure that looks like this:
D Npm_Desc_t ds qualified
D based(desc_ptr)
D type 3i 0
D datatype 3i 0
D inf1 3i 0
D inf2 3i 0
D len 10i 0
Info about these fields can be learned by reading about the CEEDOD (Get
Operational Descriptor) API.
But anyway, I'm going off on a tangent talking about descriptors...
back to the main portion of the parameter list... the parameter list
itself is completely variable and immediately follows the 16-byte work
area from the first structure I showed you.
Now, when parameters are passed by VALUE, their contents is placed
directly into the parameter list. But when they are passed by
reference, a pointer is placed in the parameter list. Since this thread
is discussing SQL UDFs, and since UDFs *only* support parameters passed
by reference, we can think of that part of the structure as an array of
pointers, I guess.
So the first structure I showed you looks like this:
D Npm_ParmList_t ds qualified
D based(the-pointer)
D desclist *
D workarea 16A
D parm * dim(varies)
Or perhaps it's clearer to lay it out like this:
D Npm_ParmList_t ds qualified
D based(the-pointer)
D desclist *
D workarea 16A
D parm1addr *
D parm2addr *
D parm3addr *
In this case, I'm showing a procedure with 3 parameters by reference.
This layout is custom built according to the data on the SQL Create
Function statement. (Or when called from ILE, it's built based on the
prototype... or CALLB or CALLPRC or whatever)
Which parameters are included or not included in the list is purely
based on that. The number of parameters listed is totally variable.
When SQL makes the call, it uses the info on the Create Function to
build this structure. When RPG reads the data, however, it uses the
info on it's PI (procedure interface) to read this data.
If your RPG PI doesn't match the SQL Create Function, THEN WHAT HAPPENS?
That's the $64k question.
Well, if the system put out 64 bytes of data in this structure (i.e.
enough for 2 parameters) and you try to read data starting at position
65 (the supposed location of the pointer for the 3rd parameter) what
will happen? You'll be reading whatever data HAPPENS to come next in
memory after this structure.
As Mark Lazarus pointed out, in ILE RPG if you do consecutive calls, it
reuses the same parameter area (which supports the idea that it's a
stack that's reused for each parameter call). In that case, there
might be a left-over value from the previous call (as Mark explained)
and then it's VERY likely that it'd be a valid pointer already.
In other cases, it's just random undefined memory. It might be all
x'00' as Birgitta implies, or it might not. You might get lucky, or you
might not.
Birgitta implies that SQL will deliberately set this memory to x'00' so
you don't have this problem. That's what I'm skeptical of. How could
SQL know that you plan to use the 3rd parameter, if your create function
told it that you only have 2 parameters??
I can't imagine that it reads your program's source code to determine
this. There's no way from outside of the procedure to know what
parameters it can potentially take.
(Yes, I know about the ability to generate PCML and store it in a
module, but SQL isn't using that functionality because that
functionality is MUCH newer than UDF capability, and it's not always
there, so that support is clearly not used.)
Since SQL can't know about this extra parameter, it can't know that it
has to set the pointer to zeros... I can't see how Birgitta could be
right. The only way it would make sense is if SQL always sets a
predefined number of pointers (maybe 399?) to *NULL... but that doesn't
seem likely to me, as that 24k of x'00' would be a fairly expensive
operation, and it's inconsistent with the way this parameter info is
filled in everywhere else.
Not to mention that, as Simon said, the developers probably didn't even
think of it this way. They aren't RPG developers, after all. They
think of overloading in the same terms that a C++ or Java developer
would think of overloading. The number of parameters on the SQL
function would have a 1-to-1 relationship with the number of parameters
in the function. They wouldn't think of them as "optional parameters",
because that's not the way overloading works in ANY language that I'm
aware of.
Anyway, you seemed interested in more technical info about how the
parameters work -- I hope I've provided that. I reserve the right to
republish this info somewhere else, as it took quite a long time to
write this message :)
midrange.com
