Re: iSeries buffer overflow immunity?

[James Rich <james@xxxxxxxxxxx>]

On Wed, 3 Nov 2004, Rich Duzenbury wrote:

> Is the iseries invulnerable to buffer overflows?  I was doing a bit of
> googling, and it seems there is a general impression from individual
> users that things are OK, but I don't see such information from large
> companies or vendors.

So far the responses to this thread have talked about overwriting the 
storage space used for variables.  Generally, these types of overflows are 
harmless (in terms of execution).  As I understand things, the problems 
exist when you have pointers to functions (as opposed to pointers to text, 
for example).

Scott's post about pointers on OS/400 is very interesting because it 
suggests that pointers cannot be created by anything other than the OS. 
For example, it is perfectly valid code to do something like this:

mypointer = 123;

However, according to Scott this pointer can't actually be used.

One of the sources of problems on Intel architecture hardware is that the 
stack (where variables are kept) is executable.  OpenBSD goes to great 
lengths to code around this shortcoming.  Fortunately, AMD's new 64-bit 
CPUs implement something the no-execute bit (NX).  This bit makes the 
stack non-executable.  Thus even if a buffer overflow occurs, the CPU will 
not run the overrun code since it isn't executable.

So all safe, right?  Not quite.  Unfortunately it isn't all that easy. 
Returning to pointers above, it is sometimes necessary to have an 
executable stack and a pointer to it.  With a pointer to the executable 
stack thoeretically we can do malicious things with it.  The compiler has 
to be smart about things.  There is a fascinating article on this very 
subject at:

http://www.freelinuxcdrom.com/LDP/LDP/LGNET/current/pramode.html

This has several code examples of how to do buffer overflows and when the 
code really does need to execute the stack.

Now if what Scott says is true (and there is no reason to believe it 
isn't), then on OS/400 things are even better.  Even in cases where an 
executable stack is required (like in the case of a function pointer) the 
pointer to that memory can't be changed and still be valid.  IOW, if an 
instruction says, "jump to memory location pointed to by X and begin 
execution" that instruction will fail if X is not a pointer generated by 
the OS.  While a program certainly could change that pointer, doing so 
would invalidate it.

However, if a pointer can be set to any value and be valid, then I think 
we would be in trouble.  Using the above article to create some test cases 
would certainly be interesting.  Imagine if it was known that the 
bootstrap code resided at address 123 on all VxRy systems.  If you could 
create a valid pointer that pointed to this address and then called that 
pointer you could potentially load any program you wanted, possibly 
bypassing all security checks (I have no idea - just an example). 
Fortunately, it appears that this is impossible.

btw - that last example would be the equivalent of being able to modify a 
kernel pointer from userspace in linux.  May be possible - but really, 
really hard.

James Rich

It's not the software that's free; it's you.
        - billyskank on Groklaw