RE: Laymans explaination for single level store?

[CWilt@xxxxxxxxxxxx]

> -----Original Message-----
> From: Walden H. Leverich [mailto:WaldenL@xxxxxxxxxxxxxxx]
> Sent: Wednesday, November 24, 2004 11:13 AM
> To: Midrange Systems Technical Discussion
> Subject: RE: Laymans explaination for single level store?
> 

Waldon,

First off, David asked for a layman's explanation.  If you want to discuss
technical details, perhaps we should start a new thread.


> And how would you handle close w/OUT save? You can simply map 
> your word
> processor document into memory on Windows today, any update to a
> "memory" location IS an update in the file on the disk, 
> effectively it's
> only in one spot -- an no, it will NOT be in the swap file. (See
> CreateFileMapping API)


Yep, memory mapped files work around the some of limitations of the
two-level storage allowing you to share across address spaces.

The difference is with single-level store, _every_ object/file is opened in
this manner.  You don't have to write the application any different to share
the objects.

Do you not agree that there is a big difference between implicitly sharing
every object and explicitly sharing a single ( or a set ) of files?

Also, a memory mapped file most certainly will be in the swap file, since
the swap file is simply where virtual-memory pages are kept when not needed
in main memory.  As the mapping is laid out in the virtual memory of the
process, it is highly likely that the pages making up the mapping will be in
the swap file at some time.  Here's a quote for you:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dngenlib/ht
ml/msdn_virtmm.asp

"All subsequent accesses to that record deal directly with the page(s) of
memory in RAM. No disk I/O is required or enforced until the file is later
closed or flushed."

So you are not writing directly to disk.  You are writing to an image of the
file in the virtual memory for the process.  The reason memory mapped files
work is that the pages that make up the mapping are appear in ( shared )
virtual memory of multiple processes.

Since you're not writing to disk directly, the only way to share a file is
to specifically write an application that way.  The OS can't share the
object without the help of every application. that uses it.

As I quoted in my first post: "Single-level store is not about a large
address space; it's about sharing."


> 
> However, if you did that, how would you close your word processing
> document w/OUT saving changes? The changes have already made their way
> to disk, IN the original document. The main reason you map documents
> into memory today isn't because we can't edit them on disk, 
> we can, it's
> for performance or operational reasons, like having the ability to
> discard changes.

Well, we've already corrected the notion that you are editing directly on
the disk.  

But you are correct that in some cases it makes sense that a copy is made so
you can discard changes.  But that's the application level you are talking
about, not the OS level.  Applications on the iSeries can be written to work
with a temporary copy of a given object.  For instance, SEU does this.

> 
> 
> OK, clearly that's not a requirement of Windows. How do you think
> SQLServer handles multi-terabyte databases? By reading the 
> entire thing
> into virtual-memory, hah!
> 

Ok, remember my original post was intended for the layman. But no, SQL
server doesn't read the entire database into VM.  Heck, unless you are
running Windows 64 the VM address space is limited to 4GB anyways and an
application can only use 2GB by default;  Though there is an option that
allows an application to have 3GB of the 4GB VM address space. 

But that doesn't change the fact that no application can access anything not
in its VM address space.  

So when dealing with large data files, you have two types of swapping going
on.
1) The application is swapping data from its VM address space to/from the
actual disk file
2) The OS is swapping VM pages of running applications between main memory
and the swap file on disk.

Now SQL attempts to minimize #2.  "When running on Microsoft® Windows NT® or
Windows® 2000, the default  memory management behavior of the SQL Server
database engine is not to acquire a specific amount of memory, but to
acquire as much memory as it can without generating excess paging I/O. The
database engine does this by acquiring as much memory as is available, while
leaving enough memory free to prevent the operating system from swapping
memory."

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/architec/8_
ar_sa_9hgw.asp

Which of course makes perfect sense.  You wouldn't want to waste resources
writing to the swap file when the application is getting ready to write the
page back into it's actual disk file.

With single level store, #2 from above is the only swapping happening. You
can see that single-level store has performance benefits.


Charles Wilt
iSeries Systems Administrator / Developer
Mitsubishi Electric Automotive America
ph: 513-573-4343
fax: 513-398-1121