Re: ADT Lib: Persistence

[Stewart Smith <ssmith@murdoch.edu.au>]

>> I have a simple (ie. naive) heap manager that allows a large number of
>> objects to be stored/retrieved from a memory-mapped file. This is the basis
>> to a simple database kernel that runs under Blackbox. There is no fancy
>
>Are we talking about object oriented databases? That would be cool
>stuff!

The project that I was referring to is a prototype data-base system (called
"Noetica") based on graphs. Technically, it is not an "object-oriented"
system (since its objects have no "methods"), but it does contain most
other features of object-oriented systems.

From my understanding, many "object-oriented" databases are simply
persistent storage systems with some added indexing capabilites and
transaction management (multiple users, database views, etc). References to
objects are usually "swizzled" (ie. mapped between object identifiers and
pointers) automatically and objects are loaded / stored on demand. In
systems based on C++ this is easy to do, since you can overload the
dereference (->) operator. This means that you can provide objects that
look and behave exactly like regular pointers but are in fact implemented
differently.

"Noetica" differs from this kind of system because it represents objects
using a meta-structure on top of Component Pascal. This means that the
internal structure of objects is always completely visible from "outside",
making it easy to automatically load, store, display, edit, and transform
the objects. By using a graph data model, references to other objects are
confined to the "top level" of objects, which simplifies the maintenance
and traversal of references. It writes objects in a format that allows them
to be interpreted directly without first "internalising" them to heap-based
representations. This means that large graph traversals can be done very
quickly by walking through a memory-mapped file (no heap operations are
necessary).

The existing Storage module provides an extensible way of reading/writing
objects. To be used for potentially large collections of objects, it needs
a way of supporting partial internalisation. As I have said, this isn't
hard to do if you use special "reference" objects. However, to provide
arbitrary queries over objects (as are found in most data-base systems)
there needs to be a way for query and indexing systems to know about
important properties of objects. In "Noetica" this is easy because the
internal structure is always visible. Within a general "AcObject" framework
it would probably be useful to include some interface by which an object
makes certain of its properties "visible". This could be done (for example)
using a simple name-space interface. You pass some kind of name (a string,
or a series of integer keys) to an accessor and it reads or writes internal
properties of the object. If objects also provided a kind of directory
function (by which the name space could be traversed) this could be used to
automatically generate views (data-entry forms, "property sheets" or
reports) for arbitrary objects. This would enable the development of some
powerful tools for indexing, querying and browsing data. Then we would
_really_ be talking about a data-base system.

>
>> unixes have mmap() functions. How widely available is mmap() under Unix? 
>
>The configure tool at least a special check for mmap (AFAIK) so don't
>exspect it on every UNIX box out there.

I know from experience that the Linux mmap() implementation was broken
prior to the version 2 kernels.

It would make more sense to design a system based simply on Files. In my
experience, with a good disk cache there is not much difference in
performance terms.

- Stewart