HashFile implements a simple mechanism to store and recover a
large quantity of data for the duration of the hosting process.
It is intended to act as a local-cache for a remote data-source,
or as a spillover area for large in-memory cache instances.
Note that any and all stored data is rendered invalid the moment
a HashFile object is garbage-collected.
The implementation follows a fixed-capacity record scheme, where
content can be rewritten in-place until said capacity is reached.
At such time, the altered content is moved to a larger capacity
record at end-of-file, and a hole remains at the prior location.
These holes are not collected, since the lifespan of a HashFile
is limited to that of the host process.
All index keys must be unique. Writing to the HashFile with an
existing key will overwrite any previous content. What follows
is a contrived example:
alias HashFile!(char[], char[]) Bucket;
auto bucket = new Bucket ("bucket.bin", HashFile.HalfK);
// insert some data, and retrieve it again
auto text = "this is a test";
bucket.put ("a key", text);
auto b = cast(char[]) bucket.get ("a key");
assert (b == text);
bucket.close;
- struct BlockSize;
- Define the capacity (block-size) of each record
- this(const(char[]) path, BlockSize block, uint initialRecords = 100);
- Construct a HashFile with the provided path, record-size,
and inital record count. The latter causes records to be
pre-allocated, saving a certain amount of growth activity.
Selecting a record size that roughly matches the serialized
content will limit 'thrashing'.
- const(char[]) path();
- Return where the HashFile is located
- ulong length();
- Return the currently populated size of this HashFile
- V get(K key, bool clear = false);
- Return the serialized data for the provided key. Returns
null if the key was not found.
Be sure to synchronize access by multiple threads
- void remove(K key);
- Remove the provided key from this HashFile. Leaves a
hole in the backing file
Be sure to synchronize access by multiple threads
- void put(K key, V data, K function(K) retain = null);
- Write a serialized block of data, and associate it with
the provided key. All keys must be unique, and it is the
responsibility of the programmer to ensure this. Reusing
an existing key will overwrite previous data.
Note that data is allowed to grow within the occupied
bucket until it becomes larger than the allocated space.
When this happens, the data is moved to a larger bucket
at the file tail.
Be sure to synchronize access by multiple threads
- void close();
- Close this HashFile -- all content is lost.
- struct Record;
- Each Record takes up a number of 'pages' within the file.
The size of these pages is determined by the BlockSize
provided during HashFile construction. Additional space
at the end of each block is potentially wasted, but enables
content to grow in size without creating a myriad of holes.
- V read(HashFile bucket, bool clear);
- This should be protected from thread-contention at
a higher level.
- void write(HashFile bucket, V data, BlockSize block);
- This should be protected from thread-contention at
a higher level.
- void createBucket(HashFile bucket, int bytes, BlockSize block);