Many applications need to keep typed objects in a local file and find them again by key, without installing or administering a database server. ObjectSpace in Sphere10 Framework is an embedded object store for that job. You register your CLR types, declare which members are indexed, and the store persists the objects in a single transactional file.

This article builds a file-backed ObjectSpace from start to finish. It covers how the store is organised, how to declare types and indexes with ObjectSpaceBuilder, how to save, fetch and delete objects, and what has to stay the same when the file is reopened.

1. A first look

A file-backed store is opened, changed, committed and then reopened in a few lines.

using (var space = SampleStore.Open(directory)) {
	// Save, update and delete objects.
	space.Commit();
}
using (var reopened = SampleStore.Open(directory)) {
	var customer = reopened.Get((Customer item) => item.Email, "[email protected]");
	var purchases = IndexQueries.By(reopened, (Purchase item) => item.CustomerId, customer.Id);
}

The first using block opens the store, changes it and publishes the work with Commit(). The second opens the same file again, fetches a customer through the unique index on Email and loads that customer's purchases through the nonunique index on CustomerId. SampleStore.Open and IndexQueries.By are the sample's own factory and lookup helper, and the sections below build both.

2. How an ObjectSpace is organised

classDiagram
    direction TB
    class ObjectSpace {
      +Dimensions
      +Save(item)
      +Get(member, key)
      +Delete(item)
    }
    class Dimension~T~ {
      +Definition
      +Container
    }
    class ObjectStream~T~ {
      +GetIndexFor(member)
      +Streams
    }
    ObjectSpace "1" *-- "1 per type" Dimension~T~
    Dimension~T~ *-- ObjectStream~T~ : slots and indexes

block-beta
    columns 1
    A["ObjectSpace: tracked objects"]
    B["ObjectStream per dimension"]
    C["ClusteredStreams container"]
    D["TransactionalStream: staged pages"]
    E[("Data file")]

3. Declare dimensions and indexes

The stored types are ordinary classes with public properties. Nothing derives from a Framework base class. The sample uses two types, and the second refers to the first through a scalar ID.

public sealed class Customer {
	public long Id { get; set; }

	public string Email { get; set; }

	public int Region { get; set; }

	public string Name { get; set; }
}

public sealed class Purchase {
	public long Id { get; set; }

	public string Reference { get; set; }

	public long CustomerId { get; set; }

	public decimal Total { get; set; }
}

Each type is registered with AddDimension<T>(), followed by its index declarations.

public static ObjectSpaceBuilder Configure(ObjectSpaceBuilder builder) {
	builder.AddDimension<Customer>(ignoreAnnotations: true)
		.WithIdentifier(item => item.Id)
		.WithUniqueIndexOn(item => item.Email)
		.WithIndexOn(item => item.Region)
		.OptimizeAssumingAverageItemSize(256)
		.Done();
	builder.AddDimension<Purchase>(ignoreAnnotations: true)
		.WithIdentifier(item => item.Id)
		.WithUniqueIndexOn(item => item.Reference)
		.WithIndexOn(item => item.CustomerId)
		.OptimizeAssumingAverageItemSize(256)
		.Done();
	return builder;
}

The three index kinds do different jobs.

Done() completes the dimension and registers an automatic serializer for the type when none was supplied. OptimizeAssumingAverageItemSize sets the dimension's cluster size from the expected size of one serialized object.

The same declarations can be written as [Identity], [UniqueIndex] and [Index] attributes on the members, which AddDimension<T>() reads by default. Passing ignoreAnnotations: true makes the fluent calls the complete definition.

4. Choose the backend and build the store

The builder also selects where the data lives and how the file is laid out.

public static FileObjectSpace Open(string directory) {
	Directory.CreateDirectory(directory);
	Directory.CreateDirectory(Path.Combine(directory, "pending-pages"));
	var builder = new ObjectSpaceBuilder()
		.UseFile(Path.Combine(directory, "sample.s10"))
		.KeepUncommittedPagesAt(Path.Combine(directory, "pending-pages"))
		.WithClusterSize(4096)
		.WithPageSize(64 * 1024)
		.WithMaxMemory(4 * 1024 * 1024)
		.WithIndexBTreeOrder(64)
		.WithEndianness(Endianness.LittleEndian)
		.AutoLoad();
	return (FileObjectSpace)SampleSchema.Configure(builder).Build();
}

Every size has a default, so a minimal builder needs only the backend and the dimensions. The sample states each value because the same values are required again on reopen, as the section on reopening explains.

5. Save, fetch and delete objects

Save inserts a new object or updates a tracked one, and maintains every index of its dimension.

space.Save(new Customer { Id = 1, Email = "[email protected]", Region = 7, Name = "Ana" });
space.Save(new Customer { Id = 2, Email = "[email protected]", Region = 7, Name = "Ben" });
space.Save(new Purchase { Id = 100, Reference = "PO-100", CustomerId = 1, Total = 125m });
space.Save(new Purchase { Id = 101, Reference = "PO-101", CustomerId = 1, Total = 40m });
space.Save(new Purchase { Id = 102, Reference = "PO-102", CustomerId = 2, Total = 15m });

Get and TryGet take a member expression and a key. They use the identifier or unique index declared on that member, so no records are scanned. Get throws when the key is absent, and TryGet returns false.

var purchase = space.Get((Purchase item) => item.Reference, "PO-101");
SampleChecks.Require(ReferenceEquals(purchase, space.Get((Purchase item) => item.Id, 101L)), "The same record resolved to two instances.");
if (space.TryGet((Purchase item) => item.Id, 999L, out var missing))
	throw new InvalidOperationException("An absent identifier was found.");

purchase.Total = 45m;
space.Save(purchase);
space.Delete(space.Get((Purchase item) => item.Reference, "PO-102"));

The second line fetches the same record through a different index and receives the same instance. Because purchase is that tracked instance, the later Save updates the existing record and its index entries. Delete recycles the record's slot. GetAll<T>() returns a snapshot of every saved object in a dimension, which is useful for small sets and for checks, although an index is the better tool for a lookup.

A nonunique index returns a group of slots for a key. The sample wraps that lookup in one generic helper that resolves the slots and then loads each object through the space, so the results are tracked like any other.

public static T[] By<T, TKey>(ObjectSpace space, Expression<Func<T, TKey>> member, TKey key) {
	using (space.EnterAccessScope()) {
		var dimension = (ObjectSpace.Dimension<T>)space.Dimensions.Single(item => item.Definition.ObjectType == typeof(T));
		var positions = dimension.Container.ObjectStream.GetIndexFor(member).Values[key].ToArray();
		return positions.Select(space.Get<T>).ToArray();
	}
}

var purchases = IndexQueries.By(reopened, (Purchase item) => item.CustomerId, customer.Id);

The indexed-query article looks at this lookup and at index maintenance in more detail.

6. Commit, dispose and reopen

In a file-backed space, Save only stages changes. Commit() publishes them to the file, and Rollback() discards them. Disposing the space does neither, so uncommitted work is lost when the space is disposed. That is why the first look at the start of this article calls Commit() before its first using block ends.

The first look also reopens the file through the same factory, and for a reason. The file does not describe its own layout unless you ask it to, so the builder must supply the original definition every time. The following have to match the values the file was created with.

• The order of the dimensions, their serialized members and their indexes.
• The B-tree order and type of the indexes.
• The cluster size, page size and byte order.

The page-cache budget is a runtime setting and may change freely. A store can also record its own definition, which the self-describing store article covers.

7. What ObjectSpace leaves to the application

8. Get the package and run the sample

ObjectSpace ships in the Sphere10.Framework NuGet package. The sample targets .NET 10.

dotnet new console --framework net10.0 --name ArticleSample
cd ArticleSample
dotnet add package Sphere10.Framework --version 3.1.3

The complete MIT-licensed sample contains every file shown in this article. Extract the download over the generated project and run it against a new directory. The sample refuses to overwrite an existing store.

dotnet run -c Release -- ./fresh-data

It creates a store with two dimensions, saves five objects, updates one and deletes another, commits, reopens the file and reads the objects back through their indexes.

Reopened Ana: 2 purchases, total 170
Identity, update, delete and reopen checks: passed

The next article in this series covers indexed lookups , and the one after it covers commit and rollback . The ObjectSpace guide is the reference for every builder option.

The Sphere10 Framework overview describes the rest of the toolkit.

Author: Herman Schoenfeld