Vernodes Technical Introduction

Vernodes is a computer storage paradigm enabling distributed collaboration, either in real time or with disconnected operation.

Overview

Nodes and Versions

A node, in vernodes, is a container for some data and metadata, with edges to other nodes, that may vary over time. In order to actually contain anything, nodes must have a version.

Versions are snapshots of a node at a particular point it its history. They contain the data and metadata, including edges, of that node at that point, along with a list of immediate predecessor versions that it replaces.

Capabilties and Edges

Nodes (and versions) are operated on with capabilities, that provide the ability to identify/summarize, read, or write versons of the node. (The capability to read includes the capability to identify/summarize and the capability to write includes the capability to read.) Identify and read capabilities are included in the metadata of each version, while read and write capabilites may be embedded into the data in versions as edges, either as a part or as a link.

Parts indicate that the referenced node is needed to fully examine or work with this version. For example, an image incorporated into a document. Identification capabilites for parts are usually exposed on the encrypted form of versions.

Links indicate that the referenced node is not needed to examine this version. For instance, a hyperlink to another document. Links are usually not exposed on the encrypted forms of versions.

Generations

Vernodes is a paradigm, not a concrete implementation, programming interface, or exchange format. There are many possibilities for storing and encrypting versions, some already in existance. These possibilites are known as generations. Different generations have different properties, including the type (and presense) of encryption and authentication, what formats and data types they support, whether they support write capabilities, and so on. In particular, generations vary in which other generations they can hold capabilites for!

Data

Versions hold structured data, not just piles of bytes. The precise details may vary by generation, but likely include:

• bytes
• text
• parts
• links
• natural numbers
• moments in time
• sequences of other values
• key-value maps

Formats

Each version also contains a list of formats that its data (allegedgly) conforms to. Typically formats are read capabilities to a node holding a description of the type or schema of the data. That description may be textual, machine-readable, or both, as indicated by the formats list on the versions of that node.

Example Programming Interface

Note: this is not a specification, merely an example.

Working with Generations

In order to create a node, we find a generation that it will be part of.

-- | Find a generation with the requested propertes.
findGeneration :: Properties Query -> m Generation
-- | Find a generation with the requested propertes.
generationProperties :: Generation -> m (Properties Identity)

data Properties f = Properties
   { dataCompressed :: f Bool
     -- ^ Whether the data is subject to a general purpose compression algorithm.
   , dataEncrypted :: f Bool
     -- ^ Whether the data is encrypted.
   , mutable :: f Bool
     -- ^ Whether nodes can have multiple versions.
   , referenceGenerations :: f (Set Generation)
     -- ^ Ensure that the generation can reference other generations.
   }

data Query a = Any | AnyOf [t] | AtLeast t

Working with Nodes and Versions

Once we have a generation, we can create a write capability for a node, and then commit a new version to it.

-- | Create a new node, but don't write any data to it yet.
createNode :: Generation -> m NodeWriteCap
-- | Commit some data to a node, and get back the capability to read it.
commit :: NodeWriteCap -> Value -> [Format] -> [Parent] -> m VersionWriteCap

data EdgeCap
   = NR NodeReadCap 
   | NW NodeWriteCap 
   | VR VersionReadCap 
   | VW VersionWriteCap 

type Format = NodeReadcap
type Parent = Either VersionIdCap VersionReadCap

Of course, to have data to put in, we need a value type.

data Value
   = Part EdgeCap
   | Link EdgeCap
   | Quantity Word
   | Bytes BytesHandle
   | Sequence SequenceHandle

Bytes and sequence handles allow manipulation of potentially large numbers of bytes and sequences.

-- | Create a new empty bytes handle for a particular generation.
createBytes :: Generation -> m BytesHandle
-- | Read some bytes from a bytes handle.
bytesRead :: BytesHandle -> Offset -> Length -> m ByteString
-- | Replace some bytes in a bytes handle, and get back a new one.
bytesReplace :: BytesHandle -> Offset -> Length -> ByteString -> m BytesHandle

-- | Create a new empty sequence handle for a particular generation.
createSequence :: Generation -> m SequenceHandle
-- | Read some values from a sequence handle.
sequenceRead :: SequenceHandle -> Offset -> Length -> m [Value]
-- | Replace some values in a sequence handle, and get back a new one.
sequenceReplace :: SequenceHandle -> Offset -> Length -> [Value] -> m SequenceHandle

With a node committed, we can open it with either capability.

-- | Open the latest (or specified) version of a node.
open :: EdgeCap -> m Version
-- | Open all the latest versions of a node.
openMany :: Either NodeReadCap NodeWriteCap -> m [Version]

data Version = Version
   { verCaps :: Either (VersionReadCap, NodeReadCap) (VersionWriteCap, NodeWriteCap)
     -- ^ The capabilities for this version and the node its on.
   , verValue :: Value
     -- ^ The data held by this version.
   , verFormats :: [Format]
     -- ^ The formats of this version.
   , verParents :: [Parent]
     -- ^ The immediate predecessors of this version.
   , verMeta :: AdditionalMetadata
     -- ^ Potentially some metadata about the version from the system.
   }

A node that is only accessible to this program doesn’t do much for us. In order to save it we either need to put it another node, or prompt the user to save it or share it.

-- | Have the system prompt the user to share or save the capability.
shareNode :: String -> EdgeCap -> m ()