Project Graph

Technically speaking a project graph at Diafunc is a finite directed multigraph with potential loops. Every project branch has exactly one project graph with exactly one project entity as root. The graph consists of so called elements (entities, links and events). The graph's nodes are called entities and represent the central units of information. Each entity is identified via a project-wide unique entity id. Entities are connected via directed links, the graph's edges. Entities can be linked in both directions. There can be multiple parallel links between the same entities with different link ids (multigraph). Events are temporary elements which exist only for a limited time period. They are always related to an entity and are removed automatically after their lifespan has expired. They can for example be used to trigger function evaluations and implement event-driven models.

Every project graph is built up by three basic elements. Entities and links are mutable elements and can be created, updated and removed anytime. They also can exist for any time period. Entities can have any binary content without of any size limitation. A link always connects two entities in a specific direction. Events are immutable and can only exist for a limited timespan.

Each link is directed and links a specific source entity to a specific target entity. However, the graph can be traversed in both directions, forwards and backwards.

A project consists of at least one project branch, called the main branch. It's possible to create a new project branch by forking any existing one.

Entity Branching allows branching specific entities or even complete subgraphs. This can be useful for example to parallelize different computation branches in simulations or to save specific states.

In the illustrated project graph the blue subgraph is forked from the "main" entity branch to the "fork" entity branch. The resulting subgraph is visualized in amber.

The garbage collector automatically removes all unreachable entities, links and events. An entity is reachable if it's linked directly or indirectly from the project entity in the forward direction. They are removed no earlier than 24 hours after creation. This allows creating temporary unreachable entities (or subgraphs) for computation purposes.

The red entities, links and events of the illustrated project graph would be removed by the garbage collector because they are not reachable from the project entity.

The graph is designed for big data use cases with linear scalability. It's stored in a distributed manner allowing read, writes and data volume to be vertically and horizontally linear scalable. This means that the graph may grow to a theoretically infinite size. Practically the graph's size is limited through the account's subscription contingents. If the contingents are exhausted then all projects are deactivated automatically until contingents are free again.

Two consistency modes are available: Weak consistency and strong consistency. While weak transactions provide the maximum scalability and performance, strong transactions may be used to achieve higher levels of consistency. However, as Diafunc is designed as an analytical platform, it's strongly recommended to use weak transactions, eventual consistency, purposeful data redundancies, idempotent operations and mappings via cloud functions between immutable data structures in most use cases.

Independent of the selected consistency mode each transaction is invisible to others until it's committed. During function evaluation a transactional view of the project graph is provided. This means that changes to the graph are seen locally within the same transaction in a single function evaluation. By committing the transaction the changes are applied to the global project graph. Transactions will be committed automatically after the function evaluation has finished if they haven't been committed or reverted manually before.

In the left panel the blue elements are added and the red elements are removed by the transaction; the right panel shows the global project graph as others see it until the transaction is committed.

The Tree View displays the project graph as a tree originating from the project entity as root.

The Graph View displays the project graph originating from the currently opened entity.