Picos and persistent state

This post provides specific detail to follow up on an earlier post about picos as the database of a web application.

Each pico holds persistent state, which is data or information specific to whatever entity it represents. Even if the pico engine which hosts it goes down temporarily, the state will be there as soon as it restarts.

The pico state is held and managed by the rulesets installed in it. Each ruleset can hold a portion of the entire state, and since there is no limit to the number of installed rulesets, the total quantity of data is also not limited.

Pico state held in a ruleset

Each ruleset can have entity variables holding the state specific to that ruleset.

An entity variable can be assigned a value, and that value can later be mutated, through use of the assignment operator (:=) in the postlude of a rule (and only there).

The value of an entity variable can be used anywhere within a function or rule of the same ruleset, just by mentioning its name.

The name of an entity variable is the prefix ent: followed by an identifier. The value is often a map, but can be of any type used in KRL.

Examples of usage are shown in code snippets in a previous post, "A stateful web app in a pico".

Pico state shared across rulesets

The scope of an entity variable is the ruleset in which it is used. This means that state cannot be shared across rulesets. There is one exception, which is the use of one ruleset as a user-defined module by another ruleset.

In rulesets used as a module

Even then, the entity variables of the module are not directly visible to the using ruleset. But the module can provide functions that the using ruleset can call, and those functions can either directly expose the current value of an entity variable in the module or be used in expressions to provide values dependent on the module's entity variables.

The value of the module's entity variables cannot directly be set by the using ruleset, but the latter can raise events which will be handled by rules in the module, and its rules can set module entity variables.

In the io.picolabs.pds ruleset

A special case is the io.picolabs.pds ruleset, whose specific purpose is to hold values to be shared across rulesets within a pico. It has no semantic role beyond supporting such sharing.

The complete source code for such a ruleset can be found here, and the bare essentials are shown below:

ruleset io.picolabs.pds {
  meta {
    name "Pico Data Store"
    provides getData
  }
  global {
    getData = function(domain,key){
      ent:pds{[domain,key]}
    }
  }
  rule setData {
    select when pds new_data_available
      domain re#(.+)#
      key re#(.+)#
      setting(domain,key)
    fired {
      ent:pds{[domain,key]} := event:attrs{"value"}
      raise pds event "data_added" attributes event:attrs
    }
  }
}

As can be seen, the module provides (line 4) a function (lines 6-8) that returns a value from the data store, and a rule (lines 10-19) that when raised will update the data store to hold a value.

A value stored in PDS is identified by a two part name, domain and key, so that many different uses of the same mechanism can co-exit among the other rulesets installed in the pico. Such rulesets use the module with code like the following (this one limiting itself to the domain named "domain1"):

ruleset user_of_domain1 {
  meta {
    use module io.picolabs.pds alias pds
  }
  global {
    ... pds:getData("domain1","key1") ...
  }
  rule storage {
    ...
    fired {
      raise pds event "new_data_available" attributes {
        "domain": "domain1", "key": "key2", "value": "value2"
      }
    }
  }
  rule listener {
    select when pds "data_added"
      domain re#^domain1$#
    ...
  }
}

The fact that the PDS will be used by this ruleset is indicated in line 3.

Actual usage of a piece of data is done with a function call to pds:getData like the one shown in line 6, and the actual storage of a value is done in a rule like the one shown in lines 8-15.

The reader may have noticed that the PDS ruleset raises a terminal event in its setData rule, and any ruleset in the pico can listen for this event with a rule like the listener shown in lines 16-20.

Pico state stored externally

Since picos are first class Internet citizens, then can use http:post to change the state of external resources, and http:get to retrieve the state of external resources.

In a sheet for reporting purposes or archiving

We have experience, reported in a document "Using a Google sheet to record data from a pico" which describes how to do this. Look for the http:post to see where the action happens.

The purpose can be either to archive older data, or to keep a running report or dashboard.

In external stores for redundancy or archiving

A pico could choose to retain a copy of some of its state in an external resource. Numerous providers offer a service often called "cloud storage." For simple cases, Amazon S3 will work well with picos.

Modules built in to every pico

There are two rulesets, designed to be used as modules, which are installed in every pico.

Wrangler

This ruleset is like the operating system of a pico, and helps in managing a pico's channels, installed rulesets, and the parent/child relationship.

For full usage information, see "Wrangler" in the Pico Labs documentation

Relationships

For non hierarchical relationships (those beyond parent/child) among picos, you can use the io.picolabs.subscription ruleset as a module, to propose, accept, and delete specific relationships.

For full usage information, see "Managing Subscriptions".