Phi: referring to sources of data

A Phi script is run when data arrives for an asset or a data pool. The Phi script is defined within a schema that describes and asset or a data pool, , so the data usually relates to the asset or data pool that the schema describes.

There are three sources of data that you can use in your Phi script:

  • For an asset, incoming data from an asset.
  • For a data pool, data from assets in that data pool, or from other other data pools under the current data pool.
  • Historic data.

The data is presented to the Phi logic as arrays, so you will see the notation with square brackets [] where it is a 1-dimensional array, or with braces {} where it is a hashed array.

For an asset: incoming data

Remember that you access Phi script when editing a schema. When data arrives from an asset that uses the schema, the Phi script defined on the schema is run.

In summary, the following arrays of data are available:

  • thisRow — the current row of data just received from the asset
  • prevRow — the previously-received row of data from the asset.

Both of these are arrays of arrays, so you will normally used a dotted notation to get to the nested arrays, string or number values.

Variable Meaning
thisRow.timestamp Number. The timestamp of the data being processed. All timestamps are in milliseconds since 1 January 1970 UTC (epoch time).
prevRow.timestamp Number. The timestamp of the previous data received, again in epoch time.
thisRow.value Array. The incoming data, with values "filled in" from previous known values if missing.*
thisRow.actual Array. The incoming data, containing exactly the values included in the current row.
thisRow.assumed Array containing all entries from thisRow.value that were assumed but not actual in the current row. By implication these values will have been received some time previously.
thisRow.assumedFrom Array. Timestamps for all of the entries in thisRow.assumed
thisRow.value.bad Boolean. Flag is set to true when the row is flagged as bad data. Use the Query data tool to view an asset's data, scroll to the right of the data row and check the "Bad" checkbox to flag. This flag is used in Phi scripts but other parts of the system ignore it.
prevRow.value Array. The previously-received row of data, with values "filled in" from previous known values if missing.*
prevRow.actual Array. The previously-received row of data, containing exactly the values included.
prevRow.assumed Array containing all entries from thisRow.value that were assumed but not actual in the previously-received row. By implication these values will have been received some time previously.
prevRow.assumedFrom Array. Timestamps for all of the entries in prevRow.assumed


* Use the value arrays as the easiest way to get to the latest value of a field. If a row of incoming data didn't contain all fields in the asset's schema (which is perfectly valid behaviour), the last-known value of the missing ones is used. When recording such data, Assetwolf fills in missing fields by copying forward field values from the value that was last received, but flags such fields as assumed.

Examples

Imagine you have some value called "activations" in incoming data. It could be a cumulative number. If you want to calculate the number of activations since the last row of data, and the rate of activations, you could use this:

activationsThisTime = thisRow.value.activations - prevRow.value.activations
activationsPerSecond = 1000 * activationsThisTime / (thisRow.timestamp - prevRow.timestamp)

 If the asset sends "lightlevel", and you want to get the most recent light level, just this thisRow.value.lightlevel. In the following you cam get more details about when the last lightlevel field was actually received:

#If you just want the most recent value for the light level, and don't care if it was assumed
lightLevel = thisRow.value.lightlevel

#If you want more details about when the value for the light level was received
if (thisRow.actual.lightlevel is not null) {
	lightLevel = thisRow.actual.lightlevel
	lightLevelTS = thisRow.timestamp
	lightLevelWasAssumed = false
} else {
	lightLevel = thisRow.lastKnown.lightlevel
	lightLevelTS = thisRow.lastKnownAt.lightlevel
	lightLevelWasAssumed = false
}

For a data pool: data from child assets and child data pools

Data pools are "big picture" views of data, and can display data that has been gathered from multiple assets, or from other data pools.

Data pools exist in a hierarchy. For example, imagine a Location (like a building) contains Areas, within Areas are Rooms, and within Rooms are real Assets that communicate. You may want to view averages, maxima, minima, and other aggregated views of the data for the entire Location. You may want to drill down into Areas and Rooms, and to drill down inside a Room to see data from individual Assets.

To achieve this, a Location data pool gathers data from Area data pools within it; and similarly Area data pools gather data from Room data pools within them. The Room data is aggregated from the Assets in each Room as they send data.

Whenever data arrives, Assetwolf's data processor looks at it and processes it according to the schema. There is a "bubbling-up" effect, so that whenever data for a set of assets is processed, the data for the data pool containing those assets is processed. The process continues up the hierarchy, whenever there is enough data at each level.

When the data processor runs, the Phi script for a data pool may contain:

  • source — the latest data "coming up" from child data pools or assets
  • prevRow — the previous row of data for this data pool (i.e. the data that was saved when the script last run).

(thisRow is not relevant for data pools, as it is empty.)

Overall, the following arrays are available.

Variable Meaning
source.values An array of arrays containing all of the data from the sources. May include assumed, rather than known data.
source.num The number of potential sources of data in the current run (e.g. number of child data pools or assets according to the data hierarchy, irrespective of whether they provided data on this run)
source.numKnown The number of sources that provided data on this run
source.numAssumed The number of sources that did not provide data on this run (and their values were assumed)
source.freshValues An array of arrays containing data values received from the sources since the last run of this Phi script. Any value that has not been retransmitted since the previous time the Phi code was run will not be included.
prevRow.value Array. The previously-create row of data for this data pool, with values "filled in" from previous known values if missing.

Examples

#Get the highest/lowest/average/total values
maxLightLevel = max(source.values.lightlevel)
minLightLevel = min(source.values.lightlevel)
totalLightLevel = sum(source.values.lightlevel)
averageLightLevel = mean(source.values.lightlevel)

//N.b. the sources are arrays of data, so you can loop through them
maxLightLevel = 0
minLightLevel = Infinity
totalLightLevel = 0
for (lightLevel in source.values.lightlevel) {
	if (maxLightLevel < lightLevel) {
		maxLightLevel = lightLevel
	}
	if (minLightLevel > lightLevel) {
		minLightLevel = lightLevel
	}
	totalLightLevel += lightLevel
}
averageLightLevel = totalLightLevel / count(source.values.lightlevel)

Getting historic data (assets and data pools)

You can get to historic data of the current data pool or asset, by using a function call.

Function Meaning
getHistoricValue(key, timePhrase)

This function can be used to look up a field value from some time in the past.

Set key to be the name of the field, and timePhrase to be either a Unix timestamp, or a friendly description (see below).

The time does not need to be an exact match to a datapoint; if it's not a perfect match, you'll receive the value that the field was assumed to be at that time.

getTimestamp(timePhrase)

getTimestamp(timePhrase, originTimestamp)

Given a description such as:

this function will return the timestamp in Unixtime.

Times will be in the default timezone for the site.

For phrases like "2 days ago" which are relative to some time, originTimestamp will be used as the time it is relative to. Defaults to thisRow.timestamp.