[Tinyos-commits] CVS: tinyos-1.x/tos/lib/VM/languages/tinysql tinysql.txt, 1.1, 1.2

Sun Oct 30 12:54:07 PST 2005

Update of /cvsroot/tinyos/tinyos-1.x/tos/lib/VM/languages/tinysql
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17792

Modified Files:
	tinysql.txt 
Log Message:
doc update
flush log output after each packet printed

Index: tinysql.txt
===================================================================
RCS file: /cvsroot/tinyos/tinyos-1.x/tos/lib/VM/languages/tinysql/tinysql.txt,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -d -r1.1 -r1.2
*** tinysql.txt	29 Oct 2005 17:26:07 -0000	1.1
--- tinysql.txt	30 Oct 2005 20:54:05 -0000	1.2
***************
*** 6,32 ****

  TinySQL allows you to view a sensor network as a database, where the
! sensors present on each node are treated as the columns ("attributes")
! of the database. Additionally, a few other important node
! characteristics, such as node id, are also attributes. This database
  is then queriable using a SQL-like language, whose major difference
  from SQL is that queries include a "sample period". This sample period
! is an interval at which the query is evaluated and results of the
! query potentially returned. The query is thus "continuous", returning
! a stream of data to the sensor network's base station until stopped.

  2) A simple example
  ===================

! This example is designed for use with mica2 nodes with a micasb (aka mts300)
! sensor board. To adapt it for other motes or sensor boards, please see
! Section XXX.

  First, because Mate's TinySQL implementation is based on compiling TinySQL
  queries to motlle code, you need to build and install an appropriately 
  configured motlle VM on your motes. Do this as follows:
!   cd $(TOSDIR)/lib/VM/languages/motlle
!   make # ensure motlle is configured
!   cd examples
!   ../tools/motllevm-build tsql.vmsf # create TinySQL VM
    cd $(TOSDIR)/../apps/TSQLVM
    make mica2
--- 6,33 ----

  TinySQL allows you to view a sensor network as a database, where the
! sensors present on each mote are treated as the columns ("attributes")
! of the database. Additionally, a few other important mote
! characteristics, such as mote id, are also attributes. This database
  is then queriable using a SQL-like language, whose major difference
  from SQL is that queries include a "sample period". This sample period
! is an interval at which the query is evaluated and results
! returned. The query is thus "continuous", returning a stream of data
! to the sensor network's base station until stopped.

  2) A simple example
  ===================

! This example is designed for use with mica2 motes with a micasb (aka
! mts300) sensor board. To adapt it for other sensor boards, please see
! Section 6. You can use TinySQL with micaz or telosb motes simply
! by replacing mica2 by micaz/telosb in the examples below (for telosb
! motes, you might want to increase the amount of RAM available -- 
! see Section 6).

  First, because Mate's TinySQL implementation is based on compiling TinySQL
  queries to motlle code, you need to build and install an appropriately 
  configured motlle VM on your motes. Do this as follows:
!   cd $(TOSDIR)/lib/VM/languages/motlle/examples
!   motlle-vmbuild tsql.vmsf # create TinySQL VM
    cd $(TOSDIR)/../apps/TSQLVM
    make mica2
***************
*** 47,51 ****
  To check that your motes have been correctly programmed, etc, you should
  execute the following command in the TSQLVM directory:
!   ./mload -e "led(1)"
  This should turn on the red LED on your base station, and shortly thereafter
  on all other motes. If this doesn't happen, check that your serial forwarder
--- 48,52 ----
  To check that your motes have been correctly programmed, etc, you should
  execute the following command in the TSQLVM directory:
!   motlle-load -e "led(1)"
  This should turn on the red LED on your base station, and shortly thereafter
  on all other motes. If this doesn't happen, check that your serial forwarder
***************
*** 53,58 ****

  You are now ready to run your first query. Try:
!   cd $(TOSDIR)/lib/VM/languages/tinysql
!   ./tsqlrun "select id, light interval 10"
  This should collect mote identity and light readings from all motes in the
  network every 10s. The results will be printed by tsqlrun until you interrupt
--- 54,58 ----

  You are now ready to run your first query. Try:
!   tinysql start "select id, light interval 10"
  This should collect mote identity and light readings from all motes in the
  network every 10s. The results will be printed by tsqlrun until you interrupt
***************
*** 60,72 ****

  Interrupting tsqlrun does not stop the query. You can resume collecting
! results by executing (all tsqlrun commands should be run from the
! $(TOSDIR)/lib/VM/languages/tinysql directory):
!   ./tsqlrun --resume
  which will resume printing results until you interrupt tsqlrun.

  You can terminate data collection by executing
!   ./tsqlrun --stop

! If you start a new query with tsqlrun, the old query will be automatically
  terminated.

--- 60,71 ----

  Interrupting tsqlrun does not stop the query. You can resume collecting
! results by executing:
!   tinysql log
  which will resume printing results until you interrupt tsqlrun.

  You can terminate data collection by executing
!   tinysql stop

! If you start a new query with tinysql, the old query will be automatically
  terminated.

***************
*** 74,83 ****
  =======================

! 4) The tsqlrun command
  ======================

  5) Extending TinySQL: User-specified attributes, aggregates
  ===========================================================

  6) Configuring a TinySQL VM
  ===========================
--- 73,377 ----
  =======================

! This implementation of TinySQL supports two kinds of queries:
! - Local queries, which collect independent data ("attributes") from each mote.
! - Global queries, which compute some aggregate over the whole sensor network.
!   Note that the syntax for global queries departs from that of TinyDB.
! 
! Both kinds of queries include:
! - An interval (in seconds), at which sensors are sampled and the query
!   evaluated. Each interval is called an epoch; epochs are numbered
!   consecutively starting at 1 when the query is installed.
! - An optional condition, evaluated on each mote at each epoch - if the
!   condition is false, that mote does not take part in the query
!   computation for the current epoch.
! 
! TinySQL can be extended by writing attributes and aggregates in motlle
! code. See Section 5 for a detailed description.
! 
! 3.1 Local queries
! -----------------
! 
! The syntax of a local query is:
!   select VALUE1, VALUE2, ... [where CONDITION] interval N
! 
! where each VALUE is 
! - the name of an attribute
!   examples: id, light
! - the name of a local aggregate, with a list of values as parameters
!   example: expdecay(light, 3)
! - an integer constant (useful in aggregate arguments and conditions)
! 
! and CONDITIONs are expressions built from VALUEs using 
!   and, or, not, < , <=, >, >=, =, <>
! and parentheses
! examples: id >= 10, (expdecay(light, 3) <= 5) and (id < 20)
! 
! Example local queries:
!   select id, light, temp interval 5
!   select id, light where temp > 20 interval 30
!   select id, light where expdecay(temp, 3) > 20 interval 30
! 
! 
! An attribute is something which can be computed or measured on a mote
! at any time. Examples include a mote's identity, or the reading of
! one of its sensors. The set of attributes available is:
! - id: the mote's identity
! - the sensors made available by the sensor board you configured in
!   your TinySQL VM (see Section 6)
! - the attributes defined in motlle code (see Section 5)
! 
! A local aggregate is a function which computes a result based on its
! arguments and some internal state. For instance, at each call,
! expdecay(X, N) returns the exponentially decayed average of all values
! of X it received since the start of the query. The decay factor is
! 1-1/2**N. The standard TinySQL VM does not include any predefined
! local aggregates; all such aggregates are defined in motlle code (see
! Section 5).
! 
! 3.2 Global queries
! ------------------
! 
! The syntax of a global query is:
!   select GLOBAL1[VALUE11, VALUE12, ...],
!          GLOBAL2[VALUE21, VALUE22, ...],
! 	 ...
!          [where CONDITION] interval N
! 
! where GLOBAL is the name of a global aggregate (e.g., avg, max).
! 
! Example global queries:
!   select avg[light] interval 10
!   select max[temp], avg[light] where light > 15 interval 30
! 
! A global aggregate is a function which is computed at each epoch over
! the value of its arguments on all motes where the CONDITION is true.
! Values of global aggregates are computed in the sensor network as
! results are collected and forwarded through the network's routing
! tree.  Computing an aggregate thus causes signifcantly less network
! traffic than collecting all the data at the root and computing the
! aggregate there. The set of global aggregates available is:
! - min, max and avg
! - the global aggregates defined in motlle code (see Section 5)
! 
! 4) The tinysql command
  ======================

+ The tinysql command is used to compile TinySQL queries, install them
+ in the network and collect their results. tinysql must always be run
+ from the directory containing your VM (e.g., apps/TSQLVM in the
+ example above). The usage for tinysql is:
+ 
+   tinysql [-I dir1 -I dir2...] COMMAND QUERY
+   where COMMAND is one of:
+     start        install QUERY followed by log
+     install      install QUERY in the sensor network
+     reinstall    reinstall current query in the sensor network
+     log          display continuous results from current query
+     stop         stop current query in sensor network
+     display      display current query
+     compile      just compile QUERY and print resulting motlle code
+ 
+     QUERY is a TinySQL query (not needed for reinstall, log, stop, display)
+ 
+     The -I directives specify directories in which user-defined aggregates
+     and attributes can be found.
+ 
+     start, install, reinstall, log and stop require that a serial forwarder,
+     talking to TinySQL mote 0 be running on the local host.
+ 
+     Examples:
+       tinysql start "select id, light interval 10"
+       tinysql compile "oops"
+       tinysql stop
+ 
+ A few comments:
+ - The motlle code which results from compiling a query is stored in
+   ~/.tinysql_query. The reinstall, log and display commands use
+   this file, so will fail if it is removed.
+ - If you wish, you can modify ~/.tinysql_query and execute 
+     tinysql reinstall
+   to install your patched code in the sensor network. However, if you
+   change the message layout, "tinysql log" will likely fail.
+ 
  5) Extending TinySQL: User-specified attributes, aggregates
  ===========================================================

+ Any time you use an attribute, local or global aggregate named X, the
+ TinySQL compiler will search for a file named X.mt in:
+ - the current directory
+ - any directories specified by -I options to tinysql
+ - tinyos-1.x/tos/lib/VM/languages/tinysql/lib
+ If it finds such a file, it will include it verbatim in the compiled
+ query output file (~/.tinysql_query).
+ 
+ This file should include motlle function definitions which implement
+ attribute, local or global aggregate X, as described below. If you
+ haven't done so already, you might want to check the motlle documentation
+ in tinyos-1.x/tos/lib/VM/languages/motlle/doc/motlle.txt.
+ 
+ 5.1 Adding attributes
+ ---------------------
+ 
+ If X is an attribute, then X.mt should define a function called X with
+ no parameters which returns the current value of X.  Example: the
+ (rather weird) light_plus_temp attribute is the sum of light and temperature:
+   any light_and_temp () {
+     return light() + temp();
+   }
+ 
+ 5.2 Adding local aggregates
+ ---------------------------
+ 
+ TinySQL assume that local aggregates have two kinds of arguments:
+ - numeric constants
+ - attributes and other local aggregates
+ 
+ For local aggregate X, X.mt should define two functions:
+ - X_make: this receives the numeric constant arguments and returns
+   a value Y. This function is called once when the query starts.
+ - X_get: at each epoch, this receives Y and the value of X's other
+   arguments. It returns X's value at this epoch. It may update Y.
+ 
+ Examples: 
+ 
+ A simple add aggregate which sums its arguments:
+   any add_make() 0; // has no useful state
+   any add_get(a, b) a + b; // just sum the two arguments
+ 
+ The code for expdecay, found in the tinysql/lib directory is:
+   // Build a two element array to remember expdecay's state.
+   //   The first element is the decay factor
+   //   The second element is the current decaying average
+   any expdecay_make(N) vector(N, 0);
+ 
+   // Update and return the decaying average
+   // the magic-looking expression computes 
+   //   sum = (sum * 1-1/2**N) + val
+   //   avg = sum / 2**N
+   //   but without keeping sum explicitly around
+   any expdecay_get(Y, val)
+     Y[1] = Y[1] - (Y[1] >> Y[0]) + (val >> Y[0]);
+ 
+ 5.3 Adding global aggregates
+ ----------------------------
+ 
+ Like local aggregates, TinySQL assume that global aggregates have two kinds 
+ of arguments:
+ - numeric constants
+ - attributes and local aggregates
+ 
+ However, global aggregates are a lot more complex to implement as they
+ have to collect results from other motes, and merge that with local
+ data.  Additionally, as communication takes time, these results may be
+ from different epochs, so global aggregates must maintain partial
+ results for many epochs.
+ 
+ For global aggregate X, X.mt should define six functions:
+ - X_make: this receives the numeric constant arguments and returns
+   a value Y. This function is called once when the query starts.
+ - X_newepoch: this function is called with Y as an argument when the
+   epoch changes.
+ - X_update: at each epoch, this receives Y and the value of X's other
+   arguments. 
+ - X_get: this function should return the value of X that should be
+   sent to our parent at this epoch. It receives Y as an argument and
+   should return a string of the same length as X_buffer.
+ - X_buffer: this function takes no arguments and must return a new string
+   whose size equals the size of the strings returned by X_get.
+ - X_intercept: this function is called when a partial result for 
+   aggregate X is received from a child of this mote. It's arguments are
+   Y and the result of X_get sent by the child mote.
+ 
+ As you can probably tell, all of this is somewhat tricky to implement. Here
+ is some code that could be used to implement the avg[] global aggregate in
+ motlle (untested, but converted from the built-in average aggregate):
+ 
+   any maxdepth = 5; // max routing tree depth supported
+   any window = 2 * maxdepth;
+ 
+   // The state of an average aggregate is an array containing, in this order:
+   //   counts for 'window' epochs
+   //   sums for 'window' epochs
+   //   the number of the oldest epoch in the window
+   any avg_make() {
+     any Y = make_vector(1 + 2 * window);
+     vector_fill!(Y, 0);
+     return Y;
+   }
+ 
+   // The epoch changed. Ensure epoch + 1 is inside the window.
+   any avg_newepoch(Y) {
+     any start = Y[2 * window];
+ 
+     if (epoch() + 1 >= start + window)
+       {
+ 	// figure out new start and how much to shift values
+ 	// from old epoch for the new start
+         any shift = epoch() + 2 - window - start, i;
+ 
+ 	start = epoch() + 2 - window, i;
+ 	Y[2 * window] = start;
+ 
+ 	if (shift > window)
+ 	  shift = window;
+ 	else
+ 	  {
+ 	    for (i = shift; i < window; i++)
+ 	      {
+ 	        Y[i - shift] = Y[i];
+ 		Y[i - shift + window] = Y[i + window];
+ 	      }
+ 	  }
+ 	// clear new values
+ 	for (i = window - shift; i < window; i++)
+ 	  {
+ 	    Y[i] = 0;
+ 	    Y[i + window] = 0;
+ 	  }
+       }
+   }
+ 
+   // Update the state for epoch 'when' with a count of 'n' and a sum of 'sum'
+   any add_avg(Y, when, n, sum) {
+     any start = Y[2 * window];
+     if (when >= start && when < start + window)
+       {
+ 	Y[when - start] += n;           // update count
+ 	Y[when - start + window] += n;  // update sum
+       }
+   }
+ 
+   // Add local result for current epoch
+   any avg_update(Y, val) add_avg(Y, epoch(), 1, val);
+ 
+   // avg_get returns a six-byte string containg encoded
+   //   epoch, count, sum (2 bytes each)
+   // where epoch is chosen so that our descendant's results have
+   // reached us before we try and send them on
+   any avg_get(Y) {
+     any start = Y[2 * window];
+     any when = epoch() - 2 * (maxdepth - 1 - depth());
+     any tosend = vector(when, 0, 0);
+ 
+     // encode the result for epoch 'when', but avoid problems if we don't 
+     // know it or if we're too deep in the tree
+     if (depth() >= maxdepth)
+       tosend[0] = epoch() - 256;
+     else if (when >= start)
+       {
+         tosend[1] = Y[when - start];          // count
+ 	tosend[2] = Y[when - start + window]; // sum
+       }
+     return encode(tosend);
+   }
+ 
+   any avg_buffer() make_string(6);
+ 
+   // Add some results from our descendants to our state
+   any avg_intercept(Y, intercepted) {
+     any decoded = decode(vector(2, 2, 2));
+     add_avg(Y, vector[0], vector[1], vector[2]);
+   }
+ 
  6) Configuring a TinySQL VM
  ===========================
***************
*** 86,89 ****
--- 380,385 ----
  # Pick a name and a directory for your TinySQL VM
  <VM NAME="TSQLVM" DESC="A TinySQL VM with micasb sensor board support." DIR="../../../../../../apps/TSQLVM">
+ # The amount of RAM available for TinySQL queries
+ # You can increase this on telosb motes
  <OPTION CAPSULE_SIZE=1024>
  <SEARCH PATH="../../../opcodes">