[net2-wg] Packet interface

[Philip Levis]

One of the things that the 2.x WG is going to start wrestling with is
the packet interface in a TinyOS system. On one hand, simple, low-rate
protocols can work fine with ActiveMessageC. On the other, the need for
better energy efficiency and protocol cooperation might push things
towards a more expressive and powerful interface.

(This is mostly a copy of a post I just sent the 2.x WG. This interface
straddles the two WGs, etc.)

The starting point for this is Joe Polastre's work on SP (in the recent
SenSys). Jonathan Hui and I sat down on Sunday and worked through a
first shot at a proposal for what an interface might look like. Here's
what we came up with:

interface SendPoolEntry {
   command error_t start(message_t* first);
   command error_t stop();
   event void stopped(message_t* last, error_t err);
   /* Returning NULL will stop the entry */
   async event message_t* next(message_t* sent);
   async command bool isRunning();

   command uint8_t getNumFutures();
   async command error_t adjustFutures(int8_t num);
   async command void clearFutures();

   command tos_addr_t getDestination();
   command error_t setDestination(tos_addr_t val);
   command bool isUrgent();
   command error_t setUrgent(bool urgent);
   command bool isReliable();
   command error_t setReliable(bool reliable);
}

The basic idea is that SendPoolEntry, like Timer, is a Service Instance.
That is, each instance of SendPoolEntry corresponds to an instance of a
service (sending packets) and its associated data structures.

Rather than tell the radio stack "send this packet," you tell it "I'd
like to send a packet, when you next get a chance." The radio stack
looks at all of the active send pool entries and makes
scheduling/transmission decisions based on their state as well as what
packets it hears. One part of this is the fact that a component can tell
the radio stack "I'd like to send N packets." This is the messageFutures
part (more on this later).

Because the send pool entry is a data structure, you're expected to
configure it. E.g., set the destination address of the pool entry, set
whether its packets are urgent or not, set whether they should be sent
reliably.

An entry can be either active or inactive (isRunning()). By default, an
entry is inactive. You activate the entry by calling start() and giving
it the first packet to send. If you've specified futures, then the stack
will start issuing next() events, which perform a buffer swap in the
transmit direction. Each next() event decrements the futures by 1. When
there are no futures left, then the stack issues the stopped() event and
the entry becomes inactive again. The stopped() event returns the last
packet that the pool entry had to send.

A component can stop the entry early by returning NULL on a next()
event, calling clearFutures(), or calling stop(). The latter two might
not stop the entry immediately. The first (next()) is the way in which
one can stop the entry immediately in an async context. It is also how
the upper layer stops the entry if it can't generate packets fast
enough.

clearFutures() sets the number of futures to 0. adjustFutures() allows a
component to either increment or decrement the number of futures.
Futures can never be below zero: anything that causes it to become
negative means zero.

Besides futures, changing the configuration of the entry (destination,
urgent, reliable) while it is active always returns FAIL. You can only
reconfigure the entry when it is inactive.

Basically, a send pool entry requests packets from above, allowing
components on top to decide on queuing policies and/or how to generate
packets.

Note that "reliable" doesn't mean guaranteed, rather "try harder" and
"tell me whether it got there."

There's also an interface for getting per-packet info:

interface SendPoolMsg {
   async command bool congestion(message_t* msg);
   async command bool received(message_t* msg);
   async command bool urgent(message_t* msg);
   async command bool reliable(message_t* msg);
   async command sp_time_t timestamp(message_t* msg);
}

So here would be a sample use. The nesC might not be perfect.

interface SendBigThing {
  command error_t send(uint8_t* ptr, uint16_t len);
  event void sendDone(uint8_t* ptr, error_t success);
}

module MyBestEffortFragmenter {
  provides interface SendBigThing;
  uses interface SendPoolEntry as Entry;
  uses interface SendPoolMsg;
  uses interface Packet;
}
implementation {
  uint8_t* data;
  uint16_t dataLen;
  uint16_t pos;

  TOS_Msg packets[2];
  TOS_MsgPtr current = &packets[0];
  TOS_MsgPtr next = &packets[1];  
  bool ready = FALSE;

  mySetupCode() {
    call Entry.setDestination(SOMEBODY);
    call Entry.setUrgent(FALSE);
    call Entry.setReliable(TRUE);
  }
 
  command void SendBigThing.send(uint8_t* ptr, uint16_t len) {
    uint8_t fragLen = call Packet.getMaxPayloadLength(current);
    data = ptr;
    dataLen = len;
    pos = 0;
   
    call Entry.clearFutures();
    call Entry.adjustFutures((dataLen + fragLen - 1) / fragLen);
    memcpy(call Packet.getPayload(current), data, fragLen);
    pos += fragLen;
    memcpy(call Packet.getPayload(next), data+pos, fragLen);
    ready = TRUE;
    call Entry.start(current);
  }
 
  task void prepareNextPacket() {
    uint8_t fragLen = call Packet.getMaxPayloadLength(next);
    memcpy(call Packet.getPayload(next), data+pos, fragLen)
    pos += fragLen;
    atomic {
      ready = TRUE;
    }
  }

  async event message_t* next(message_t* last) {
     TOS_MsgPtr tmp = next;
     next = last;
     if (!ready) {
       current = tmp;
       return NULL;
     }
     ready = FALSE;
     post prepareNextPacket();
     return tmp;
  }

  event void stopped(message_t* last) {
    ready = FALSE;
    if (last != NULL) {
      current = tmp;
    }
    // Some logic in here that decides what SUCCESS means (all
fragments, etc.
  }    
}

Thoughts? Again, this is kind of complicated, but it's the powerful
interface that performance-critical protocols can use. ActiveMessageC is
basically just a pool entry which never has futures.

Phil