[Tinyos Core WG] Fwd: [Tinyos-devel] message_t

[Miklos Maroti]

Guys,

> Fixed format packets: the goal of message_t and mx structs was to
> eliminate error prone runtime parsing and to avoid copies.  The
> challenge is that it is extremely difficult to represent a series of
> TLVs or the analogous sequence of fields is very difficult to fit into
> the struct static parsing model.  We are seeing this at layer 2 with
> short and long mac addresses, layer 2.5 with lowpan, layer 3 and 4
> with IP, and layer 7 as soon as you include a sequence of attribute/
> values like in almost all the industrial instrumentation protocols.
> You need at least an iterator to run down the sequence of fields.  I
> would love to not reproduce the tons of broken code that exists
> outside tinyos with all the broken htons and noths and bogus parsers.
> But I don't see how just nested structs is enough.

Simple nested structs are not enough. Conceptually, the networking
stack is built up from a bunch of components that form a tree. The
root of the tree at the top is the physical layer, and the leafs at
the bottom are application components. Each component wants to add
some extra header, footer and metadata fields to the message (e.g the
ActiveMessage component is a simple fan out based on the am type, and
the packet link layer is jut wants to add some the retry count
metadata). The send command propagates up in this tree, the receive
event propagates down, at each branch some decision is made to which
branch to follow or to call both in case of snoop. Each component
provides a set of interfaces with which others can manipulate the
header/footer and metadata sections (e.g. PacketLink,
PacketAcknowledgements, LowPowerListening) and interacts with other
components via these packet interfaces (e.g. PacketLink uses
PacketAcknowledgement). The AMPacket interface is special because, as
David pointed out, it is a mix of the link layer (address part) and
service and point (am type), but that layer/interface should be broken
up into two.

The nested structs 1) forces a rigid structure on the networking stack
(linear tree) thus prevents other interesting composition of the
networking components, and 2) components with variable sized
header/footer/metadata cannot be built. I have proposed a solution to
these two problems at the core workgroup:

- Each networking component should use and provide a simple interface
expressing the size of the header/payload/metadata of the message as
viewed from that point in the tree (e.g.
http://tinyos.cvs.sourceforge.net/viewvc/*checkout*/tinyos/tinyos-2.x/tos/chips/rf2xx/util/RadioPacket.nc)

- Variable sized headers/footers/metadata can be parsed since the size
of the header/payload/metadata getter methods take a message_t
pointer, which works perfectly fine for the received messages. There
are two solutions to change the length of the header for transmitted
messages: 1) memmove the payload to make room, or 2) implement add a
bitfield parameter to the clear() command, that describes the optional
parameters (e.g. 64-bit addresses) that will be used, and formats the
message accordingly.

- All send interfaces should be moved to a BareSend that does not take
any parameters other than the message (e.g.
http://tinyos.cvs.sourceforge.net/viewvc/*checkout*/tinyos/tinyos-2.x/tos/chips/rf2xx/util/BareSend.nc)
It requires a fully formatted message (thus not even the payload
length parameter is passed. This allows new networking components to
be seamlessly inserted into the network stack at any point without
breaking any of the other interfaces. Also avoids the practice used
e.g. at AMSenderC where the am type first is set in the message but
will be read again just to be able to call the lower AMSend interface
which again will set it. The length parameter is also problematic: the
message is perfectly capable to store what the length is, but we e.g.
in PacketLink we store the length separately just to be able to resend
the message because the lower level requires/resets the length again
and again and again.

- The use of XXXSend interfaces is just not flexible, you cannot stack
them well (since if you add a new layer in between two existing ones
then you need yet another YYYSend interface that takes the extra
parameters), and you cannot store the parameters with the message when
you need to call a lower layer. Instead I think all messages should be
fully assembled and then passed around (with no extra parameters, if
you need one then put it into the metadata just like packet link).

- The circular buffer is not a solution, since when you need to
enlarge [header2] in an existing message [header 1][header
2][header3][payload], e.g. when you resend the message to a 64-bit
address, then you either need to move the [header3[[payload] part or
you need to move the [header 1] part. Admittedly, the [header 1] part
is short for Ieee154, but does this worth the extra overhead of getter
functions for all fields in the header that needs to know about byte
boundary wrapover for 32-bit fields? It is not enough to add push/pop
interfaces because you want to access the header fields even when you
are not sending/receiving the message.

Miklos