[Tinyos-devel] Storage Working Group

[David Moss]

I would like to join the storage group.

I have developed many tools for both the internal and external flash on all
mica* mote types and telosb mote types in TinyOS 1.x and 2.x.

Here's what I have developed in the way of storage for TinyOS 2.x.  These
tools were developed for use in deployable systems meeting my team's project
goals, but meant to be general enough to be used by anybody for any
deployment with any project:

--External Memory--
1. DirectStorage component
Developed a "DirectStorage" abstraction that is compliant with the
architecture described in TEP103 and cross-platform compatible, currently
running on mica* and telosb mote types.  This sits along side the other
storage "abstractions" defined in TEP103: BlockStorage, ConfigStorage, and
LogStorage; although, those other storage abstractions could be easily
implemented on top of DirectStorage with no performance impacts.
DirectStorage allows an application to directly access bytes on a volume of
flash to perform the following functions:

 DirectStorage (each of these commands returns an event):
 * read(addr, *buf, len)
 * write(addr, *buf, len)
 * flush()
 * erase(eraseUnit)
 * crc(addr, len)

The DirectStorage component also provides the interface "VolumeSettings",
which provides the following functions to extract information about the type
of flash applications are running on.  Knowing the properties of the flash
chip your app is usually essential to developing cross-platform compatible
flash tools, libraries, and applications.  Instead of hard-coding settings,
just ask the VolumeSettings interface instead:

 VolumeSettings:
 * uint32_t getVolumeSize()
 * uint32_t getTotalEraseUnits()
 * uint32_t getEraseUnitSize()
 * uint32_t getTotalWriteUnits()
 * uint32_t getWriteUnitSize()
 * uint8_t getFillByte()
 * uint8_t getEraseUnitSizeLog2()
 * uint8_t getWriteUnitSizeLog2()

Furthermore, some memory types can be modified - i.e. EEPROM or flash with
small erase unit sizes.  DirectStorage provides the interface "DirectModify"
to allow applications to modify already-written bytes:

 DirectModify:
 * modify(addr, *buf, len)    => modified(...)
 * bool isSupported()

The DirectStorage abstraction and its interfaces (DirectStorage,
DirectModify, and VolumeSettings) are meant to be implementable on any media
type, including EEPROM, NOR-flash, and NAND-flash.  Again, right now it's
working on EEPROM and NOR-flash just fine, and motes with NAND flash should
find the implementation requirements easy to meet.


2. Blackbook File System
Ported the Blackbook file system from TinyOS 1.x to 2.x, taking advantage of
the new DirectStorage abstraction.  Blackbook provides several core
interfaces, and because it was built on top of DirectStorage, it is
cross-platform compatible.  Blackbook will store any kind of data
(configuration, sensor, regular files, etc.), but is not necessarily meant
to be used as a circular-log (although it could).

 Blackbook Interfaces
 * BFileRead
 * BFileWrite
 * BFileDir
 * BFileDelete
 * BDictionary

There are other interfaces it allows an application to access, such as
Bclean, but those are the main ones.  Read, Write, Delete, Dir all work as
you would expect a file system to work.  The Dictionary is a special file
type that allows an application to store and update key-value pairs of
information on flash.  A key is any uint16_t, and a value is any type and
size.  This allows for storing lots of information, like configuration data
or indexed information.

Because Blackbook is a general tool for storing information on flash, it is
currently being used in at least two deployments my team is working on.  To
make sure it is reliable enough for deployment, we are in the process of
developing a suite of automated regression tools to exercise Blackbook to
the maximum extent possible.


--Internal Memory--
3. Configuration Manager
The internal flash on the MSP430 and internal EEPROM on the atmega's is a
good place to store configuration data; however, it is difficult for more
than one component to share.  The Configurator component provides a way for
many components to share the internal memory and store small amounts of
configuration data.

Here's how it works:  On boot, the Configurator requests each of its clients
to login through an event.  Each component logs into the Configurator when
asked, specifying a place in RAM where configuration data should be stored
from and loaded into, and the amount of data it needs to store to memory.
The Configurator then divides up the internal memory appropriately.  If the
node is restarted, the manager starts up the same way so the internal memory
is still divided up the same way.

After all components have logged in, the Configurator is ready to go.  Each
component can then call store() and load(), which will retrieve that
component's data from memory or store data to memory.  The Configurator
component itself keeps track of where data should come from on internal
memory and where it should go to in RAM.


Alright, I think that's all I've got in the way of storage.  Accessing
external storage:  DirectStorage, DirectModify, VolumeSettings, Blackbook;
Internal storage: Configurator.  All of these libraries work on both mica*
and telosb.  I can provide these components as needed.

-David Moss




-----Original Message-----
From: tinyos-devel-bounces at Millennium.Berkeley.EDU
[mailto:tinyos-devel-bounces at Millennium.Berkeley.EDU] On Behalf Of Deepak
Ganesan
Sent: Thursday, November 30, 2006 3:39 PM
To: tinyos-devel at Millennium.Berkeley.EDU
Subject: [Tinyos-devel] Storage Working Group



All,

I would like to form a working group on storage. A draft charter of  
the working group is below.

Charter: To define storage interfaces and abstractions, reusable non- 
volatile data structures, interoperability requirements for  
components that share flash memory, and reference implementations of  
storage systems.

Please let me know if you are interested in participating in this WG  
and if you have thoughts on its charter/goals.

thanks
Deepak

_______________________________________________
Tinyos-devel mailing list
Tinyos-devel at Millennium.Berkeley.EDU
https://mail.millennium.berkeley.edu/cgi-bin/mailman/listinfo/tinyos-devel