Behaviors, Policies, and Activities

Behaviors, Policies, and Activities
by Jonathan Connell

It is often useful to have a more structured control system than just free-form C code. What we have found convenient is a three level hierarchy of behaviors, policies, and activities. These granularities correspond well to various types of code reusability and they modularize the functionality in a way that makes it easy to debug a controller one piece at a time.

The basic level of control is a behavior. Typically a behavior only controls one actuator modality (i.e. NOT both steering and translation speed). This makes it easier to re-use them in other controllers. For a specific sensory pattern a behavior recommends a particular actuator control value. This part of the code is the "transfer function". If a behaviors does not know what to do it should remain silent and generate no control value. This gating is accomplished by a binary "applicability predicate". Sometimes it is useful to have some time constants controlling how long a behavior is active. These are implemented through self-decaying state variables called "monostables". A monostable returns to zero after its triggering condition has been false for a given amount of time.

Above behaviors are policies. These are collections of behaviors which in combination give the robot some particular skill. Policies principally fuse the commands from different behaviors. Some of this command arbitration occurs automatically via the individual applicability predicates of the behaviors. Still, it is not unusual for different behaviors to generate conflicting commands. To resolve this we impose a fixed priority "suppression network" in which one behavior always wins for each actuator. This is typically implemented just by the sequence of behavior calls -- they are called in order from least important to most important. Each behavior affects some global control variables and the most important behavior for each modality thus has the "last word" concerning what values are set.

Finally there are activities. These are sequences of policies which the robot shifts between based on certain discrete events. They are implemented like finite state automata, typically with a few bits of persistent state. They can also be recursively embedded so a high level activity can be composed of lower level activities. A particularly useful interaction is to have the high level activity automatically terminate the lower level one when some goal condition has been achieved or when a timeout occurs. In general, the actual details of the policies and transition rules used by an activity are hidden from its caller, however its current state is visible. This is useful for determining whether progress is being made. It also allows us to use an activity to determine the condition of the environment -- the final state it arrives at codes the condition.

Many behaviors take raw sensory values as their inputs and produce hardware specific control values as outputs. Yet in certain cases it makes sense to connect to higher-level virtual sensors and actuators instead. These can be constructed using policies which work with specific global variables instead of base-level I/O commands. For instance, for a legged robot it is very useful to have a virtual actuator like "travel-direction". This way strategic guidance behaviors do not have to understand all the interactions between legs or worry about dynamic balance. Similarly, it is often advantageous to have a generic virtual sensor like "target-location". Many different policies can set this variable based on the current circumstances and goals, yet the same back-end policies for pursuing the target can be used irregardless of its derivation.

There are a number of examples of this style of programming that run on the Phoenix mobile robot. The file wander2 shows the use of behaviors in a simple policy for travelling around without hitting obstacles. The file virtual shows how a higher-level sensor and actuator interface can be grafted on top of an existing I/O system. Finally, the file stalk shows a complicated multi-phase controller using activities and the virtual interface. This program waits for a button to be pressed then roams around looking for a dark place. When a suitably dark location is found the robot turns around to face outward. Then, after a brief pause, it charges out and deliberately tries to smash into things.

Return to Phoenix - Back to Johuco