abp
Class AdaptiveProcess

java.lang.Object
  abp.AdaptiveProcess

All Implemented Interfaces:

java.io.Serializable

public final class AdaptiveProcess
extends java.lang.Object
implements java.io.Serializable

An adaptive process is a group of adaptives that work in tandem towards some common goal. Each adaptive might represent a specific choice we have to make in order to progress towards some goal.

Users construct an apdaptive process via the init methods. The init(File) method gives the adaptive process a backing store so that upon shutdown, the process is saved to file (see the init methods for details).

New adaptives are created via the initAdaptive method.

Users indicate success or failure of the adaptation process via the reward method.

See Also:

Serialized Form

Method Summary

void	disableAutosave() In general persisted adaptive processes (those constructed with init(File)) will be automatically saved when the program shuts down; this disables that feature.
void	disableLearning() Disables learning on all adaptives managed by this process.
void	dumpTable(java.io.OutputStream os) Dumps some representation of the current values in the table (as HTML) WARNING: This method is experimental and may be removed in later releases.
void	enableLearning() Enables learning on all adaptives managed by this process.
void	endEpisode() Separates one sequence of choices and rewards from another independent sequence of choices and rewards in the same process during learning.
static AdaptiveProcess	init() Initialize a temporary adaptive process that is not automatically persisted to file when the program terminates.
static AdaptiveProcess	init(java.io.File file) Constructs a new adaptive from the contents of a file.
<C,A> Adaptive<C,A>	initAdaptive(java.lang.Class<C> contextClass, java.lang.Class<A> actionClass) This constructs a new adaptive or binds to an existing one.
void	reward(double r) Specifies a numeric value as feedback for the choices made during learning.
void	save(java.io.File f) Persists the adaptive process to file explicitly.
void	setSeed(long seed) Sets this initial random seed to be used in all freshly constructed random number generators in the system.
long	totalEpisodes() The total number of times that endEpisode has been called.
double	totalReward() Returns the total reward this process is accumulated for all program runs.
double	totalRewardSingleRun() Returns the total reward sum on this run.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

init

public static AdaptiveProcess init()

Initialize a temporary adaptive process that is not automatically persisted to file when the program terminates. (However, the save method may still be used explicitly.)

See Also:

init(File)

init

public static AdaptiveProcess init(java.io.File file)
                            throws java.io.IOException

Constructs a new adaptive from the contents of a file. Adaptives are created via initAdaptive. Note, if the file does not exist, it will be created upon shutdown. If the program terminates normally and the shutdown handlers run, then the save method is automatically called and this adaptive is persisted to file. Note, if you use this construction process, you must create your adaptives in a deterministic order each time the program is run. That is, adaptives are stored positionally in the order they were created the first time the program is run.

Throws:

java.io.IOException - if the file exists, but is somehow malformed or some other IO error is encountered

See Also:

save(java.io.File), init()

initAdaptive

public <C,A> Adaptive<C,A> initAdaptive(java.lang.Class<C> contextClass,
                                        java.lang.Class<A> actionClass)

This constructs a new adaptive or binds to an existing one. If the adaptive process is not persisted, this will return fresh adaptives. But if the process just reloaded via init(File), then this will return the adaptives in their initial creation order.

totalEpisodes

public long totalEpisodes()

The total number of times that endEpisode has been called.

See Also:

endEpisode()

totalRewardSingleRun

public double totalRewardSingleRun()

Returns the total reward sum on this run. This does not include the sum from previous program runs, if an adaptive is reloaded, this is zeroed. WARNING: This method is experimental and may be removed in later releases.

totalReward

public double totalReward()

Returns the total reward this process is accumulated for all program runs. WARNING: This method is experimental and may be removed in later releases.

setSeed

public void setSeed(long seed)

Sets this initial random seed to be used in all freshly constructed random number generators in the system. Note, this does not affect random number generators already constructed just new ones (deserialized adaptives are not affected). The intended use is to set this value after constructing the adaptive process, but before it's use. A value of -1L indicates a non-deterministic seed (e.g. the system timer) should be used.

save

public void save(java.io.File f)
          throws java.io.IOException

Persists the adaptive process to file explicitly. This saves all adaptives and their current learning mode (see disableLearning()). If the program terminates normally and the adaptive process was created via AdaptiveProcess.init(File), then calling this is unnecessary as we add a shutdown hook to save back to that file upon program shutdown.

Throws:

java.io.IOException

See Also:

init()

disableLearning

public void disableLearning()

Disables learning on all adaptives managed by this process. See

Adaptive.disableLearning

for a discussion of what this means.

See Also:

Adaptive.enableLearning(), enableLearning()

enableLearning

public void enableLearning()

Enables learning on all adaptives managed by this process. See

Adaptive.disableLearning

for a discussion of what this means.

See Also:

Adaptive.disableLearning(), disableLearning()

disableAutosave

public void disableAutosave()

In general persisted adaptive processes (those constructed with init(File)) will be automatically saved when the program shuts down; this disables that feature.

See Also:

init(File)

reward

public void reward(double r)

Specifies a numeric value as feedback for the choices made during learning. Higher values are considered positive feedback and lower values indicate. The exact method assigning the reward to suggests is implementation depedent and discussed in the technical report.

See Also:

Adaptive.suggest(C, java.util.Set)

endEpisode

public void endEpisode()

Separates one sequence of choices and rewards from another independent sequence of choices and rewards in the same process during learning. This operation can be considered a barrier that prevents further rewards from being attributed to previous suggestions.

Suppose you have an adaptive process that represents some sequence of choices in a game (like tic-tac-toe or connect four), but want to be able to play multiple rounds. Then you might place a call to this method between each game so that rewards in later games are not applied to the choices made in prior rounds. (There are some games where you might use this construct multiple times in one round.)

Note, the effect of misuse, or failure to use this construct can vary between inefficiency of the learning algorithm or even inability to learn.

For more details see the technical report.

dumpTable

public void dumpTable(java.io.OutputStream os)
               throws java.io.IOException

Dumps some representation of the current values in the table (as HTML) WARNING: This method is experimental and may be removed in later releases.

Throws:

java.io.IOException