A preliminary study was conducted to test the effectiveness of DERSNLP+EBL's method of plan merging through replay. This experiment compared DERSNLP+EBL both with and without increasing the justification for replay. The experimental setup consisted of training DERSNLP+EBL on a set of 20 randomly generated 4-goal training problems, and testing on a different set of 30 4-goal test problems. The initial state of each problem contained 12 locations (6 post offices and 6 airports) and 12 transport devices (6 planes and 6 trucks). In the training phase, the planner solved problems and stored the successful plan derivations in the case library. During the testing phase, the planner retrieved multiple stored plan derivations and used these as guidance in solving the test problems. DERSNLP+EBL was tested on the same 30 problems in both replay and from-scratch modes. Replay was either with (DERSNLP+EBL) or without (DERSNLP+EBL-IJ) increased justification. The results are shown in Table 3.
Although overall performance was poorer, the quality of plans in terms of number of steps improved with DERSNLP+EBL's strategy of increasing the justification for step addition. This result suggests that that DERSNLP+EBL's method of plan merging serves to reduce the redundancy in the plans produced through multi-case replay. Recently, Munoz-Avilla and Weberskirch (1997) have tested this non-redundant merging strategy in a process planning domain and have found a similar improvements in plan size. The next section describes an evaluation of the full DERSNLP+EBL system.