(a) Linear model.
Issue CL1500 Issue SDES1500 Issue COC
(b) Quadratic model.
6.2 OPTIMAL POLICY
The optimal policy was computed using a unit NMAC cost, an alert cost of 0.01, a reversal cost of 0.01, a strengthening cost of 0.009, and a clear-of-con.ict reward of 1 × 10.6. Five terminal cycles were
used.
Figure
22
depicts
two
sections
of
the
optimal
policy
when
both
aircraft
are
.ying
level,
DES1500 is active, and the pilot is responding to it.
Figure
22(a)
shows
the
optimal
policy
computed
using
the
linear
pilot
response
model.
The
blue region indicates the best action is to continue issuing the descend advisory. The descend advisory is typically maintained when the intruder is above the own aircraft. However, for large values of τ , the descend advisory is continued even when the own aircraft is above because there is su.cient time for the own aircraft to safely pass below the intruder. In the teal region, the optimal policy is to reverse the descend to a climb when the own aircraft is above the intruder, rendering the continued descent likely to induce NMAC. This region widens and narrows as τ changes. In certain regions of the state space, depicted in white, the best action is to discontinue the advisory.
Figure
22(b)
is
the
optimal
policy
computed
using
the
quadratic
pilot
response
model.
The
policy is similar to that of the .rst plot. Because the pilot will continue to follow the descend advisory with probability 3/4 even after it is switched to a climb, the reversal region has been expanded to allow additional time for the pilot to reverse and prevent NMAC. The reversal region is smaller for the linear model because the pilot will become unresponsive to all advisories with probability 3/4 when the reversal is issued, which proves to be a much safer course of action than continuing the descent.
The
purple
region
in
Fig.
22(b)
marks
the
places
in
the
state
space
where
the
optimal
action
is
to strengthen the descent. This region is absent in the .rst plot because the response of the pilot to strengthenings, together with the strengthening cost, makes continuing the advisory more advanta-geous than strengthening. Namely, because the pilot ignores all advisories with probability 3/4 and responds only with probability 1/4 after the strengthening is issued, it is unlikely that strengthen-ing can improve safety, especially when NMAC is imminent. The probability of responding to the strengthening is also 1/4 in the quadratic model. However, because the pilot continues his descent with probability 3/4, the probability of strengthening, though the same, causes strengthening to have a lower expected cost than continuing the descend advisory.
6.3 BELIEF STATE FILTERING
During the online execution of the logic, determining the alert to issue at each time step requires knowledge of the state of the resolution advisory, sRA. Unlike deterministic pilot response, even in the absence of sensor noise, there is uncertainty as to whether the pilot is responding to an advisory, thus making sRA not fully observable. Instead, a probability distribution, or belief state, over possible values of sRA must be maintained to summarize the beliefs regarding the response of the pilot to advisories. As new observations of the aircraft state are made each time step, the belief state b(sRA) is recursively updated using standard model-based .ltering techniques. The belief state is initialized at t =0 to b0(COC/COC) = 1. For all subsequent time steps t> 0, the
