2.4 DISCUSSION
Before adopting an MDP framework for deriving collision avoidance logic, it is important to assess whether the assumptions made by the model are valid. One of the most signi.cant assumptions it makes is that the dynamics are Markovian, which means that the current state must contain all the information relevant to predict the distribution of states at the next decision point. In some problems, it is not practical to add all the state variables necessary to make the dynamics Markovian because it makes the discretized state space intractably large. Depending on the problem, it may be possible to approximate a seemingly non-Markovian system with a simpli.ed Markovian model and still obtain acceptable performance. One of the primary questions to be answered in this report is whether the collision avoidance problem can be adequately represented by an MDP that is simple enough to be solved in reasonable time while providing the desired safety and operational performance.
2In contexts where reward is to be maximized, as opposed to cost to be minimized, Q is often used in the literature
[35]
to
represent
the
state-action
value
function.
In
general,
Q(s, a)= .J(s, a).
3. COLLISION AVOIDANCE IN TWO SPATIAL DIMENSIONS
The previous section introduced MDPs as a framework for modeling sequential decision problems and DP as a solution method. This section presents an MDP formulation to collision avoidance that makes the following assumptions:
.
Decisions at 1Hz. Decisions are made once per second. This is exactly the same frequency at which the current version of TCAS makes decisions.
.
No horizontal maneuvering. The intruder is approaching head-on with a constant closure rate, and neither aircraft turns. Because the closure rate is assumed constant and there is no horizontal maneuvering during the encounter, the motion is essentially in two spatial dimensions:
altitude
and
horizontal
range.
This
assumption
is
relaxed
in
Section
5.
.
Deterministic pilot response. It is assumed that the pilot responds deterministically to the resolution advisories issued by the system. In reality, there is signi.cant variability in the response
of
pilots.
Section
6
shows
how
to
accommodate
pilot
response
variability
into
the
model.
.
Perfect sensors. The collision avoidance system has perfect state information. In reality, the sensor
system
is
likely
to
be
noisy.
This
assumption
is
relaxed
in
Section
7.
.
Uncoordinated. The intruder is not equipped with a collision avoidance system. If the intruder is also equipped with a collision avoidance system, it is important that their maneuvers be coordinated,
as
discussed
in
Section
8.
.
Single intruder. There is a single intruder approaching the own aircraft. Although multithreat scenarios
are
relatively
rare,
this
assumption
is
relaxed
in
Section
9.
Although this model is very simple, it serves as a base from which a more sophisticated collision avoidance system can be built, as discussed later in this report.
3.1 ACTION SPACE
The current version of TCAS issues advisories to the pilot through an aural annunciation, such as “climb, climb,” and through a visual display. The visual display varies, but it is typically implemented on an instantaneous vertical speed indicator or a vertical speed tape or pitch cues on the primary .ight display. The set of advisories issued by TCAS can be interpreted as target vertical rate ranges. If the current vertical rate is outside the target vertical rate range, the pilot should maneuver to come within the required range. If the current vertical rate is within the target range, a corrective maneuver is not required, but the pilot should be careful not to maneuver outside the range.
中国航空网 www.aero.cn
航空翻译 www.aviation.cn
本文链接地址：Robust Airborne Collision Avoidance through Dynamic Programm(14)
