A Spatially and Temporally Attentive Joint Trajectory Prediction Framework for Modeling Vessel Intent

Ships, or vessels, often sail in and out of cluttered environments over the course of their trajectories. Safe navigation in such cluttered scenarios requires an accurate estimation of the intent of neighboring vessels and their effect on the self and vice-versa well into the future. We propose a spatially and temporally attentive LSTM-based encoder-decoder model that is able to predict future trajectories jointly for all ships in the frame.

For three agents in a frame as shown below, the trajectory of the red agent is influenced by that of the other two. The spatial attention mechanism hence weighs the hidden states of these neighbors based on their influence and uses the weighted sum at the next time step.

Spatial Attention Mechanism

To model the spatial influence of neighbors on a vessel of interest and incorporate the influence on the vessel's trajectory, we introduce a spatial attention mechanism .

Temporal Attention Mechanism

In the decoder, we also interleave a temporal attention mechanism with the spatial attention mechanism, to enable the model to inform prediction using previously observed spatial situations.

Spatial Influence

We define a trainable parameter, called domain in our spatial attention mechanism. For an agent attempting to navigate safely in a crowded environment, the agent’s domain can be defined as the safe space surrounding the agent, the intrusion of which by any neighboring agent would cause both to have a direct impact on each other’s future intent.

On training on AIS Data (https://marinecadastre.gov/ais/) from January 2017, our model infers the ship domain as:

Below is an example of the spatial influence computed by our model for 2 nearly similar scenarios. The size of the blue circle is directly proportional to the model inferred spatial influence of that vessel on the neighbor.

Implementation Details

Step 1: Download Data

To download data run:

sh download_data.sh

This downloads AIS Data from January 2017 from https://marinecadastre.gov/data/ and saves it in data_processing/raw_data/.

Step 2: Preprocess the data

The raw data contains duplicate MMSIs, missing/invalid heading values, etc. that need to be removed. Further, all the vessels transmit AIS data at different frequencies, but for feeding the data into our model, we need to resample all data to 1 minute intervals. For doing this, run:

python preprocess_data.py --zone

Every file contains data corresponding to a Zone. Each vessel is associated with its trajectory over timestamps, i.e. latitude and longitude positions and speed and heading values. Each UTM Zone spans 6° of longitude and 8° of latitude (approximately grid squares of 100 km). For simplicity, we (optionally) split each zone into smaller grids. For example:

python grid.py --zone=11 --grid_size=0.05 

Step 3: Train a model

There are four models to choose from: a vanilla LSTM , a spatially attentive LSTM, a temporally attentive LSTM, a spatially and temporally attentive LSTM.

To train a new model with our best hyper-parameters, run:

sh scripts/train.sh <model_type> 

Step 4: Test a trained model

To test a trained model, run:

sh scripts/test.sh <model_type>