Self-Learning

Note: I developed this project as part of a challenge project suggested by Dr. Alfredo Canziani in my Deep Learning course at New York University. I took the initial code that sets up a simulated environment, an agent (truck), and the agent’s dynamics within the environment from here. I designed/developed the training process and built two models that learn to back up the agent to the target position from any initial point. The training process, the agent’s behavior before and after the training, and the instructions for how to run the code are explained below.

Truck Kinematics

Truck

Training Emulator

Truck

Training Controller

Truck

Simulation Before Training

Simulation After Training

Simulation Beyond Training Boundaries

Trajectories

Trajectory 1

Trajectory 2

Trajectory 4

Trajectory 5

Trajectory 6

Trajectory 7

Trajectory 8

Trajectory 9

Trajectory 10

Run the Simulation

Create and activate the conda environment with:

conda env create -f conda_env.yaml

conda activate truck_backer_upper

To test the models inside the training region, run:

python truck-backer-upper.py

To test the models outside the training region, run:

python truck-backer-upper.py\
    --env_x_range 0 100\
    --env_y_range -30 30\
    --test_x_cab_range 40 90\
    --test_y_cab_range -25 25

To train both emulator and controller models, run:

python truck-backer-upper.py --train_emulator True

To train only controller models, run:

python truck-backer-upper.py --train_controller True

References

Nguyen, D., & Widrow, B. (1989). The truck backer-upper: an example of self-learning in neural networks. In International 1989 Joint Conference on Neural Networks (pp. 357–363, vol. 2). https://doi.org/10.1109/IJCNN.1989.118723
Schoenauer, M., & Ronald, E. (1994). Neuro-genetic truck backer-upper controller. In Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence (pp. 720–723, vol. 2). https://doi.org/10.1109/ICEC.1994.349969