Jhonatan da Silva

Jhonatan da Silva



  • In progress

Unity Model Analysis

With the modelling of the system done inside Unity. An analysis of the model was needed, such as changing the parameters, such as mass, drag, angular drag, both in the cart and in the pendulum, and check if it behave as we expect from the mathematical modelling.

The base parameters were

  Mass [kg] Drag Angular Drag
Cart 1 0 0.05
Pendulum 1 0.5 0.05

And the same force was applied to all the tests. With the characteristics of an impulse response.

Cart parameters analysis

Mass of the cart

The first simulation was regarding the mass of the cart. A simulation with the mass of the cart as 1kg, 5kg, 10kg and 50kg was made and this was the result both for the position of the cart and the angle of the pendulum in those simulations.

As expected, as we increase the mass of the car, applying the same force will result in different positions, as the accelaration of the cart will be less than the previous one. Regarding the pendulum, the same logic applies, when the mass of the cart increases, the displacement of the cart decreases and the angle of the pendulum decreases as well.

Drag of the cart

Sidenote: In Unity, as we are using rigid bodies, we can interact with the drag and the angular drag.

Pendulum parameter analysis

Length of the pendulum

Angular drag of the pendulum

Model training - Stabilization

mlagents-learn ./trainer_0.16.1.yaml --run-id stable_004 --resume
tensorboard --logdir summaries 

The final parameters of mass, drag and angular drag for the training

  Mass [kg] Drag Angular Drag
Cart 5 2 0
Pendulum 1 0 0.5

The file configuration for the training

  trainer_type: ppo
    batch_size: 64
    buffer_size: 12000
    learning_rate: 0.0003
    beta: 0.001
    epsilon: 0.2
    lambd: 0.95
    num_epoch: 3
    learning_rate_schedule: linear
    normalize: true
    hidden_units: 128
    num_layers: 2
    vis_encode_type: simple
      gamma: 0.99
      strength: 1.0
  keep_checkpoints: 5
  max_steps: 5.0e6
  time_horizon: 1000
  summary_freq: 1000
  threaded: true

Cumulative reward


Training response

number of points in the graph? frequency

Parameter variation

The model was trained according to the table x. In order to test if the model is robust, changes to the parameters of the cart and pendulum were made to test if it could still stabilize the model without retraining.

Cart parameter variation

Mass of the cart

Drag of the cart

Pendulum parameter variation

Length of the pendulum

Mass of the pendulum

Angular drag of the pendulum

Model training Swing Up