Controlling the cart pole manually
Series of tutorials
This is a series of posts describing how to create an inverted pendulum (or cart pole system) with Unity and ML Agents.
Assumptions
- Finished reading second tutorial in the series.
- Have a Box and Cylinder connected by a Hinge Joint
What is the idea?
Now that we have our very basic model, to start implementing a Reinforcement Learning algorithm, we need to need two things, a way to make actions on the environment, and a way to observe the states that matter.
In our case, the action will be a force applied to the cart, just like in the mathematical model. And the states that we need to observe, are the position and angle of the pendulum.
Unity physics
Regarding the force, as we've added a rigidbody component both in the cart and the pendulum, Unity will give us a way to add forces, torque, movement to the objects in a very direct way.
To achieve this, you need to create a C# script. After that, just add another component in the cart, and search for the name of the script.
C# Moving the cart around
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class Test : MonoBehaviour
{
private Rigidbody cartRB;
void Start()
{
cartRB = GetComponent<Rigidbody>();
Vector3 testForce = new Vector3(100.0f, 0.0f, 0.0f);
cartRB.AddForce(testForce,ForceMode.Force);
}
void Update()
{
}
}
With this, unity will get the rigidbody component and then it will add a force, in this case
Vector3 testForce = new Vector3(100.0f, 0.0f, 0.0f);
The testForce will be 100 in the x direction, 0 on the y direction e 0 on the z direction.
Final Considerations
A part from just adding a force to the pendulum, it could also be done a way to control the pendulum with the arrow keys of the keyboard, this will be done once we'll move on to ML-Agents, as we can implement a Heuristics, where you try to train the pendulum.
A part from the actions, we also need to take into consideration how to observe the states, the position is straight forward, as unity give us out of the box the position of a rididbody, but the angle, most engines, such as [Unity] and Unreal Engine 4, use quaternions to get the angle of a rigidbody. So in the next post of the series, we'll be exploring quaternions and how to get those values in Unity.