Approximate Reinforcement Learning Methods for Cartpole Control - OpenAI gym
​
Tabular reinforcement learning methods like dynamic programming and monte-carlo simulations are suitable for simple and discrete environments, but when the states of the environment are continuous and/or very large, the tabular methods are limited and unpractical in terms of computational efficiency and memory storage. Quite frankly, most of our real-world environments are not discrete so approximate RL methods are better for these scenarios.
​
PROJECT INFO
PRODUCT
Software Solution
TIMELINE
4 Weeks
ROLE
Developer
IMPLEMETATION DETAILS
Approximate RL methods use supervised machine learning (traditional and deep learning) methods to approximate the state-action values. In this repo, traditional machine learning algorithms are used to approximate the state values.
More specifically, linear models (linear regression) is combined with RL methods (Q-learning and Monte-Carlo) to approximate the state-action values, the radial basis function (RBF) is used for feature engineering to make the linear model approximate non-linear functions. These algorithms are implemented on the cartpole environment in open-AI gym.
Batch gradient descent with Monte Carlo is also applied to the cartpole environment in this repo.
​
​
TOOLS AND ALGORITHMS
Programming Language
-
Python NumPy Stack (NumPy, SciPy, Pandas, Matplotlib), Object-oriented programming.
ML Algorithms
-
Reinforcement Learning: Q-learning & Monte-Carlo, Linear regression model, Radial Basis Function.
ML Frameworks
-
Scikit learn, OpenAI gym.