top of page
Abstract Futuristic Background

Approximate Reinforcement Learning Methods for Cartpole Control - OpenAI gym

​

Tabular reinforcement learning methods like dynamic programming and monte-carlo simulations are suitable for simple and discrete environments, but when the states of the environment are continuous and/or very large, the tabular methods are limited and unpractical in terms of computational efficiency and memory storage. Quite frankly, most of our real-world environments are not discrete so approximate RL methods are better for these scenarios.

​

PROJECT INFO

PRODUCT

Software Solution

TIMELINE

4 Weeks

ROLE

Developer

IMPLEMETATION DETAILS

Approximate RL methods use supervised machine learning (traditional and deep learning) methods to approximate the state-action values. In this repo, traditional machine learning algorithms are used to approximate the state values.

More specifically, linear models (linear regression) is combined with RL methods (Q-learning and Monte-Carlo) to approximate the state-action values, the radial basis function (RBF) is used for feature engineering to make the linear model approximate non-linear functions. These algorithms are implemented on the cartpole environment in open-AI gym.

Batch gradient descent with Monte Carlo is also applied to the cartpole environment in this repo.

​

​

TOOLS AND ALGORITHMS

Programming Language

  • Python NumPy Stack (NumPy, SciPy, Pandas, Matplotlib), Object-oriented programming.

ML Algorithms

  • Reinforcement Learning: Q-learning & Monte-Carlo, Linear regression modelRadial Basis Function.

ML Frameworks

  • Scikit learn, OpenAI gym.

bottom of page