Robotic Arm Forward Dynamics Learning Using Deep Q-Networks and Proximal Policy Optimization

This project involves developing advanced models for learning the forward dynamics of a robotic arm using reinforcement learning algorithms, specifically Deep Q-Networks (DQN) and Proximal Policy Optimization (PPO). The objective is to accurately predict the future states of a 2-link robotic arm based on applied torques and current states.

Key Features:
Deep Q-Learning (DQN) Implementation:
- Designed and implemented a Q-network to approximate Q-values for discrete actions. The network consists of three fully connected layers. Implemented a replay buffer to store and sample experience tuples for training.
- Developed a method to convert discrete actions to continuous actions suitable for the arm environment.
- Created a training loop to optimize the Q-network using experience replay and target network updates. Trained the network over multiple episodes, periodically updating the target network and saving model checkpoints.
- Evaluated the trained DQN model in the arm environment, assessing its accuracy in predicting the arm's future states.

Proximal Policy Optimization (PPO) Implementation:
- Utilized Stable-Baselines3 to create parallel environments, enhancing training efficiency.
- Implemented PPO using the provided MlpPolicy from Stable-Baselines3. Trained the PPO model over a specified number of timesteps, saving the model checkpoint at the end.
- Tested the trained PPO model in the arm environment, evaluating its performance in predicting the forward dynamics of the robotic arm.