IBM Torcs AI racing competition

Hello readers!

Me and my friend Patryk are taking part in the 2026 edition of IBM AI Racing Competition.

The goal of it is to develop an AI agent capable of driving a car in Torcs simulator on "Corkscrew" track and perform a full lap.

For our solution, we trained an TD3 RL neural network agent to drive the car.

Reinforcement Learning networks can control things and learn thru points - in a nutshell, they recieve positive points (reward) for doing something correctly, and negative points (punishment) for doing something incorrectly. They learn how to behave to maximize the reward. Of course, they lack understanding of purpose, so we have to program constrains and how the point system works ourselves. It is challenging, as they like to find workarounds to get free points or stop losing them. If the grading system is designed wrong, the neural network may end up learning to quickly fail or never do anything, for example. 

First, we pre-trained our network to drive thru the whole circuit without falling off-track from start to finish, and reduced maximum speed to a low one. After many hours of trial and error, TD3 learned to finish a lap.

Since it was driving very chaotically, next fine-tuning session focused on stability. The neural network learned to drive straight instead of turning all the time.

For next fine-tuning, we unlocked the max speed and implemented automatic gear switching. The AI model learned how to control speed, but became less effective in finishing a lap.

Final fine-tuning restored its ability to finish the whole track. The AI car could finally finish a whole race!

Since some random failures were still occuring, we decided to add a safety envelope for the AI system in form of a deterministic fallback. If the neural network stopped responding or the car fell off the track and the AI was unable to return back onto the road, the deterministic program takes control until the AI can drive again.

We used IBM Granite LLM model, more specifically Granite 4.0 H 32B, to help us understand how Torcs works, optimize code and consult solutions and grading system for the TD3.

We are thankful for the opportunity to take part in the competition. We wish best of luck to all other participants, and hope our solution will perform well.

Comments

Post a Comment

Popular posts from this blog

ISKRA experiment 02_10_2024-1 - "Defeat the Minecraft creeper" [reupload]

Image
ISKRA experiment 02_10_2024-1 - "Defeat the Minecraft Creeper" Hello everyone! In this post, I will describe my recent and first public ISKRA Project experiment. I have conducted some experiments before, but I believe they are not worth publishing. Let's consider this one the first official ISKRA Project experiment. Let's begin with the description of it. I've built an installation in Minecraft Bedrock Edition: This was the starting point and viewport of our AI: The goal of the AI was to defeat the Creeper by using a set of pre-defined control commands: W - move forward 1 block S - move backwards 1 block A - strafe left 1 block D - strafe right 1 block P - place a brown dirt block in front of you R - destroy block in front of you F - punch to attack/fight The commands P and R were never actually implemented since they were unnecessary for this experiment. The Creeper was trapped in a structure to prevent...
Read more

Pre-training a GPT-2 AI model on 32 GB of data. I made it both draw and recognise sketches!

Image
Hello everyone! In this post I will tell you about how I trained a GPT-2 "Small" AI model from scratch on a huge dataset of around 32 GB of raw text files with use of Torch and HuggingFace's Transformers libraries. I will explain steps of such a project, tell you about how it went for me and how I recommend doing these things now, and of course I will show you the code that pre-trains the AI model from a dataset too. Let's start from the beginning: 1. The idea, purpose and raw dataset First and foremost, a project like this needs an idea and should have an purpose. I pre-trained (trained from scratch) the AI model just to learn more about the process, but pre-training LLM AI models of that scale is done rarely and is usually pointless, as there are lot of base models already trained on specific languages that can then just be fine-tuned into a specific response format for example. But let's assume that we want to pre-train such an model anyway, for example as an a...
Read more

Making ChatGPT capable of video analysis - part 2 - adding speech recognition and more

Image
Hello. In this follow-up, we will continue development of the solution to let ChatGPT describe videos. In my approach, the video is split into fragments and then these fragments are turned into "comics". Then, every fragment gets described and all the descriptions are sent to the master AI agent, which interacts with the user.  Since the last post, I updated the code to make it more modular and move AI processiong out of initalizer: from moviepy.video.io.VideoFileClip import VideoFileClip from PIL import Image import os import easyapiopenai from WojSafeAPI import * #My own library for loading API keys safely, you cannot use it. class Video_ChatGPT: def __init__(self,openai_api_key:str, video_path: str, time_between_frames_seconds: float, frames_per_scene: int, process_audio=True, frame_height = 480, watcher_model='gpt-4o-mini',master_model='gpt-4o', watcher_token_limit = 500, master_token_limit=15*10...
Read more