This is an update for v0.3 of Space War RL! You can see a summary of my changes in my releases. You can view my kanban here.
Making Ship Rotation more Discrete
In the emulator, I noticed each keypress of the rotation keybinding would rotate the ship by 22.5 degrees (4 key presses for a 90 degree turn).
The current rotation handling logic is being called from the player's update function, which gets called every frame. I was using pygame.key.get_pressed, which is only used to check which keys are currently pressed down. What I want is to detect when the rotation key is pressed down and increment or decrement the angle by 22.5. I will need to check the rotation keybinding in the event loop.
while running:
# event loop
# pygame.QUIT event means the user clicked X to close your window
for event in pygame.event.get():
player_1.handle_events(event)
if event.type == pygame.QUIT:
running = False
def handle_events(self, event: Event):
"""Handles keyboard input to update ship's rotation and position state"""
if event.type == KEYDOWN:
...
if event.key == K_a:
self.ang -= 22.5
if event.key == K_d:
self.ang += 22.5
self.ang = self.ang % 360
I also added pygame.key.set_repeat(100, 100) in main.py#L14 to allow the KEYDOWN event to be repeated every 100ms, so I can hold down the rotation keybinding for multiple rotations. Otherwise it is only triggered once per keypress.
Demo
A possible alternate implementation is using numpy and indexing through the array.
>>> import numpy as np
>>> np.arange(0, 360, 22.5)
array([ 0. , 22.5, 45. , 67.5, 90. , 112.5, 135. , 157.5, 180. ,
202.5, 225. , 247.5, 270. , 292.5, 315. , 337.5])
I noticed my implementation didn't handle multiple keypresses and release correctly. I took a look at the emulator and it's not perfect either.
It looks like the check for D is overriding the check for A. This is the pseudocode I'm thinking that's causing this behavior in the emulator.
IF (D is down) THEN
rotate clockwise
ELSE IF (A is down) THEN
rotate counterclockwise
The check for D appears to be first causing inconsistent behavior. I want to see if I can check my Dad's computer to see if this is in the original game.
I decided not to copy this behavior and fixed it in my implementation.
def handle_events(self, event: Event, check_key_event: Event):
"""Handles keyboard input to update ship's rotation and position
state
"""
if event.type == KEYDOWN:
# Handle ship movement and rotation
if event.key == K_a:
self.rotate_ccw_lock = True
if event.key == K_d:
self.rotate_ccw_lock = False
if event.type == KEYUP:
if event.key == K_a:
self.rotate_ccw_lock = False
if event.key == K_d:
self.rotate_ccw_lock = True
if event.type == check_key_event:
keys = key.get_pressed()
rotate_ccw = keys[K_a] and self.rotate_ccw_lock
rotate_cc = keys[K_d] and not self.rotate_ccw_lock
...
# Update rotation
if rotate_ccw:
self.ang -= 22.5
elif rotate_cc:
self.ang += 22.5
self.ang %= 360
I am using the the self.rotate_ccw_lock field as a way to detect when the A or D is first pressed or released.
For input check polling, I replaced the pygame.key.set_repeat function with a custom user event to check user input.
# create custom event to check user input
check_key_event = pygame.USEREVENT + 1
pygame.time.set_timer(check_key_event, CHECK_KEYS_TIME_DELAY_MS)
...
while running:
# event loop
for event in pygame.event.get():
player_one.handle_events(event, check_key_event)
Demo
Adding Zero Gravity Physics
From my previous blog post, I discovered this formula to calculate the new relative x and y coordinates:
delta_y = dist * sin(angle)
delta_x = dist * cos(angle)
To add velocity, the player just needs to keep track of its net x and y velocity and each frame would add the net velocities to the player's current position.
def handle_events(self, event: Event, check_key_event: Event):
"""Handles keyboard input to update ship's rotation and position
state
"""
...
if event.type == check_key_event:
keys = key.get_pressed()
...
accelerate = keys[K_s]
# Update acceleration
if accelerate:
self.x_vel += math.cos(self.ang * math.pi / 180)
self.y_vel += math.sin(self.ang * math.pi / 180)
...
def update(self, *args, **kwargs):
"""Entrypoint for updating the player state each frame"""
x_pos, y_pos = self.pos
...
# Apply velocity to position
x_pos += self.x_vel
y_pos += self.y_vel
self.rect.x = x_pos
self.rect.y = y_pos
self.pos = (x_pos, y_pos)
...
After making these changes, I noticed a design pattern has emerged.
- In the event loop, the
handle_eventsfunction is used to update the player's internal state from user input. E.g. calculate changes in the netxandyvelocity as well as theangle. - The
updatefunction is used to apply the values of the internal state variables to the environment. E.g. updating the player's position from net velocity and applying rotation to the surface from the angle.
Demo
Whoops, it is easy to go out of bounds now! I will be adding screen wrap-around for this for now, since that is present in the original game. In other environment simulations, the bounds are restricted, so it is like the agent is in a box. Trying to move beyond the boundaries will place the agent at the boundary. Another method in RL is to punish the agent from going out of bounds via a negative reward. The agent can still go out of bounds, but hopefully it learns not to do that with training. The latter is more realistic.
Adding Screen Wrap-Around
def update(self, *args, **kwargs):
"""Entrypoint for updating the player state each frame"""
x_pos, y_pos = self.pos
# Conditions for wrapping around the screen
if x_pos >= SCREEN_WIDTH + 10:
x_pos = 0
elif x_pos <= -10:
x_pos = SCREEN_WIDTH
if y_pos >= SCREEN_HEIGHT + 10:
y_pos = 0
elif y_pos <= -10:
y_pos = SCREEN_HEIGHT
Demo
Thank you for reading my blog! My next blog post will discuss adding weapons to the agent.
Top comments (0)