What's new

Proper way of stepping through frames as fast as possible


New member

I am currently developing an environment for using mupen64 with reinforcement learning techniques, similar to the one described by deep mind.

For that I need to do the following in a loop:
  • emulate one frame
  • wait until done, read screen, read memory
  • do some processing with the result

Everything is working pretty well so far (never really used C before in my life, but the api was really quite simple to understand), however I encountered a problem stepping through the frames. The whole thing is written in python, since it is intended to be used with tensorflow.
I mostly used the existing python front-end (m64py) as an example. My current approach looks roughly like this:

  • create a thread that loads the core, plugins, etc (as described in the wiki, "High-level Usage") and finally send the M64CMD_EXECUTE command
  • create a (synchronized) "frame_ready" flag
  • register a frame callback that sets the frame_ready flag
  • in another thread (pseudocode):
  frame_ready <- False
  wait until frame_ready = True

This works in most cases, however after some time the program gets stuck waiting for the frame_ready flag to be set. The problem becomes more pronounced under high CPU load, in my case when running many instances of the emulator at the same time. Manually unpausing the emulator fixes the problem for a short time until it occurs again.

My question is:
Is this the proper way to this in principle and these issues are likely due to my implementation (since it just happens sometimes and seemingly unpredictable, I suspect some kind of threading issue, but I am not the best programmer :))
Or am I using the emulator in the wrong way here, e.g. the frame callback is not intended to be used like this?

I noticed that when I use the state callback that can be passed to the core and instead of waiting for the frame ready flag simply wait for the emulator state to change to "paused" after i call advance frame, everything seems to work fine. However, this seems not-as-nice, and potentially less stable, a solution as the other one

If anybody has any advice regarding this problem, I would greatly appreciate it.
Thanks in advance, best regards and thanks for making such an easy to use emulator interface,