Getting Game Boy Sound Emulation Right

[Sergio89]

Hello.
I'm currently working on a Game Boy/Game Boy Color emulator. More information can be found here: realboyemulator.wordpress.com

It is fairly stable (well, not so much for the Game Boy Color part), and it even passes the tests cpu_instrs, instr_timing and mem_timing. It also implements channels 1, 2 and 4, but I think I'm really missing something and I want to understand what it is. If you could test the emulator, you would notice that sound indeed is not very good.

I'll try now to describe where my doubts arises:

I use the SDL libraries, but this doesn't really matter. SDL creates a thread that calls a 'callback' function every time audio need updating. So, because sample sizes are 1470 (quite arbitrary) bytes, the function gets called 30 times per second when sampling at 44100hz. Right now, the 'callback' function just calls another function that fills the sound buffer according to the values in the sound registers.
Now, I have seen some implementations (VisualBoy's, for instance), and all of them seem to do some kind of synchronization that I can't figure out. For example, when writing to a sound register, a mutex is locked so the 'callback' function does not update sound state. Then, before actually writing the new value to the sound register, the update function is called (the same one that the 'callback' function calls to update sound state). But this time, the whole buffer (the 1470 bytes, for example) are not filled (maybe less samples are needed at that particular point); instead, a small portion is filled depending on the time that has passed since last update. Also, a variable of the likes of 'buffer_position' is used, so for the next update (either through the 'callback' function or the last-mentioned mechanism), the buffer starts filling at 'buffer_position' offset.

This synchronization mechanism is what I fail to understand. What could go wrong with my current implementation, which currently only updates the sound buffer when the 'callback' function is called, and not when a sound register is being updated?

Thanks a lot in advance.

 

[Cyberman]

Hello.

Greets

Hello. I use the SDL libraries, but this doesn't really matter. SDL creates a thread that calls a 'callback' function every time audio need updating. So, because sample sizes are 1470 (quite arbitrary) bytes, the function gets called 30 times per second when sampling at 44100hz. Right now, the 'callback' function just calls another function that fills the sound buffer according to the values in the sound registers.

This is in another thread correct? Do you have thread synchronization going on?

Now, I have seen some implementations (VisualBoy's, for instance), and all of them seem to do some kind of synchronization that I can't figure out. For example, when writing to a sound register, a mutex is locked so the 'callback' function does not update sound state. Then, before actually writing the new value to the sound register, the update function is called (the same one that the 'callback' function calls to update sound state). But this time, the whole buffer (the 1470 bytes, for example) are not filled (maybe less samples are needed at that particular point); instead, a small portion is filled depending on the time that has passed since last update. Also, a variable of the likes of 'buffer_position' is used, so for the next update (either through the 'callback' function or the last-mentioned mechanism), the buffer starts filling at 'buffer_position' offset.

Time is off the essence. Consider the data in the buffer needs to start when the registers change not when the potentially arbitrary thread gets executed.

Hello. This synchronization mechanism is what I fail to understand. What could go wrong with my current implementation, which currently only updates the sound buffer when the 'callback' function is called, and not when a sound register is being updated? Thanks a lot in advance.

Desynchronization, you are assuming that the sound thread will always be in sync with the register changes but it is entirely possible they might not (could be arbitrarily doing 'stuff').

A semaphore is oft handy too keep things in lock step. Remember the thread is a coroutine you are expecting to get called every 30th of a second (actually it's probably more like every 1/32 because windows does not sync things to the fast tick in the system but the slow one so 1024 is actually 64 time slices per second, and thus why the standard timer in windows can't fire off under 16ms (because it's the same as zero to the system which is OFF)). Anyhow basic thing is they have to remain in sync or strange things can happen.

They are just trying to make sure any sound played is actually because of something the emulator thread has done and not a glitch in the system (or in this case the thread that feeds the audio system). My thought on this is if the register that affects the sound changes during the sound buffer time slices you set a 'dirty' bit saying "this bit changed". That bit is cleared each time the thread is called and the audio update happens. You keep track of states of the registers locally to the thread so you know what register changed (they may not necessarily be different at all however hardware can behave in interesting ways when you write to a register just the act may for example reset a position in a buffer or something). Anyhow not an expert at the GBC so be sure you understand what would happen if a register was updated with the same information on a real GBC for example.

Not sure it's the best advice but I hope it can give you possible insite as to the why.

Cyb

 

[Sergio89]

Greets This is in another thread correct? Do you have thread synchronization going on?

Indeed; SDL creates this thread and then just calls the 'callback' function requesting more data according to the sampling rate and sample size. There is currently no communication/synchronization whatsoever between the (two) threads.

Time is off the essence. Consider the data in the buffer needs to start when the registers change not when the potentially arbitrary thread gets executed. Desynchronization, you are assuming that the sound thread will always be in sync with the register changes but it is entirely possible they might not (could be arbitrarily doing 'stuff').

I'm a bit confused on the details. You are essentially telling me that with my current implementation it is possible to 'miss' some sound? For example, suppose the 'callback' function has filled the buffer, copies it to the sound buffer, and returns. Shortly after, a sound register is updated (written to), a while later, another update occurs to a sound register... all this before the 'callback' function is called again for new data. Now, when the 'callback' function is called again, it fills the buffer according to the registers' values at that point, but all the data (sound) that was written (and perhaps overwritten several times) before it got called again was 'lost'. Is this indeed the scenario you are suggesting? I want to be sure I get this part right before continuing (this is what I have suspected, but wasn't really sure it was the case).
Thank you for your time.

 

[Cyberman]

Indeed; SDL creates this thread and then just calls the 'callback' function requesting more data according to the sampling rate and sample size. There is currently no communication/synchronization whatsoever between the (two) threads.

That can create strange problems especially when updating things such as GUI elements you can get a deadly embrace or odd behavior.

I'm a bit confused on the details. You are essentially telling me that with my current implementation it is possible to 'miss' some sound? For example, suppose the 'callback' function has filled the buffer, copies it to the sound buffer, and returns. Shortly after, a sound register is updated (written to), a while later, another update occurs to a sound register... all this before the 'callback' function is called again for new data. Now, when the 'callback' function is called again, it fills the buffer according to the registers' values at that point, but all the data (sound) that was written (and perhaps overwritten several times) before it got called again was 'lost'. Is this indeed the scenario you are suggesting? I want to be sure I get this part right before continuing (this is what I have suspected, but wasn't really sure it was the case).
Thank you for your time.

Well yes, that is kind of what I was refering too. It's less obvious on a high performance machine (or more depending on how synchronise the speed of each thread is).

Unfortunately thier is no easy way to do this. If I remember correctly what you want to do is have a "function" that is locks buffer access by the audio buffer stuffer thread whilst changes to registers are made. To prevent epic fails (stuttering) you may wish to double buffer this. That is have 2 buffers one that is currently feeding the audio and one that can have data put in it at anytime. The buffers are swaped when data is available in the next buffer and it's not being accessed (otherwise it's ok to have a stutted as it can indicate the problem). The hard part comes with register modifications and the buffer status. Not sure off hand what you could do with that. However that is the fun of emulating things

Sometimes emulation can have ... weird problems? I need to get back to my PS1 project. SIGH so many projects and no ... time whatsoever.

Cyb

 

[Sergio89]

That can create strange problems especially when updating things such as GUI elements you can get a deadly embrace or odd behavior.


Well yes, that is kind of what I was refering too. It's less obvious on a high performance machine (or more depending on how synchronise the speed of each thread is).

Unfortunately thier is no easy way to do this. If I remember correctly what you want to do is have a "function" that is locks buffer access by the audio buffer stuffer thread. To prevent epic fails (stuttering) you may wish to double buffer this. That is have 2 buffers one that is currently feeding the audio and one that can have data put in it at anytime. The buffers are swaped when data is available in the next buffer and it's not being accessed (otherwise it's ok to have a stutted as it can indicate the problem).

Sometimes emulation can have ... weird problems? I need to get back to my PS1 project. SIGH so many projects and no ... time whatsoever.

Cyb

Heh, no problem, man; thank you for your time.

I'm using the SDL interface. SDL takes care of double-buffering. It creates a thread that calls a callback function at fixed intervals depending on the frequency rate and sample size (in my case, 44100hz and 2048 bytes, respectively). This callback function fills the buffer for the current sample and returns.
So, it is possible that in between calls to this callback function, the audio state (the registers) is updated more than once, and hence, when the callback is called again, some samples are 'lost'. This is what I understood from your explanation. Is it ok?

Sorry to bother you too much; I just want to be sure to really grasp this

 

[Cyberman]

Heh, no problem, man; thank you for your time.

I'm using the SDL interface. SDL takes care of double-buffering. It creates a thread that calls a callback function at fixed intervals depending on the frequency rate and sample size (in my case, 44100hz and 2048 bytes, respectively). This callback function fills the buffer for the current sample and returns.
So, it is possible that in between calls to this callback function, the audio state (the registers) is updated more than once, and hence, when the callback is called again, some samples are 'lost'. This is what I understood from your explanation. Is it ok?

Sorry to bother you too much; I just want to be sure to really grasp this

The best way I know this is happening is to have a time stamp buffer with an event that is added to it. For example when a register is modified or when the call back function requests data for the buffer. Each event gets a time stamp, and perhaps some debug data (IE how much data is copied from a buffer etc.) You can see if you are getting short buffers or weird stuff like that by dumping the buffer at regular intervals into a file. (this prevents a lot of small writes killing your over all emulation operation).

Example:

1002.033 {2|BUF READ}1353
1002.066 {2|BUF READ}1353
1002.100 {2|BUF READ}1353
1002.111 {1|REG MOD}FF36:7F->7E
1002.133 {2|BUF READ}321
1002.167 {2|BUF READ}1353

If for example you get a short read after a register modification in the sound hardware then I would say you are right. ;D
Catching the problem in the act however can be quite difficult (threads are difficult to debug because of this}. This is what I used to watch what was going on in a thread (seriously).

You need to get the time stamp from the high resolution time function call into WIN.NT or if under linux the appropriate API call.
Second the data needs to be stored as binary data (as ascii is not particularly efficient to store data in and could really over run your memory quick if you have a lot of event types being monitored).

You also need to enumerate event types (heh) and thread contexts for the events
Something as exciting as

typedef struct
{
 uint16_t id_for_thread;
 uint32_t epoch;
 uint32_t sub_second_time;
 uint16_t event_id;
 uint32_t event_data[4];
}
event_stamp;

In this case you have some data spots for storing information from the event an id (IE what was the event) the internal ID of the program regarding what thread was doing what, an epoch time stamp and the system sub second time for it. The epoch time is the time from when the program started (available in Win NT).
As I said it's been a while, I used that to debug a protocol between a different system and the one I was working with that used a thread attached to a COM port to feed the primary application. (So think 3 things operating concurently).
The data 'stuff' can be anything relevant to the event IE address bytes written register state before register state after etc.

You main thread should append this too a file say every second (IE flush the fifo out that's filled during the second).
You can use a seperate program to dump the data (or convert it to ascii and just use a large buffer for the file). Anyhow 65535 events should cover almost anything happening inside your hardware etc.

I personally would make a function that gathers events and dumps them into the queue based on a filter. IE only look for these kinds of events (in this case audio register mods and the buffer stuffing call back function).

You don't want to monitor everything as that can become quite difficult to follow something in the noise from other stuff happening.

This should get you a sequence of events to following "what's going on". As I said it can be VERY hard to debug threaded programs (heh).

Cyb

 

[Sergio89]

The best way I know this is happening is to have a time stamp buffer with an event that is added to it. For example when a register is modified or when the call back function requests data for the buffer. Each event gets a time stamp, and perhaps some debug data (IE how much data is copied from a buffer etc.) You can see if you are getting short buffers or weird stuff like that by dumping the buffer at regular intervals into a file. (this prevents a lot of small writes killing your over all emulation operation).

Example:

1002.033 {2|BUF READ}1353
1002.066 {2|BUF READ}1353
1002.100 {2|BUF READ}1353
1002.111 {1|REG MOD}FF36:7F->7E
1002.133 {2|BUF READ}321
1002.167 {2|BUF READ}1353

If for example you get a short read after a register modification in the sound hardware then I would say you are right. ;D
Catching the problem in the act however can be quite difficult (threads are difficult to debug because of this}. This is what I used to watch what was going on in a thread (seriously).

You need to get the time stamp from the high resolution time function call into WIN.NT or if under linux the appropriate API call.
Second the data needs to be stored as binary data (as ascii is not particularly efficient to store data in and could really over run your memory quick if you have a lot of event types being monitored).

You also need to enumerate event types (heh) and thread contexts for the events
Something as exciting as

typedef struct
{
 uint16_t id_for_thread;
 uint32_t epoch;
 uint32_t sub_second_time;
 uint16_t event_id;
 uint32_t event_data[4];
}
event_stamp;

In this case you have some data spots for storing information from the event an id (IE what was the event) the internal ID of the program regarding what thread was doing what, an epoch time stamp and the system sub second time for it. The epoch time is the time from when the program started (available in Win NT).
As I said it's been a while, I used that to debug a protocol between a different system and the one I was working with that used a thread attached to a COM port to feed the primary application. (So think 3 things operating concurently).
The data 'stuff' can be anything relevant to the event IE address bytes written register state before register state after etc.

You main thread should append this too a file say every second (IE flush the fifo out that's filled during the second).
You can use a seperate program to dump the data (or convert it to ascii and just use a large buffer for the file). Anyhow 65535 events should cover almost anything happening inside your hardware etc.

I personally would make a function that gathers events and dumps them into the queue based on a filter. IE only look for these kinds of events (in this case audio register mods and the buffer stuffing call back function).

You don't want to monitor everything as that can become quite difficult to follow something in the noise from other stuff happening.

This should get you a sequence of events to following "what's going on". As I said it can be VERY hard to debug threaded programs (heh).

Cyb

Thanks, man.