Bad Apple!! CD+G

In 2023 I got my first CD Graphics disc. These are standard audio CDs, but they also have graphics data in space that otherwise goes unused. I was drawn to the idea of a hidden unobtrusive bonus for those who know where to look, so since then I've been digging into the technical details and collecting every CD+G disc I can find.

Soon I hit on the idea of using it for animation, specifically the famous Bad Apple!! music video, common fodder for demos on underpowered hardware. CD+G mostly fell into a niche of displaying karaoke lyrics, and I thought it'd be fun to show the video running on a karaoke machine. I got that working well enough in November 2024, but I wanted to present it with a bit of background and explanation of the techniques. Bear with me and I'll try to avoid waxing encyclopedic. (Or skip ahead to the video if you prefer.)

Drawing less, less often

This project was a challenge because CD+G is very much not designed for animation. There's bandwidth for only 300 commands per second, which take effect immediately, and if a slot isn't used for a command it's wasted. The visible display is 48 x 16 blocks, and each of those takes at least one command to write, so if you were to draw a full screen image it would take at least 2.5 seconds. [aside-scrolling]

If we want to update the display at animation rates we'll need to cut some corners. The main compromise is to use a comically low resolution, I'm rendering only 11 x 4 blocks, 66 x 48 pixels. But even updating that small part of the screen in monochrome can only happen at 6.8 FPS. A fundamental animation technique is to only update the parts of the image that change from one frame to the next; a lot of the image is background or within solid-colored figures. Only making updates for changed blocks runs at an average 17 FPS. It increases to 20 FPS if we allow some jitter, starting the next frame a little early if there's time after the previous one.

But there's still a major problem: Even if we were able to update at the full 30 FPS of the source video, it still often takes a noticeable amount of time to update the image, and that whole time the frame is being displayed. If we could hold for a bit on the finished image it might not be too bad, but to hit that framerate we need to be constantly updating. It becomes a flickering mess. We want to hold on one frame while we're drawing the next one; we want to double buffer. However CD+G only has the one buffer… [CD+EG]

Invisible updates

Let's talk about colors. Each pixel is 4 bits, CD+G has a 16 color lookup table (CLUT, a.k.a. palette) that gives the RGB value to use for each color index. We can freely update the CLUT, it only takes two commands to set all 16 entries, and that will affect the whole screen. This allows us to use some very powerful techniques, all of which are based on setting multiple entries to the same color.

A common effect is to set all the CLUT entries to the same RGB value (e.g. black), draw an image (however long that takes), and then make rapid updates to the CLUT to transition the colors from black to the proper value for the image. This hides the initial drawing and creates a fade-in effect, often this is used to make a good first impression with a fancy title screen. The downside is that nothing is displayed on screen while the image is being drawn invisibly.

If we're dealing with an image where we don't need 16 distinct colors, we can do some more tricks. Each of the 4 lines of text in my Bad Apple!! demo is drawn with a different color index, though they are all only displayed in white. The CLUT has only 1 of those colors set to white at a time, the other 3 colors are set to the same black as the background, thus rendering them invisible while they're being drawn. When it's time to display the next line of text I issue a single CLUT command, changing which line is white.

Creating a double buffer

The text trick only works easily because I have the text lines draw at non-overlapping locations. For our double buffered movie we want to reuse the same location in the middle of the screen. This requires a different technique: bitplane extraction. [bitplane-term]

The color index of each pixel will be set depending on the combination of colors in two independent images, call them A and B:

image
A		image
B		CLUT
index		binary
blackblack000
blackwhite101
whiteblack210
whitewhite311

We can choose which image is displayed just by setting the CLUT. To display image A we set the colors as in the image A column: If image A is black at a pixel it could have value either 0 or 1, depending on B; it won't matter which because they both are displayed as black. We could switch to displaying image B by setting the CLUT to the colors in the B column.

So image A is displayed, and we want to update image B. I mentioned that it takes at least one command to draw a block of graphics. An unusual feature of CD+G is that the command to set a block can use any pattern, but only two colors. If you want to have more than two colors in one block you need to issue a series of XOR commands, which allows flipping bits of the color index in certain pixels. Switching the color of image B is just an XOR with 1, that toggles between 0 and 1 (both A black), or between 2 and 3 (both A white). This XOR is the same regardless of what color is in A, we just need to specify what pixels we want to toggle in image B. [aside-corruption]

So, now we're double buffering! This has a subtle impact on framerate. We're updating two buffers, but we only need to update each for half the frames, and we're still just using one command per block update. But one buffer has the even frames and the other has the odd frames, so successive frames are further apart in time; the separate buffers need to change more on each update because generally more distant frames have less in common. In this case we go from 20 to 18.5 FPS.

Final touches

I ended up using triple buffering. When I tested the CD+G in the ares Mega CD emulator it seemed like the palette change didn't take effect immediately, so drawing the next frame was often visible. With three buffers there can be one frame drawing and the previous two possibly displaying. This supports a wider range of player implementations because it doesn't matter exactly when the display switch happens between the two complete frames. The bitplane XOR update works the same as with two buffers, and the 8 colors needed can all be updated by a single CLUT command. The only downside is this further increases the distance between successive frames in a buffer, making the partial updates slower; the average drops to 17.3 FPS.

Finally, I added in the lyrics. These work more like subtitles than karaoke lyrics, but I think it gives a good impression. Mainly the lyrics are loaded when there is bandwidth left over from the animation. I split it across 4 lines because it needed to have 3 lines saved up; there's little bandwidth left over during busy scenes, but the words keep coming! Even with that pre-buffered it needs to sometimes preempt the animation, stealing command slots from frames that otherwise would render the fastest. This puts the final average at 16.3 FPS.

In motion!

Here it is, I hope it's a little more interesting with that background!

Also check out the rainbow version which uses unique colors for all the color indexes, this shows all the stuff going on behind the scenes. [bonus]

Resources

[aside-scrolling] CD+G is at its best with long scrolling images. It is possible to render invisibly in the narrow offscreen border, while continuously scrolling the image. This maximizes use of bandwidth, hides drawing, keeps the whole screen in motion, and allows the completed image to persist long enough to be viewed.
The first CD+G I owned, Super Dungeon Master, had some great examples of this, as well as many great CLUT effects I didn't get to discuss in the article. I put together a web viewer for it: Super Dungeon Master [VINL-1] CD+G

[CD+EG] There's an extension to CD+G, CD Extended Graphics, which adds a second buffer. This is especially helpful to hide loading: draw to the offscreen buffer and then switch to display it instantly. There's some other neat features: the RGB colors on the two buffers can be added together for transparency and independent scroll, or they can be combined into a 256 color mode with 18-bit color. It can be a bit awkward because it reuses CD+G commands, to retain compatibility with CD+G players without encoding all the graphics twice.
There's a Mashin Hero Wataru 3 drama album that makes good use of these effects, I made a few comparison videos showing side-by-side the CD+G and CD+EG decode: disc 1 and disc 2 (using my CD+EG branch of cdgraphics)
CD+EG was even less successful than CD+G, and quite obscure today. I've cataloged the few known discs at my site extended.graphics.

[bitplane-term] Bit-plane extraction is the term used in Techniques for Frame Buffer Animation (Booth & MacKay 1982). They cite Color Table Animation (Shoup 1979), which mentioned this as a special case of alternate color animation, the general idea of using different colors for different images (as I used for the text).
I had earlier misused the term bitplane animation for this. That refers to a somewhat different technique, covered in depth in Michael Abrash's Graphics Programming Black Book, Chapter 43. There the planes are used to represent overlapping layers, all visible at once. The bitplanes are stored separately in some VGA modes, so this makes it efficient to animate them independently, but the animation isn't accomplished by changing the palette.

[aside-corruption] We can update the image because we know what it was when we last wrote it, so we can use XOR to flip a bit if needed. This is a little risky, though, because we may not know precisely what image was actually on screen. CD+G has some error correction, but sometimes commands are lost or corrupted, and when seeking (which includes pausing) commands will be skipped or repeated. The longer it has been since the block was set, or the whole screen was cleared (preset), the greater the chance of corruption.
This is probably a big reason why the XOR partial update technique is rarely used. Usually a block is Set, immediately XORed to introduce any needed colors, and then not touched again until it gets Set next.

[bonus] Here's an older, even smaller version with a higher framerate:

Still not sure which is preferable. There's a bug in the text at the end, if not for that I might have left it this way.