Stop Using "Nonwhite" as a Signal and Freeze Palette Contracts (DMG + CGB) with Robust Metrics + Clean-Room Tests

Summary

Replacement of the "nonwhite" metric with robust metrics that measure real frame diversity (unique_rgb_count, dominant_ratio, frame_hash). Implementation of equivalent metrics in headless and UI for comparison. Creation of DMG palette clean-room tests (BGP, OBP0/OBP1) that validate that palettes correctly map color indices to RGB and are reorderable.Critical finding:The palette tests fail because the framebuffer is completely flat (only 1 single color), confirming that the problem is in the index→RGB conversion or in the application of palettes.

Hardware Concept

DMG Palettes (BGP, OBP0, OBP1):In DMG mode, the Game Boy uses 4-color palettes to map color indices (0-3) to RGB values. The BGP register (0xFF47) controls the Background and Window palette, while OBP0 (0xFF48) and OBP1 (0xFF49) control the sprite palettes. Each palette is an 8-bit byte where each pair of bits (bits 0-1, 2-3, 4-5, 6-7) maps a color index to one of the 4 colors of the DMG palette (white, light gray, dark gray, black).

Robust Metrics:The "nonwhite" metric is insufficient because a framebuffer can be "full" (nonwhite=23040) but be completely flat (only 1-2 unique colors). Robust metrics measure:

• unique_rgb_count:Number of unique RGB colors in the frame (16x16 grid sampling = 256 pixels)
• dominant_ratio:Proportion of the most frequent color (if > 0.90 and unique_rgb_count ≤ 2, the frame is flat)
• frame_hash:MD5 hash of a frame sample to detect changes
• hash_changed:Indicates if the hash changed compared to the previous frame

"Flat frame" criterion:unique_rgb_count ≤ 2 and dominant_ratio > 0.90 for several frames indicates that the framebuffer is completely flat, suggesting a problem in palettes or RGB conversion.

Fountain:Pan Docs - "LCD Control Register", "Palettes (BGP, OBP0, OBP1)", "Color Palettes"

Implementation

Robust metrics were implemented in headless and UI, and palette clean-room tests were created:

Phase A: Robust Headless Metrics

Function added_calculate_robust_metrics()intools/rom_smoke_0442.pythat:

• Sample the framebuffer with a 16x16 grid (256 pixels)
• Calculate unique_rgb_count (number of unique RGB colors)
• Calculates dominant_ratio (ratio of the most frequent color)
• Generate frame_hash (sample MD5) and detect changes

The metrics are printed in frames logged with the tag[ROBUST-METRICS].

Phase B: Robust UI Metrics

Function added_calculate_unique_rgb_count_surface()insrc/gpu/renderer.pythat:

• Sample Pygame surface after blit with 16×16 grid
• Calculate unique_rgb_count_after_blit and dominant_ratio
• It is printed in frames logged with the tag[UI-ROBUST-METRICS]

This allows comparing headless vs UI: if headless has high unique_rgb_count but UI has low after_blit, the problem is in the presenter (blit/format/surface).

Phase C: DMG BGP Palette Clean-Room Test

was createdtests/test_palette_dmg_bgp_0454.pythat:

• Write a tile to VRAM with a 4-color pattern (indexes 0,1,2,3)
• Place the tile on the tilemap and render 1 frame
• Validates that there are ≥3 unique RGB colors (not flat)
• Change BGP and validate that the colors change (reorderable palette)

Phase D: Clean-Room Sprite + OBP0/OBP1 Test

was createdtests/test_palette_dmg_obj_0454.pythat:

• Create a sprite in OAM with a color pattern
• Render 1 frame and validate that there are ≥2 unique colors
• Change OBP0 and validate that the colors change

Phase E: CGB Test (Optional)

was createdtests/test_palette_cgb_sanity_0454.pymarked asxfailbecause CGB palettes are not implemented yet.

Affected Files

• tools/rom_smoke_0442.py- Added function_calculate_robust_metrics()and statistics in summary
• src/gpu/renderer.py- Added function_calculate_unique_rgb_count_surface()and robust metrics logging
• tests/test_palette_dmg_bgp_0454.py- BGP palette clean-room test (new)
• tests/test_palette_dmg_obj_0454.py- OBP0/OBP1 pallet clean-room test (new)
• tests/test_palette_cgb_sanity_0454.py- Test sanity CGB (new, xfail)

Tests and Verification

The paddle tests were executed with the following results:

• BGP Test:❌ FAIL - AssertionError: Flat frame: only 1 unique colors (expected ≥3)
• Test OBP0/OBP1:❌ FAIL - AssertionError: Flat sprite: only 1 unique colors (expected ≥2)
• CGB Test:⚠️ XFAIL - CGB palettes not implemented yet

Critical result:The tests fail because the RGB framebuffer only contains a single color (black: 0,0,0), which confirms that the problem is with the index→RGB conversion or the palette application.

Command executed:

pytest tests/test_palette_dmg_bgp_0454.py tests/test_palette_dmg_obj_0454.py tests/test_palette_cgb_sanity_0454.py -v

Result:2 failed, 1 xpassed in 0.70s

Test code (key fragment):

# BGP Test - Palette Validation
framebuffer = ppu.get_framebuffer_rgb()
pixels_rgb = []
for x in [0, 2, 4, 6]: # Pixels with indices 0,1,2,3
    idx = (0 * 160 + x) * 3
    r = framebuffer[idx]
    g = framebuffer[idx + 1]
    b = framebuffer[idx + 2]
    pixels_rgb.append((r, g, b))

unique_colors = set(pixels_rgb)
assert len(unique_colors) >= 3, f"Flat frame: only {len(unique_colors)} unique colors"

Native Validation:Compiled C++ module validation (PPU.get_framebuffer_rgb()).

Sources consulted

• Pan Docs: "LCD Control Register" - LCDC Register (0xFF40)
• Pan Docs: "Palettes (BGP, OBP0, OBP1)" - DMG Palettes
• Pan Docs: "Color Palettes" - CGB Palettes
• Pan Docs: "Tile Data" - 2bpp tile format

Educational Integrity

Next Steps

• [ ] Investigate index→RGB conversion in PPU.get_framebuffer_rgb()
• [ ] Verify that BGP/OBP0/OBP1 are read correctly during rendering
• [ ] Run headless and UI with robust metrics to compare unique_rgb_count
• [ ] If headless has high unique_rgb_count but low UI → fix presenter
• [ ] If both have low → fix core (palettes/RGB conversion)