Framebuffer Rendering Investigation and Fix

Summary

Detailed investigation of rendering issue where the framebuffer contains correct data (80/160 non-white pixels according to C++ logs) but the screen displays completely white. Added diagnostic logs at three critical points in the rendering pipeline: (1) diagnostics of the framebuffer received in the renderer, (2) verification of palette application to specific pixels, and (3) detailed verification of the framebuffer copy in Python. These logs will allow you to identify exactly where color information is lost in the rendering pipeline.

Hardware Concept

Rendering Pipeline: From Indices to RGB

The PPU framebuffer containscolor indices (0-3), not direct RGB colors. These indices must be converted to RGB colors using the BGP (Background Palette) before being displayed on the screen.

The complete pipeline is:

1. C++ (PPU):Writes color indices (0-3) to the framebuffer during rendering of each scanline
2. Cython:Expose the framebuffer asmemoryviewfor access from Python
3. Python (viboy.py):Copy the framebuffer tobytearrayto protect it from changes in C++
4. Python (renderer.py):Convert indices to RGB using the debug palette:
   • Index 0 → (255, 255, 255) - White
   • Index 1 → (170, 170, 170) - Light Gray
   • Index 2 → (85, 85, 85) - Dark Gray
   • Index 3 → (8, 24, 32) - Black
5. Pygame:Draw RGB pixels on the screen

Identified Issue: Discrepancy between Framebuffer and Rendering

The logs from Step 0331 show that:

• Framebuffer (C++):80/160 non-white pixels, distribution: 0=80, 1=0, 2=0, 3=80
• Rendering (Python/Pygame):Completely white screen

This suggests that the problem is somewhere in the pipeline between C++ and Pygame. The possible causes are:

• Framebuffer gets corrupted during copy in Python
• The renderer is not receiving the correct data
• There is a problem with converting indices to RGB
• The renderer is using the wrong palette or not applying it

Fountain:Pan Docs - Background Palette (BGP), Pixel Rendering, Rendering Pipeline

Implementation

Aggregated Diagnostic Logs

Added three diagnostic logging systems to track the framebuffer through the pipeline:

1. Diagnosis of the Received Framebuffer (renderer.py)

Inrenderer.py, after receivingframebuffer_data, logs are added that:

• Count indices in the first 100 pixels of the framebuffer
• They show the first 20 indices to verify the checkerboard pattern
• They verify that index 3 is correctly converted to black (8, 24, 32)

# --- STEP 0332: Framebuffer Diagnostic Received ---
if framebuffer_data is not None and len(framebuffer_data) > 0:
    if not hasattr(self, '_framebuffer_diagnostic_count'):
        self._framebuffer_diagnostic_count = 0
    if self._framebuffer_diagnostic_count< 5:
        self._framebuffer_diagnostic_count += 1
        
        # Contar índices en el framebuffer
        index_counts = {0: 0, 1: 0, 2: 0, 3: 0}
        for idx in range(min(100, len(framebuffer_data))):
            color_idx = framebuffer_data[idx] & 0x03
            if color_idx in index_counts:
                index_counts[color_idx] += 1
        
        logger.info(f"[Renderer-Framebuffer-Diagnostic] Frame {self._framebuffer_diagnostic_count} | "
                   f"Index counts (first 100): 0={index_counts[0]} 1={index_counts[1]} "
                   f"2={index_counts[2]} 3={index_counts[3]}")
        
        # Verificar primeros 20 píxeles
        first_20 = [framebuffer_data[i] & 0x03 for i in range(min(20, len(framebuffer_data)))]
        logger.info(f"[Renderer-Framebuffer-Diagnostic] First 20 indices: {first_20}")
        
        # Verificar que el índice 3 se convierte a negro
        if index_counts[3] >0:
            test_index = 3
            test_color = palette[test_index]
            logger.info(f"[Renderer-Framebuffer-Diagnostic] Index 3 -> RGB: {test_color} (should be black: (8, 24, 32))")
            if test_color != (8, 24, 32):
                logger.warning(f"[Renderer-Framebuffer-Diagnostic] ⚠️ PROBLEM: Index 3 does not convert to black!")
# -------------------------------------------

2. Palette Application Verification (renderer.py)

Added logs that verify that the palette is correctly applied to specific pixels:

• Top left corner (0, 0)
• Screen center (80, 72)
• Bottom right corner (159, 143)

These logs show the color index and the resulting RGB for each pixel, allowing you to verify that the conversion is working correctly.

# --- STEP 0332: Palette Application Verification ---
if not hasattr(self, '_palette_apply_check_count'):
    self._palette_apply_check_count = 0
if self._palette_apply_check_count< 5:
    self._palette_apply_check_count += 1
    
    test_pixels = [
        (0, 0),    # Esquina superior izquierda
        (80, 72),  # Centro de pantalla
        (159, 143) # Esquina inferior derecha
    ]
    
    for x, y in test_pixels:
        idx = y * 160 + x
        if idx < len(frame_indices):
            color_index = frame_indices[idx] & 0x03
            rgb_color = palette[color_index]
            logger.info(f"[Renderer-Palette-Apply] Pixel ({x}, {y}): index={color_index} ->RGB={rgb_color}")
            
            if color_index == 3 and rgb_color != (8, 24, 32):
                logger.warning(f"[Renderer-Palette-Apply] ⚠️ PROBLEM: Index 3 does not convert to black on ({x}, {y})!")
# -------------------------------------------

3. Detailed Verification of Framebuffer Copy (viboy.py)

Logs were improved inviboy.pyto verify that the framebuffer copy is identical to the original:

• Show the first 20 indexes before and after copying
• Verify that the copy is identical to the original
• Count indexes in the copy to compare with C++ logs

# --- Step 0332: Detailed Framebuffer Copy Verification ---
# Check first 20 pixels before copying
first_20_before = [raw_view[i] & 0x03 for i in range(min(20, len(raw_view)))]

if not hasattr(self, '_framebuffer_copy_detailed_count'):
    self._framebuffer_copy_detailed_count = 0
if self._framebuffer_copy_detailed_count< 5:
    self._framebuffer_copy_detailed_count += 1
    logger.info(f"[Viboy-Framebuffer-Copy-Detailed] Frame {self._framebuffer_copy_detailed_count} | "
               f"First 20 indices before copy: {first_20_before}")

# Hacer copia profunda
fb_data = bytearray(raw_view)

# Verificar primeros 20 píxeles después de copiar
first_20_after = [fb_data[i] & 0x03 for i in range(min(20, len(fb_data)))]

# Verificar que la copia es idéntica
if first_20_before != first_20_after:
    logger.warning(f"[Viboy-Framebuffer-Copy-Detailed] ⚠️ DISCREPANCIA: "
                  f"Before={first_20_before}, After={first_20_after}")

# Contar índices en la copia
index_counts = {0: 0, 1: 0, 2: 0, 3: 0}
for idx in range(len(fb_data)):
    color_idx = fb_data[idx] & 0x03
    if color_idx in index_counts:
        index_counts[color_idx] += 1

logger.info(f"[Viboy-Framebuffer-Copy-Detailed] Index counts in copy: "
           f"0={index_counts[0]} 1={index_counts[1]} "
           f"2={index_counts[2]} 3={index_counts[3]}")
# -------------------------------------------

Design Decisions

• Log limit:Logs are only shown in the first 5 frames to avoid context saturation and maintain performance
• Checking specific pixels:Pixels in known positions (corners and center) are checked for clear reference points
• Before/after comparison:Data before and after copy is compared to detect corruption during transfer

Affected Files

• src/gpu/renderer.py- Added received framebuffer diagnostic logs and palette application verification
• src/viboy.py- Improved detailed framebuffer copy verification logs

Tests and Verification

Diagnostic logs will run automatically when running the emulator with any ROM. To analyze the results:

# Run tests with the 5 ROMs (2.5 minutes each)
timeout 150 python3 main.py roms/pkmn.gb 2>&1 | tee logs/test_pkmn_step0332.log
timeout 150 python3 main.py roms/tetris.gb 2>&1 | tee logs/test_tetris_step0332.log
timeout 150 python3 main.py roms/mario.gbc 2>&1 | tee logs/test_mario_step0332.log
timeout 150 python3 main.py roms/pkmn-amarillo.gb 2>&1 | tee logs/test_pkmn_amarillo_step0332.log
timeout 150 python3 main.py roms/Oro.gbc 2>&1 | tee logs/test_oro_step0332.log

# Analyze the logs (use commands that do not clutter the context)
grep "\[Renderer-Framebuffer-Diagnostic\]" logs/test_*_step0332.log | head -n 15
grep "\[Renderer-Palette-Apply\]" logs/test_*_step0332.log | head -n 15
grep "\[Viboy-Framebuffer-Copy-Detailed\]" logs/test_*_step0332.log | head -n 20

Expected validation:

• The logs show what indices the renderer receives from the framebuffer
• The logs show how the indices are converted to RGB
• The cause of the problem is identified (where color information is lost)

Sources consulted

• Pan Docs: Background Palette (BGP) - Converting indexes to colors
• Pan Docs: Pixel Rendering - Rendering Pipeline
• Python documentation:bytearrayandmemoryviewfor binary data transfer

Educational Integrity

Next Steps

• [ ] Run tests with the 5 ROMs and collect diagnostic logs
• [ ] Analyze logs to identify where color information is lost
• [ ] Step 0333:Implement correction based on log findings
• [ ] Step 0334:Final render check after correction