Investigation of Conversion of Indices to RGB and Pixel Order

Summary

Extensive investigation of pixel order in the framebuffer, converting indices to RGB, and drawing pixels in Pygame. 4 blocks of diagnostic logs were implemented to verify the format of the framebuffer (pixel order), the application of the palette (conversion of indices to RGB), the drawing of pixels on the surface of Pygame and the scaling of the image. The goal is to identify why visual content displays vertical stripes instead of the expected checkerboard, investigating possible issues in pixel order, index conversion to RGB, or drawing in Pygame.

Hardware Concept

Framebuffer format

The framebuffer is in 1D format:[y * 160 + x]where:

• andis the line (0-143)
• xis the column (0-159)
• idx = y * 160 + xgives the index in the 1D array
• To convert from index to coordinates:y = idx // 160, x = idx % 160

This format allows an image of 160x144 pixels to be stored in a linear array of 23040 elements (160x144 = 23040). Each element contains a color index (0-3).

Color Palette

The debug palette maps indices 0-3 to RGB colors:

• Index 0:(255, 255, 255) - White
• Index 1:(170, 170, 170) - Light Gray
• Index 2:(85, 85, 85) - Dark Gray
• Index 3:(8, 24, 32) - Black

Converting indices to RGB should be direct:rgb_color = palette[color_index]. This palette allows you to display the contents of the framebuffer with high contrast, independently of the actual hardware palette (BGP/OBP).

Rendering in Pygame

Pygame offers several ways to draw pixels:

• NumPy:Usesurfarray.blit_array()with format (width, height, channels). The numpy array must be in format (160, 144, 3) for RGB.
• PixelArray:Usepx_array[x, y] = colorto draw individual pixels. The order is (x, y), not (y, x).
• Scaling:Usepygame.transform.scale()to scale the surface to the screen resolution. Scaling can cause interpolation that changes colors slightly.

Fountain:Pan Docs - "LCD Timing", "Frame Rate", "Tile Data", "LCD Control Register"

Implementation

4 blocks of diagnostic logs were implemented insrc/gpu/renderer.pyto investigate pixel order, index conversion to RGB, pixel drawing and scaling.

1. Checking the Pixel Order in the Framebuffer

Added logs to verify that the order of the pixels in the framebuffer is correct (format[y * 160 + x]).

• Horizontal line check:Checks pixels on a horizontal line (y=0, x=0 to x=10) to confirm that the horizontal order is correct
• Vertical column verification:Checks pixels in a vertical column (x=0, y=0 to y=10) to confirm that the vertical order is correct
• Checkerboard pattern verification:Checks adjacent pixels to confirm that the expected checkerboard pattern is correctly reflected in the framebuffer
• Tag: [Renderer-Pixel-Order]

2. Detailed Verification of Indices to RGB Conversion

Logs were added to verify that the conversion of indices to RGB is correct and that the palette is applied correctly.

• Pallet check:Verify that the palette has the correct values ​​(0=White, 1=Light Gray, 2=Dark Gray, 3=Black)
• Checking specific pixels:Check some specific pixels (corners, center, random pixels) to confirm the conversion is correct
• Index validation:Verify that the indices are in a valid range (0-3) and that the resulting RGB is valid
• Tag: [Renderer-RGB-Conversion]

3. Checking Pixel Drawing in Pygame

Added logs to verify that pixels are drawn correctly on the Pygame surface after applying the palette.

• Reading surface colors:Read the surface color after drawing the pixels usingsurface.get_at((x, y))
• Comparison with expected values:Compare the color read from the surface to the color expected from the palette
• Correspondence verification:Verify that the color on the surface matches the expected color (±5 tolerance for interpolation)
• Tag: [Renderer-Pixel-Draw]

4. Scaling Verification and Final Display

Added logs to verify that scaling does not cause significant artifacts and that the scaled content matches the original.

• Comparison before/after scaling:Compare pixel color before scaling (original surface 160x144) and after scaling (scaled surface)
• Interpolation check:Verify that the scaled colors are similar to the originals (±20 tolerance for scaling interpolation)
• Tag: [Renderer-Scale-Visualization]

Affected Files

• src/gpu/renderer.py- Added 4 diagnostic log blocks to investigate pixel order, RGB conversion, pixel drawing and scaling

Tests and Verification

Diagnostic logs were implemented that are activated in the first 10 frames to avoid context saturation. Logs are generated during the normal execution of the emulator.

Test Commands

To run tests with the 5 ROMs and generate logs:

timeout 150 python3 main.py roms/pkmn.gb 2>&1 | tee logs/test_pkmn_step0341.log
timeout 150 python3 main.py roms/tetris.gb 2>&1 | tee logs/test_tetris_step0341.log
timeout 150 python3 main.py roms/mario.gbc 2>&1 | tee logs/test_mario_step0341.log
timeout 150 python3 main.py roms/pkmn-amarillo.gb 2>&1 | tee logs/test_pkmn_amarillo_step0341.log
timeout 150 python3 main.py roms/Oro.gbc 2>&1 | tee logs/test_oro_step0341.log

Log Analysis

To analyze the logs without cluttering the context:

# Check pixel order
grep "\[Renderer-Pixel-Order\]" logs/test_*_step0341.log | head -n 50

# Verify conversion of indices to RGB
grep "\[Renderer-RGB-Conversion\]" logs/test_*_step0341.log | head -n 50

# Check pixel drawing
grep "\[Renderer-Pixel-Draw\]" logs/test_*_step0341.log | head -n 30

# Check scaling
grep "\[Renderer-Scale-Visualization\]" logs/test_*_step0341.log | head -n 30

Validation

The logs allow you to verify:

• That the order of pixels in the framebuffer is correct (format[y * 160 + x])
• That the conversion of indices to RGB is correct and the palette is applied correctly
• That pixels are drawn correctly on the surface of Pygame
• That scaling does not cause significant artifacts

Sources consulted

• Pan Docs: "LCD Timing", "Frame Rate", "Tile Data", "LCD Control Register"
• Pygame documentation:pygame.Surface, pygame.PixelArray, pygame.transform.scale()

Educational Integrity

Next Steps

• [ ] Run full tests with the 5 ROMs (2.5 minutes each)
• [ ] Analyze the generated logs to identify the cause of the visual problem
• [ ] If the cause is identified, implement the correction in the next step
• [ ] If the problem persists, perform further analysis and workaround