Get list of distinct pixel RGB values

[thetalkietoaster]

Sorry for the very late reply - managed to completely forget about this after I fixed it :/. The code's on GitHub, and it's been revised quite a bit since I posted this. Originally, I was calculating the sum RGB for each pixel and then using that as the dict key, but that was clearly a daft move when I could just do the summing afterwards.

The code I ended up with is below, with changes to account for 1. some palettes having multiple colours of the same sum RGB, and 2. some of the images having 1-2 stray pixels out of the palette. Fixing those meant switching to using pixel RGB as key like you, which then solved the performance issues.

Code:

def extract_sorted_palette(
    layer, include_transparent, count_threshold,
    current_progress, progress_fraction,
):
    """
    Extracts a palette from an image, by finding the discrete RGB values
    and then sorting them by total R+G+B value.
    """
    palette_counts = {}    
    progress_step = progress_fraction / layer.height

    region = layer.get_pixel_rgn(
        0, 0, layer.width, layer.height
    )

    for index_row in range(0, layer.height):
        for pixel in RowIterator(region[0:layer.width, index_row], layer.bpp):
            colour_rgb = pixel[0:3]

            if layer.has_alpha and pixel[3] == 0 and not include_transparent:
                continue

            elif colour_rgb not in palette_counts:
                palette_counts[colour_rgb] = 1

            else:
                palette_counts[colour_rgb] += 1

        gimp.progress_update(current_progress + progress_step * index_row)

    # Now we've counted all the pixel colours, discard outliers and sort
    palette = {}
    for colour_rgb, colour_count in palette_counts.items():
        colour_sum = sum(colour_rgb)

        if colour_count > count_threshold:
            if colour_sum in palette:
                if colour_rgb != palette[colour_sum]:
                    colour_duplicate = palette[colour_sum]
                    raise KeyError(
                        "Multiple colours in layer with same total RGB values: " + \
                        str(colour_rgb) + "(" + str(colour_count) + " pixels) and " + \
                        str(colour_duplicate) + "(" + str(palette_counts[colour_duplicate]) + " pixels). "
                        "Cannot automatically sort colours by brightness. " + \
                        "Try increasing the 'ignore colours with less than this many pixels' setting " + \
                        "to drop stray pixels."
                    )
            else:
                palette[colour_sum] = colour_rgb

    sorted_palette = [
        palette[key] for key in sorted(list(palette.keys()))
    ]
    return sorted_palette

Though looking at your example I can't believe I forgot about defaultdict, I'll fix that. Should also be using actual perceived brightness rather than RGB intensity too.