Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

background_color Transparency Fixes #2484

Open
wants to merge 61 commits into
base: main
Choose a base branch
from

Conversation

proneon267
Copy link
Contributor

@proneon267 proneon267 commented Apr 6, 2024

Fixes #767, Fixes #2425, Fixes #2430

With reference to the issues identified in #2478, I have implemented the related fixes separately in this PR. Reposting from #2478 for reference:
The reason for why only the divider widget triggered the transparency bug on Android and not the existing widgets is that the divider widget is explicitly setting a background color on initialization:

# Background color needs to be set or else divider will not be visible.
self.native.setBackgroundColor(Color.LTGRAY)

Hence, its default background is also not transparent whereas for other widgets, the default background was TRANSPARENT.
I have also fixed a bug on winforms progressbar, which was encountered when I enabled test_background_color_transparent in test_progressbar.

On the backends, there are 3 different possible values for set_background_color()/set_background_color_simple() - None, TRANSPARENT, color(Actual color). I want to propose the following interpretation for them(for the internal backend calls):

  • None - Default background color
  • TRANSPARENT - Actual transparency and default background color as fallback on widgets not supporting transparency.
  • Color - Actual color

Also, as per: https://developer.mozilla.org/en-US/docs/Web/CSS/background-color#formal_definition, the initial value for background_color is transparent, so I also propose to make TRANSPARENT the initial value for background_color.

Let me know what you think and whether I have left out any widgets where the transparency test should be enabled.

  • All new features have been tested
  • All new features have been documented
  • I have read the CONTRIBUTING.md file
  • I will abide by the code of conduct

@proneon267
Copy link
Contributor Author

Regarding the new test, I have added the missing canvas image data test, but since the internal method to patch for testing are platform specific, hence the test is on the probe (Also, on android, canvas._impl.native.getBackground() method cannot be directly monkey patched).
I know this might be a red flag but the other way for the test to be on testbed was to use:

if toga.platform.current_platform is "android":

which you had previously told me to avoid. But, I think that using if toga.platform.current_platform is "android": and keeping the test on the testbed would be more suitable here.

@proneon267 proneon267 mentioned this pull request Apr 6, 2024
4 tasks
@proneon267 proneon267 changed the title Transparency test fixes background_color Transparency Fixes Apr 6, 2024
Copy link
Member

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your proposed interpretation of background color matches what I would expect.

Your proposal to change the default background color also largely makes sense; it's been proposed as a fix for at least 2 issues; see #767 and #2425. However, it will likely have a widespread impact, so we need to be very careful making this change.

As noted inline, the mock-based approach is a non starter; I suspect once that has been removed, the issue about the new test becomes a non-issue.

@@ -236,14 +237,20 @@ def _text_paint(self, font):
paint.setTextSize(self.scale_out(font.size()))
return paint

# This has been separated out, so that it can be mocked during testing.
def _native_get_background(self):
return self.native.getBackground()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't an acceptable approach. Mocking is a last resort for testing. It's used in a handful of places related to permissions and hardware APIs because using those APIs requires passing control to a separate process, at which point the test suite loses the ability to execute. Background color is an entirely internal mechanism, and isn't encumbered by this restriction.

It may be difficult to interrogate - but that doesn't mean we can mock it, because we're trying to verify the actual behavior.

Copy link
Contributor Author

@proneon267 proneon267 Apr 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During the test, coverage reported missing coverage for:

background = self.native.getBackground()
if background:
background.draw(canvas)
self.native.draw(canvas)

I/python.stdout: Name                                                                                                        Stmts   Miss Branch BrPart  Cover   Missing
I/python.stdout: -------------------------------------------------------------------------------------------------------------------------------------------------------
I/python.stdout: data/data/org.beeware.toga.testbed/files/chaquopy/AssetFinder/requirements/toga_android/widgets/canvas.py     153      0     28      1  99.4%   245->247
I/python.stdout: -------------------------------------------------------------------------------------------------------------------------------------------------------
I/python.stdout: TOTAL                                                                                                        2190      0    324      1  99.9%

So, 245->247 means it is reporting missing coverage for the missing else branch. The else branch will be triggered when self.native.getBackground() returns None. Since, it is an internal platform method, I cannot make it return None without mocking it. The other way was to ignore the missing else branch with #pragma: no branch, which you have told me to avoid. So, is there any other way to test the missing else branch or am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVM, I just realized there was a much simpler way to test this. Thanks :) So, the question remains, should I move the test to testbed or keep it in the probe, since it is implementation specific?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be a specific edge case that is needed to exercise that test, but if the other backends don't have an equivalent "if background" branch, then they will run the code without branching, giving double coverage of one specific code path. That's not a problem - it's OK to test something more than once on iOS (et al) if it means we get 100% coverage on Android as well; if nothing else, it's an indication that there are two different test configurations, and we need to test both, even if the "default" implementation works for both cases on iOS.

However, I'm confused because that line did have coverage prior to this change, and it has coverage on every other platform. This, to me, indicates that there's a bigger problem here. Either this branch of code was covering over an edge case that no longer exists, or there's a case that isn't being tested.

# Set the background color to something different
widget.style.background_color = RED
await probe.redraw("Widget background color should be RED")
assert_color(probe.background_color, named_color(RED))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this test for? We already have a background color test - why do we need to test a normal background color in the transparent background color test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, this is a debugging artifact, that I had put on for visual inspection of widgets with transparent as default background color. I'll remove it. Thanks.

@proneon267
Copy link
Contributor Author

proneon267 commented Apr 7, 2024

While implementing this PR, I discovered a bug in the WinForms Divider widget.

I have corrected the WinForms Divider implementation to use a WinForms.Panel instead of a WinForms.Label, as WinForms.Panel is more suitable for a Divider widget and allows setting a background color.

The previous implementation did not produce any visual results when background color was set. This is because it was producing the Divider widget by squeezing the borders of the WinForms.Label and setting its height to 2 px, which hid its background, resulting in just the borders being visible.

The reason why it was passing tests was because it was setting the background color of the WinForms.Label. However, the probe only checks through the native APIs, which returned the set background color. Since its background was hidden, there were no visual results.

@proneon267
Copy link
Contributor Author

proneon267 commented Apr 7, 2024

Also, discovered another bug in Label widget transparent background_color on WinForms.
WinForms.Label although allows setting BackColor to Color.Transparent(doesn't raise transparent background exception), but it doesn't actually work, the background color will be white when set to Color.Transparent. Also tried:

label1.Parent = pictureBox1
label1.BackColor = Color.Transparent

But it only works if the label is inside/above a PictureBox and not for other cases. So, the best bet for transparency is to make label background color same as the container inside which the widget is present. The other way would be to mark Label as _background_supports_alpha = False. I have currently implemented the former method. Which do you think should we choose?

@freakboy3742
Copy link
Member

Also, discovered another bug in Label widget transparent background_color on WinForms. WinForms.Label although allows setting BackColor to Color.Transparent(doesn't raise transparent background exception), but it doesn't actually work, the background color will be white when set to Color.Transparent.

Unless I'm misunderstanding, this is what #2425 is describing.

Also tried:

label1.Parent = pictureBox1
label1.BackColor = Color.Transparent

But it only works if the label is inside/above a PictureBox and not for other cases.

If winforms.PictureBox allows transparency but winforms.Panel doesn't, and there's no notable performance hit from using a winforms.PictureBox with no picture in it, this might be a viable option. It also sounds like it might provide a path to putting a background image on toga.Box, which is a request we get occasionally.

So, the best bet for transparency is to make label background color same as the container inside which the widget is present. The other way would be to mark Label as _background_supports_alpha = False. I have currently implemented the former method. Which do you think should we choose?

Is it setting the background to white, or "default background"? We need to be very careful that we're differentiating between those two cases, especially given that background has been getting closer and closer to white over time.

I think the solution may be actually halfway between the two answers you've suggested. My read of #767 and #2425 is that the current _background_supports_alpha=False implementation is problematic. It's currently using the default background color as a fallback, but it should be using the color of the parent container as a fallback, on the basis that if a (for example) button is in a box, it's the box's background color that should be used.

In the case of widgets like Button, we also need to be careful about the use of background - are we describing the button background color, or are we describing the color of the padding around the button.

@proneon267
Copy link
Contributor Author

proneon267 commented Apr 9, 2024

Apologies for the delayed response. So, I researched more and did more tests, and found that WinForms doesn't actually support true transparency. The important bit for setting up transparency is:

self.native.Parent = self.interface.parent._impl.native
self.native.BackColor = Color.Transparent

WinForms actually just copies the BackColor of the widget's parent to the widget. Which means, every time the widget's parent changes, we need to reapply background color to copy the new parent's BackColor to the widget. However, if we use the above code, then the behavior becomes highly inconsistent, and even glitches out when the parent is changed. Most of the time transparency works like a fluke. I also tried to use a PictureBox to achieve transparency, but the behavior was same as that of a Panel.

I have found that manually setting a widget's BackColor same as that of the parent widget works most stably, and that is what I have opted to implement. With it, I have fixed #2425(the bug were non transparent Label and Box widgets).

Another bug I have noticed is that when a button's background_color is manually set to another color, like in the above image, then a white border appears. Maybe it is not clearly visible in a white background, but if we change the background to a contrasting color, then it becomes apparent.

The only way to remove it seems to be to change the button's FlatStyle to Flat:

Button.FlatStyle = FlatStyle.Flat
Button.FlatAppearance.BorderSize = 0

But then, it doesn't look like a native button anymore.

As for the new failing canvas tests:

tests/widgets/test_canvas.py::test_background_color_transparent
tests/widgets/test_canvas.py::test_transparency

They are somehow connected to the new set_background_color() method of Box widget, as when I disable the set_background_color() method of Box widget, the error disappears.

@freakboy3742
Copy link
Member

I have found that manually setting a widget's BackColor same as that of the parent widget works most stably, and that is what I have opted to implement.

What happens if I:

  • Set the parent to Red
  • Set the child to Transparent
  • Set the parent to Blue?

My read of this code is that the child will end up with a red background. Don't we also need to propagate any background color to all children?

Also - what about alpha blending? The approach you've described will only work for a true TRANSPARENT color. If I set my background to "50% gray, 50% alpha", and my parent is red, then my background shouldn't be red - it should be a 50% blend of gray and red.

This last case is definitely more complex to handle (although, interestingly, a blend with TRANSPARENT should be a redundant case of blending with any parent color...). Unless you can find an easy way to implement it, I'd be OK with noting this as an edge case in the docs.

Another bug I have noticed is that when a button's background_color is manually set to another color, like in the above image, then a white border appears. Maybe it is not clearly visible in a white background, but if we change the background to a contrasting color, then it becomes apparent.
...
But then, it doesn't look like a native button anymore.

A red background doesn't look very native either :-) There's an argument to be made that the border is desirable specifically because it ensures the button edge is visually distinct. If I have a red background and a red button, how do I know the extent of the button's pressable surface?

As for the new failing canvas tests:

tests/widgets/test_canvas.py::test_background_color_transparent
tests/widgets/test_canvas.py::test_transparency

They are somehow connected to the new set_background_color() method of Box widget, as when I disable the set_background_color() method of Box widget, the error disappears.

Well sure - you're messing around with transpareny, so the tests of transparency are breaking. My guess will be either the expected results are incorrect once you correctly handle transparency, or the canvas probe isn't correctly introspecting transparency.

@proneon267
Copy link
Contributor Author

proneon267 commented Apr 13, 2024

Apologies for the delayed response - I had gotten busy with my university assignments.

I've addressed background color and transparency issues across all widgets, eliminating the need of _background_supports_alpha and background_supports_alpha.

Additionally, I've ensured proper handling of rgba() for setting background colors, replicating CSS's rgba handling with alpha compositing. Tests and probes have been fixed, and I've double-checked my equations. I've also prepared a sample app to compare WinForms' background color implementation with CSS's:

"""
My first application
"""

import toga
from toga.style import Pack
from toga.colors import rgba


class HelloWorld(toga.App):
    def validate_rgba(self, value):
        try:
            parts = value.split(",")
            if len(parts) != 4:
                raise ValueError
            r, g, b, a = map(float, parts)
            if not (0 <= r <= 255 and 0 <= g <= 255 and 0 <= b <= 255 and 0 <= a <= 1):
                raise ValueError
        except ValueError:
            print("Invalid RGBA string format. Example: 0,255,0,0.5")
        return None

    def do_change_place(self, widget, **kwargs):
        color_tuple = tuple(map(float, self.color_input.value.split(",")))
        self.widget_box.style.background_color = rgba(*color_tuple)

        if getattr(self, "widgets_in_upper_box", True):
            self.lower_box.add(self.widget_box)
            self.widgets_in_upper_box = False
        else:
            self.upper_box.add(self.widget_box)
            self.widgets_in_upper_box = True

    def startup(self):
        self.main_window = toga.MainWindow(title=self.formal_name)
        self.toga_box = toga.Box(style=Pack(flex=1, direction="column"))

        self.upper_box = toga.Box(style=Pack(flex=1, background_color="red"))
        self.lower_box = toga.Box(style=Pack(flex=1, background_color="blue"))

        self.widget_box = toga.Box(
            style=Pack(height=60, flex=1, background_color=rgba(21, 21, 21, 0.6))
        )
        self.button = toga.Button(
            text="Change Place",
            on_press=self.do_change_place,
            style=Pack(
                padding=10,
                padding_top=15,
                padding_right=5,
                background_color="transparent",
            ),
        )
        self.label = toga.Label(
            text="WinForms RGBA:",
            style=Pack(
                padding_top=20,
                padding_bottom=20,
                color="white",
                background_color="transparent",
            ),
        )
        self.color_input = toga.TextInput(
            placeholder="21,21,21,0.6",
            value="21,21,21,0.6",
            validators=[self.validate_rgba],
            style=Pack(padding_top=15, padding_left=5),
        )
        self.widget_box.add(self.button, self.label, self.color_input)

        self.upper_box.add(self.widget_box)

        self.toga_box.add(self.upper_box, self.lower_box)
        self.web_box = toga.Box(
            style=Pack(flex=1),
            children=[
                toga.WebView(
                    url="https://proneon267.github.io/proneon267-tests/transparency-test.html",
                    style=Pack(flex=1),
                )
            ],
        )

        self.main_window.content = toga.SplitContainer(
            content=[self.toga_box, self.web_box]
        )
        self.main_window.show()

        self.widget_box.refresh()


def main():
    return HelloWorld()

The WebView HTML file can be found https://proneon267.github.io/proneon267-tests/transparency-test.html. Please try out the sample app and let me know if you encounter any inconsistencies.

Based on this, it's evident that the expected reference image of the canvas is incorrect and requires updating.

I haven't made TRANSPARENT the default value for background_color yet, as I want to fix the implementations across all backends first.

Copy link
Member

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core of the Winforms transparency changes make sense; I've flagged a couple of issues inline.

It doesn't surprise me that the reference image needs to be changed. When the test fails locally, it will save a copy of the "actual" image. The log of the test failure should provide the image that's been generated; the image generated in CI is also uploaded as an artefact attached to the CI run.

The biggest issue at this point is the new android "warning" branch, and the test that you've added to get coverage - the approach you've taken seems to be the "make the test be quiet" approach, rather than actually attempting to understand what is going on, and why that branch isn't being covered (and, more importantly - why it was covered before).

There's also an issue with the way you've implemented the background color probe; details inline.

else:
self.native.BackColor = native_color(color)

if isinstance(self.interface, toga.Box):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be an explicit type check for Box. self.interface.children is None is a better check for widgets that cannot have children; if children is not None, then there are children to work with.

)
CACHE[c] = color
CACHE[toga_color] = color
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toga_color is also a method, so there's an inherent ambiguity here.

return rgba(native_color.R, native_color.G, native_color.B, native_color.A / 255)


def alpha_blending_over_operation(child_color, parent_color):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use of child and parent nomenclature here presupposes the usage. Alpha blending is a generic task; back and front (or similar) would be better terminology.

I'd also be inclined to add this as a utility in core (or possibly even in Travertino). Although we're not actually using this anywhere other than Winforms, the generic ability to do alpha blending isn't a bad thing to have, and it would allow us to explicitly test the blending API with core tests, rather than implicitly testing it through usage in Winforms.



def alpha_blending_over_operation(child_color, parent_color):
# The blending operation I have implemented here is the "over" operation and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code comments generally shouldn't refer to the author. We're describing the current state of the code, which isn't dependent on any one author.

)
self.native.BackColor = native_color(blended_color)
else:
self.native.BackColor = native_color(color)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This branch is dealing with setting the color on the root element (i.e, parent is None); what happens if color has an alpha value (i.e., set the background color of the root element to 0.5 opacity red)? Do we need to do an alpha blend with SystemColors.Control?

return TRANSPARENT
else:
return toga_color(self.native.BackColor)
parent_color = toga_color(self.widget.parent._impl.native.BackColor)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again - this requires that parent exists.

parent_color = toga_color(self.widget.parent._impl.native.BackColor)
blended_color = alpha_blending_over_operation(
self.widget.style.background_color, parent_color
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above - the value of background_color shouldn't matter; we're trying to establish ground truth.

@@ -244,6 +245,8 @@ def get_image_data(self):
background = self.native.getBackground()
if background:
background.draw(canvas)
else:
warnings.warn("Failed to get canvas background")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't make any sense to me. Why couldn't the background be obtained?

self.native.setBackground(None)
with pytest.warns(match="Failed to get canvas background"):
self.impl.get_image_data()
self.native.setBackground(original_background)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the comment on the implementation - why is this occurring? Why is the test suite passing with 100% branch coverage prior to this change?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably needs to be broken up into a couple of release notes - one for each underlying issue that we're resolving.

@proneon267
Copy link
Contributor Author

proneon267 commented Apr 23, 2024

I've identified the cause of the missing coverage on the Android canvas:

The missing coverage surfaced after I had made the following change:

if not hasattr(self, "_default_background"):
self._default_background = self.native.getBackground()
if value in (None, TRANSPARENT):
self.native.setBackground(self._default_background)

to:
if not hasattr(self, "_default_background"):
self._default_background = self.native.getBackground()
if value is None:
self.native.setBackground(self._default_background)
elif value is TRANSPARENT:
self.native.setBackgroundColor(Color.TRANSPARENT)

The missing coverage was reported for the lines 245->247:

background = self.native.getBackground()
if background:
background.draw(canvas)
self.native.draw(canvas)

It seemed that a test was setting the widget's background to None, causing it to skip over the if statement. This resulted in coverage for the case of a direct jump to line 247. However, when I changed set_background_simple to set the background color to Color.TRANSPARENT, the background was never None, so it never directly jumped to line 247. As a result, coverage now complains about missing coverage for the case of a direct jump to line 247.

The test that is covering the case of direct jump to line 247 is test_transparency:

async def test_transparency(canvas, probe):
"Transparency is preserved in captured images"
canvas.style.background_color = TRANSPARENT
# Draw a rectangle. move_to is implied
canvas.context.begin_path()
canvas.context.rect(x=20, y=20, width=120, height=120)
canvas.context.fill(color=REBECCAPURPLE)
canvas.context.begin_path()
canvas.context.rect(x=60, y=60, width=120, height=120)
canvas.context.fill(color=rgba(0x33, 0x66, 0x99, 0.5))
await probe.redraw("Image with transparent content and background")
# 0.1 is a big threshold; it's equivalent to 400 pixels being 100% the wrong color.
# This occurs because pixel aliasing around the edge of the squares generates
# different colors due to image scaling and alpha blending differences on each
# platform. You could also generate 0.1 threshold error by moving the entire image 1
# px to the left. However, it's difficult to find a measure that passes the edge
# issue without also passing a translation error.
assert_reference(probe, "transparency", threshold=0.1)

At the end of this test, we do:

assert_reference(probe, "transparency", threshold=0.1)

And in assert_reference we do:
def assert_reference(probe, reference, threshold=0.0):
"""Assert that the canvas currently matches a reference image, within an RMS threshold"""
# Get the canvas image.
image = probe.get_image()

So, for a canvas widget, it calls get_image()=>get_image_data() and in Canvas.get_image_date():
background = self.native.getBackground()
if background:
background.draw(canvas)
self.native.draw(canvas)

Here, background = self.native.getBackground() returned None and it directly jumped to line 247

I have also identified that, in base.py=>set_background_simple:

self._default_background = self.native.getBackground()

Here, self.native.getBackground() returned None since there was no background previously set on the widget.

So, to account for the missing coverage of the background color being set to None, should we introduce a new test test_background_color_none similar to test_background_color_transparent?

@proneon267
Copy link
Contributor Author

Yes - I read that comment. It describes, in great detail, what has changed - but doesn't describe why. If there's an issue with rendering the background of Selection, I've never seen that reported anywhere. The context for changes is important.

I agree with you. I should have provided the context.

Yes - but it should be. If you return a deblended color, the entire redirection function isn't needed at all.
I understand how the redirection works - what I don't understand is why you've opted for a complex dependency injection approach, rather than implementing a probe that returns the value you need, and asserting that value. The deblended implementation is simpler, the test is simpler... there is, as far as I can make out, no down side.

The problem I am facing is that the native background color of a widget cannot be just deblended to get the deblended color. For example, if the native widget returns a background color(self.native.BackColor) value of rgb(127,0,128) then we cannot just deblend the parent color from it because we don't know if the background color was actually set to rgba(127,0,128,1) or if it was set to rgba(0,0,255,0.5) on a rgb(255,0,0) parent background.

With my previous attempt, I had tried to use the interface(widget.style.background) to deduce if the color was actually set as is or was produced due to blending. But, as you had noted earlier, I cannot use the interface to deduce this. Hence, I have taken this approach.

@proneon267
Copy link
Contributor Author

@freakboy3742 Also, do let me know what I should do with ContainedWidget class. Additionally, if there's any other way to assert the background color with the deblended color without encountering the issue I mentioned, I would appreciate your guidance.

@freakboy3742
Copy link
Member

The problem I am facing is that the native background color of a widget cannot be just deblended to get the deblended color. For example, if the native widget returns a background color(self.native.BackColor) value of rgb(127,0,128) then we cannot just deblend the parent color from it because we don't know if the background color was actually set to rgba(127,0,128,1) or if it was set to rgba(0,0,255,0.5) on a rgb(255,0,0) parent background.

Ok - that makes sense as a technical limitation; but I don't think the interface is right.

From a API design perspective, the test needs to assert_background_color(). On most platforms, the ground truth for this is "what is the background color of the widget?"; but on Windows (and, in theory, any other platform that needs to do color blending), this isn't a single color - it's a color and a background.

So - modify the probe interface to do that. Make the probe for background_color return 2 values on Windows - the background color of the widget, and the background color of the parent.

On most platforms, the background color is a single value, so assert_background_color does a simple color comparison.

On Windows (or any platform that returns a tuple), you do the mix between the parent background and the expected value to confirm it matches the actual value.

This keeps the API for assert_background_color entirely testbed-side, doesn't require any dependency injection into the probes, and (theoretically) allows for other colour-mixing backends to use the same blending logic.

@freakboy3742 Also, do let me know what I should do with ContainedWidget class. Additionally, if there's any other way to assert the background color with the deblended color without encountering the issue I mentioned, I would appreciate your guidance.

My inclination is that this is a completely separate problem, and should be a separate bug report (with the details from your previous reply), and a separate set of fixes. I can't comment yet on whether ContainedWidget is the right fix - but my immediate impression is that while it may work, there's almost certainly a simpler solution. I find it difficult to believe that the official Android Material suggestion for "changing the background color of a widget" involves embedding the widget in another widget.

A key part of pulling this specific bugfixes into a separate PR will tests that demonstrate the problem - or, at least, changes to the test probe that validate the right thing is happening.

@proneon267
Copy link
Contributor Author

proneon267 commented Jun 5, 2024

So - modify the probe interface to do that. Make the probe for background_color return 2 values on Windows - the background color of the widget, and the background color of the parent.
On most platforms, the background color is a single value, so assert_background_color does a simple color comparison.
On Windows (or any platform that returns a tuple), you do the mix between the parent background and the expected value to confirm it matches the actual value.

We would still hit the limitation. The key part that is needed for this to work is the actual alpha value of the widget, which is missing from both the background color of the widget and the parent.

This is because we are manually calculating the blended color and the blended color is set with an alpha value of 1. If we only have the resultant color of a widget and the parent color, then without the alpha value of the widget, the suggested approach still won't work.

Basically the blending on the WinForms Backend blends the colors specified on the interface like the following:

Case 1:

On the Interface On the WinForms Backend
Parent Background rgba(255,0,0,1) rgba(255,0,0,1)
Widget Background rgba(0,0,255,0.5) rgba(127,0,128,1)

Case 2:

On the Interface On the WinForms Backend
Parent Background rgba(255,0,0,1) rgba(255,0,0,1)
Widget Background rgba(127,0,128,1) rgba(127,0,128,1)

Case 3:

On the Interface On the WinForms Backend
Parent Background rgba(0,255,0,1) rgba(0,255,0,1)
Widget Background rgba(127,0,128,1) rgba(127,0,128,1)

So, the probe.background_color would need to return a tuple only in the Case 1 and not in Case 2 & 3(i.e., the cases where widget's alpha value on the interface is 1).

You will notice that in Cases 1 & 2, the values on the winforms backend are same in both cases.

Since, on the probe we can only use the backend and not the interface, so we cannot decide if probe.background_color should return only the widget color or the tuple of the parent & widget's color (i.e., should we directly compare the widget color or do blending operations on the returned parent and widget's color).

There also will be cases like Cases 2 & 3 where the widget's color will have no relation to the parent color(i.e., widget background color having alpha value of 1 on the interface).

The important information to differentiate between the cases where the color was set directly(i.e., color set with an explicit alpha value of 1) vs the cases where the color was obtained due to blending operation(i.e., color set with an explicit alpha of value other than 1), is stored only on the interface.

And as you had noted, we cannot use the interface on the probe, so the values that we would have access to are the values listed on On WinForms Backend column in the 3 cases. So, we cannot differentiate between the possible cases.


The only other way, I can think of differentiating between the cases, is by setting a boolean flag like _background_alpha_not_1 on the backend of each widget to indicate cases where there would be a relation between the parent and widget's background color(i.e., the cases where the alpha value of the widget's background is not 1).

@freakboy3742
Copy link
Member

So - modify the probe interface to do that. Make the probe for background_color return 2 values on Windows - the background color of the widget, and the background color of the parent.
On most platforms, the background color is a single value, so assert_background_color does a simple color comparison.
On Windows (or any platform that returns a tuple), you do the mix between the parent background and the expected value to confirm it matches the actual value.

We would still hit the limitation. The key part that is needed for this to work is the actual alpha value of the widget, which is missing from both the background color of the widget and the parent.

I'm not sure if I'm missing something, or you are.

You've got an implementation of a working background color comparison function in the backend. What I'm suggesting is that rather than have the comparison logic in the backend probe, you have the same logic in the testbed, conditional on the probed value of the background color returning a tuple. You then modify the winforms backend (and only the winforms backend) to return a tuple that has all the underlying values needed to do the existing comparison. All the existing backends require no changes; the existing testbed tests that assume assert_color can be used for background color checks are updated to use assert_background_color.

This will:

  • remove the need for the dependency injection of assert_color;
  • keep the probe API as "return ground truth that can be used for comparisons"
  • constrain the scope of backend_color probe changes to just the backend that has this alpha blending problem; and
  • open the door for any other backend with the same issue to use shared comparison logic.

@proneon267
Copy link
Contributor Author

proneon267 commented Jun 7, 2024

I have also been wanting to tell you that the current implementation that I have is also incorrect and is actually not checking alpha blending properly.

def assert_background_color(self, color, assert_color_function):
widget_back_color = self.background_color
if self.widget.parent:
parent_back_color = toga_color(
self.widget.parent._impl.native.BackColor
).rgba
else:
parent_back_color = toga_color(SystemColors.Control).rgba
if (color is TRANSPARENT) and (self.native.BackColor != Color.Transparent):
assert_color_function(widget_back_color, parent_back_color)
elif (color is TRANSPARENT) or (color.a == 1):
assert_color_function(widget_back_color, color)
else:
requested_color = color.rgba
blended_color = alpha_blending_over_operation(
requested_color, parent_back_color
).rgba
# Both of them should also have an alpha value of 1.
assert_color_function(widget_back_color, blended_color)

The important part in the above snippet is:
else:
requested_color = color.rgba
blended_color = alpha_blending_over_operation(
requested_color, parent_back_color
).rgba
# Both of them should also have an alpha value of 1.
assert_color_function(widget_back_color, blended_color)

As I had mentioned earlier, both the requested_color and parent_color have an alpha value of 1, so when we do

blended_color = alpha_blending_over_operation(requested_color, parent_back_color)

then it just returns the requested_color since the requested_color has an alpha value of 1. So, when we are doing:

assert_color_function(widget_back_color, blended_color) 

Then it is just checking the same color with itself. So, there is no proper checking of the blended color.

I agree with you about moving the assert_background_color() to the testbed and removing dependency injection of assert_color.

But, I believe that explaining the issue directly, rather than demonstrating it with code, is causing additional confusion. So, I will first write the code that I think would work properly, and from there we can eliminate what is not needed.

@proneon267
Copy link
Contributor Author

proneon267 commented Jun 8, 2024

I have made the changes as we had discussed. The important changes are:

@property
def background_color(self):
return (
toga_color(self.native.BackColor),
toga_color(self.impl.interface.parent._impl.native.BackColor),
(
# self.impl.interface.style.background_color can be None or TRANSPARENT
# and so there will be no alpha value on them. In such cases return 0
# as the original alpha value.
self.impl.interface.style.background_color.a
if getattr(self.impl.interface.style.background_color, "a", None)
is not None
else 0
),
)

I’ve only used the interface in one specific location to obtain the original alpha value. As I have previously mentioned, this value is necessary for deblending during background color assertion.

As you had suggested, I have made assert_background_color API to work entirely on the testbed itself.

def assert_background_color(actual, expected):
# For platforms where alpha blending is manually implemented, the
# probe.background_color property returns a tuple consisting of:
# - The widget's background color
# - The widget's parent's background color
# - The widget's original alpha value - Required for deblending
if isinstance(actual, tuple):
actual_widget_bg, actual_parent_bg, actual_widget_bg_alpha = actual
deblended_actual_widget_bg = reverse_alpha_blending_over(
actual_widget_bg, actual_parent_bg, actual_widget_bg_alpha
)
if isinstance(expected, tuple):
expected_widget_bg, expected_parent_bg, expected_widget_bg_alpha = expected
deblended_expected_widget_bg = reverse_alpha_blending_over(
expected_widget_bg, expected_parent_bg, expected_widget_bg_alpha
)
assert actual_widget_bg_alpha == expected_widget_bg_alpha
if actual_widget_bg_alpha != 1 and expected_widget_bg_alpha != 1:
assert_color(deblended_actual_widget_bg, deblended_expected_widget_bg)
else:
assert_color(actual_widget_bg, expected_widget_bg)
else:
if expected == TRANSPARENT or expected.a == 0:
assert_color(actual_widget_bg, actual_parent_bg)
elif expected.a != 1:
assert_color(deblended_actual_widget_bg, expected)
else:
assert_color(actual_widget_bg, expected)
# For other platforms
else:
assert_color(actual, expected)

Do let me know what you think! 😊

Copy link
Member

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've done another quick review pass; the refactor of the background color assertion is matches the general shape we discussed, but I've got some questions inline about the implementation.

A quick survey of some other widgets has raised some additional questions; and there's still the outstanding issue from my previous light review.

@@ -55,6 +56,13 @@ def set_value(self, value):
if self.interface.on_change and value != old_value:
self.interface.on_change()

def set_background_color(self, color):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This implementation yields the wrong default background color:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not even sure why I had changed the implementation on cocoa. Since, it is working incorrectly, I will restore the previous implementation.

gtk/src/toga_gtk/colors.py Show resolved Hide resolved
self.native.BackColor = native_color(self._default_background_color)
else:
color = self._default_background_color
elif (color != TRANSPARENT) and (color.a == 1):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this redundant? if color.a == 1, it can't be TRANSPARENT.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I have changed it.

self.native.BackColor = self._default_background
if color is None:
if self._default_background_color != TRANSPARENT:
self.native.BackColor = native_color(self._default_background_color)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't default_background_color always be a native color? If not... why not?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In WinForms, since we manually perform alpha blending, so there’s no suitable native color to indicate that we’re not using Color.Transparent for transparent color. Instead, we use rgba(0,0,0,0) for transparent color during alpha blending calculation.

The native TRANSPARENT color (Color.Transparent) isn’t used by any widget for setting the background to transparent (except the Canvas widget).

Using a toga color allows for easy identification of cases where Color.Transparent should and shouldn't be used for TRANSPARENT.

if self._default_background_color != TRANSPARENT:
self.native.BackColor = native_color(self._default_background_color)
else:
color = self._default_background_color
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow what is going on here - won't this be a no-op?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I have changed set_background_color() method to be more simpler.

return toga_color(self.native.BackColor)
return (
toga_color(self.native.BackColor),
toga_color(self.impl.interface.parent._impl.native.BackColor),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.impl.interface can be simplified to self.widget.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

self.impl.interface.style.background_color.a
if getattr(self.impl.interface.style.background_color, "a", None)
is not None
else 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seem overly complicated. background_color is a Toga color. If it's an RGB declaration, it must have an alpha (it will default to 1). If it's a HSL definition, there's a .rgba property to convert to RGB format (but for testbed purposes, we don't use HSL, so this is a moot point). That means the value should be either TRANSPARENT, or it will have an alpha value - so getattr(bg, "a", 0) (or 0 if bg == TRANSPARENT else bg.a if you want to be more explicit) should be all you need here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks much simpler.

deblended_actual_widget_bg = reverse_alpha_blending_over(
actual_widget_bg, actual_parent_bg, actual_widget_bg_alpha
)
if isinstance(expected, tuple):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can the expected value be a tuple? That's always going to be the reference color, which will be a Toga color literal.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are instances in the test, where

  • expected value can be a tuple:
original = probe.background_color
widget.style.background_color = RED
...
del widget.style.background_color
assert_background_color(probe.background_color, original)
  • expected value can be a single value object:
widget.style.background_color = RED
assert_background_color(probe.background_color, named_color(RED))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah - I'd missed the case where the expected color is coming from a probe. In that case - why not just compare the tuples directly? Surely it's a trivial case, same as when both expected and actual are actual colors? Why use the deblending path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could compare the tuples directly, but we have keep in mind that the probe returned tuple contains blended colors. In cases where the tuple values wouldn't be equal due to different widget color - then it might not be obvious to debug a blended color.
For example:

On the Interface Expected Tuple Value Probe Returned Tuple Value
Parent Background rgba(255,0,0,1) rgba(255,0,0,1) rgba(255,0,0,1)
Widget Background rgba(0,0,255,0.5) rgba(127,0,128,1) rgba(0,255,0,1)

Now when the expected widget color value is compared with the probe returned widget color value directly, then it would report the failure to something similar like:

assert (actual.r, actual.g, actual.b, actual.a) == (
AssertionError: assert (0,255,0,1) == (127,0,128,1)

Then it might not be immediately obvious from where the expected value rgba(127,0,128,1) has come.

Compared this to asserting deblended colors, the error message would be something like:

assert (actual.r, actual.g, actual.b, actual.a) == (
AssertionError: assert (0,255,0,1) == (0,0,255,0.5)

Which clearly specifies that rgba(0,0,255,0.5) is the expected color. This would be easy to debug since the expected value would be the same as the one set by the test on the interface.

else:
assert_color(actual_widget_bg, expected_widget_bg)
else:
if expected == TRANSPARENT or expected.a == 0:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get why TRANSPARENT is a special case here (because it's going to be a string); but why differentiate "a=0"? Why not convert TRANSPARENT into "rgba(0,0,0,0)" and then use a single based on deblending? For that matter - shouldn't the math for a deblended color with alpha=1 result in the original color?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...but why differentiate "a=0"?...

Because we cannot deblend a color having an alpha value of 0 to get the exact original color:

front_color = rgba(
round(
(
(blended_color.r * blended_color.a)
- (back_color.r * back_color.a * (1 - original_alpha))
)
/ original_alpha

As, we can see in the above snippet, original_alpha with a value of 0 - will lead to a division by zero error. For preventing this, we are returning the back_color(parent color) in such cases:
if original_alpha == 0:
return back_color

This works for asserting cases where the blended color returned by the probe can be deblended to the original color (i.e., the cases where the original alpha has a value in the range (0,1] ) - since the widget color would be the same as the parent color on alpha blending with alpha value of 0.

But there is another set of tests, where the expected value is not a tuple returned by the probe, but rather it is a predetermined test color data.

async def test_background_color(widget, probe):
"The background color of a widget can be set"
for color in COLORS:
widget.style.background_color = color
await probe.redraw("Widget background color should be %s" % color)
assert_background_color(probe.background_color, color)

In the above test, when there will be the case of color being rgba(50, 128, 200, 0.0) then we cannot use the following branch for assertion:
elif expected.a != 1:
assert_color(deblended_actual_widget_bg, expected)

because here deblended_actual_widget_bg would have the parent's background color instead of the predetermined expected color value.

Hence, we need to use this branch for such cases:

if expected == TRANSPARENT or expected.a == 0:
assert_color(actual_widget_bg, actual_parent_bg)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...but why differentiate "a=0"?...

Because we cannot deblend a color having an alpha value of 0 to get the exact original color:

It took me a moment to fully understand this; it would be worth capturing this detail in a comment for future code explorers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a comment explaining the reasoning behind the choice.

@proneon267
Copy link
Contributor Author

Quick question, if I move the fixes, I have made to the Android backend to a separate PR(like "Android background_color fixes") and change the description of this PR, then can we merge this PR?

Also, a meta question, often times when working on a PR, I encounter several related bugs that I can fix. But by doing so, I increase the size of the PR. Moreover, if I decide to breakdown each of those bugfixes into new PRs, then I just start to pile up PRs. What should I do in such situations? Do I log an issue about each of those bugs & not fix the bugs, and leave it at that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants