Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[macOS Sonoma] Control.setBackgroundImage - image is upside down and flipped #772

Open
Phillipus opened this issue Aug 19, 2023 · 28 comments · May be fixed by #860
Open

[macOS Sonoma] Control.setBackgroundImage - image is upside down and flipped #772

Phillipus opened this issue Aug 19, 2023 · 28 comments · May be fixed by #860
Labels
help wanted Extra attention is needed macOS happens on macOS

Comments

@Phillipus
Copy link
Contributor

Phillipus commented Aug 19, 2023

Describe the bug
Background images on SWT controls are inverted.

This was reported here but I've opened a dedicated issue as it's not to do with the splash screen but rather Control#setBackgroundImage.

To Reproduce

import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.layout.FillLayout;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;

public class ImageTest {

    public static void main(String[] args) {
        Display display = new Display();
        Shell shell = new Shell(display);
        shell.setText("Image Test");
        shell.setLayout(new FillLayout());
        
        Image image = new Image(display, ImageTest.class.getResourceAsStream("splash.png"));
        shell.setBackgroundImage(image);
        
        shell.setSize(image.getBounds().width, image.getBounds().height);
        shell.open();
        
        while(!shell.isDisposed()) {
            if(!display.readAndDispatch()) display.sleep();
        }

        display.dispose();
    }
}

Use this image for testing (rename to splash.png), or your own image if you prefer:

splash

Expected behavior
The image should display the right way.

Screenshots
Run the snippet and see this:

inverted

Environment:

  1. Select the platform(s) on which the behavior is seen:
    • All OS
    • Windows
    • Linux
    • macOS
  1. Additional OS info (e.g. OS version, Linux Desktop, etc)
    macOS Sonoma 14
    Mac Mini M1

  2. JRE/JDK version
    Temurin 17

@Phillipus Phillipus changed the title [macOS Sonoma] Shell.setBackgroundImage - image is upside down [macOS Sonoma] Control.setBackgroundImage - image is upside down Aug 19, 2023
@Phillipus Phillipus changed the title [macOS Sonoma] Control.setBackgroundImage - image is upside down [macOS Sonoma] Control.setBackgroundImage - image is upside down anf flipped Aug 19, 2023
@Phillipus Phillipus changed the title [macOS Sonoma] Control.setBackgroundImage - image is upside down anf flipped [macOS Sonoma] Control.setBackgroundImage - image is upside down and flipped Aug 19, 2023
@Phillipus
Copy link
Contributor Author

Phillipus commented Aug 20, 2023

In the case of displaying the splash screen when launching Eclipse or an RCP app, the image initially appears correctly but is inverted after a second or two. AFAICS, this is because the initial display of the splash is done natively in the org.eclipse.equinox.launcher.Main class which calls the org.eclipse.equinox.launcher.JNIBridge method showSplash which then calls the native _show_splash method. Presumably whatever is happening in _show_splash works OK.

But then the trail goes to org.eclipse.ui.internal.Workbench and the method createSplashWrapper. Here is where the Shell's background is set at line 786:

splashShell.setBackgroundImage(background);

Then spinEventQueueToUpdateSplash(display) is called by Workbench#createAndRunWorkbench to dispatch messages which updates the splash Shell. This is where the underlying ObjC message from calling setBackgroundImage is dispatched and the image is painted inverted.

Control#setBackgroundImage only sets the Image object and requests an update. Painting the background image is done in Control#fillBackground. The line that actually draws the background image is at line 1323:

NSColor.colorWithPatternImage(image.handle).setFill();

The code for this call is in org.eclipse.swt.internal.cocoa.NSColor:

public static NSColor colorWithPatternImage(NSImage image) {
    long result = OS.objc_msgSend(OS.class_NSColor, OS.sel_colorWithPatternImage_, image != null ? image.id : 0);
    return result != 0 ? new NSColor(result) : null;
}

And another at line 1345:

NSBezierPath.fillRect(rect);

So, there's either a bug in Apple's code or something has changed and a workaround is needed in SWT code.

I can't do any more than offer this analysis, so please feel free to help. :-)

@Phillipus
Copy link
Contributor Author

The same problem with using a Pattern with image:

import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.graphics.Pattern;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;

public class ImageTestWithPattern {

    public static void main(String[] args) {
        Display display = new Display();
        Shell shell = new Shell(display);
        shell.setText("Image Test with Pattern");
        shell.setLayout(new GridLayout());
        
        Image image = new Image(display, ImageTestWithPattern.class.getResourceAsStream("splash.png"));
        shell.setSize(image.getBounds().width, image.getBounds().height);
        
        Pattern pattern = new Pattern(display, image);
        
        shell.addPaintListener(e -> {
            e.gc.setBackgroundPattern(pattern);
            e.gc.fillRectangle(0, 0, shell.getSize().x, shell.getSize().y);
        });
        
        shell.open();
        
        while(!shell.isDisposed()) {
            if(!display.readAndDispatch()) display.sleep();
        }

        display.dispose();
    }
}

@Phillipus
Copy link
Contributor Author

Phillipus commented Sep 12, 2023

This will be in macOS Sonoma final as the behaviour exists in the Release Candidate.

@Phillipus
Copy link
Contributor Author

A self-contained snippet:

import org.eclipse.swt.graphics.Color;
import org.eclipse.swt.graphics.GC;
import org.eclipse.swt.graphics.Image;
import org.eclipse.swt.layout.GridLayout;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;

public class ImageTest {

    public static void main(String[] args) {
        Display display = new Display();
        Shell shell = new Shell(display);
        shell.setText("Image Test");
        shell.setLayout(new GridLayout());
        
        shell.setSize(200, 200);
        
        Image image = new Image(Display.getDefault(), 200, 200);
        GC gc = new GC(image);
        gc.setForeground(new Color(null, 0, 0, 0));
        gc.drawText("Hello World", 50, 100);
        gc.dispose();
        
        shell.setBackgroundImage(image);

        shell.open();
        
        while(!shell.isDisposed()) {
            if(!display.readAndDispatch()) display.sleep();
        }

        display.dispose();
    }
}

hello

@TIBCOeddie
Copy link

I'm hoping this can be addressed asap; it will hold up our adoption of this latest release

@Phillipus
Copy link
Contributor Author

I'm hoping this can be addressed asap; it will hold up our adoption of this latest release

I've documented my analysis of the problem in the previous comments, but I don't know if this is an Apple bug (I have reported it) or something that needs to change in SWT. It needs someone else to answer that.

@Phillipus Phillipus added the help wanted Extra attention is needed label Oct 2, 2023
@DonaldWills
Copy link

I am experiencing the same problem on Sonoma/MacMini M2 Pro. It exists in Eclipse 2023-03 as well as 2023-09.

In addition, the name of the application at the left side of the Menu Bar (just to the right of the Apple icon) says "New Application" instead of "Eclipse".

@Phillipus
Copy link
Contributor Author

Phillipus commented Oct 5, 2023

I am experiencing the same problem on Sonoma/MacMini M2 Pro. It exists in Eclipse 2023-03 as well as 2023-09.

Yes, that's because the problem lies in macOS 14.

In addition, the name of the application at the left side of the Menu Bar (just to the right of the Apple icon) says "New Application" instead of "Eclipse".

Yes, and that has been reported (and fixed) here - #779

knime-github pushed a commit to knime/knime-product that referenced this issue Oct 11, 2023
On macOS Sonoma 14.0, images that are set as background image on SWT controls are upside-down.
See eclipse-platform/eclipse.platform.swt#772.

The workaround checks for the presence of the bug and flips the splash screen background image such that it gets flipped
back by the bug to its normal position.

AP-21237 (Splash screen is displayed upside down)
knime-github pushed a commit to knime/knime-product that referenced this issue Oct 12, 2023
On macOS Sonoma 14.0, images that are set as background image on SWT controls are upside-down.
See eclipse-platform/eclipse.platform.swt#772.

The workaround checks for the presence of the bug and flips the splash screen background image such that it gets flipped
back by the bug to its normal position.

AP-21237 (Splash screen is displayed upside down)
knime-github pushed a commit to knime/knime-product that referenced this issue Oct 12, 2023
On macOS Sonoma 14.0, images that are set as background image on SWT controls are upside-down.
See eclipse-platform/eclipse.platform.swt#772.

The workaround checks for the presence of the bug and flips the splash screen background image such that it gets flipped
back by the bug to its normal position.

AP-21237 (Splash screen is displayed upside down)
knime-github pushed a commit to knime/knime-product that referenced this issue Oct 12, 2023
On macOS Sonoma 14.0, images that are set as background image on SWT controls are upside-down.
See eclipse-platform/eclipse.platform.swt#772.

The workaround checks for the presence of the bug and flips the splash screen background image such that it gets flipped
back by the bug to its normal position.

AP-21237 (Splash screen is displayed upside down)
@Phillipus
Copy link
Contributor Author

Phillipus commented Oct 15, 2023

One of referenced commits from @enplotz above has a nice workaround for the splash screen if anyone is interested:

knime/knime-product@095c88e

The code flips the image which you can set in the splash handler's background image. I've re-purposed it to use in our RCP app's splash handler:

private Image flipImage(Display display, Image srcImage) {
    Rectangle bounds = srcImage.getBounds();
    Image newImage = new Image(display, bounds.width, bounds.height);
    
    GC gc = new GC(newImage);
    gc.setAdvanced(true);
    gc.setAntialias(SWT.ON);
    gc.setInterpolation(SWT.HIGH);
    
    Transform transform = new Transform(display);
    transform.setElements(1, 0, 0, -1, 0, 0);
    transform.translate(0, -bounds.height);
    gc.setTransform(transform);
    
    gc.drawImage(srcImage, 0, 0, bounds.width, bounds.height,
                           0, 0, bounds.width, bounds.height);
    
    gc.dispose();
    transform.dispose();
    
    return newImage;
}

Note that the method there isFlipBugPresent() doesn't work so you'll have to test for whether the bug is present in another way.

I test for it like this:

if(PlatformUtils.isMac() && PlatformUtils.compareOSVersion("14.0") >= 0) {
    // Flip the image
    shell.setBackgroundImage(flipImage(shell.getDisplay(), shell.getBackgroundImage()));
    
    // We are now responsible for disposing of the new image
    shell.addDisposeListener(e -> {
        shell.getBackgroundImage().dispose();
    });
}

@swtdev
Copy link

swtdev commented Oct 15, 2023

@Phillipus unfortunately the workaround doesn't work. I was testing on pre-Sonoma and seems flip-detection won't do the trick, unfortunately - see discussion ->

@Phillipus
Copy link
Contributor Author

Phillipus commented Oct 15, 2023

@swtdev Yes, that's why I said "Note that the method there isFlipBugPresent() doesn't work so you'll have to test for whether the bug is present in another way."

@swtdev
Copy link

swtdev commented Oct 15, 2023

Lol would help if I'd read the whole comment 🙃

@Phillipus
Copy link
Contributor Author

But, it would be better if a Mac expert could say why the image is flipped when calling the native ObjectiveC methods. Perhaps the co-ordinate system changed for colorWithPatternImage?

@SyntevoAlex
Copy link
Member

I suspect that it's related to flipping in Image, see for example

NSGraphicsContext flippedContext = NSGraphicsContext.graphicsContextWithGraphicsPort(context.graphicsPort(), true);
context = flippedContext;
context.retain();
if (data != null) data.flippedContext = flippedContext;

@zulus
Copy link

zulus commented Nov 1, 2023

Reason might be even simpler. When application starts, NSView created by native code have isFlipped marked as false, but later, all views created via SWTCanvasView.alloc() have isFlipped marked as true (which is strange, I cannot find reason why). So looks like there is "isFlipped" mix between views and display context

@Phillipus
Copy link
Contributor Author

Reason might be even simpler. When application starts, NSView created by native code have isFlipped marked as false, but later, all views created via SWTCanvasView.alloc() have isFlipped marked as true (which is strange, I cannot find reason why). So looks like there is "isFlipped" mix between views and display context

Are you able to determine where this isFlipped is set?

sratz added a commit that referenced this issue Nov 6, 2023
Image flipping was originally implemented back in 2008 [1,2]. Possibly
as a workaround.

It appears something has changed with macOS Sonoma (14.0), and this is
no longer necessary.

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=254797
[2] ca92157

Resolves #772.
@sratz sratz linked a pull request Nov 6, 2023 that will close this issue
@sratz
Copy link
Member

sratz commented Nov 6, 2023

There is another flipped creation of the context in Control:

NSGraphicsContext flippedContext = NSGraphicsContext.graphicsContextWithGraphicsPort(graphicsContext.graphicsPort(), true);
graphicsContext = flippedContext;

I have provided a pull-request that fixes both of the snippets in #772 (comment) and #772 (comment)

@sratz
Copy link
Member

sratz commented Nov 6, 2023

Ok, well, turns out this is way more complicated and doesn't solve all the issues... In fact, it causes line numbers in the editor to be flipped now.

I found this:
https://developer.apple.com/forums/thread/736618

Looks like we are also in that case 3, as we configure isFlipped:


for some of the widgets:

  • SWTCanvasView:
    className = "SWTCanvasView";
    cls = OS.objc_allocateClassPair(OS.class_NSView, className, 0);
    OS.class_addIvar(cls, SWT_OBJECT, size, (byte)align, types);
    //NSTextInput protocol
    OS.class_addProtocol(cls, OS.protocol_NSTextInput);
    OS.class_addMethod(cls, OS.sel_hasMarkedText, proc2, "@:");
    OS.class_addMethod(cls, OS.sel_markedRange, markedRangeProc, "@:");
    OS.class_addMethod(cls, OS.sel_selectedRange, selectedRangeProc, "@:");
    OS.class_addMethod(cls, OS.sel_setMarkedText_selectedRange_, setMarkedText_selectedRangeProc, "@:@{NSRange}");
    OS.class_addMethod(cls, OS.sel_unmarkText, proc2, "@:");
    OS.class_addMethod(cls, OS.sel_validAttributesForMarkedText, proc2, "@:");
    OS.class_addMethod(cls, OS.sel_attributedSubstringFromRange_, attributedSubstringFromRangeProc, "@:{NSRange}");
    OS.class_addMethod(cls, OS.sel_insertText_, proc3, "@:@");
    OS.class_addMethod(cls, OS.sel_characterIndexForPoint_, characterIndexForPointProc, "@:{NSPoint}");
    OS.class_addMethod(cls, OS.sel_firstRectForCharacterRange_, firstRectForCharacterRangeProc, "@:{NSRange}");
    OS.class_addMethod(cls, OS.sel_doCommandBySelector_, proc3, "@::");
    //NSTextInput protocol end
    OS.class_addMethod(cls, OS.sel_canBecomeKeyView, proc2, "@:");
    OS.class_addMethod(cls, OS.sel_isFlipped, isFlippedProc, "@:");
  • SWTImageView:
    className = "SWTImageView";
    cls = OS.objc_allocateClassPair(OS.class_NSImageView, className, 0);
    OS.class_addIvar(cls, SWT_OBJECT, size, (byte)align, types);
    OS.class_addMethod(cls, OS.sel_isFlipped, isFlippedProc, "@:");
  • SWTController:
    className = "SWTScroller";
    cls = OS.objc_allocateClassPair(OS.class_NSScroller, className, 0);
    OS.class_addIvar(cls, SWT_OBJECT, size, (byte)align, types);
    OS.class_addMethod(cls, OS.sel_sendSelection, proc2, "@:");
    addEventMethods(cls, proc2, proc3, drawRectProc, hitTestProc, setNeedsDisplayInRectProc);
    addFrameMethods(cls, setFrameOriginProc, setFrameSizeProc);
    addAccessibilityMethods(cls, proc2, proc3, proc4, accessibilityHitTestProc);
    OS.objc_registerClassPair(cls);
    /* Note that isFlippedProc is used for performance and convenience */
    long metaClass = OS.objc_getMetaClass(className);
    OS.class_addMethod(metaClass, OS.sel_isCompatibleWithOverlayScrollers, isFlippedProc, "@:");
  • SWTView:
    className = "SWTView";
    cls = OS.objc_allocateClassPair(OS.class_NSView, className, 0);
    OS.class_addIvar(cls, SWT_OBJECT, size, (byte)align, types);
    OS.class_addMethod(cls, OS.sel_canBecomeKeyView, proc2, "@:");
    OS.class_addMethod(cls, OS.sel_isFlipped, isFlippedProc, "@:");

The corresponding native code is

#ifndef NO_isFlipped_1CALLBACK
static BOOL isFlippedProc(id obj, SEL sel)
{
return YES;
}
JNIEXPORT jlong JNICALL OS_NATIVE(isFlipped_1CALLBACK)
(JNIEnv *env, jclass that)
{
return (jlong)isFlippedProc;
}
#endif

@Phillipus
Copy link
Contributor Author

That clearly affects things. I commented out this line and the image displays correctly in the Snippets:

OS.class_addMethod(cls, OS.sel_isFlipped, isFlippedProc, "@:");

@zulus
Copy link

zulus commented Nov 9, 2023

In general looks like flipped property isn't correctly overriden in SWTCanvasView and others (overriding getter is not enough in sonoma probably due SWIFT bindings), class_addProperty should be used

elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Dec 1, 2023
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Dec 1, 2023
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
sravanlakkimsetti pushed a commit to eclipse-platform/eclipse.platform.ui that referenced this issue Dec 1, 2023
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
@lshanmug lshanmug reopened this Dec 1, 2023
DedunuKarunarathne added a commit to DedunuKarunarathne/integration-studio-public that referenced this issue Feb 29, 2024
elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Mar 19, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Mar 19, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Mar 19, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to eclipse-platform/eclipse.platform.ui that referenced this issue Mar 20, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to eclipse-platform/eclipse.platform.ui that referenced this issue Mar 20, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to eclipse-platform/eclipse.platform.ui that referenced this issue Mar 20, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to elsazac/eclipse.platform.ui that referenced this issue Mar 22, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
elsazac added a commit to eclipse-platform/eclipse.platform.ui that referenced this issue Mar 22, 2024
Fixes: eclipse-platform/eclipse.platform.swt#772
Due to a bug in MacOS
Sonoma(eclipse-platform/eclipse.platform.swt#772)
,Splash Screen gets flipped.As a workaround the image is flipped and
returned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed macOS happens on macOS
Projects
None yet
Development

Successfully merging a pull request may close this issue.