Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Impeller] Replace Impeller opacity peephole delegate with DL variant. #52707

Merged
merged 15 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
63 changes: 0 additions & 63 deletions impeller/aiks/aiks_unittests.cc
Original file line number Diff line number Diff line change
Expand Up @@ -494,30 +494,6 @@ TEST_P(AiksTest, CanEmptyPictureConvertToImage) {
ASSERT_TRUE(OpenPlaygroundHere(canvas.EndRecordingAsPicture()));
}

TEST_P(AiksTest, CanRenderGroupOpacity) {
Canvas canvas;

Paint red;
red.color = Color::Red();
Paint green;
green.color = Color::Green().WithAlpha(0.5);
Paint blue;
blue.color = Color::Blue();

Paint alpha;
alpha.color = Color::Red().WithAlpha(0.5);

canvas.SaveLayer(alpha);

canvas.DrawRect(Rect::MakeXYWH(000, 000, 100, 100), red);
canvas.DrawRect(Rect::MakeXYWH(020, 020, 100, 100), green);
canvas.DrawRect(Rect::MakeXYWH(040, 040, 100, 100), blue);

canvas.Restore();

ASSERT_TRUE(OpenPlaygroundHere(canvas.EndRecordingAsPicture()));
}

TEST_P(AiksTest, CoordinateConversionsAreCorrect) {
Canvas canvas;

Expand Down Expand Up @@ -1634,45 +1610,6 @@ TEST_P(AiksTest, PaintWithFilters) {
ASSERT_FALSE(paint.HasColorFilter());
}

TEST_P(AiksTest, OpacityPeepHoleApplicationTest) {
auto entity_pass = std::make_shared<EntityPass>();
auto rect = Rect::MakeLTRB(0, 0, 100, 100);
Paint paint;
paint.color = Color::White().WithAlpha(0.5);
paint.color_filter =
ColorFilter::MakeBlend(BlendMode::kSourceOver, Color::Blue());

// Paint has color filter, can't elide.
auto delegate = std::make_shared<OpacityPeepholePassDelegate>(paint);
ASSERT_FALSE(delegate->CanCollapseIntoParentPass(entity_pass.get()));

paint.color_filter = nullptr;
paint.image_filter = ImageFilter::MakeBlur(Sigma(1.0), Sigma(1.0),
FilterContents::BlurStyle::kNormal,
Entity::TileMode::kClamp);

// Paint has image filter, can't elide.
delegate = std::make_shared<OpacityPeepholePassDelegate>(paint);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be the case where I differ in assumptions. DL allows IF+opacity, but it looks like AIKS expects it to be disallowed...?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We always fold the opacity into the image filter compositing, so in effect its as if the peephole wasn't applied.

ASSERT_FALSE(delegate->CanCollapseIntoParentPass(entity_pass.get()));

paint.image_filter = nullptr;
paint.color = Color::Red();

// Paint has no alpha, can't elide;
delegate = std::make_shared<OpacityPeepholePassDelegate>(paint);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh? Why can't opaque colors be peepholed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just old overly defensive code.

ASSERT_FALSE(delegate->CanCollapseIntoParentPass(entity_pass.get()));

// Positive test.
Entity entity;
entity.SetContents(SolidColorContents::Make(
PathBuilder{}.AddRect(rect).TakePath(), Color::Red()));
entity_pass->AddEntity(std::move(entity));
paint.color = Color::Red().WithAlpha(0.5);

delegate = std::make_shared<OpacityPeepholePassDelegate>(paint);
ASSERT_TRUE(delegate->CanCollapseIntoParentPass(entity_pass.get()));
}

TEST_P(AiksTest, DrawPaintAbsorbsClears) {
Canvas canvas;
canvas.DrawPaint({.color = Color::Red(), .blend_mode = BlendMode::kSource});
Expand Down
20 changes: 12 additions & 8 deletions impeller/aiks/canvas.cc
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@ void Canvas::Save(bool create_subpass,
entry.transform = transform_stack_.back().transform;
entry.cull_rect = transform_stack_.back().cull_rect;
entry.clip_height = transform_stack_.back().clip_height;
entry.distributed_opacity = transform_stack_.back().distributed_opacity;
if (create_subpass) {
entry.rendering_mode = Entity::RenderingMode::kSubpass;
auto subpass = std::make_unique<EntityPass>();
Expand Down Expand Up @@ -827,6 +828,7 @@ void Canvas::AddRenderEntityToCurrentPass(Entity entity, bool reuse_depth) {
++current_depth_;
}
entity.SetClipDepth(current_depth_);
entity.SetInheritedOpacity(transform_stack_.back().distributed_opacity);
GetCurrentPass().AddEntity(std::move(entity));
}

Expand All @@ -838,8 +840,16 @@ void Canvas::SaveLayer(const Paint& paint,
std::optional<Rect> bounds,
const std::shared_ptr<ImageFilter>& backdrop_filter,
ContentBoundsPromise bounds_promise,
uint32_t total_content_depth) {
uint32_t total_content_depth,
bool can_distribute_opacity) {
TRACE_EVENT0("flutter", "Canvas::saveLayer");
if (can_distribute_opacity) {
FML_DCHECK(!backdrop_filter);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@flar I'm surprised but I hit this check in some tests. Do I need to assume that even if can_distribute_opacity is true, there may be other paint states (blend mode/ filters) that can still require a save layer?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which tests? I can take a look and see if there might be something wrong with the DL checks. In particular, I think I only pass SrcOver for now even though other blend modes might cause problems. I disallow ColorFilters, but I allow ImageFilters because according to my investigations the opacity should be applied to the output of the ImageFilter so it shouldn't matter. (It's applied to the input of the ColorFilter, though, so I just disallow them even though I could maybe look at their details and find that some of them are compatible).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, are you specifically referring to backdrop filter vs opacity flag? I notice that I don't turn off the flag in that case, but perhaps I should. The flag is more about the contents and whether the contents are compatible.

I also don't think I check the attributes that apply the SL to the parent. If they aren't SrcOver then I don't think we can do the opacity stuff.

I also don't believe the "old code" (pre-reorg) does this either. Let me get the reorg back in and then I can revisit layer properties and how they affect the flag.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Save(false, total_content_depth, paint.blend_mode, backdrop_filter);
transform_stack_.back().distributed_opacity *= paint.color.alpha;
return;
}

Save(true, total_content_depth, paint.blend_mode, backdrop_filter);

// The DisplayList bounds/rtree doesn't account for filters applied to parent
Expand All @@ -861,13 +871,7 @@ void Canvas::SaveLayer(const Paint& paint,
new_layer_pass.SetRequiredMipCount(mip_count_visitor.GetRequiredMipCount());
}

// Only apply opacity peephole on default blending.
if (paint.blend_mode == BlendMode::kSourceOver) {
new_layer_pass.SetDelegate(
std::make_shared<OpacityPeepholePassDelegate>(paint));
} else {
new_layer_pass.SetDelegate(std::make_shared<PaintPassDelegate>(paint));
}
new_layer_pass.SetDelegate(std::make_shared<PaintPassDelegate>(paint));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this transfer the existing distributed opacity from the transform_stack.back() to the new layer pass?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, if this saveLayer didn't pass along the distributed opacity above, then it needs to apply it to the blit-back to the parent and I don't see where that is happening. Should PaintPassDelegate carry along an extra parameter for the distributed opacity from outside the saveLayer?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, this is fixed and a test case is added.

}

void Canvas::DrawTextFrame(const std::shared_ptr<TextFrame>& text_frame,
Expand Down
4 changes: 3 additions & 1 deletion impeller/aiks/canvas.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ struct CanvasStackEntry {
size_t clip_height = 0u;
// The number of clips tracked for this canvas stack entry.
size_t num_clips = 0u;
Scalar distributed_opacity = 1.0f;
Entity::RenderingMode rendering_mode = Entity::RenderingMode::kDirect;
};

Expand Down Expand Up @@ -75,7 +76,8 @@ class Canvas {
std::optional<Rect> bounds = std::nullopt,
const std::shared_ptr<ImageFilter>& backdrop_filter = nullptr,
ContentBoundsPromise bounds_promise = ContentBoundsPromise::kUnknown,
uint32_t total_content_depth = kMaxDepth);
uint32_t total_content_depth = kMaxDepth,
bool can_distribute_opacity = false);

virtual bool Restore();

Expand Down
11 changes: 10 additions & 1 deletion impeller/aiks/experimental_canvas.cc
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@ void ExperimentalCanvas::Save(uint32_t total_content_depth) {
entry.transform = transform_stack_.back().transform;
entry.cull_rect = transform_stack_.back().cull_rect;
entry.clip_depth = current_depth_ + total_content_depth;
entry.distributed_opacity = transform_stack_.back().distributed_opacity;
FML_CHECK(entry.clip_depth <= transform_stack_.back().clip_depth)
<< entry.clip_depth << " <=? " << transform_stack_.back().clip_depth
<< " after allocating " << total_content_depth;
Expand All @@ -207,7 +208,13 @@ void ExperimentalCanvas::SaveLayer(
std::optional<Rect> bounds,
const std::shared_ptr<ImageFilter>& backdrop_filter,
ContentBoundsPromise bounds_promise,
uint32_t total_content_depth) {
uint32_t total_content_depth,
bool can_distribute_opacity) {
if (can_distribute_opacity) {
Save(total_content_depth);
transform_stack_.back().distributed_opacity *= paint.color.alpha;
return;
}
// Can we always guarantee that we get a bounds? Does a lack of bounds
// indicate something?
if (!bounds.has_value()) {
Expand Down Expand Up @@ -393,6 +400,8 @@ void ExperimentalCanvas::DrawTextFrame(

void ExperimentalCanvas::AddRenderEntityToCurrentPass(Entity entity,
bool reuse_depth) {
entity.SetInheritedOpacity(transform_stack_.back().distributed_opacity);

auto transform = entity.GetTransform();
entity.SetTransform(
Matrix::MakeTranslation(Vector3(-GetGlobalPassPosition())) * transform);
Expand Down
3 changes: 2 additions & 1 deletion impeller/aiks/experimental_canvas.h
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,8 @@ class ExperimentalCanvas : public Canvas {
std::optional<Rect> bounds,
const std::shared_ptr<ImageFilter>& backdrop_filter,
ContentBoundsPromise bounds_promise,
uint32_t total_content_depth) override;
uint32_t total_content_depth,
bool can_distribute_opacity) override;

bool Restore() override;

Expand Down
103 changes: 0 additions & 103 deletions impeller/aiks/paint_pass_delegate.cc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
#include "impeller/aiks/paint_pass_delegate.h"

#include "impeller/core/formats.h"
#include "impeller/core/sampler_descriptor.h"
#include "impeller/entity/contents/contents.h"
#include "impeller/entity/contents/texture_contents.h"
#include "impeller/entity/entity_pass.h"
Expand Down Expand Up @@ -54,106 +53,4 @@ std::shared_ptr<FilterContents> PaintPassDelegate::WithImageFilter(
Entity::RenderingMode::kSubpass);
}

/// OpacityPeepholePassDelegate
/// ----------------------------------------------

OpacityPeepholePassDelegate::OpacityPeepholePassDelegate(Paint paint)
: paint_(std::move(paint)) {}

// |EntityPassDelgate|
OpacityPeepholePassDelegate::~OpacityPeepholePassDelegate() = default;

// |EntityPassDelgate|
bool OpacityPeepholePassDelegate::CanElide() {
return paint_.blend_mode == BlendMode::kDestination;
}

// |EntityPassDelgate|
bool OpacityPeepholePassDelegate::CanCollapseIntoParentPass(
EntityPass* entity_pass) {
// Passes with enforced bounds that clip the contents can not be safely
// collapsed.
if (entity_pass->GetBoundsLimitMightClipContent()) {
return false;
}

// OpacityPeepholePassDelegate will only get used if the pass's blend mode is
// SourceOver, so no need to check here.
if (paint_.color.alpha <= 0.0 || paint_.color.alpha >= 1.0 ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the alpha is 0 then we should skip the pass entirely, so I'm not sure why it is saying that it "can't collapse into parent" here (i.e. return false).

Also, if the paint is opaque, then that isn't a reason not to collapse the pass...? It just means you have no partial opacity to distribute, but you can still collapse it and distribute the opacity of 1.0 (i.e. NOP on modifying the opacity).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I was just being overly conservative at the time.

paint_.image_filter || paint_.color_filter) {
return false;
}

// Note: determing whether any coverage intersects has quadradic complexity in
// the number of rectangles, and depending on whether or not we cache at
// different levels of the entity tree may end up cubic. In the interest of
// proving whether or not this optimization is valuable, we only consider very
// simple peephole optimizations here - where there is a single drawing
// command wrapped in save layer. This would indicate something like an
// Opacity or FadeTransition wrapping a very simple widget, like in the
// CupertinoPicker.
if (entity_pass->GetElementCount() > 3) {
// Single paint command with a save layer would be:
// 1. clip
// 2. draw command
// 3. restore.
return false;
}
bool all_can_accept = true;
std::vector<Rect> all_coverages;
auto had_subpass = entity_pass->IterateUntilSubpass(
[&all_coverages, &all_can_accept](Entity& entity) {
const auto& contents = entity.GetContents();
if (!entity.CanInheritOpacity()) {
all_can_accept = false;
return false;
}
auto maybe_coverage = contents->GetCoverage(entity);
if (maybe_coverage.has_value()) {
auto coverage = maybe_coverage.value();
for (const auto& cv : all_coverages) {
if (cv.IntersectsWithRect(coverage)) {
all_can_accept = false;
return false;
}
}
all_coverages.push_back(coverage);
}
return true;
});
if (had_subpass || !all_can_accept) {
return false;
}
auto alpha = paint_.color.alpha;
entity_pass->IterateUntilSubpass([&alpha](Entity& entity) {
entity.SetInheritedOpacity(alpha);
return true;
});
return true;
}

// |EntityPassDelgate|
std::shared_ptr<Contents>
OpacityPeepholePassDelegate::CreateContentsForSubpassTarget(
std::shared_ptr<Texture> target,
const Matrix& effect_transform) {
auto contents = TextureContents::MakeRect(Rect::MakeSize(target->GetSize()));
contents->SetLabel("Subpass");
contents->SetTexture(target);
contents->SetSourceRect(Rect::MakeSize(target->GetSize()));
contents->SetOpacity(paint_.color.alpha);
contents->SetDeferApplyingOpacity(true);

return paint_.WithFiltersForSubpassTarget(std::move(contents),
effect_transform);
}

// |EntityPassDelgate|
std::shared_ptr<FilterContents> OpacityPeepholePassDelegate::WithImageFilter(
const FilterInput::Variant& input,
const Matrix& effect_transform) const {
return paint_.WithImageFilter(input, effect_transform,
Entity::RenderingMode::kSubpass);
}

} // namespace impeller
37 changes: 0 additions & 37 deletions impeller/aiks/paint_pass_delegate.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,43 +43,6 @@ class PaintPassDelegate final : public EntityPassDelegate {
PaintPassDelegate& operator=(const PaintPassDelegate&) = delete;
};

/// A delegate that attempts to forward opacity from a save layer to
/// child contents.
///
/// Currently this has a hardcoded limit of 3 entities in a pass, and
/// cannot forward to child subpass delegates.
class OpacityPeepholePassDelegate final : public EntityPassDelegate {
public:
explicit OpacityPeepholePassDelegate(Paint paint);

// |EntityPassDelgate|
~OpacityPeepholePassDelegate() override;

// |EntityPassDelgate|
bool CanElide() override;

// |EntityPassDelgate|
bool CanCollapseIntoParentPass(EntityPass* entity_pass) override;

// |EntityPassDelgate|
std::shared_ptr<Contents> CreateContentsForSubpassTarget(
std::shared_ptr<Texture> target,
const Matrix& effect_transform) override;

// |EntityPassDelgate|
std::shared_ptr<FilterContents> WithImageFilter(
const FilterInput::Variant& input,
const Matrix& effect_transform) const override;

private:
const Paint paint_;

OpacityPeepholePassDelegate(const OpacityPeepholePassDelegate&) = delete;

OpacityPeepholePassDelegate& operator=(const OpacityPeepholePassDelegate&) =
delete;
};

} // namespace impeller

#endif // FLUTTER_IMPELLER_AIKS_PAINT_PASS_DELEGATE_H_
1 change: 1 addition & 0 deletions impeller/display_list/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ impeller_component("display_list") {
template("display_list_unittests_component") {
target_name = invoker.target_name
predefined_sources = [
"aiks_dl_opacity_unittests.cc",
"aiks_dl_path_unittests.cc",
"dl_golden_unittests.cc",
"dl_playground.cc",
Expand Down