-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
People detect Lighthouse to cheat its performance score #15829
Comments
I will take care of making our client hints not signal Lighthouse, and changing the PSI UA header. |
Next Lighthouse release will address the
I'm not convinced we should do this for purposes of evading detection. This is the same value that an actual Linux user would have. Are you seeing this used to detect Lighthouse? It'd be a pretty big false signal.. but, maybe worth doing to match the fake UA we set for mobile/desktop. @paulirish wdyt |
I am generally not a fan of removing the ability to identify LH traffic. It is important to able to identify Lighthouse traffic in monitoring and firewall contexts. As we know, cheating Lighthouse doesn't improve Core web vitals for visitors. Yes, there are bad actors selling webperf plugins that intentionally mislead site owners. My take: Abuse reports should be delivered to the platforms who host problematic plugins. Services like Wordpress and Shopify must maintain quality within their own marketplace. Identification of traffic is important to software teams, agencies, consultants and all the folks who are focused on making positive webperf outcomes. |
I figured it may be more appropriate for DevTools to handle it as a part of the device emulation. I created a bug against Chromium: https://issues.chromium.org/issues/326791407 @benschwarz In principle I agree with your statement. This is yet another case of "this is why we can't have nice things". We spent months to uncover as many cases as possible and I believe there are still more that we didn't identify. In reality, most platforms don't have enough resources to address this at all. At the same time, there are real non-technical people paying $ to "performance experts" taking advantage of this. |
@krzksz I'm not sure I agree this is a case of "this is why we can't have nice things". From my perspective it's a case of "Shopifys job is difficult because they have a large addressable market." or "Shopify must protect customers who use their marketplace". |
Summary
Hey! I work as a performance technical architect at Shopify. Lighthouse (PSI) has always been one of the go-to tools for our merchants when it comes to checking the performance of their stores. Unfortunately this ended up as an opportunity for some bad 3rd party actors to offer "speed optimisation services" that heavily strip down the page content when the Lighthouse test is detected. This generates a false image of the actual experience real users are getting, preventing store owners from even learning it's worth to improve.
Most popular techniques
Here are the most popular techniques that I found during the investigation:
Lighthouse
string in the user agent.Linux x86_64
value ofnavigator.platform
.moto g power
string in the user agent.After Lighthouse is detected, there are usually scripts in place that do a mix of the following:
document.close()
.Proposed solution
While we're working internally to address this issue, we probably won't be able to find and fix all of the affected pages. The malicious JavaScript is usually heavily obfuscated so it's not always as simple as looking for one of the strings mentioned above. That's why I thought it would be good to fix the problem at its root instead.
When it comes to the UA, I can see it was already removed by @paulirish in this PR. That said, it seems like PSI still appends the
Chrome-Lighthouse
string. The information still leaks through theSec-Ch-Ua
header as well through:lighthouse/core/lib/emulation.js
Line 32 in d251755
My first proposal would be to remain consistent and remove those completely.
I think that the
navigator.platform
issue can be addressed by adding the following logic somewhere inside the emulate function:This should keep its value consistent with the rest of the user agent.
For the
moto g power
I don't think there are any elegant ways to address it. If someone wants to risk breaking the page for the actual users of this device then so be it.Let me know what you think. I'm happy to prepare a PR with the changes mentioned above so we can make sure it's not so easy to cheat people with those shady practices in the future.
The text was updated successfully, but these errors were encountered: