Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeing Warning Logs :: Cannot refresh Redis Cluster topology in 6.2.6.RELEASE #2843

Open
subrajitdhal opened this issue Apr 30, 2024 · 4 comments
Labels
status: waiting-for-feedback We need additional information before we can continue

Comments

@subrajitdhal
Copy link

subrajitdhal commented Apr 30, 2024

Bug Report

Current Behavior

I've found that the WARN log message is "Cannot refresh Redis Cluster topology", after AWS Elasticache instance reboot.
Although the log was printed every topology refresh period, all requests was fine.NO connection issue though.it just print warning logs.

Unable to connect to [xxxxxxx:port]: PKIX path validation failed: java.security.cert.CertPathValidatorException: Unable to determine revocation status due to network error

Stack trace
i.l.c.c.t.DefaultClusterTopologyRefresh$CannotRetrieveClusterPartitions: Cannot retrieve cluster partitions from [rediss://********************@xxxxxx:port]

Details:
	[rediss://********************@xxxxx:port]: PKIX path validation failed: java.security.cert.CertPathValidatorException: Unable to determine revocation status due to network error

	Suppressed: java.io.EOFException: not enough content
		at sun.security.util.DerValue.<init>(DerValue.java:425)
		at sun.security.util.DerValue.<init>(DerValue.java:340)
		at s.s.provider.certpath.OCSPResponse.<init>(OCSPResponse.java:200)
		at sun.security.provider.certpath.OCSP.check(OCSP.java:196)
		at s.s.provider.certpath.RevocationChecker.checkOCSP(RevocationChecker.java:785)
		at s.s.provider.certpath.RevocationChecker.check(RevocationChecker.java:369)

Input Code

Input Code
  private val topologyRefreshOptions: ClusterTopologyRefreshOptions = ClusterTopologyRefreshOptions
    .builder()
    .enablePeriodicRefresh(RedisClusterTopologyRefreshInterval)
    .enableAllAdaptiveRefreshTriggers()
    .dynamicRefreshSources(true)
    .build()

Expected behavior/code

"Cannot refresh Redis Cluster topology" log should no longer be printed.

Environment

  • Lettuce version(s): 6.2.6.RELEASE

Possible Solution

Additional context

@tishun
Copy link
Collaborator

tishun commented May 23, 2024

Hey @subrajitdhal ,

How do you verify that "all requests are fine. NO connection issue though."?

From the code I see in Lettuce the WARN message you are seeing is indicating that the topology refresh has failed.
The indicated issue is that the revocation status of the certificate could not be established due to network error.

Have in mind that a topology refresh failing would not stop the driver from operating and using the topology it already has from previous attempts, but the topology information would not be updated until the network issue is resolved.

@tishun tishun added the status: waiting-for-feedback We need additional information before we can continue label May 23, 2024
@subrajitdhal
Copy link
Author

Hi @tishun ,
Thanks for checking it.
This error we are seeing suddenly though there is no change from application.We have also tried use correct java trust store location.Any pointer would be greatly help.

@tishun
Copy link
Collaborator

tishun commented Jun 3, 2024

I am afraid I do not have enough information to help diagnose the issue.

The only pointer I have is that, the Lettuce driver, while attempting to refresh the topology by connecting to rediss://********************@xxxxx:port fails to establish a connection because there is some network connectivity issue.

If otherwise the driver is able to connect to the same instance then I have no clue why this specific network connection fails.

@subrajitdhal
Copy link
Author

Hi @tishun ,
Up on further checking we see this warning logs are more frequent in app which are using AWS based Correto Java 17 base images.Where as app which are using Alepine java 11 this warning logs are very very less

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: waiting-for-feedback We need additional information before we can continue
Projects
None yet
Development

No branches or pull requests

2 participants