Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support premature termination of listing #928

Merged
merged 3 commits into from
Apr 15, 2024

Conversation

EndzeitBegins
Copy link
Contributor

Adds the notion of a RemoteResourceSelector which is a more powerful alternative of the existing RemoteResourceFilter allowing to terminate a listing prematurely.

An improvement to Apache NiFi aims to support listing a configured maximum of remote file resources.
As of now, this can be achieved by providing a RemoteResourceFilter that returns false for all RemoteResourceInfo after the configured threshold has been reached. However, sshj will list all files under the hood nonetheless, which has huge affect on listing performance, especially for directories a with large amount of files.

The PR also resolved a TODO to remove the GOTO that resided in the code touched.

Please let me know if this is a good trajectory for addressing the underlying problem stated or whether it should be addressed differently or any other changes should be incorporated.

@EndzeitBegins EndzeitBegins closed this by deleting the head repository Feb 24, 2024
@EndzeitBegins EndzeitBegins reopened this Feb 24, 2024
@hierynomus hierynomus merged commit 624fe83 into hierynomus:master Apr 15, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants