Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

help with libuv support #56

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

help with libuv support #56

wants to merge 1 commit into from

Conversation

certxlm
Copy link

@certxlm certxlm commented Dec 26, 2020

Hello,

While testing async mode with python binding (async-get-ip.py), it clearly becomes apparent that the script does not scale well. For 100 domains, some requests fall in timeout. For 500 (see examples/samples.txt), all of them fall in timeout.

./examples/async-get-ip.py $(cat examples/samples.txt|xargs)

As stated in getdns issue 257, one of the possible cause is that the provided loop should be avoided, and an external event loop should be used: issuecomment-276311860

Hence this draft pull request. Which implements an "getdns.UVContext" subtype, to build with the following command:

python ./setup.py build --with-ext-uv

However, much to our dismay, the performances are equally .... not satisfying. The same test with 500 domains ends up with every requests falling in timeout, again.

./examples/async-uv-get-ip.py $(cat examples/samples.txt|xargs)

So the question is, what exactly are we doing wrong? Is there a way to add queries to a context and have the underlying event loop resolve them without ending up in timeout?
Our goal is to have a python script testing then of thousands of domains continuesly, as part of a monitoring tool.

Many thanks for any help you could provide...

Regards,
CERT-XML

@MelindaShore
Copy link
Contributor

Happy New Year, and thanks for adding the libuv event loop! I'll review it tonight after I'm done with $DAYJOB.

TBH it's been a couple of years since I've looked at this closely and fortunately Python profiling tools have improved over that time, so I think we're in better shape to track down problems in the module. That said, I'm not that optimistic about being able to isolate the problem to the getdns code, given the findings in this paper. However, you've inspired me to start digging into it again.

Thanks again, and I'll keep you posted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants