powerdns recursor uses only 1 core

[chelezo]

powerdns recursor for some reason uses only one core out of 4 available (on stress test one core was loaded at 100% while the others were free)

[rgacogne]

Please share your configuration..

[chelezo]

dnssec:
  validation: off
incoming:
  listen:
  - 10.8.2.144:5301
logging:
  facility: '1'
  loglevel: 7
recursor:
   forward_zones:
     - zone: tests.lans
       forwarders:
       - 10.8.2.141:5350
     - zone: mps.lan
       forwarders:
       - 10.8.2.141:5350
   forward_zones_recurse:
     - zone: .
       recurse: true
       forwarders:
       - 10.10.10.4:53
   threads: 4

[omoerbeek]

Check https://docs.powerdns.com/recursor/performance.html#threading-and-distribution-of-queries

[omoerbeek]

Also of importance: how did you stress test? If you use a single client it is likely the kernel will distribute queries to a single thread only.

[chelezo]

I use dnsperf on the highest possible settings.

[omoerbeek]

Please show the exact command line.

[chelezo]

dnsperf -s 10.8.2.140 -d dnsperf-example.txt -t 5 -l 1000 -c 256 -q 65536

[omoerbeek]

• Try adding -T 8 or 16 to dnsperf
• Try running multiple instances

Testing from a single client remains a poor substitute for a real world test.

If this all doesn't work for you for some yet unknown reason (I'll try to reproduce, but for that I need to know your distribution version and kernel version), you can as a workaround make the recursor do the thread picking instead of the kernel by setting

• https://docs.powerdns.com/recursor/yamlsettings.html#incoming-pdns-distributes-queries to true and
• https://docs.powerdns.com/recursor/yamlsettings.html#incoming-reuseport to false.

[omoerbeek]

Not that I made an error here, which has been corrected: the second bullet talks bout reuseport

[omoerbeek]

Oh, important: I noticed you use dnsperf -s 10.8.2.140 but the recursor listens on 10.8.2.144:5301. What's in-between?
Describe your test environment in detail, these things matter.

[chelezo]

I have 5 identical recursors) in a cluster)

[omoerbeek]

Sigh... I asked to describe your setup in detail to be able to understand and maybe reproduce what you are seeing. You keep on given us vague information. What kind of cluster? What OS version and what other software is involved?

[chelezo]

dnsdist (2 instances combined vrrp) --- followed by 5 powerdns outch and 5 powerdns recursor. do you need detailed configuration of each? they all have the same settings, only the listing changes.

[omoerbeek]

A lot of questions remain open. Some examples:

• Which OS?
• Bare metal? VMs? Docker? General description of (network) setup.
• Test setup details: do you run dnsperf against a recursor directly or against dnsdist which talks to the recursor? Do you run the dnsperf on a different machine?
• I suggested experiments with settings, no feedback on that

Please read https://blog.powerdns.com/2016/01/18/open-source-support-out-in-the-open and act on it

You are making this way more time consuming than needed. This is really frustrating for someone trying to help.

[chelezo]

dnsdist auth recursor. Installed on VM (host machine ESXi) communication inside the powerdns complex is carried out via a dedicated internal network (10.8.2.0/24). The recursor has 2 network interfaces 1 internal 1 looks into the network with Internet access! (I suggested experiments with settings, no feedback on that). Yes, we are currently testing your proposal for a week and I will write about the results.

[chelezo]

Thanks! If there are multiple sources it uses all cores!