Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What about high latency of wrk access interface on ubuntu? #527

Open
ponponon opened this issue Sep 22, 2023 · 1 comment
Open

What about high latency of wrk access interface on ubuntu? #527

ponponon opened this issue Sep 22, 2023 · 1 comment

Comments

@ponponon
Copy link

I have an api service http://192.168.38.223:9100/

Deployed using docker on machine 192.168.38.223

╰─➤  docker ps -a | grep proxy         
12b8b5801e73   ponponon/proxy-king:2023.09.21.1           "python api.py"          38 minutes ago   Up 38 minutes         0.0.0.0:9100->9100/tcp, :::9100->9100/tcp                                                                                                                                                      proxy-king

I then use wrk on mac to stress test the http://192.168.38.223:9100/ interface

The wrk is installed by brew install wrk.

╰─➤  wrk --version                                  
wrk 4.2.0 [kqueue] Copyright (C) 2012 Will Glozer
Usage: wrk <options> <url>                            
Options:                                            
 -c, --connections <N>  Connections to keep open   
 -d, --duration    <T>  Duration of test           
 -t, --threads     <N>  Number of threads to use   

 -s, --script      <S>  Load Lua script file       
 -H, --header      <H>  Add header to request      
     --latency          Print latency statistics   
     --timeout     <T>  Socket/request timeout     
 -v, --version          Print version details      

Numeric arguments may include a SI unit (1k, 1M, 1G)
Time arguments may include a time unit (2s, 2m, 2h)
╰─➤  wrk http://192.168.38.223:9100/ -t32 -c64 -d 30                                                                                                                     130 ↵
Running 30s test @ http://192.168.38.223:9100/
  32 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.51ms    2.99ms  64.81ms   91.05%
    Req/Sec   644.17    204.94     1.16k    66.50%
  615969 requests in 30.04s, 92.72MB read
Requests/sec:  20503.44
Transfer/sec:      3.09MB

The result is very good, the RPS can reach 2w+.

I then used the wrk tool on 192.168.38.223 to stress test the http://192.168.38.223:9100/ interface again

192.168.38.223 The machine is using ubuntu 22.04.

╰─➤  uname -a
Linux T4GPU 5.15.0-84-generic #93-Ubuntu SMP Tue Sep 5 17:16:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

The installed wrk is apt installed apt install wrk.

╰─➤  wrk --version                                                                                                                                                         1 ↵
wrk debian/4.1.0-3build1 [epoll] Copyright (C) 2012 Will Glozer
Usage: wrk <options> <url>                            
Options:                                            
 -c, --connections <N>  Connections to keep open   
 -d, --duration    <T>  Duration of test           
 -t, --threads     <N>  Number of threads to use   

 -s, --script      <S>  Load Lua script file       
 -H, --header      <H>  Add header to request      
     --latency          Print latency statistics   
     --timeout     <T>  Socket/request timeout     
 -v, --version          Print version details      

Numeric arguments may include a SI unit (1k, 1M, 1G)
Time arguments may include a time unit (2s, 2m, 2h)

But something unexpected happened, the RPS became very low, only 1.4k, and the latency became very high, 45ms! This is unusual!

╰─➤  ./wrk http://192.168.38.223:9100/ -t32 -c64 -d 30
Running 30s test @ http://192.168.38.223:9100/
  32 threads and 64 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    44.67ms    2.24ms  52.34ms   81.85%
    Req/Sec    44.78      6.01    60.00     94.41%
  43079 requests in 30.09s, 6.49MB read
Requests/sec:   1431.51
Transfer/sec:    220.89KB

I ran a python script on the 192.168.38.223 machine

import time
import requests


s=time.time()
response=requests.get('http://192.168.38.223:9100/')
e=time.time()

print(f'{round((e-s)*1000,2)} ms')

output result

4.34 ms

As you can see, there are no latency issues with accessing using python scripts, but why is the latency of wrk access so high? What are the ways to troubleshoot this?

@qsantos
Copy link

qsantos commented Jan 13, 2024

You might be observing #488. Add -H 'Connection: close' to your wrk invocation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants