Skip to content

Commit

Permalink
release 0.5.22
Browse files Browse the repository at this point in the history
12ea894b7022e1a7136b968a01994289b5c5d725
  • Loading branch information
oke11o committed Apr 8, 2024
1 parent 116bcc8 commit 2593395
Show file tree
Hide file tree
Showing 30 changed files with 239 additions and 37 deletions.
5 changes: 5 additions & 0 deletions .changes/v0.5.22.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## v0.5.22 - 2024-03-26
### Added
* `shared-client` for gRPC and HTTP generators
### Changed
* Refactoring HTTP generators. Now they use common components
26 changes: 26 additions & 0 deletions .changie.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
changesDir: .changes
unreleasedDir: unreleased
headerPath: header.tpl.md
changelogPath: CHANGELOG.md
versionExt: md
versionFormat: '## {{.Version}} - {{.Time.Format "2006-01-02"}}'
kindFormat: '### {{.Kind}}'
changeFormat: '* {{.Body}}'
kinds:
- label: Added
auto: minor
- label: Changed
auto: major
- label: Deprecated
auto: minor
- label: Removed
auto: major
- label: Fixed
auto: patch
- label: Security
auto: patch
newlines:
afterChangelogHeader: 1
beforeChangelogVersion: 1
endOfVersion: 1
envPrefix: CHANGIE_
14 changes: 10 additions & 4 deletions .mapping.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@
".changes/v0.5.19.md":"load/projects/pandora/.changes/v0.5.19.md",
".changes/v0.5.20.md":"load/projects/pandora/.changes/v0.5.20.md",
".changes/v0.5.21.md":"load/projects/pandora/.changes/v0.5.21.md",
".changes/v0.5.22.md":"load/projects/pandora/.changes/v0.5.22.md",
".changie.yaml":"load/projects/pandora/.changie.yaml",
".github/workflows/release.yml":"load/projects/pandora/.github/workflows/release.yml",
".github/workflows/test.yml":"load/projects/pandora/.github/workflows/test.yml",
".gitignore":"load/projects/pandora/.gitignore",
Expand Down Expand Up @@ -249,7 +251,9 @@
"debian/rules":"load/projects/pandora/debian/rules",
"debian/source/format":"load/projects/pandora/debian/source/format",
"docs/eng/architecture.md":"load/projects/pandora/docs/eng/architecture.md",
"docs/eng/best_practices/rps_per_instance.md":"load/projects/pandora/docs/eng/best_practices/rps_per_instance.md",
"docs/eng/best-practices.md":"load/projects/pandora/docs/eng/best-practices.md",
"docs/eng/best_practices/rps-per-instance.md":"load/projects/pandora/docs/eng/best_practices/rps-per-instance.md",
"docs/eng/best_practices/shared-client.md":"load/projects/pandora/docs/eng/best_practices/shared-client.md",
"docs/eng/config.md":"load/projects/pandora/docs/eng/config.md",
"docs/eng/custom.md":"load/projects/pandora/docs/eng/custom.md",
"docs/eng/grpc-generator.md":"load/projects/pandora/docs/eng/grpc-generator.md",
Expand All @@ -274,8 +278,10 @@
"docs/images/scn_cases.png":"load/projects/pandora/docs/images/scn_cases.png",
"docs/index.md":"load/projects/pandora/docs/index.md",
"docs/rus/architecture.md":"load/projects/pandora/docs/rus/architecture.md",
"docs/rus/best-practices.md":"load/projects/pandora/docs/rus/best-practices.md",
"docs/rus/best_practices.md":"load/projects/pandora/docs/rus/best_practices.md",
"docs/rus/best_practices/rps_per_instance.md":"load/projects/pandora/docs/rus/best_practices/rps_per_instance.md",
"docs/rus/best_practices/rps-per-instance.md":"load/projects/pandora/docs/rus/best_practices/rps-per-instance.md",
"docs/rus/best_practices/shared-client.md":"load/projects/pandora/docs/rus/best_practices/shared-client.md",
"docs/rus/config.md":"load/projects/pandora/docs/rus/config.md",
"docs/rus/custom.md":"load/projects/pandora/docs/rus/custom.md",
"docs/rus/grpc-generator.md":"load/projects/pandora/docs/rus/grpc-generator.md",
Expand Down Expand Up @@ -368,7 +374,7 @@
"tests/acceptance/http_test.go":"load/projects/pandora/tests/acceptance/http_test.go",
"tests/acceptance/testdata/connect/connect-check-limit.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect-check-limit.yaml",
"tests/acceptance/testdata/connect/connect-check-passes.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect-check-passes.yaml",
"tests/acceptance/testdata/connect/connect-pool-size.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect-pool-size.yaml",
"tests/acceptance/testdata/connect/connect-shared-client.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect-shared-client.yaml",
"tests/acceptance/testdata/connect/connect-ssl.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect-ssl.yaml",
"tests/acceptance/testdata/connect/connect.yaml":"load/projects/pandora/tests/acceptance/testdata/connect/connect.yaml",
"tests/acceptance/testdata/connect/payload.uri":"load/projects/pandora/tests/acceptance/testdata/connect/payload.uri",
Expand All @@ -378,7 +384,7 @@
"tests/acceptance/testdata/http/http-check-limit.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http-check-limit.yaml",
"tests/acceptance/testdata/http/http-check-passes.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http-check-passes.yaml",
"tests/acceptance/testdata/http/http.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http.yaml",
"tests/acceptance/testdata/http/http2-pool-size.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http2-pool-size.yaml",
"tests/acceptance/testdata/http/http2-shared-client.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http2-shared-client.yaml",
"tests/acceptance/testdata/http/http2.yaml":"load/projects/pandora/tests/acceptance/testdata/http/http2.yaml",
"tests/acceptance/testdata/http/https.yaml":"load/projects/pandora/tests/acceptance/testdata/http/https.yaml",
"tests/acceptance/testdata/http/payload.uri":"load/projects/pandora/tests/acceptance/testdata/http/payload.uri",
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html),
and is generated by [Changie](https://github.com/miniscruff/changie).


## v0.5.22 - 2024-03-26
### Added
* `shared-client` for gRPC and HTTP generators
### Changed
* Refactoring HTTP generators. Now they use common components

## v0.5.21 - 2024-02-02
### Added
* parameterizable grpc reflection port
Expand Down
2 changes: 1 addition & 1 deletion cli/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
"go.uber.org/zap/zapcore"
)

const Version = "0.5.21"
const Version = "0.5.22"
const defaultConfigFile = "load"
const stdinConfigSelector = "-"

Expand Down
26 changes: 16 additions & 10 deletions components/guns/grpc/core.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,16 @@ type GrpcDialOptions struct {
}

type GunConfig struct {
Target string `validate:"required"`
ReflectPort int64 `config:"reflect_port"`
Timeout time.Duration `config:"timeout"` // grpc request timeout
TLS bool `config:"tls"`
DialOptions GrpcDialOptions `config:"dial_options"`
AnswLog AnswLogConfig `config:"answlog"`
PoolSize int `config:"pool-size"`
Target string `validate:"required"`
ReflectPort int64 `config:"reflect_port"`
Timeout time.Duration `config:"timeout"` // grpc request timeout
TLS bool `config:"tls"`
DialOptions GrpcDialOptions `config:"dial_options"`
AnswLog AnswLogConfig `config:"answlog"`
SharedClient struct {
ClientNumber int `config:"client-number,omitempty"`
Enabled bool `config:"enabled"`
} `config:"shared-client,omitempty"`
}

type AnswLogConfig struct {
Expand Down Expand Up @@ -134,14 +137,17 @@ func (g *Gun) prepareMethodList(opts *warmup.Options) (map[string]desc.MethodDes
}

func (g *Gun) prepareClientPool() (*clientpool.Pool[grpcdynamic.Stub], error) {
if g.Conf.PoolSize <= 0 {
if !g.Conf.SharedClient.Enabled {
return nil, nil
}
clientPool, err := clientpool.New[grpcdynamic.Stub](g.Conf.PoolSize)
if g.Conf.SharedClient.ClientNumber < 1 {
g.Conf.SharedClient.ClientNumber = 1
}
clientPool, err := clientpool.New[grpcdynamic.Stub](g.Conf.SharedClient.ClientNumber)
if err != nil {
return nil, fmt.Errorf("create clientpool err: %w", err)
}
for i := 0; i < g.Conf.PoolSize; i++ {
for i := 0; i < g.Conf.SharedClient.ClientNumber; i++ {
conn, err := g.makeConnect()
if err != nil {
return nil, fmt.Errorf("makeGRPCConnect fail %w", err)
Expand Down
20 changes: 13 additions & 7 deletions components/guns/http/base.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,13 @@ const (
)

type BaseGunConfig struct {
AutoTag AutoTagConfig `config:"auto-tag"`
AnswLog AnswLogConfig `config:"answlog"`
HTTPTrace HTTPTraceConfig `config:"httptrace"`
PoolSize int `config:"pool-size"`
AutoTag AutoTagConfig `config:"auto-tag"`
AnswLog AnswLogConfig `config:"answlog"`
HTTPTrace HTTPTraceConfig `config:"httptrace"`
SharedClient struct {
ClientNumber int `config:"client-number,omitempty"`
Enabled bool `config:"enabled"`
} `config:"shared-client,omitempty"`
}

// AutoTagConfig configure automatic tags generation based on ammo URI. First AutoTag URI path elements becomes tag.
Expand Down Expand Up @@ -121,11 +124,14 @@ func (b *BaseGun) createSharedDeps(opts *warmup.Options) (*SharedDeps, error) {
}

func (b *BaseGun) prepareClientPool() (*clientpool.Pool[Client], error) {
if b.Config.PoolSize <= 0 {
if !b.Config.SharedClient.Enabled {
return nil, nil
}
clientPool, _ := clientpool.New[Client](b.Config.PoolSize)
for i := 0; i < b.Config.PoolSize; i++ {
if b.Config.SharedClient.ClientNumber < 1 {
b.Config.SharedClient.ClientNumber = 1
}
clientPool, _ := clientpool.New[Client](b.Config.SharedClient.ClientNumber)
for i := 0; i < b.Config.SharedClient.ClientNumber; i++ {
client := b.ClientConstructor()
clientPool.Add(client)
}
Expand Down
4 changes: 4 additions & 0 deletions docs/eng/best-practices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Практики использования

- [RPS per instance](./best_practices/rps-per-instance.md)
- [Shared client](best_practices/shared-client.md)
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[Home](../../index.md)

---

# RPS per instance

Usually in tests, when we increase the speed of requests submitted to the target service by specifying the `line`, `const`, `step` scheme in the rps section,
Expand Down Expand Up @@ -38,3 +42,7 @@ pools:
log:
level: error
```
---
[Home](../../index.md)
37 changes: 37 additions & 0 deletions docs/eng/best_practices/shared-client.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
[Home](../../index.md)

---

# Use shared client

## General principle

### Transport client

By default, Pandora components automatically assign it a transport client, such as for http, grpc, or tcp, when creating
a new Instance. When the Instance starts, it opens a connection, such as a tcp connection.
Normally, clients can use multiple connections at the same time, but in the case of Pandora,
each Instance opens only one connection as the Instance makes requests one after the other.

It's interesting to note that creating a connection doesn't mean that requests will go through that connection. Why?
In the test configuration, you can specify a large number of instances and a small number of RPSs.
The Pandora provider generates requests with the frequency specified in the RPS settings
and sends them to a random instance so that the instance will execute the request.

## `shared-client`.
In the [http](../http-generator.md) and [grpc](../grpc-generator.md) generator settings,
you can specify the `shared-client.enabled=true` parameter. If you enable this setting,
all instances will use a shared transport client and each will not have to create its own.

## `shared-client.client-number`.

The transport client uses to connect the connection. For example, HTTP and gRPC use a tcp connection.
A single client uses multiple connections and can create additional connections if needed.

But under heavy loads there may be a situation when the client does not have time to create connections.
You can increase the speed of connection creation by a common client by increasing the `shared-client.client-number` parameter.
By default `shared-client.client-number=1`.

---

[Home](../../index.md)
9 changes: 9 additions & 0 deletions docs/eng/grpc-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ gun:
target: '[hostname]:443'
timeout: 15s # Grpc request timeout. Default: 15s
tls: false # If true, Pandora accepts any certificate presented by the server and any host name in that certificate. Default: false
shared-client:
enabled: false # If TRUE, the generator will use a common transport client for all instances
client-number: 1 # The number of shared clients can be increased. The default is 1
dial_options:
authority: some.host # Specifies the value to be used as the :authority pseudo-header and as the server name in authentication handshake
timeout: 1s # Timeout for dialing GRPC connect. Default: 1s
Expand Down Expand Up @@ -48,6 +51,12 @@ But to unify reports it converts them into HTTP codes.
| unknown | - | 500 |
# References
- Best practices
- [RPS per instance](best_practices/rps-per-instance.md)
- [Shared client](best_practices/shared-client.md)
---
[Home](index.md)
9 changes: 9 additions & 0 deletions docs/eng/http-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ gun:
idle-conn-timeout: 90s # Maximum amount of time an idle (keep-alive) connection will remain idle before closing itself. Zero means no limit. Default: 90s
response-header-timeout: 0 # Amount of time to wait for a server's response headers after fully writing the request (including its body, if any). Zero means no timeout. Default: 0
expect-continue-timeout: 1s # Amount of time to wait for a server's first response headers after fully writing the request headers if the request has an "Expect: 100-continue" header. Zero means no timeout. Default: 1s
shared-client:
enabled: false # If TRUE, the generator will use a common transport client for all instances
client-number: 1 # The number of shared clients can be increased. The default is 1
dial:
timeout: 1s # TCP connect timeout. Default: 3s
dns-cache: true # Enable DNS cache, remember remote address on first try, and use it in the future. Default: true
Expand All @@ -39,6 +42,12 @@ gun:
trace: true # calculate different request stages: connect time, send time, latency, request bytes
```
# References
- Best practices
- [RPS per instance](best_practices/rps-per-instance.md)
- [Shared client](best_practices/shared-client.md)
---
[Home](../index.md)
1 change: 1 addition & 0 deletions docs/eng/scenario-grpc-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ Follow - [Variable sources](scenario/variable_source.md)
- [HTTP generator](http-generator.md)
- Best practices
- [RPS per instance](best_practices/rps-per-instance.md)
- [Shared client](best_practices/shared-client.md)

---

Expand Down
2 changes: 2 additions & 0 deletions docs/eng/scenario-http-generator.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,6 +409,8 @@ Follow - [Variable sources](scenario/variable_source.md)
- [HTTP generator](http-generator.md)
- Best practices
- [RPS per instance](best_practices/rps-per-instance.md)
- [Shared client](best_practices/shared-client.md)


---

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ write your own load scenarios in Go, compiling them just before your test.
- [gRPC generators](eng/grpc-generator.md)
- [Scenario generator / gRPC](eng/scenario-grpc-generator.md)
- [Custom guns](eng/custom.md)
- [Best practices](eng/best_practices.md)
- [Best practices](eng/best-practices.md)
- [Pandora’s performance](eng/performance.md)
- [Architectural overview](eng/architecture.md)

Expand Down
4 changes: 4 additions & 0 deletions docs/rus/best-practices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Практики использования

- [RPS на инстанс](best_practices/rps-per-instance.md)
- [Общий транспорт](best_practices/shared-client.md)
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[Домой](../index.md)

---

# RPS на инстанс

Обычно в тестах, когда мы увеличиваем скорость запросов, подаваемых на тестируемый севис, указывая схему `line`, `const`, `step` в секции rps,
Expand Down Expand Up @@ -37,3 +41,7 @@ pools:
log:
level: error
```
---
[Домой](../index.md)
37 changes: 37 additions & 0 deletions docs/rus/best_practices/shared-client.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
[Домой](../index.md)

---
# Использовать общий транспорт

## Общий принцип

### Транспортный клиент

По умолчанию, компоненты Pandora при создании нового Инстанса автоматически назначают ему транспортного клиента,
например, для http, grpc или tcp. Когда Инстанс запускается, он открывает соединение, например, tcp соединение.
Обычно, клиенты могут использовать несколько соединений одновременно, но в случае с Pandora,
каждый Инстанс открывает только одно соединение, так как Инстанс выполняет запросы один за другим.

Стоит заметить, что создание соединения не означает, что запросы будут идти через это соединение. Почему?
В настройках теста можно указать большое кол-во инстансов и маленькое кол-во RPS.
Провайдер Пандора генерирует запросы с частотой указаной в настройках RPS и передает на случайный инстанс, чтобы
инстанс выполнил этот запрос.

## `shared-client`

В настройках генераторов [http](../http-generator.md) и [grpc](../grpc-generator.md) можно указать параметр `shared-client.enabled=true`.
Если включить данную настройку, то все инстансы будут использовать общий транспортный клиент и каждый не будет
создавать собственный.

## `shared-client.client-number`

Транспортный клиент использует для подключения соединения. Например, HTTP и gRPC используют tcp соединение.
Один клиент использует нескоклько соединений. И может создавать дополнительные при непобходимости.

Но при больших нагрузках может возникнуть ситуация, когда клиент не успевает создать соединения.
Можно увеличить, скорость создания соединений общим клиентом, увеличив параметро `shared-client.client-number`.
По умолчанию `shared-client.client-number=1`

---

[Домой](../index.md)
Loading

0 comments on commit 2593395

Please sign in to comment.