Release 11: When do we branch out?

[amarts]

Release 10.0 was tagged on Nov 2, 2021. If we consider 1 year cycle for releases, the next release (ie, 11.0) should be happening in Nov 2022. If we do a cycle of 1month of branch testing (ie, branch out early, and only take critical patches from then on based on testing), we should be very close to branch date.

There are many PRs targeted for 'release 11' (https://github.com/gluster/glusterfs/pulls?q=is%3Apr+is%3Aopen+label%3A%22release+11%22), any plans around them?

Good to start thinking about this, we have made significant bug fixes in the devel branch since last release.

[mykaul]

I hope any bug fix we've made in devel since last release was also backported to 10.x at least.

I'd be happy to see some of the 'release 11' PRs getting in, but there's really nothing urgent, though I think many are nice cleanups (#3544 for example).
We really do not want to get into the situation where we push too many in at the same time, regress and it's challenging to find why, so I'm in favor of keeping the steady state of PRs getting in until start-mid October or so.

[edrock200]

In my testing of gluster 11 so far, Ive seen some behavior I didn't see in 10.3. Primarily, under heavy write operations to include rebalancing, bricks will randomly drop/disconnect. I've been digging through the logs but can't find any smoking guns yet. When a brick drops, its not all bricks on a node, but anywhere from 1-4 bricks out of 10. The bricks that claim to be disconnected are reachable on their port from other nodes. A restart of glusterd seems to resolve the issue but it resurfaces every few days. I haven't been able to get through a full successful rebalance yet due to bricks dropping.

I can say the cpu utilization on the nodes is significantly better, CPU utilization is nearly cut in half.

[mohit84]

In my testing of gluster 11 so far, Ive seen some behavior I didn't see in 10.3. Primarily, under heavy write operations to include rebalancing, bricks will randomly drop/disconnect. I've been digging through the logs but can't find any smoking guns yet. When a brick drops, its not all bricks on a node, but anywhere from 1-4 bricks out of 10. The bricks that claim to be disconnected are reachable on their port from other nodes. A restart of glusterd seems to resolve the issue but it resurfaces every few days. I haven't been able to get through a full successful rebalance yet due to bricks dropping.

I can say the cpu utilization on the nodes is significantly better, CPU utilization is nearly cut in half.

Can you please file a bug and share the /var/log/glusterfs dump from all the nodes?

[edrock200]

Yes of course. Been meaning too. Will try to post it this evening.

[edrock200]

@mohit84 do you know if there's a way to share all those logs a bit more discreetly with the devs? I can scrub gluster command outputs but scrubbing multiple logs from 15 nodes would take quite a while. Thank you in advance.

[mohit84]

@mohit84 do you know if there's a way to share all those logs a bit more discreetly with the devs? I can scrub gluster command outputs but scrubbing multiple logs from 15 nodes would take quite a while. Thank you in advance.

It's ok you can share the logs from 1 node as of now.

[edrock200]

I had a few bricks drop today so I will share one of those nodes

[edrock200]

@mohit84 do you know if there's a way to share all those logs a bit more discreetly with the devs? I can scrub gluster command outputs but scrubbing multiple logs from 15 nodes would take quite a while. Thank you in advance.

It's ok you can share the logs from 1 node as of now.

Issue created #4028

[edrock200]

is it possible, or safe, to roll back from 11 to 10.3?

[edrock200]

Or 10.4?

[edrock200]

I guess that's a no :)
Anxiously awaiting 11.1 :)

[vvarga007]

Same here.