-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Significant performance degradation after migration from 3.5.3 to 4.26.0 #5801
Comments
@Fedoop1, thank you for your thorough investigation and writeup of this issue. I will add this issue to the next sprint for review. Someone on our team will reach out if they have questions or need to discuss this further. |
Hi @Fedoop1, I've been trying to reproduce this issue using the below environment:
And having the below heap size:
I tried executing the changelog you provided Was your heap size set to 4GB? If not can you tell us what's the memory size you have been using? So I can set mine in the same way and try to reproduce it with similar characteristics. On the other, I know you mentioned you can share some of the changelogs you are using because it contains sensitive data, but I was wondering if this performance degradation issue was more noticeable with one of those changelogs. If yes, could you please tell us what were the most change types it contains? That way myself or someone from the team can craft a changelog and try with it. Thanks in advance, |
Hi @MalloD12, Thanks for the feedback! To avoid confusion: the screenshots I've attached before were from different changelog and Here is what I get when running With the heap size Dump: Even with this one changeset, it consumes 1.5GB of RAM which is several times bigger than expected. In my case, memory consumption is the key which leads to slow execution due to memory pressure. I'd focus on RAM consumption rather than execution time. Best regards, |
Search first
Description
After liquibase migration from 3.5.3 to 4.26.0 we noticed that the total
liquibase update
time became very unstable.First of all overall memory consumption increased and now there are a few spikes that vary from 4 GB to 5 GB per execution (most of them are byte arrays) after which process private memory doesn't shrink. On 3.5.3 we didn't have such a problem.
The second problem is when there is memory pressure and ram utilization is close to the end, the total execution time can be several times longer than when there is enough memory for the execution. Before migration, it was stable for 3 minutes, but now it can last for 7-9 minutes. I'd guess that it's due to the problem mentioned above and probably OS can't dedicate so much memory to the liquibase process.
Steps To Reproduce
Please find the archive attached. You can run a test.ps1 script and monitor performance. There is only one big migration without sensitive data. NB: on the screenshot, you can see the performance overview from JMC of test db creation with the attached transaction included. Unfortunately, I can't attach all migrations due to their sensitivity.
liquibase-test.zip
Expected/Desired Behavior
Predictable average memory consumption. Shrink after spike. Predictable performance (execution time)
Liquibase Version
4.26.0
Database Vendor & Version
SQL Server 13.0.5893.48
Liquibase Integration
CLI
Liquibase Extensions
No response
OS and/or Infrastructure Type/Provider
Windows 10 Enterprise
Additional Context
I tried to look into this problem and found a few migrations after which I usually see some spikes. They are huge bulk inserts.
I moved SQL code from .xml changelog files to .sql and group insert statements into batches, spikes reduced from 4-5 GB to 2.5-3 GB per average execution, but it's still very high.
Sometimes I see strange runs when process memory doesn't exceed 1-2 GB and can't explain it to myself.
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: