Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mango tests using custom db name #5341

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jiahuili430
Copy link
Contributor

Overview

When running mango tests, sometimes get
org.apache.lucene.store.NoSuchDirectoryException error. This is because each test in the same class used the same database name, and in the setUp function always delete and recreate it, which caused a race condition.

Using custom database names should solve it.

Testing recommendations

make mango-test

Related Issues or Pull Requests

Checklist

  • Code is written and works correctly
  • Changes are covered by tests
  • Any new configurable parameters are documented in rel/overlay/etc/default.ini
  • Documentation changes were made in the src/docs folder
  • Documentation changes were backported (separated PR) to affected branches

When running mango tests, sometimes get
`org.apache.lucene.store.NoSuchDirectoryException` error. This is because
each test in the same class used the same database name, and in the setUp
function always delete and recreate it, which caused a race condition.

Using custom database names should solve it.
@@ -189,6 +189,7 @@ def test_uses_partial_index_with_non_indexable_selector(self):
@unittest.skipUnless(mango.has_text_service(), "requires text service")
class IndexSelectorText(mango.DbPerClass):
def setUp(self):
self.db = mango.Database(f"{mango.random_db_name()}_{str(self).split()[0]}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many tests files which use self.db.recreate() did you consider modifying that function?

Copy link
Contributor Author

@jiahuili430 jiahuili430 Nov 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If using the same db name, 0.13 is the shortest time to avoid 500 locally.
This approach also increased the overall Mango test time to 60.730s, compared to 49.936s using custom database name.

def recreate(self):
-    NUM_TRIES = 10
+    NUM_TRIES = 3

-    for k in range(NUM_TRIES):
+    for k in range(1, NUM_TRIES):
         r = self.sess.get(self.url)
         if r.status_code == 200:
             db_info = r.json()
             docs = db_info["doc_count"] + db_info["doc_del_count"]
                 if docs == 0:
                     # db exists and it is empty -- exit condition is met
                     return
                 self.delete()
+        time.sleep(k * 0.13)
         self.create()
-        time.sleep(k * 0.1)
     raise Exception(
         "Failed to recreate the database after {} tries".format(NUM_TRIES)
     )

@@ -189,6 +189,7 @@ def test_uses_partial_index_with_non_indexable_selector(self):
@unittest.skipUnless(mango.has_text_service(), "requires text service")
class IndexSelectorText(mango.DbPerClass):
def setUp(self):
self.db = mango.Database(f"{mango.random_db_name()}_{str(self).split()[0]}")
self.db.recreate()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was recent work done on the recreate function, but it didn't seem to address the raciness of reusing db names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the link!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants