Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libsvm instances during the unit tests are influenced by other coexisting instances and cause test failures #12

Open
JasonShin opened this issue May 11, 2019 · 2 comments
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@JasonShin
Copy link
Member

JasonShin commented May 11, 2019

Context:
While trying to cover tests for WASM loaded SVM's predict, I've encountered an issue of incorrect prediction results between running a test by itself vs running tests in-band.

Problematic test:

it('should predict bodyfat scale dataset using C_SVC and LINEAR', () => {
      const rawData = fs.readFileSync(path.join(__dirname, '../samples/bodyfat_scale.txt'), 'utf-8');
      const data = rawData.split('\n').map((line) => line.split(' ').filter((el) => el));
      const labels = data.map((line) => +line.splice(0, 1)[0]);
      const features = data.map((line) => line.map((el) => +el.split(':')[1]));

      const svm = new SVM({
        type: SVMTypes.C_SVC,
        kernel: KernelTypes.LINEAR,
        epsilon: 0.001,
        quiet: true,
        probabilityEstimates: true,
      });
      return svm.loadWASM().then((loadedSVM) => {
        loadedSVM.train({ samples: features, labels });
        const predResult = loadedSVM.predict({
          samples: [features[0], features[1]],
        });
        loadedSVM.free();
        // TODO: It seems like other tests are influencing the result of this.
        // I don't know why so far, we should figure it out
        expect(predResult).toEqual([1, 1]);
      });
    });

When you run this test on its own, it passes without an issue since the loadedSVM instance is isolated. However, if you run this test in parallel/sequentially along with other tests, the prediction starts to return different values.

We should investigate why this is happening and devise a solution.

@JasonShin JasonShin added bug Something isn't working help wanted Extra attention is needed good first issue Good for newcomers labels May 11, 2019
@JasonShin
Copy link
Member Author

The issue occurs in both WASM and ASM.

@JasonShin JasonShin changed the title libsvm instances during the unit tests are influenced by other coexisting instances and causes test failures libsvm instances during the unit tests are influenced by other coexisting instances and cause test failures May 11, 2019
@leo-p-labs
Copy link

I'm having a bug that comes from libsvm-ts/out/asm/libsvm.js even if there is no SVM instance loaded. I'm just calling accuracy_score(). I post it here since I think it's related to the same problem. If not I'll open a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants