Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: ENOENT: no such file or directory, lstat '/crawls' #14

Open
juansalvatella opened this issue Jan 25, 2022 · 0 comments
Open

Error: ENOENT: no such file or directory, lstat '/crawls' #14

juansalvatella opened this issue Jan 25, 2022 · 0 comments

Comments

@juansalvatella
Copy link

juansalvatella commented Jan 25, 2022

While running I get this error:

2022-01-25T03:48:27.877Z (node:8) UnhandledPromiseRejectionWarning: Error: ENOENT: no such file or directory, lstat '/crawls'
2022-01-25T03:48:27.879Z (node:8) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1970)

I'm not really sure where it comes from but it seems to be related with:

// load persistent storage
    if (stream != null) {
        fs.writeFileSync('downloaded.tgz', stream);
        try { execSync('rm -r ./crawls/'); } catch (err) {}
        fs.createReadStream('downloaded.tgz').pipe(tarfs.extract('./'));
    }

and

// update spider state every 5 seconds
    const storeJobsInterval = setInterval(() => {
        tar.c({ gzip: false, file: 'jobdir.tgz' }, ['crawls/']).then(() => {
          Apify.setValue('jobdir.tgz', fs.readFileSync('jobdir.tgz'), { contentType: 'application/tar+gzip' });
        });
      }, 5000);

https://github.com/apify/actor-scrapy-executor/blob/2da6656b231545c085627ce65df355beced402d0/main.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant