-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run parallel browsers from one node process #22
Comments
At this point in time, the launch of browsers is synchronized, but after that you can work with them in parallel in any way. Parallel unsynchronized launch isn't yet supported due to some limitations - we may fix this in the future, so we'll leave the ticket open for now. Launching the browser takes longer than default one, as it requires additional configuration and the presence of an engine that is downloaded once. If a timeout error occurs, you can change the default timeout to any value you like via the If you're having trouble launching multiple browsers and then working with them, or if I misunderstood you, please describe the problem in more detail and attach sample code. |
So I should use some As far as I understood correctly, the problem is that the browsers uses the same |
I just tested my code which didn't work with |
I still don't understand what your problem is and why you need to use additional mutex inside require('dotenv').config();
const { plugin } = require('puppeteer-with-fingerprints');
async function main(index) {
const browser = await plugin.launch();
const page = await browser.newPage();
await page.goto(`https://jsonplaceholder.typicode.com/todos/${index + 1}`, { waitUntil: 'domcontentloaded' });
const todo = JSON.parse(await page.$eval('pre', (pre) => pre.innerText));
await browser.close();
return todo;
}
Promise.all([0, 1, 2].map(main)).then((todos) => {
console.log({ todos });
}); I tested it many times and it worked. Try running it and let me know what specific problems you're having with it. |
but this example doesn't use |
@ClassAxion can you attach minimum runnable source code and what error do you get. Please avoid using FingerprintSwitcher key in your code. |
Hello, I'm having the same problem. project_folder -> index.js export const index = async (plugin) => {
const fingerprint = await plugin.fetch('key', {
tags: ['Microsoft Windows', 'Chrome'],
});
log.yellow("STEP: fingerprints fetched");
plugin.useProxy(proxy, {
changeGeolocation: true,
});
log.white("STEP: proxy setted");
plugin.useFingerprint(fingerprint);
log.white("STEP: figerprint using");
const browser = await plugin.launch({
headless: isHeadless,
// minBrowserVersion: 102,
// minWidth: 1024,
// minHeight: 768,
// maxWidth: 1920,
// maxHeight: 1080,
});
log.white("STEP: browser launched");
} project_folder -> sub_folder1 import { plugin } from "puppeteer-with-fingerprints";
import {index} from "../index.js";
await index(plugin); It is installed in the puppeteer-with-fingerprints subdirectory, so each browser is used from a different directory. At the same place, I sometimes encounter the following error. node:events:491
throw er; // Unhandled 'error' event
^
Error: read ECONNRESET
at TCP.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on Socket instance at:
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
errno: -4077,
code: 'ECONNRESET',
syscall: 'read'
}
Node.js v19.0.0 |
@emretulek I don't see any issues here, your code works fine for me - please, describe your problem in more details. Also keep in mind that you must close browsers properly whether you are using a plugin or not. Otherwise, it can lead to other problems, such as excessive consumption of resources, and the process can remain open, including because of this. |
@CheshireCaat When I open multiple browsers, the most noticeable problem is that processes freeze in relation to one another. For example, if 5 processes are started at the same time, and the 2nd process is frozen, it cannot be terminated even with ctr+c. But it freezes in conjunction with one of the processes 1 - 3 - 4 - 5. When 1-4-5 is closed, nothing changes, while 3 is turned off, 2 continues from where it left off. (It doesn't have to be 3, it can be any one specifically.) |
Does this package support for run multiple browsers at once? Like 10-50 parallel browsers from the same node process?
I'm trying but browsers starting very very long or even timeout at 300s and freezes.
How to do this correctly? I'm using
plugin.launch
withheadless: 'new'
anduseFingerprint
for launching browsersThe text was updated successfully, but these errors were encountered: