Passing data_user_dir to docker container

[Valleymu]

Hi,

Thanks @mdmintz for awsome work!

Wanted to ask you Guys about storing data_user_dir to save data usage through proxy using CDP mode. I'm using with statement like this:

with SB(
            uc=True,
            incognito=True,
            headed=True,
            xvfb=True,
            proxy=proxy,
            user_data_dir="/app/user_data_dir"
) as sb:

and running this in docker container. Works fine, but I'd like to store user_data_dir so it wouldn't be recreated everytime I create container. So I thought about creating volume to save user_data_dir on host but it didn't work, selenium wasn't able to connect to chrome "cannot connect to chrome at 127.0.0.1:9222".

I didn't understand why it would have problem with this approach, I assumed maybe read/write was too slow so I ended up with coping user_data_dir from docker container before removing it but whenever docker container would provide any data in user_data_dir during container start I always get
"cannot connect to chrome at 127.0.0.1:9222". Container starts only when user_data_dir is empty.

Anyone knows why that happens?

[Valleymu]

I think this is chromedriver bug and seeing how many issues are already open I'm affraid I can't reuse user data dir...

[Randul-Malinhara]

Hi!

It seems like the problem comes from how Chrome uses the user_data_dir when running inside a Docker container. Here's what could be happening and some ways to fix it:

Why This Might Be Happening

1. Permission Issues: The volume you’re mounting on the host might not have the right permissions. Chrome inside the container may not have access to read or write the files correctly.

2. Leftover Session Data: Chrome's user_data_dir may contain data from a previous session, and when the container starts again, it might not match the environment Chrome expects.

3. Conflicting Configurations: Things like extensions, cached files, or mismatched Chrome versions in the directory could cause Chrome to crash when trying to start.

---

How to Fix It

1. Check Permissions:

   • Make sure the mounted directory on your host machine has the same user permissions as the one Chrome uses inside the container. You can fix this by running:

     sudo chown -R $(id -u):$(id -g) /path/to/host/volume

2. Clean Up Problem Files:

   • Chrome creates temporary files like SingletonLock and SingletonSocket that can cause conflicts. Delete these before starting your container:

     rm -f /path/to/host/volume/Singleton*

3. Mount the Volume Correctly:

   • When starting the container, mount the volume to ensure it persists:

     docker run -v /path/to/host/volume:/app/user_data_dir your_image

4. Use a Fresh State for Each Run:

   • Start with a clean user_data_dir every time. If you need persistence, copy the data after the container finishes running and clean it before the next run.

5. Enable Debugging:

   • Add Chrome's verbose logging to understand why it fails to connect:

     options = webdriver.ChromeOptions()
     options.add_argument("--verbose")

6. Test with tmpfs:

   • If you suspect the issue is slow read/write speeds, you can use a faster in-memory filesystem:

     docker run --tmpfs /app/user_data_dir your_image

---

Alternatively,
If none of this works, here’s a safe workaround:

1. Start with an empty user_data_dir every time.
2. After the container finishes running, copy the directory to your host machine.
3. Before starting the container again, delete files like Singleton* from the saved directory to avoid conflicts.

[Valleymu]

removing Singleton* files helped