Cookies

[ggforces]

Hello, assuming I want to access the example.com website, how can I use the cookies of the example.com website before entering it? I won't be visiting any other site before example.com, so I need to add this before starting the Chrome new tab or before the webdriver has initiated.

[AlexPaiva]

If you mean from a different session save the cookies to then load them later on you can use the pickle, read this article which should tell you everything you need: https://www.scrapingbee.com/webscraping-questions/selenium/how-to-save-cookies-selenium/

Or do you mean manually 'forcing' cookies to be added to the browser without being on it's domain?

[ggforces]

What I want is to start a WebDriver with a previously saved cookie, loading the cookie when starting. Visiting example.com, then loading the cookie and refreshing the page to see the page with the loaded cookie, hinders me from keeping my program's total runtime within the desired limit. I'm looking for a different method because loading the same page twice affects my program's execution time.

[mdmintz]

That's called preloading cookies. You can use the Chrome Devtools Protocol (CDP) for that:

driver.execute_cdp_cmd('Network.setCookie', cookie)

There's a nice Stack Overflow post about that here: https://stackoverflow.com/a/63220249/7058266

See https://chromedevtools.github.io/devtools-protocol/tot/Network/ for more related CDP methods for that, eg:

• https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setCookie
• https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setCookies
• https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-getCookies
• https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-getAllCookies

[OpsecGuy]

Hello,
The way I follow looks like this when I want to use preloaded cookies:

1. Navigate to the destination URL
2. Dump cookies set by website
3. Save that cookies then append or override that cookies with predefined names and values
4. Clear original cookies and load new list of cookies in proper format.
5. Navigate to destination URL with new cookies data.

Here is an older piece of code I wrote that I used and worked perfectly fine in my case (Maybe it will be useful for someone else too):

def import_cookies(self, cookies: dict=None) -> list:
        protocol, domain, port, path = Helper().fetch_host_data(self.target_url)
        try:
            browser_cookies = self.get_cookies()
        except Exception as err:
            print(err)

        if len(cookies) != 0:
            self.delete_cookies()
        try:
            cookies = ''.join(cookies['Cookie']).split(';')
            for cookie in cookies:

                if cookie.startswith(' '):
                    cookie = cookie.removeprefix(' ')

                cookie_name, cookie_value = cookie.split('=')
                build_cookie = {
                    'domain': domain,
                    'name': cookie_name,
                    'path': '/',
                    'value': cookie_value
                }
                browser_cookies.append(build_cookie)
        except Exception as err:
            pass
        
        if len(cookies) != 0:
            for cookie in browser_cookies:
                self.add_cookie(cookie)

            self.session.get(self.target_url)
        return browser_cookies