Skip to main content

Selenium

Doc​

Private Mode​

stackoverflow:

  • First of all, since selenium by default starts up a browser with a clean, brand-new profile, you are actually already browsing privately.
  • But you can strictly enforce/turn on incognito/private mode anyway.
  • For firefox, set browser.privatebrowsing.autostart to True:
from selenium import webdriver

firefox_profile = webdriver.FirefoxProfile()
firefox_profile.set_preference("browser.privatebrowsing.autostart", True)

driver = webdriver.Firefox(firefox_profile=firefox_profile)

Examples​

Data Scrapping from a Lazy Loading Website​

Sometimes you want to get all URLs that are on the completely loaded page after scrolling down to the bottom of the page. However, just downloading the HTML source code and greping the URLs does not work, since the HTML you get by using Firefox' "Save Page As..." function does not contain all URLs. For cases like this you can use the following Selenium example: