In Selenium 4 for Python, find an element with driver.find_element(By.<STRATEGY>, "value") after from selenium.webdriver.common.by import By. It returns the first match or raises NoSuchElementException; find_elements returns a list, empty when nothing matches. The old find_element_by_* methods are gone in Selenium 4.
We ran every locator below on 20 November 2026 with Python 3.12.10, selenium 4.53.0 and Microsoft Edge 157 in headless mode, on quotes.toscrape.com, a public site made for scraping practice, through a QuanticData mobile proxy. The comments are the real output.
The eight By locators
| Locator | Call | Our result |
|---|---|---|
| By.ID | find_element(By.ID, "username") | the input on /login |
| By.NAME | find_element(By.NAME, "password") | input of type password |
| By.CLASS_NAME | find_element(By.CLASS_NAME, "quote") | first quote, by Albert Einstein |
| By.CSS_SELECTOR | find_elements(By.CSS_SELECTOR, "div.quote span.text") | 10 elements |
| By.XPATH | find_element(By.XPATH, "//small[@class='author' and text()='Jane Austen']") | Jane Austen |
| By.LINK_TEXT | find_element(By.LINK_TEXT, "Login") | href https://quotes.toscrape.com/login |
| By.PARTIAL_LINK_TEXT | find_element(By.PARTIAL_LINK_TEXT, "Next") | href .../page/2/ |
| By.TAG_NAME | find_element(By.TAG_NAME, "h1") | Quotes to Scrape |
from selenium import webdriver
from selenium.webdriver.common.by import By
opts = webdriver.EdgeOptions()
opts.add_argument("--headless=new")
driver = webdriver.Edge(options=opts)
driver.get("https://quotes.toscrape.com/")
first = driver.find_element(By.CLASS_NAME, "quote")
print(first.find_element(By.CLASS_NAME, "author").text) # Albert Einstein
texts = driver.find_elements(By.CSS_SELECTOR, "div.quote span.text")
print(len(texts)) # 10
Calling find_element on an element, as with first.find_element(...), searches inside that element only. That is how you keep each quote's text with its own author.
Find an element by text
The most searched variant, and there is no By.TEXT. Use XPath:
# exact text
driver.find_element(By.XPATH, "//small[@class='author' and text()='Jane Austen']").text
# 'Jane Austen'
# text that contains a word (the dot means all text inside the element)
driver.find_element(By.XPATH, "//span[contains(@class,'text') and contains(., 'miracle')]").text
# '"There are only two ways to live your life. One is ...'
text()='...' needs the exact string, spaces included; contains(., '...') is the forgiving form. For links only, By.LINK_TEXT and By.PARTIAL_LINK_TEXT do the same without XPath. More expressions in XPath expressions.
Find by attribute, by class, by href
driver.find_element(By.CSS_SELECTOR, "[itemprop='keywords']").get_attribute("content")
# 'change,deep-thoughts,thinking,world'
driver.find_element(By.CSS_SELECTOR, "a[href*='/page/2']")
driver.find_element(By.XPATH, "//a[@href='/login']")
- By attribute:
[name='value']in CSS,[@name='value']in XPath.get_attribute()reads any attribute, includinghrefandvalue. - By class:
By.CLASS_NAMEtakes one class. Given two,"row header-box", it raised InvalidSelectorException in our run;By.CLASS_NAME, "header-box"alone found the element, and CSS joins classes:By.CSS_SELECTOR, ".row.header-box"found it too. - By href:
a[href*='part']matches a part of the link,a[href='exact']the whole value.
Which of CSS or XPath to prefer is in CSS selector vs XPath.
After you find it: click, type, read
find_element returns a WebElement, and the useful work happens on it. On the same site:
driver.find_element(By.PARTIAL_LINK_TEXT, "Next").click()
print(driver.current_url) # https://quotes.toscrape.com/page/2/
print(driver.find_element(By.CSS_SELECTOR, "div.quote small.author").text) # Marilyn Monroe
tags = driver.find_elements(By.CSS_SELECTOR, "div.quote:first-child a.tag")
print([t.text for t in tags])
# ['friends', 'heartbreak', 'inspirational', 'life', 'love', 'sisters']
driver.get("https://quotes.toscrape.com/login")
user = driver.find_element(By.ID, "username")
user.send_keys("demo")
print(user.get_attribute("value")) # demo
| On a WebElement | Does |
|---|---|
.text | The visible text, as the user sees it |
.get_attribute("href") | Any attribute or property, such as href, value, class |
.click() | Clicks; links and buttons navigate as in a browser |
.send_keys("text") | Types into an input |
.is_displayed(), .tag_name | Visibility and tag; our h1 returned True, a quote block "div" |
Relative locators: find by position
Selenium 4 can also find an element by where it sits next to another one, useful when a field has no stable id or class:
from selenium.webdriver.support.relative_locator import locate_with
below = driver.find_element(locate_with(By.TAG_NAME, "input").below(user))
print(below.get_attribute("id"), below.get_attribute("type")) # password password
Other directions are above, to_left_of, to_right_of and near. They depend on the page layout, so keep them for forms and fixed layouts rather than long lists.
When nothing is found
driver.find_elements(By.CLASS_NAME, "no-such-class") # []
driver.find_element(By.ID, "no-such-id")
# NoSuchElementException: no such element: Unable to locate element:
# {"method":"css selector","selector":"[id="no-such-id"]"}
The message shows that Selenium turns By.ID into a CSS selector. Use find_elements and check the length for optional elements; use find_element when a missing element is a real error.
Wait before you find: dynamic pages
Most "element not found" bugs are timing. quotes.toscrape.com/js-delayed/ adds its quotes after a delay:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver.get("https://quotes.toscrape.com/js-delayed/")
print(len(driver.find_elements(By.CLASS_NAME, "quote"))) # 0
WebDriverWait(driver, 20).until(
EC.presence_of_element_located((By.CLASS_NAME, "quote")))
print(len(driver.find_elements(By.CLASS_NAME, "quote"))) # 10, after 10.8 s
WebDriverWait polls until the condition is true or the timeout passes. Prefer it to time.sleep(), which waits too long or not long enough. Other conditions: visibility_of_element_located, element_to_be_clickable. The wider picture is in scraping dynamic web pages in Python.
Selenium with an authenticated proxy
Edge takes the proxy as --proxy-server=http://host:port and does not take a username and password inside it. So we ran a small local forwarder in Python: it listens on 127.0.0.1 without a password, adds the Proxy-Authorization header, and passes everything on to the QuanticData gateway. Save it as local_proxy.py:
import base64, socket, threading
def start_local_proxy(gateway, port, user, password, listen_port=8899):
auth = base64.b64encode(f"{user}:{password}".encode()).decode()
def pipe(src, dst):
try:
while data := src.recv(65536):
dst.sendall(data)
except OSError:
pass
finally:
src.close()
dst.close()
def handle(client):
head = b""
while b"\r\n\r\n" not in head:
head += client.recv(4096)
first_line, rest = head.split(b"\r\n", 1)
upstream = socket.create_connection((gateway, port))
upstream.sendall(first_line + b"\r\nProxy-Authorization: Basic "
+ auth.encode() + b"\r\n" + rest)
threading.Thread(target=pipe, args=(upstream, client), daemon=True).start()
pipe(client, upstream)
def serve():
srv = socket.socket()
srv.bind(("127.0.0.1", listen_port))
srv.listen(50)
while True:
conn, _ = srv.accept()
threading.Thread(target=handle, args=(conn,), daemon=True).start()
threading.Thread(target=serve, daemon=True).start()
Start it with your gateway, port, username and password, then point the browser at it:
from local_proxy import start_local_proxy
start_local_proxy("mobile.quanticdata.io", 8080, "USER-country-us-type-mobile-sid-s01-ttl-30m", "PASS")
opts.add_argument("--proxy-server=http://127.0.0.1:8899")
driver = webdriver.Edge(options=opts)
driver.get("https://ipinfo.io/json") # exit: Baltimore, Maryland, US
driver.get("https://quotes.toscrape.com/")
print(len(driver.find_elements(By.CLASS_NAME, "quote"))) # 10
The forwarder runs in background threads for as long as your script runs. It handles HTTPS through the CONNECT method, which is what browsers use for https:// pages.
A mobile line suits sites that serve phones differently; residential lines give home IPs by country. Changing the IP between sessions is covered in rotating a proxy in Selenium Python. Plans: mobile proxies, residential proxies and rotating proxies.


