Selenium No Element Found Exception even after copying element from source code

Question

I am trying to simply login to this page to access LexisNexis. Here is my code:

from selenium import webdriver
browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
type(browser)

browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
linkElem2 = browser.find_element_by_link_text('LEXIS NEXIS DATABASES') 
type(linkElem2)
linkElem2.click()
alert = browser.switch_to.alert
alert.accept()

loginElem = browser.find_element_by_id('j_username')
loginElem.send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

And here is the html source:

div class="form_row"

label for="j_username"
Enter your strongNetID:/strong/label

input id="j_username" name="j_username" autocomplete="OFF" autocapitalize="off" autocorrect="off" value="" type="text"
/div

I am not sure what I am doing wrong :/

Answer 1

Clicking on that link opens a new tab, so you need to switch to new tab before locating username and password fields. Try this (indentation may be broken):

from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
linkElem2 = browser.find_element_by_link_text('LEXIS NEXIS DATABASES') 
linkElem2.click()
alert = browser.switch_to.alert  
alert.accept()
time.sleep(5) #to add 5 sec wait, you can use other wait methods for window handle. e.g. https://stackguides.com/questions/26641779/python-selenium-how-to-wait-for-new-window-opens
browser.switch_to.window(browser.window_handles[1])
WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, 'j_username')))
loginElem = browser.find_element_by_id('j_username')
loginElem.send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

you will need following imports

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Answer 2

1. Problem "LEXIS NEXIS DATABASES" link open new window and you have to switch to it to login, Handle multiple windows in Python:

from selenium import webdriver

browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.implicitly_wait(5)

browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
browser.find_element_by_link_text('LEXIS NEXIS DATABASES').click()

browser.close()
browser.switch_to.window(browser.window_handles[0])

browser.find_element_by_id('j_username').send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

2. Problem Link you share in comments is have error when you open for the first time(new browser profile). All you need is open page and click "Login with your Illinois NetID" to open login page successfully. This way much better, because you don't need to handle windows, check code below:

from selenium import webdriver

browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.implicitly_wait(5)

browser.get('https://compass2g.illinois.edu/webapps/login/')
browser.find_element_by_link_text('Login with your Illinois NetID').click()

browser.implicitly_wait(5)

browser.find_element_by_id('j_username').send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()