Using Selenium on a Dynamic Webpage

Automating tasks on a webpage using Selenium can be straightforward when dealing with static elements. However, when you encounter a dynamic webpage — where the XPath or elements change after each action — the challenge increases significantly. Imagine changing the password for 2,000 customers and you found that the XPath would alter each time a password is changed? 🤷🏾‍♂️

So, how do I ensure my script runs smoothly without breaking? Here’s a step-by-step breakdown of how I tackled this challenge.🥰🥂

Step 1: Identify the Pattern of the XPath
The first step I took was to analyze how the XPath changes after each password update. In my case, I noticed that the IDs incremented by 2 with each iteration. For example, the XPaths looked like this:

xpath1 = f'//*[@id="pt1:MA:0:d2:1:pt1:pma_pt1:it1::content"]'
xpath2 = f'//*[@id="pt1:MA:0:d4:1:pt1:pma_pt1:it1::content"]'
xpath3 = f'//*[@id="pt1:MA:0:d6:1:pt1:pma_pt1:it1::content"]'

Step 2: Initialize the Current ID
Next, I defined a variable at the beginning of my script to keep track of the current ID (Just a variable name I chose). This helps to dynamically generate the XPath for each iteration.

def relogin(driver, url, wait):
    current_id = 2

Step 3: Generate the Dynamic XPath
For each element I needed to interact with, I constructed the XPath using the current ID. This ensures that I always reference the correct element.

def relogin(driver, url, wait):
    current_id = 2

enter_Charge_xpath = f'//*[@id="pt1:MA:0:d{current_id}:1:pt1:it1::content"]'
enter_Charge_Id = wait.until(EC.element_to_be_clickable((By.XPATH, enter_Charge_xpath)))

clickDebit_xpath = f'//*[@id="pt1:MA:0:d{current_id}:1:pt1:pma_pt1:createChargeOffer:t1:0:socPriceTyp::content"]/option[2]'
clickDebit = wait.until(EC.visibility_of_element_located((By.XPATH, f'{clickDebit_xpath}')))

Step 4: Increment the Current ID
After I complete the action for the current customer, I increment the current_id by 2 for the next iteration. This step is crucial to ensure that the script references the correct XPath for the next password change.

current_id += 2

Complete Sample Code
Here’s how all the pieces come together in a complete function:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

def relogin(driver, url):
    wait = WebDriverWait(driver, 10)
    current_id = 2
    
    # Loop through 2000 customers
    for _ in range(2000):
        driver.get(url)

        # Generate XPath dynamically
        enter_Charge_xpath = f'//*[@id="pt1:MM:0:d{current_id}:1:pt1:it1::content"]'
        enter_Charge_Id = wait.until(EC.element_to_be_clickable((By.XPATH, enter_Charge_xpath)))
        
        # Perform actions (e.g., enter new password)
        enter_Charge_Id.clear()
        enter_Charge_Id.send_keys("new_secure_password")

        # Increment the ID for the next iteration
        current_id += 2

Conclusion

By identifying the pattern in the changing XPaths and dynamically generating them within my script, I was able to automate the change process without breaking the workflow. If you’re working with dynamic web pages, I hope this method proves helpful in your Selenium projects!🥰🥂