Iterating Through Table Rows in Selenium (Python)

Iterating Through Table Rows in Selenium (Python)

XPath is fragile. Its better to use CSS selectors or classes:

mytable = find_element_by_css_selector(table.datadisplaytable)
for row in mytable.find_elements_by_css_selector(tr):
    for cell in row.find_elements_by_tag_name(td):
        print(cell.text)

If you want to go row by row using an xpath, you can use the following:

h  = <table class=datadisplaytable>
<tr>
<td class=dddefault>16759</td>
<td class=dddefault>MATH</td>
<td class=dddefault>123</td>
<td class=dddefault>001</td>
<td class=dddefault>Calculus</td>
<td class=dddefault></td>
<td class=dddead></td>
<td class=dddead></td>
</tr>
<tr>
<td class=dddefault>16449</td>
<td class=dddefault>PHY</td>
<td class=dddefault>456</td>
<td class=dddefault>002</td>
<td class=dddefault>Physics</td>
<td class=dddefault></td>
<td class=dddead></td>
<td class=dddead></td>
</tr>
</table>

from lxml import html
xml = html.fromstring(h)
# gets the table
table =  xml.xpath(//table[@class=datadisplaytable])[0]


# iterate over all the rows   
for row in table.xpath(.//tr):
     # get the text from all the tds from each row
    print([td.text for td in row.xpath(.//td[@class=dddefault][text()])

Which outputs:

[16759, MATH, 123, 001, Calculus]
[16449, PHY, 456, 002, Physics]

Using td[text()] will avoid getting any Nones returned for the tds that hold no text.

So to do the same using selenium you would:

table =  driver.find_element_by_xpath(//table[@class=datadisplaytable])

for row in table.find_elements_by_xpath(.//tr):
    print([td.text for td in row.find_elements_by_xpath(.//td[@class=dddefault][1]])

For multiple tables:

def get_row_data(table):
   for row in table.find_elements_by_xpath(.//tr):
        yield [td.text for td in row.find_elements_by_xpath(.//td[@class=dddefault][text()]])


for table in driver.find_elements_by_xpath(//table[@class=datadisplaytable]):
    for data in get_row_data(table):
        # use the data

Iterating Through Table Rows in Selenium (Python)

Correction of the Selenium part of @Padraic Cunninghams answer:

table = driver.find_element_by_xpath(//table[@class=datadisplaytable])

for row in table.find_elements_by_xpath(.//tr):
    print([td.text for td in row.find_elements_by_xpath(.//td[@class=dddefault])])

Note: there was one missing round bracket at the end; also removed the [1] index, to match the first XML example.

Another note: Though, the example with the index [1] should also be preserved, to show how to extract individual elements.

Leave a Reply

Your email address will not be published.