如何使用SeleniumPython提取 NewEgg上的图形卡发布的所有标题文本

我目前正在开发一个 Python 脚本来获取 RTX 3080 图形卡上的信息,并且我遇到了让我的脚本抓住页面上每个帖子的每个图形卡标题的问题。我的脚本如下

我目前正在开发一个 Python 脚本来获取 RTX 3080 图形卡上的信息,并且我遇到了让我的脚本抓住页面上每个帖子的每个图形卡标题的问题。我的脚本如下

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
import time
options = Options()
PATH = 'C://Program Files (x86)//chromedriver.exe'
driver = webdriver.Chrome(PATH)
url = 'https://www.newegg.com/p/pl?d=RTX+3080'
actions = ActionChains(driver)
driver.maximize_window()
driver.get(url)
card_path = '/html/body/div[8]/div[3]/section/div/div/div[1]/div/dl[1]/dd/ul[2]/li/a'
desktop_graphics_cards = driver.find_element(By.XPATH, card_path)
desktop_graphics_cards.click()
time.sleep(5)
graphics_card = []
shipping_cost = []
price = []
total_cost = []
card_data = driver.find_elements(By.XPATH, '//div[@class = "item-cell"]')
print(card_data)
for card in card_data:
    graphics_card0 = card.find_element(By.XPATH, '//a[@title = "View Details"]').get_attribute('innerText')
    print(graphics_card0)

每当我运行脚本时,它都会给我第一个图形卡的标题,但对于其余图形卡标题的每个帖子都会重复相同的标题。例如,当我进行打印(len(card_data))时,它会返回 46 个帖子(页面上每个“item-cell”的图形卡数量),但我的打印(graphics_card0)会返回类似“MSI Gaming GeForce RTX 30RT

截图 1:

enter image description here

截图 2:

enter image description here

1

要使用Selenium提取所有标题,例如MSI Gaming GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX 视频卡 RTX 3080 GAMING Z TRIO 10G LHR等,您可以使用List Comprehension,也可以使用以下任一Locator Strategies

Usingcss_selectorandget_attribute("innerHTML"):
driver.get("https://www.newegg.com/p/pl?d=RTX+3080")
print([my_elem.get_attribute("innerHTML") for my_elem in driver.find_elements(By.CSS_SELECTOR, "a[title = 'View Details']")])

使用xpathtext属性:

driver.get("https://www.newegg.com/p/pl?d=RTX+3080")
print([my_elem.text for my_elem in driver.find_elements(By.XPATH, "//a[@title = 'View Details']")])

理想情况下,您需要为visibility_of_all_elements_located()引入WebDriverWait,并且可以使用以下Locator Strategies之一:

使用CSS_SELECTORtext属性:

driver.get("https://www.newegg.com/p/pl?d=RTX+3080")
print([my_elem.text for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "a[title = 'View Details']")))])
UsingXPATHandget_attribute("innerHTML"):
driver.get("https://www.newegg.com/p/pl?d=RTX+3080")
print([my_elem.get_attribute("innerText") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//a[@title = 'View Details']")))])

控制台输出:

['GIGABYTE A520M S2H mATX AM4 4+3 Phases Digital PWM, GIGABYTE Gaming GbE LAN, NVMe PCIe 3.0 x4 M.2, 3 Display Inteces, Q-Flash Plus, RGB Fusion 2.0, Motherboard', 'MSI Gaming GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 GAMING Z TRIO 10G LHR', 'Goldshell LT5 PRO 2455MH/S ASIC LTC Scrypt algorithm mining Dogecoin + Litecoin miner power consumption of 3100W', 'Goldshell Mini-DOGE 185MH/S Ordinary version (without psu)DOGE& LTC Mining Machine Low noise Small&simple Home Mining Home Riching', 'Goldshell LT5 PRO 2455MH/S ASIC LTC Scrypt algorithm mining Dogecoin + Litecoin miner power consumption of 3100W', 'Goldshell Mini-DOGE 185MH/S Ordinary version (without psu)DOGE& LTC Mining Machine Low noise Small&simple Home Mining Home Riching', 'ASUS TUF Gaming NVIDIA GeForce RTX 3080 V2 OC Edition Graphics Card (PCIe 4.0, 10GB GDDR6X, LHR, HDMI 2.1, DisplayPort 1.4a, Dual Ball Fan Bearings, Military-grade Certification, GPU Tweak II)', 'ABS Gladiator Gaming PC - Windows 10 Home - Intel i7 11700KF - GeForce RTX 3080 - G.Skill TridentZ RGB 16GB DDR4 3200MHz - 1TB Intel M.2 NVMe SSD', 'ABS ROG Gundam Limited Edition Gaming PC - Windows 10 Home - Intel i7 11700K - STRIX Gundam GeForce RTX 3080 - G.Skill 32GB 3200MHz - 1TB Intel M.2 - STRIX 360mm AIO', 'ABS Gladiator Gaming PC - Windows 10 Home - Intel i9 11900KF - GeForce RTX 3080 - G.Skill TridentZ RGB 16GB DDR4 3200 MHz - 1TB Intel M.2 NVMe SSD', 'Goldshell HS-BOX 235GH/S(without PSU)BOX & HNSB Mining Machine Low noise Small&simple Home Mining Home Riching', 'Goldshell CK-BOX 1050GH/S(without psu) CKB Mining Machine Low noise Small&simple Home Mining Home Riching', 'Adamant Custom 32-Core Video Editing Rendering Modelling Workstation Computer AMD Threadripper 3970X 3.7Ghz TRX40 2 ZENITH 256Gb DDR4 2x2TB NVMe 1800MBs SSD 10TB HDD 1000W Geforce RTX 3080', 'Adamant Custom 14-Core Rendering Modelling Workstation Computer Intel i9 10940X 3.3Ghz X299 AORUS 128Gb DDR4 2x2TB NVMe SSD 10TB HDD 850W Toughpower PSU Geforce RTX 3080 10Gb', 'GIGABYTE AORUS GeForce RTX 3080 MASTER 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080AORUS M-10GD (rev. 3.0) (LHR)', 'ABS Legend Gaming PC - Windows 10 Home - Intel i9 11900KF - GeForce RTX 3080 Ti - G.Skill TridentZ RGB 16GB DDR4 3200 MHz - 1TB Intel M.2 NVMe SSD', 'GIGABYTE Gaming OC GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080GAMING OC-10GD (rev. 2.0) (LHR)', 'MSI Suprim GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 Video Card RTX 3080 SUPRIM X 10G LHR', 'MSI Ventus GeForce RTX 3080 PCI Express 4.0 Video Card RTX 3080 VENTUS 3X PLUS 10G OC LHR', 'ASUS ROG Strix NVIDIA GeForce RTX 3080 Ti OC Edition Gaming Graphics Card (PCIe 4.0, 12GB GDDR6X, HDMI 2.1, Axial-tech Fan Design, 2.9-Slot, Super Alloy Power II, ASUS Auto-Extreme Technology)', 'ASUS TUF Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card TUF-RTX3080TI-O12G-GAMING', 'MSI Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card RTX 3080 Ti Gaming X Trio 12G', 'NVIDIA GeForce RTX 3080 Ti Founders Edition 12GB GDDR6X', 'ZOTAC AMP Holo GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card ZT-A30810F-10P', 'GIGABYTE AORUS GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TAORUS M-12GD', 'ASUS ROG STRIX GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 x16 ATX Video Card ROG-STRIX-RTX3080-O10G-WHITE-V2 (LHR)', 'MSI Gaming Desktop Aegis RS 11TF-223US Intel Core i7 11th Gen 11700K (3.60 GHz) 32 GB DDR4 2 TB HDD 2 TB PCIe Gen3 SSD NVIDIA GeForce RTX 3080 Ti Windows 10 Home 64-bit', 'MSI Gaming Desktop Aegis R 11TE-097US Intel Core i7 11th Gen 11700 (2.50GHz) 16GB DDR4 3TB HDD 1 TB PCIe SSD NVIDIA GeForce RTX 3080 Windows 10 Home 64-bit', 'Skytech PRISM II Gaming PC Desktop - AMD Ryzen 7 5800X, RTX 3080 10GB GDDR6X, 16GB DDR4 3200, 1TB Gen4 SSD, RGB Fans, AC Wi-Fi, Windows 10 Home 64-bit, Black', 'MSI Ventus GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 Video Card RTX 3080 VENTUS 3X 10G OC LHR', 'GIGABYTE AORUS GeForce RTX 3080 XTREME WATERFORCE WB 10G (rev. 2.0) Graphics Card, WATERFORCE Water Block Cooling System, 10GB 320-bit GDDR6X, GV-N3080AORUSX WB-10GD Rev2.0 Video Card (LHR)', 'Skytech Shiva Gaming PC Desktop - AMD Ryzen 5 5600X 3.7 GHz, RTX 3080 10 GB GDDR6X, 16 GB DDR4 3200, 1 TB NVMe SSD, B550 Motherboard, 750W Gold PSU, AC WiFi, Windows 10 Home 64-bit', 'CLX SET VR-Ready Gaming Desktop - Liquid Cooled Intel Core i9 10900KF 3.7Ghz 10-Core Processor, 32GB DDR4 Memory, GeForce RTX 3080 10GB GDDR6X Graphics, 960GB SSD, 4TB HDD, WiFi, Win 10 Home 64-bit', 'AORUS MODEL X Gaming PC Computer Desktop (Intel i9-11900K, NVIDIA GeForce RTX 3080 10GB GDDR6X, 16GB DDR4 RAM, 3TB M.2 SSD) - GB-AMXI9N8A-2051', 'GIGABYTE AORUS GeForce RTX 3080 MASTER 10G (rev. 2.0) Graphics Card, Max Covered Cooling, 10GB 320-bit GDDR6X, GV-N3080AORUS M-10GD Rev2.0 Video Card', 'iBUYPOWER Gaming Desktop Revolt 3 i7BG Intel Core i7 11th Gen 11700KF (3.60GHz) 16GB DDR4 1 TB NVMe SSD NVIDIA GeForce RTX 3080 Windows 10 Home 64-bit', 'ASUS ROG Strix G15CE Gaming & Entertainment Desktop PC (Intel i7-11700KF 8-Core, 32GB RAM, 1TB m.2 SATA SSD + 3TB HDD (3.5), NVIDIA GeForce RTX 3080 Ti, Wifi, Bluetooth, 6xUSB 3.1, Win 10 Pro)', 'iBUYPOWER Gaming Desktop Trace5MR1003Ti Intel Core i7 11th Gen 11700KF (3.60GHz) 16GB DDR4 1 TB PCIe SSD NVIDIA GeForce RTX 3080 Ti Windows 10 Home 64-bit', 'GIGABYTE Vision GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TVISION OC-12GD', 'GIGABYTE AORUS GeForce RTX 3080 XTREME 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080AORUS X-10GD (rev. 2.0) (LHR)', 'ZOTAC Trinity OC GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card ZT-A30810J-10P', 'CLX SET VR-Ready Gaming Desktop - Liquid Cooled AMD Ryzen 9 5900X 3.7Ghz 12-Core Processor, 64GB DDR4 Memory, GeForce RTX 3080 10GB GDDR6X Graphics, 1TB SSD, 6TB HDD, WiFi, Windows 10 Home 64-bit', 'ASUS ROG STRIX GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 x16 Video Card ROG-STRIX-LC-RTX3080TI-O12G-GAMING', 'GIGABYTE Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TGAMING OC-12GD']

注意:您必须添加以下导入:

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Outro

链接到有用的文档:

get_attribute()methodGets the given attribute or property of the element. textattribute returnsThe text of the element. Difference between text and innerHTML using Selenium
1

我改变了你的代码的一部分:

card_data = driver.find_elements(By.XPATH, '//a[@title = "View Details"]')
for card in card_data:
    graphics_card0 = card.get_attribute('innerText')
    print(graphics_card0)

然后输出变为:

ASUS TUF Gaming NVIDIA GeForce RTX 3080 V2 OC Edition Graphics Card (PCIe 4.0, 10GB GDDR6X, LHR, HDMI 2.1, DisplayPort 1.4a, Dual Ball Fan Bearings, Military-grade Certification, GPU Tweak II)
MSI Gaming GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 GAMING Z TRIO 10G LHR
GIGABYTE AORUS GeForce RTX 3080 MASTER 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080AORUS M-10GD (rev. 3.0) (LHR)
GIGABYTE Gaming OC GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080GAMING OC-10GD (rev. 2.0) (LHR)
MSI Suprim GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 Video Card RTX 3080 SUPRIM X 10G LHR
MSI Ventus GeForce RTX 3080 PCI Express 4.0 Video Card RTX 3080 VENTUS 3X PLUS 10G OC LHR
GIGABYTE Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TGAMING OC-12GD
ASUS TUF Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card TUF-RTX3080TI-O12G-GAMING
MSI Gaming GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card RTX 3080 Ti Gaming X Trio 12G
ASUS ROG Strix NVIDIA GeForce RTX 3080 Ti OC Edition Gaming Graphics Card (PCIe 4.0, 12GB GDDR6X, HDMI 2.1, Axial-tech Fan Design, 2.9-Slot, Super Alloy Power II, ASUS Auto-Extreme Technology)
GIGABYTE Geforce RTX 2060 6G Graphics Card, 2 x WINDFORCE Fans, 6GB 192-Bit GDDR, GV-N2060D6-6GD Video Card
Colorful AXE RTX 3060Ti 12G LHR Video Card PCI Express 4.0 Nvidia Graphics Card GPU Video Cards for Desktop Graphics Cards
LISM GTX 650ti 2GB (Computer display is 1050ti4G) GDDR5 128 Bit for PC Low-Noise and High-Definition Desktop Discrete Video Card for PC
UNIKA Radeon RX 560 million Drawing 4GD5 V2 RX 560 4G D5 Video Card for Gaming
NVIDIA GeForce RTX 3080 Ti Founders Edition 12GB GDDR6X
ZOTAC AMP Holo GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 Video Card ZT-A30810F-10P
ASUS ROG STRIX GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 x16 ATX Video Card ROG-STRIX-RTX3080-O10G-WHITE-V2 (LHR)
GIGABYTE AORUS GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TAORUS M-12GD
Graphics HD7450 2GB GDDR3 64 Bit PCIE 2.0 HDMI-Compatible+VGA+DVI Intece Video Card with Cooling Fan
MSI Ventus GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 Video Card RTX 3080 VENTUS 3X 10G OC LHR
GIGABYTE AORUS GeForce RTX 3080 XTREME WATERFORCE WB 10G (rev. 2.0) Graphics Card, WATERFORCE Water Block Cooling System, 10GB 320-bit GDDR6X, GV-N3080AORUSX WB-10GD Rev2.0 Video Card (LHR)
GIGABYTE AORUS GeForce RTX 3080 MASTER 10G (rev. 2.0) Graphics Card, Max Covered Cooling, 10GB 320-bit GDDR6X, GV-N3080AORUS M-10GD Rev2.0 Video Card
GIGABYTE Vision GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TVISION OC-12GD
GIGABYTE AORUS GeForce RTX 3080 XTREME 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080AORUS X-10GD (rev. 2.0) (LHR)
ASUS ROG STRIX GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 x16 Video Card ROG-STRIX-LC-RTX3080TI-O12G-GAMING
ZOTAC Trinity OC GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card ZT-A30810J-10P
EVGA GeForce RTX 3080 Ti 12GB XC3 ULTRA 12G-P5-3955-KR Video Conference Bundle (w/ Speaker Mic, Monitor, Webcam)
GIGABYTE Vision OC GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080VISION OC-10GD (rev. 2.0) (LHR)
GIGABYTE AORUS GeForce RTX 3080 XTREME WATERFORCE 10GB GDDR6X PCI Express 4.0 ATX Video Card GV-N3080AORUSX W-10GD (rev. 2.0) (LHR)
MSI Gaming GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 GAMING Z TRIO 10G LHR
GIGABYTE AORUS GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TAORUSX W-12GD
MSI Ventus GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 Ti Ventus 3X 12G OC
ASUS ROG Strix GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 Video Card ROG-STRIX-RTX3080-O10G-GAMING
ZOTAC GAMING GeForce RTX 3080 Trinity OC White Edition ZT-A30800K-10P
MSI Ventus GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 Ti VENTUS 3X 12G
MSI Suprim GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 Ti SUPRIM X 12G
GIGABYTE GeForce RTX 3080 Gaming OC V2 D6X 10GB
ASUS ROG STRIX GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 x16 ATX Video Card ROG-STRIX-RTX3080-O10G-WHITE-V2 (LHR)
MSI Gaming GeForce RTX 3080 10GB GDDR6X PCI Express 4.0 ATX Video Card RTX 3080 GAMING Z TRIO 10G LHR
GIGABYTE GeForce RTX 3080 VISION OC 10GB Video Card, GV-N3080VISION OC-10GD
GIGABYTE AORUS GeForce RTX 3080 Ti 12GB GDDR6X PCI Express 4.0 ATX Video Card GV-N308TAORUSX WB-12GD

本站系公益性非盈利分享网址,本文来自用户投稿,不代表码文网立场,如若转载,请注明出处

(8)
如何在 python中安装一个阶跃函数
上一篇
在命令行上使用 Firefox进行整页截图
下一篇

相关推荐

发表评论

登录 后才能评论

评论列表(44条)