3

golang爬虫 Linux + chromedriver + chrome

 1 year ago
source link: https://studygolang.com/articles/29861?fr=sidebar
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

golang爬虫 Linux + chromedriver + chrome

少群hughPGman · 2020-07-14 22:32:48 · 977 次点击 · 预计阅读时间 2 分钟 · 大约8小时之前 开始浏览    
这是一个创建于 2020-07-14 22:32:48 的文章,其中的信息可能已经有所发展或是发生改变。

Linux下chrome 安装

下载地址:https://www.google.com/chrome/?platform=linux

yum install google-chrome-stable_current_x86_64.rpm

查看版本:

google-chrome --version

chromedriver安装

wget https://chromedriver.storage.googleapis.com/84.0.4147.30/chromedriver_linux64.zip

注意:chrome和 chromedriver 版本对应。不然会出现版本不兼容的情况哦!

加载selenium包

go get github.com/tebeka/selenium

golang代码:

<pre>

package main

import (

"fmt"

"log"

"github.com/tebeka/selenium"

"github.com/tebeka/selenium/chrome"

// StartChrome 启动谷歌浏览器headless模式

func StartChrome() {

opts := []selenium.ServiceOption{}

caps := selenium.Capabilities{

"browserName": "chrome",

// 禁止加载图片,加快渲染速度

imagCaps := map[string]interface{}{

"profile.managed_default_content_settings.images": 2,

chromeCaps := chrome.Capabilities{

Prefs: imagCaps,

Path:  "",

Args: []string{

"--headless", // 设置Chrome无头模式

"--no-sandbox",

"--user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_2) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0.2 Safari/604.4.7", // 模拟user-agent,防反爬

caps.AddChrome(chromeCaps)

// 启动chromedriver,端口号可自定义

service, err := selenium.NewChromeDriverService("./chromedriver", 9516, opts...)

if err != nil {

log.Printf("Error starting the ChromeDriver server: %v", err)

// 调起chrome浏览器

webDriver, err := selenium.NewRemote(caps, fmt.Sprintf("http://localhost:%d/wd/hub", 9516))

if err != nil {

panic(err)

//目标网站

targeUrl = "https://www.toutiao.com/i6846744256028082696"

// 导航到目标网站

err = webDriver.Get(targeUrl)

if err != nil {

panic(fmt.Sprintf("Failed to load page: %s\n", err))

log.Println(webDriver.GetCookies())

defer service.Stop()  // 停止chromedriver

defer webDriver.Quit() // 关闭浏览器

func main() {

StartChrome()

</pre>


有疑问加站长微信联系(非本文作者)

280

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK