Surfer
Package surfer is a high level concurrency http client.
It has surf
and phantom
download engines, highly simulated browser behavior, the function of analog login and so on.
Features
- Both
surf
andphantomjs
engines are supported - Support random User-Agent
- Support cache cookie
- Support http/https
Usage
package main
import (
"github.com/andeya/surfer"
"io/ioutil"
"log"
)
func main() {
// Use surf engine
resp, err := surfer.Download(&surfer.Request{
Url: "http://github.com/andeya/surfer",
})
if err != nil {
log.Fatal(err)
}
b, err := ioutil.ReadAll(resp.Body)
log.Println(string(b), err)
// Use phantomjs engine
surfer.SetPhantomJsFilePath("Path to phantomjs.exe")
resp, err = surfer.Download(&surfer.Request{
Url: "http://github.com/andeya",
DownloaderID: 1,
})
if err != nil {
log.Fatal(err)
}
b, err = ioutil.ReadAll(resp.Body)
log.Println(string(b), err)
resp.Body.Close()
surfer.DestroyJsFiles()
}
License
Surfer is under Apache v2 License. See the LICENSE file for the full license text.