Discover oxylabs/mechanicalsoup-proxy-integration Open Source project

Use Google News API to obtain the latest global news for your project, including a wide range of sources, headlines, URLs, and publication dates from the Google News platform.

1,042

google-news-scraper

Google Maps Scraper for collecting data from various Google Maps listings, including business profiles.

1,027

google-maps-scraper

Real time google image scraper. The process is automated by sending HTTP requests to retrieve image data which is then parsed and saved.

996

how-to-scrape-google-images

Scrapes flight data from Google Flights and analyzes flight prices with Python and Oxylabs’ Google Flights API.

351

how-to-scrape-google-flights

Etsy Scraper for collecting prices, titles, descriptions, specifications, reviews, and other product information from etsy.com

120

etsy-scraper

The basics of forming an input code for scraping travel industry pages with Tripadvisor Scraper API + an example of results.

104

tripadvisor-scraper

Netflix Scraper for easy collection of titles, descriptions, cast, ratings, and other public data from Netflix.

103

netflix-scraper

Gather public Craigslist data like jobs, items, services, and ads with Craigslist Scraper.

craigslist-scraper

OxyParser

A tutorial for web scraping using Playwright headless browser

playwright-web-scraping

Free trial Web Unblocker - an AI-powered proxy solution that can bypass even the most sophisticated anti-bot systems.

web-unblocker

Easily collect public AliExpress data like prices, product details, and ratings with AliExpress Scraper API

aliexpress-scraper

A guide on how to scrape JavaScript rendered websites with Python and BeautifulSoup.

Scraping-Dynamic-JavaScript-Ajax-Websites-With-BeautifulSoup

Web scraping, data parsing and automation tutorials. Suited for both beginners and intermediate/advanced programmers.

web-scraping-tutorials

Jupyter Notebook

youtube-scraper

YouTube Scraper for effortless public YouTube data collection, including video and channel information.

eBay Scraper for extracting product titles, descriptions, prices, specifications, customer reviews, and ratings from eBay

ebay-scraper

In this Python Web Scraping Tutorial, we will outline everything needed to get started with web scraping. We will begin with simple examples and move on to relatively more complex.

Python-Web-Scraping-Tutorial

Learn about how to rotate proxies by using Python.

Rotating-Proxies-With-Python

A guide on how to deal with pagination via Python.

Pagination-With-Python

In this guide on how to web scrape with Selenium, we will be using Python 3. The code should work with any version of Python above 3.6

Web-Scraping-With-Selenium

Code examples and general information

product-integrations

PHP

best-buy-scraper

Extract exact prices, titles, reviews, and product descriptions with our Best Buy Scraper.

OxyCon 2021 Active Fingerprinting with Pyppeteer Code Samples

oxycon2021-active-fingerprinting-pyppeteer

Yelp Scraper for hassle-free public Yelp data collection, including search results, comments, profiles, posts, and more.

yelp-scraper

Backlink checker is a simple tool, which checks backlink quality, identifies problematic backlinks, and outputs them to a specific Slack channel

backlink-monitoring

Tutorial for integrating Oxylabs' Residential Proxies with Selenium in Python

selenium-proxy-integration-python

Learn about web scraping news articles using Python and JavaScript.

News-Article-Scraper

Everything you need to know about parsing prices in Python.

Price-Parsing-Tutorial

Date-Parser-Tutorial

Everything you need to know about parsing dates in Python.

news-scraping

A tutorial for scraping news

curl-with-proxy

A tutorial on using cURL command with proxies

Shell

aiohttp-proxy-integration

Python tutorial for implementing Residential Proxies with AIOHTTP

A tutorial for building a web scraper in Golang

golang-web-scraper

proxy-integrations

Oxylabs proxy integration tutorials for Selenium, Playwright, Puppeteer, aiohttp and more.

A tutorial and code samples of web scraping with PHP

web-scraping-php

PHP

selenium-proxy-integration-java

Oxylabsʼ Residential Proxies integration with Selenium using Java

How to Build a Price Tracker With Python

how-to-build-a-price-tracker

Learn how to gather accurate real-time data from Google Shopping hassle-free.

scrape-google-shopping

A tutorial for implementing Oxylabs` Residential and Datacenter proxies with Puppeteer using JavaScript

puppeteer-proxy-integration-js

Tutorial for building SEO Monitoring System using Python, Celery, and a SERP Scraper API

seo-monitoring

Oxy Proxy Manager for Chrome is a free Chrome browser proxy extension that has all of the essential proxy session features in your browser.

proxy-chrome-extension

web-scraping-powershell

A comprehensive guide on using PowerShell and PowerHTML for designing reliable web scrapers.

PowerShell

how-to-make-web-scraping-faster

How to Make Web Scraping Faster - Python Tutorial

Guide to Using Google Sheets for Basic Web Scraping

web-scraping-google-sheets

parse-html-pyquery

Learn to parse HTML using PyQuery, a Python library for web scraping and manipulating HTML.

Free Trial Real Estate Scraper to extract publicly available propertydata

real-estate-scraper-api-free-trial-guide

webscraping-with-ruby

A tutorial for web scraping with Ruby

Ruby

lxml-tutorial

A tutorial on parsing webpages with lxml

The Kayak Scraper API is an effective tool for extracting valuable data from Kayak, a leading travel service. Ideal for developers, travel agencies or data analysts seeking up-to-date travel insights.

kayak-scraper

Google Shopping Scraper can scrape and parse various Google Shopping page types to collect structured e-commerce data.

google-shopping-scraper

Free trial of your SERP Scraper API. How to make the most of it?

serp-scraper-api-free-trial-guide

walmart-scraper

Walmart Scraper for extracting product prices, titles, descriptions, specifications, ratings and reviews from walmart.com

Web Crawler is a tool used to discover target URLs, select the relevant content, and have it delivered in bulk. It crawls websites in real-time and at scale to quickly deliver all content or only the data you need based on your chosen criteria.

web-crawler

A tutorial for implementing Oxylabs` Residential and Datacenter proxies with Playwright using JavaScript

playwright-proxy-integration-js

Web Scraping and Data Parsing Using Beautiful Soup

web-scraping-data-parsing-beautiful-soup

Learn how to rotate proxies using JavaScript.

rotating-proxies-javascript

A tutorial for web scraping with R

web-scraping-r

python-script-service-guide

A guide on running a Python script as a service on Windows & Linux.

A tutorial for automating competitors’ & benchmark analysis using Python

automate-competitors-benchmark-analysis

Web Scraping with Python Selenium: Tutorial for Beginners

web-scraping-selenium-python

Amazon ASIN Scraper for effortless product ASIN extraction.

amazon-asin-scraper

Leads Scraper API is a powerful tool for extracting quality leads data. Utilized for advanced web scraping, data consolidation, and lead generation.

leads-scraper

Learn how to build your own Google Jobs scraper that simultaneously scrapes Google Jobs for multiple search queries and geo-locations with Python and Oxylabs’ Google Jobs Scraper API. https://oxylabs.io/blog/how-to-scrape-google-jobs

how-to-scrape-google-jobs

google-play-scraper

Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.

Data center proxies is a powerful and cost-effective solution for public data gathering on the web.

datacenter-proxies

PHP

web-scraper-api-guide

A quick-start guide on using Web Scraper API

beautiful-soup-parsing-tutorial

Using Python & Beautiful Soup to Parse Data

A tutorial on using Oxylabs' E-Commerce Scraper

ecommerce-scraper-api-guide

regex-web-scraping

Web Scraping with RegEx

Scrape Images From a Website with Python

scrape-images-from-website

Using Apache Airflow to Build a Pipeline for Scraped Data

building-scraping-pipeline-apache-airflow

Web Scraping With Java. Let’s examine this library to create a Java website scraper.

web-scraping-with-java

python-requests

Learn how to use Python Requests module

Wayfair scraper for extracting precise titles, prices, reviews, and descriptions from wayfair.com

wayfair-scraper

shopify-scraper

Shopify Scraper for seamless Shopify data extraction, including prices, product information, reviews, and more.

Learn how to use wget command with proxies

wget-proxy

Shell

pricing-data-collection-from-ecommerce-stores

Appache Airflow DAGs for e-commerce pricing collection.

Learn the difference between concurrency and parallelism

concurrency-vs-parallelism

The Rockauto Scraper API enables efficient extraction of vehicle data from Rockauto. It delivers accurate, reliable information for developers and businesses in the auto industry.

rockauto-scraper

Learn the fundamentals of writing parsing instructions with Oxylabs' Custom Parser.

csharp-web-scraping

Web Scraping With C#

custom-parser-instructions

A step-by-step tutorial on extracting data from Wayfair’s product pages at scale and in real time. The guide details actionable code and considers various aspects before and during the scraping process.

how-to-scrape-wayfair

Use this tutorial and learn how to perform web scraping using a headless browser.

puppeteer-tutorial

A quick-start guide on using SERP Scraper API

serp-scraper-api-guide

python-parse-json

A tutorial for parsing JSON data with Python

A practical guide to reading Python syntax errors and fixing them.

python-syntax-errors

A guide to caching web scraping scripts in Python.

python-cache-tutorial

This article covers everything you need to get started with Crawlee. Learn more about its benefits and see a working example of scraping a website with Crawlee.

crawlee-web-scraping-tutorial

Step by step guide to building a web scraper with Python

how-to-build-web-scraper

Amazon Review Scraper for hassle-free review data extraction, including authors, titles, descriptions, ratings, dates, and more.

amazon-review-scraper

Weedmaps Scraper API is a powerful toolkit designed to extract crucial data from Weedmaps, a leading cannabis directory. Ideal for data analysts in the cannabis industry.

weedmaps-scraper

The Twitch Scraper API allows you to extract live and historical data from Twitch.tv. It provides user profiles, chat history, streaming stats, and more in a structured format.

twitch-scraper

Try Oxylabs' proxies with a 7-day free trial for businesses, including easy integration, detailed guides, and a 3-day money-back guarantee.

Free-Trial-Proxies

contact-details-scraper

The Contact Details Scraper API is a powerful tool for extracting contact information from various online sources. It simplifies data retrieval, ensuring efficiency, accuracy, and rapidity. Ideal for lead generation and data mining tasks.

A comparison of asynchronous and synchronous web scraping methods with practical examples.

asynchronous-web-scraping-python

Binance Scraper API is a powerful tool designed for effective web scraping of the Binance cryptocurrency data. It's efficient, easy to implement, and provides reliable market insights.

binance-scraper

Learn how to set up and get the most of your e-commerce, web scraper and serp scraper free trial.

free-trial-guides

pandas-read-html-tables

A tutorial on parsing HTML tables with pandas

Jupyter Notebook

100

jobs-scraper

The Jobs Scraper API is an innovative tool designed to scrape job postings from different platforms. It delivers reliable, accurate, and real-time job market data for business analysis.