• Stars
    star
    197
  • Rank 197,722 (Top 4 %)
  • Language
    Python
  • Created over 5 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Simple scripts for Level UP your scraping Skills, and source code for Level UP playlist on Youtube

SouqScraper

Script for scarping items from souq.com using BeautifulSoup and Python3 you need to install Python3 and BeautifulSoup

the code is very simple

import requests
import urllib.request
import time
from bs4 import BeautifulSoup
import json
import csv
filecsv = open('SouqDataapple.csv', 'w',encoding='utf8')
file = open('SouqDataapple.json','w',encoding='utf8')
# Set the URL you want to webscrape from
url = 'https://saudi.souq.com/sa-ar/apple/new/a-c/s/?section=2&page='
file.write('[\n')
data = {}
csv_columns = ['name','price','img']
for page in range(1000):
    print('---', page, '---')
    r = requests.get(url + str(page))
    print(url + str(page))
    soup = BeautifulSoup(r.content, "html.parser")
    ancher=soup.find_all('div',{'class' : 'column column-block block-grid-large single-item'})
    writer = csv.DictWriter(filecsv, fieldnames=csv_columns)
    i=0
    writer.writeheader()
    for pt in  ancher:        
file.write("\n]")
filecsv.close()
file.close()

and the output will be :

[
{"name": "سماعات ابل اير بودز اللاسلكية، ابيض - MMEF2                ", "price": "665.00", "img": "https://cf4.s3.souqcdn.com/item/2016/10/06/11/64/54/16/item_M_11645416_16747749.jpg"},
{"name": "ابل ايفون 6 مع فيس تايم - 32 جيجا، الجيل الرابع ال تي اي، رمادي                ", "price": "1,160.34", "img": "https://cf4.s3.souqcdn.com/item/2017/03/06/22/15/33/89/item_M_22153389_29502098.jpg"},
{"name": "ابل ايفون X مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, رمادي                ", "price": "3,199.00", "img": "https://cf5.s3.souqcdn.com/item/2018/01/30/24/05/14/26/item_M_24051426_102956405.jpg"},
{"name": "ابل ايفون 8 مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, ذهبي                ", "price": "2,224.99", "img": "https://cf3.s3.souqcdn.com/item/2017/09/12/24/05/14/31/item_M_24051431_35103527.jpg"},
{"name": "ابل ايفون 8 Plus مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, ذهبي                ", "price": "2,548.99", "img": "https://cf5.s3.souqcdn.com/item/2017/09/12/24/05/14/47/item_M_24051447_35103542.jpg"},
{"name": "ابل ايفون 6 بدون فيس تايم- 32 جيجا، الجيل الرابع ال تي اي، ذهبي                ", "price": "1,148.00", "img": "https://cf3.s3.souqcdn.com/item/2017/03/06/22/15/34/81/item_M_22153481_29502385.jpg"},
{"name": "ابل ايفون 6s مع فيس تايم - 64 جيجا، الجيل الرابع LTE، رمادي                ", "price": "3,270.00", "img": "https://cf3.s3.souqcdn.com/item/2015/09/10/89/92/27/3/item_M_8992273_9434431.jpg"},
{"name": "ساعة كورين للرجال - رياضية بسوار من الجلد الصناعي - 8139                ", "price": "23.50", "img": "https://cf2.s3.souqcdn.com/item/2014/05/02/68/93/69/6/item_M_6893696_4618742.jpg"},
{"name": "عطر كالفن كلاين ذا ون للرجال و النساء، متوسط التركيز، 200مل                ", "price": "101.78", "img": "https://cf2.s3.souqcdn.com/item/2015/07/21/46/37/21/4/item_M_4637214_8674641.jpg"},
{"name": "عطر بيوتي النسائي من كالفن كلاين - او دي بارفان، 100 مل                ", "price": "109.85", "img": "https://cf5.s3.souqcdn.com/item/2017/02/20/45/50/82/3/item_M_4550823_28793157.jpg"},
{"name": "سماعات ابل اير بودز اللاسلكية، ابيض - MMEF2                ", "price": "665.00", "img": "https://cf4.s3.souqcdn.com/item/2016/10/06/11/64/54/16/item_M_11645416_16747749.jpg"},
{"name": "ابل ايفون 6 مع فيس تايم - 32 جيجا، الجيل الرابع ال تي اي، رمادي                ", "price": "1,160.34", "img": "https://cf4.s3.souqcdn.com/item/2017/03/06/22/15/33/89/item_M_22153389_29502098.jpg"},
{"name": "ابل ايفون X مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, رمادي                ", "price": "3,199.00", "img": "https://cf5.s3.souqcdn.com/item/2018/01/30/24/05/14/26/item_M_24051426_102956405.jpg"},
{"name": "ابل ايفون 8 مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, ذهبي                ", "price": "2,224.99", "img": "https://cf3.s3.souqcdn.com/item/2017/09/12/24/05/14/31/item_M_24051431_35103527.jpg"},
{"name": "ابل ايفون 8 Plus مع فايس تايم - 64 جيجا, الجيل الرابع ال تي اي, ذهبي                ", "price": "2,548.99", "img": "https://cf5.s3.souqcdn.com/item/2017/09/12/24/05/14/47/item_M_24051447_35103542.jpg"},
{"name": "ابل ايفون 6 بدون فيس تايم- 32 جيجا، الجيل الرابع ال تي اي، ذهبي                ", "price": "1,148.00", "img": "https://cf3.s3.souqcdn.com/item/2017/03/06/22/15/34/81/item_M_22153481_29502385.jpg"},
{"name": "ابل ايفون 6s مع فيس تايم - 64 جيجا، الجيل الرابع LTE، رمادي                ", "price": "3,270.00", "img": "https://cf3.s3.souqcdn.com/item/2015/09/10/89/92/27/3/item_M_8992273_9434431.jpg"},
{"name": "ساعة كورين للرجال - رياضية بسوار من الجلد الصناعي - 8139                ", "price": "23.50", "img": "https://cf2.s3.souqcdn.com/item/2014/05/02/68/93/69/6/item_M_6893696_4618742.jpg"},
{"name": "عطر كالفن كلاين ذا ون للرجال و النساء، متوسط التركيز، 200مل                ", "price": "101.78", "img": "https://cf2.s3.souqcdn.com/item/2015/07/21/46/37/21/4/item_M_4637214_8674641.jpg"},
{"name": "عطر بيوتي النسائي من كالفن كلاين - او دي بارفان، 100 مل                ", "price": "109.85", "img": "https://cf5.s3.souqcdn.com/item/2017/02/20/45/50/82/3/item_M_4550823_28793157.jpg"},
{"name": "سماعات ابل اير بودز اللاسلكية، ابيض - MMEF2                ", "price": "665.00", "img": "https://cf4.s3.souqcdn.com/item/2016/10/06/11/64/54/16/item_M_11645416_16747749.jpg"},
]

More Repositories

1

whats-gpt-firebase

This is simple connecter for WhatsApp & ChatGPT& Firebase; it can be used to remember conversation using firebase.
JavaScript
20
star
2

web2appIonic1

this is simple app for converting your website to app easily
JavaScript
10
star
3

FreeStuffGeoApp

Simple ionic 4 app uses maps
JavaScript
9
star
4

LiveStreamURLApp

This simple app for saving live stream urls (NBA,UFC, NFL ...etc)
JavaScript
9
star
5

QRPay

TypeScript
5
star
6

Create_db_faker

This is a repo for innopolis Students for Phase 3 in DMD 1
Python
5
star
7

youtube-list-embed-lit-element

This is web component build by Lit Element
HTML
4
star
8

PolymerShopwithNode

JavaScript
4
star
9

freeuploader

https://ipfs.io/ipfs/Qmbn33BW35oCgCHD7q2mGK9LH2vxa97R2RzEHtkkCt6Zu6
HTML
4
star
10

WebScrapingNodeJS

web scraping script for scraping urls from any website
JavaScript
4
star
11

data-minning

Jupyter Notebook
3
star
12

pwa-html-js-example

JavaScript
3
star
13

coronaAappGeoLocation

JavaScript
3
star
14

ionic1-firebase3-stater

JavaScript
3
star
15

SmartContracts

This is a Copy of ERC20 contract
3
star
16

e

CSS
3
star
17

docker-flask-image

Python
3
star
18

ads-listing-dapp

ads listing dapp
Solidity
3
star
19

CodePiratesBlockChain

JavaScript
2
star
20

platechain

platechain website
HTML
2
star
21

DataAnalysisJupyter

Simple Example from CS dojo on youtube
Jupyter Notebook
2
star
22

learn-language-while-coding

This is website that will help you to learn any language when you are Coding
HTML
2
star
23

16personalitys

JavaScript
2
star
24

UderstandGitFast

HTML
2
star
25

Handy_Man

Source Code ofr My youtube Channel
JavaScript
2
star
26

KrokaAppStarter

This is starter app for Kroka project demonstrated in my youtube channel
TypeScript
2
star
27

TCS2-675733677

1
star
28

Book_search

JavaScript
1
star
29

AugmentReailtyRasengan

Java
1
star
30

polymer-3-starter-kit-with-login-using-node-js

polymer-3-starter-kit with login using node-js
JavaScript
1
star
31

truffle-project-example

Basic Example for Truffle Lib to deply on repsten
JavaScript
1
star
32

Gizmo

Gizmo MSD Project
Java
1
star
33

belikebelal

JavaScript
1
star
34

localstorageApp

JavaScript
1
star
35

ionicAddNote

JavaScript
1
star
36

RodeYourSelfi

JavaScript
1
star
37

MapsApp

JavaScript
1
star
38

ml-node-classifier

JavaScript
1
star
39

FacebookloginFeed

JavaScript
1
star
40

simpleDraw

JavaScript
1
star
41

YoutubePlaylistComponent

This is a web component build by polymer 3.0 to embed playlist form youtube.
JavaScript
1
star
42

twilio-node-js-send-sms

simple node js app to send message from twilio
JavaScript
1
star
43

AugmentReailtyDancingopama

JavaScript
1
star