• Stars
    star
    169
  • Rank 224,453 (Top 5 %)
  • Language
    Python
  • License
    MIT License
  • Created over 3 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Bulk Downloader for Reddit

PyPI version license

saveddit is a bulk media downloader for reddit

pip3 install saveddit

Setting up authorization

        

        

These registrations will authorize you to use the Reddit and Imgur APIs to download publicly available information.

User configuration

The first time you run saveddit, you will see something like this:

foo@bar:~$ saveddit
Retrieving configuration from ~/.saveddit/user_config.yaml file
No configuration file found.
Creating one. Would you like to edit it now?
> Choose Y for yes and N for no

Once you choose 'yes', the program will request you to enter these credentials:

  • Your imgur client ID
  • Your reddit client ID
  • Your reddit client secret
  • Your reddit username

In case you choose 'no', the program will create a file which you can edit later, this is how to edit it:

  • Open the generated ~/.saveddit/user_config.yaml
  • Update the client IDs and secrets from the previous step
  • If you plan on using the user API, add your reddit username as well
imgur_client_id: '<YOUR_IMGUR_CLIENT_ID>'
reddit_client_id: '<YOUR_REDDIT_CLIENT_ID>'
reddit_client_secret: '<YOUR_REDDIT_CLIENT_SECRET>'
reddit_username: '<YOUR_REDDIT_USERNAME>'

Download from Subreddit

foo@bar:~$ saveddit subreddit -h
Retrieving configuration from /Users/pranav/.saveddit/user_config.yaml file

usage: saveddit subreddit [-h] [-f categories [categories ...]] [-l post_limit] [--skip-comments] [--skip-meta] [--skip-videos] -o output_path subreddits [subreddits ...]

positional arguments:
  subreddits            Names of subreddits to download, e.g., AskReddit

optional arguments:
  -h, --help            show this help message and exit
  -f categories [categories ...]
                        Categories of posts to download (default: ['hot', 'new', 'rising', 'controversial', 'top', 'gilded'])
  -l post_limit         Limit the number of submissions downloaded in each category (default: None, i.e., all submissions)
  --skip-comments       When true, saveddit will not save comments to a comments.json file
  --skip-meta           When true, saveddit will not save meta to a submission.json file on submissions
  --skip-videos         When true, saveddit will not download videos (e.g., gfycat, redgifs, youtube, v.redd.it links)
  --all-comments        When true, saveddit will download all the comments in a post instead of just downloading the top ones.)
  -o output_path        Directory where saveddit will save downloaded content
foo@bar:~$ saveddit subreddit pics -f hot -l 5 -o ~/Desktop
foo@bar:~$ tree -L 4 ~/Desktop/www.reddit.com
/Users/pranav/Desktop/www.reddit.com
└── r
    └── pics
        └── hot
            ├── 000_Prince_Philip_Duke_of_Edinburgh_...
            ├── 001_Day_10_of_Nobody_Noticing_the_Ap...
            ├── 002_First_edited_picture
            ├── 003_Reorganized_a_few_months_ago_and...
            └── 004_Van_Gogh_inspired_rainy_street_I...

You can download from multiple subreddits and use multiple filters:

foo@bar:~$ saveddit subreddit funny AskReddit -f hot top new rising -l 5 -o ~/Downloads/Reddit/.

The downloads from each subreddit to go to a separate folder like so:

foo@bar:~$ tree -L 3 ~/Downloads/Reddit/www.reddit.com
/Users/pranav/Downloads/Reddit/www.reddit.com
└── r
    ├── AskReddit
    │   ├── hot
    │   ├── new
    │   ├── rising
    │   └── top
    └── funny
        ├── hot
        ├── new
        ├── rising
        └── top

Download from anonymous Multireddit

To download from an anonymous multireddit, use the multireddit option and pass a number of subreddit names

foo@bar:~$ saveddit multireddit -h
usage: saveddit multireddit [-h] [-f categories [categories ...]] [-l post_limit] [--skip-comments] [--skip-meta] [--skip-videos] -o output_path subreddits [subreddits ...]

positional arguments:
  subreddits            Names of subreddits to download, e.g., aww, pics. The downloads will be stored in <OUTPUT_PATH>/www.reddit.com/m/aww+pics/.

optional arguments:
  -h, --help            show this help message and exit
  -f categories [categories ...]
                        Categories of posts to download (default: ['hot', 'new', 'random_rising', 'rising', 'controversial', 'top', 'gilded'])
  -l post_limit         Limit the number of submissions downloaded in each category (default: None, i.e., all submissions)
  --skip-comments       When true, saveddit will not save comments to a comments.json file
  --skip-meta           When true, saveddit will not save meta to a submission.json file on submissions
  --skip-videos         When true, saveddit will not download videos (e.g., gfycat, redgifs, youtube, v.redd.it links)
  -o output_path        Directory where saveddit will save downloaded content
foo@bar:~$ saveddit multireddit EarthPorn NaturePics -f hot -l 5 -o ~/Desktop

Anonymous multireddits are saved in www.reddit.com/m/<Multireddit_names>/<category>/ like so:

tree -L 4 ~/Desktop/www.reddit.com
/Users/pranav/Desktop/www.reddit.com
└── m
    └── EarthPorn+NaturePics
        └── hot
            ├── 000_Banning_State_Park_Minnesota_OC_...
            ├── 001_Misty_forest_in_the_mountains_of...
            ├── 002_One_of_the_highlights_of_my_last...
            ├── 003__OC_Japan_Kyoto_Garden_of_the_Go...
            └── 004_Sunset_at_Mt_Rainier_National_Pa...

Download from User's page

foo@bar:~$ saveddit user -h
usage: saveddit user [-h] users [users ...] {saved,gilded,submitted,multireddits,upvoted,comments} ...

positional arguments:
  users                 Names of users to download, e.g., Poem_for_your_sprog
  {saved,gilded,submitted,multireddits,upvoted,comments}

optional arguments:
  -h, --help            show this help message and exit

Here's a usage example for downloading all comments made by Poem_for_your_sprog

foo@bar:~$ saveddit user "Poem_for_your_sprog" comments -s top -l 5 -o ~/Desktop

Here's another example for downloading kemitche's multireddits:

foo@bar:~$ saveddit user kemitche multireddits -n reddit -f hot -l 5 -o ~/Desktop

User-specific content is downloaded to www.reddit.com/u/<Username>/... like so:

foo@bar:~$ tree ~/Desktop/www.reddit.com
/Users/pranav/Desktop/www.reddit.com
└── u
    ├── Poem_for_your_sprog
    │   ├── comments
    │   │   └── top
    │   │       ├── 000_Comment_my_name_is_Cow_and_wen_its_ni....json
    │   │       ├── 001_Comment_It_stopped_at_six_and_life....json
    │   │       ├── 002_Comment__Perhaps_I_could_listen_to_podca....json
    │   │       ├── 003_Comment__I_don_t_have_regret_for_the_thi....json
    │   │       └── 004_Comment__So_throw_off_the_chains_of_oppr....json
    │   └── user.json
    └── kemitche
        ├── m
        │   └── reddit
        │       └── hot
        │           ├── 000_When_posting_to_my_u_channel_NSF...
        │           │   ├── comments.json
        │           │   └── submission.json
        │           ├── 001_How_to_remove_popular_near_you
        │           │   ├── comments.json
        │           │   └── submission.json
        │           ├── 002__IOS_2021_13_0_Reddit_is_just_su...
        │           │   ├── comments.json
        │           │   └── submission.json
        │           ├── 003_The_Approve_User_button_should_n...
        │           │   ├── comments.json
        │           │   └── submission.json
        │           └── 004_non_moderators_unable_to_view_su...
        │               ├── comments.json
        │               └── submission.json
        └── user.json

Search and Download

saveddit support searching subreddits and downloading search results

foo@bar:~$ saveddit search -h
usage: saveddit search [-h] -q query [-s sort] [-t time_filter] [--include-nsfw] [--skip-comments] [--skip-meta] [--skip-videos] -o output_path subreddits [subreddits ...]

positional arguments:
  subreddits       Names of subreddits to search, e.g., all, aww, pics

optional arguments:
  -h, --help       show this help message and exit
  -q query         Search query string
  -s sort          Sort to apply on search (default: relevance, choices: [relevance, hot, top, new, comments])
  -t time_filter   Time filter to apply on search (default: all, choices: [all, day, hour, month, week, year])
  --include-nsfw   When true, saveddit will include NSFW results in search
  --skip-comments  When true, saveddit will not save comments to a comments.json file
  --skip-meta      When true, saveddit will not save meta to a submission.json file on submissions
  --skip-videos    When true, saveddit will not download videos (e.g., gfycat, redgifs, youtube, v.redd.it links)
  -o output_path   Directory where saveddit will save downloaded content

e.g.,

foo@bar:~$ saveddit search soccer -q "Chelsea" -o ~/Desktop

The downloaded search results are stored in www.reddit.com/q/<search_query>/<subreddits>/<sort>/.

foo@bar:~$ tree -L 4 ~/Desktop/www.reddit.com/q
/Users/pranav/Desktop/www.reddit.com/q
└── Chelsea
    └── soccer
        └── relevance
            ├── 000__Official_Results_for_UEFA_Champ...
            ├── 001_Porto_0_1_Chelsea_Mason_Mount_32...
            ├── 002_Crystal_Palace_0_2_Chelsea_Chris...
            ├── 003_Post_Match_Thread_Chelsea_2_5_We...
            ├── 004_Match_Thread_Porto_vs_Chelsea_UE...
            ├── 005_Crystal_Palace_1_4_Chelsea_Chris...
            ├── 006_Porto_0_2_Chelsea_Ben_Chilwell_8...
            ├── 007_Post_Match_Thread_Porto_0_2_Chel...
            ├── 008_UCL_Quaterfinalists_are_Bayern_D...
            ├── 009__MD_Mino_Raiola_and_Haaland_s_fa...
            ├── 010_Chelsea_2_5_West_Brom_Callum_Rob...
            ├── 011_Chelsea_1_2_West_Brom_Matheus_Pe...
            ├── 012__Bild_Sport_via_Sport_Witness_Ch...
            ├── 013_Match_Thread_Chelsea_vs_West_Bro...
            ├── 014_Chelsea_1_3_West_Brom_Callum_Rob...
            ├── 015_Match_Thread_Chelsea_vs_Atletico...
            ├── 016_Stefan_Savi�\207_Atlético_Madrid_str...
            ├── 017_Chelsea_1_0_West_Brom_Christian_...
            └── 018_Alvaro_Morata_I_ve_never_had_dep...

Supported Links:

  • Direct links to images or videos, e.g., .png, .jpg, .mp4, .gif etc.
  • Reddit galleries reddit.com/gallery/...
  • Reddit videos v.redd.it/...
  • Gfycat links gfycat.com/...
  • Redgif links redgifs.com/...
  • Imgur images imgur.com/...
  • Imgur albums imgur.com/a/... and imgur.com/gallery/...
  • Youtube links youtube.com/... and yout.be/...
  • These sites supported by youtube-dl
  • Self posts
  • For all other cases, saveddit will simply fetch the HTML of the URL

Contributing

Contributions are welcome, have a look at the CONTRIBUTING.md document for more information.

License

The project is available under the MIT license.

More Repositories

1

awesome-hpp

A curated list of awesome header-only C++ libraries
3,468
star
2

indicators

Activity Indicators for Modern C++
C++
3,004
star
3

argparse

Argument Parser for Modern C++
C++
2,655
star
4

tabulate

Table Maker for Modern C++
C++
1,926
star
5

pprint

Pretty Printer for Modern C++
C++
911
star
6

csv2

Fast CSV parser and writer for Modern C++
C++
552
star
7

alpaca

Serialization library written in C++17 - Pack C++ structs into a compact byte-array without any macros or boilerplate code
C++
474
star
8

structopt

Parse command line arguments by defining a struct
C++
455
star
9

fccf

fccf: A command-line tool that quickly searches through C/C++ source code in a directory based on a search string and prints relevant code snippets that match the query.
C++
359
star
10

glob

Glob for C++17
C++
246
star
11

csv

[DEPRECATED] See https://github.com/p-ranav/csv2
C++
234
star
12

criterion

Microbenchmarking for Modern C++
C++
211
star
13

binary_log

Fast binary logger for C++
C++
207
star
14

hypergrep

Recursively search directories for a regex pattern
C++
201
star
15

PhotoLab

AI-Powered Photo Editor (Python, PyQt6, PyTorch)
Python
161
star
16

box

box is a text-based visual programming language inspired by Unreal Engine Blueprint function graphs.
Python
120
star
17

cppgit2

Git for Modern C++ (A libgit2 Wrapper Library)
C++
116
star
18

psched

Priority-based Task Scheduling for Modern C++
C++
84
star
19

repr

repr for Modern C++: Return printable string representation of a value
C++
83
star
20

fswatch

File/Directory Watcher for Modern C++
C++
79
star
21

envy

envy: Deserialize environment variables into type-safe structs
C++
66
star
22

pipeline

Pipelines for Modern C++
C++
57
star
23

iris

Lightweight Component Model and Messaging Framework based on ØMQ
C++
53
star
24

merged_depth

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models
Python
47
star
25

unicode_display_width

Displayed width of UTF-8 strings in Modern C++
C++
44
star
26

task_system

Task System presented in "Better Code: Concurrency - Sean Parent"
C++
39
star
27

cgol

Conway's Game of Life in the Terminal
C++
35
star
28

small_vector

"Small Vector" optimization for Modern C++: store up to a small number of items on the stack
C++
33
star
29

jsonlint

Lightweight command-line tool for validating JSON
C++
33
star
30

result

Result<T, E> for Modern C++
C++
32
star
31

container_traits

Container Traits for Modern C++
C++
28
star
32

lexer

Hackable Lexer with UTF-8 support
C++
21
star
33

lc

Fast multi-threaded line counter in Modern C++ (2-10x faster than `wc -l` for large files)
C++
18
star
34

oystr

oystr recursively searches directories for a substring.
C++
10
star
35

walnut.v1

The Walnut programming language
C++
8
star
36

line-detector

OpenCV-based Hough Transform Line Detection
C++
8
star
37

ttt

Terminal Typing Test
C++
7
star
38

OpenGL-Engine

OpenGL 3D Rendering Engine
C++
7
star
39

wxPython-text-editor

wxPython Text Editor
Python
6
star
40

Vulkan-Earth

Vulkan-based 3D Rendering of Earth
HTML
6
star
41

strcpp.old

String Manipulation API for C++
C++
6
star
42

DiverseDepth

The code and data of DiverseDepth
Python
6
star
43

ImageViewer-Qt6

Minimalist image viewer in Qt6
C++
6
star
44

any_of_trait

Type traits for any_of and any_but
C++
5
star
45

zcm

A Lightweight Component Model using ZeroMQ
C++
4
star
46

StaticAnalysis

GitHub action for C++ static analysis
Python
4
star
47

video_device_discovery

Find all video devices connected to Linux-based embedded platform
C++
3
star
48

krpci

C++ client to kRPC for communication with Kerbal Space Program (KSP)
C++
2
star
49

activity-plotter

Linux Scheduler Thread Activity Plotter
Python
2
star
50

python-zcm

ZeroMQ-based Component Model in Python
Python
2
star
51

emacs_config

Emacs configuration
Emacs Lisp
1
star
52

plexil-analysis

Timing Analysis for the Plan Interchange Language (Plexil)
Python
1
star
53

object-tracker

OpenCV-based Real-time Object Tracking
C++
1
star
54

json.old

JSON Manipulation Library for C++
C++
1
star
55

phd-dissertation

TeX
1
star
56

OpenGL-Engine-II

OpenGL 3D Rendering Engine II - Alternate Architecture
C++
1
star
57

arangit

Python program that can scan a .git folder and reconstruct a git version control property graph in ArangoDB
Python
1
star
58

ros-installer

Script to install ROS Indigo from source
Python
1
star