• Stars
    star
    2,027
  • Rank 22,840 (Top 0.5 %)
  • Language
    Python
  • License
    Other
  • Created over 12 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Python m3u8 Parser for HTTP Live Streaming (HLS) Transmissions

m3u8

Python m3u8 parser.

Documentation

Loading a playlist

To load a playlist into an object from uri, file path or directly from string, use the load/loads functions:

import m3u8

playlist = m3u8.load('http://videoserver.com/playlist.m3u8')  # this could also be an absolute filename
print(playlist.segments)
print(playlist.target_duration)

# if you already have the content as string, use

playlist = m3u8.loads('#EXTM3U8 ... etc ... ')

Dumping a playlist

To dump a playlist from an object to the console or a file, use the dump/dumps functions:

import m3u8

playlist = m3u8.load('http://videoserver.com/playlist.m3u8')
print(playlist.dumps())

# if you want to write a file from its content

playlist.dump('playlist.m3u8')

Supported tags

Encryption keys

The segments may or may not be encrypted. The keys attribute list will be a list with all the different keys as described with #EXT-X-KEY:

Each key has the next properties:

If no #EXT-X-KEY is found, the keys list will have a unique element None. Multiple keys are supported.

If unencrypted and encrypted segments are mixed in the M3U8 file, then the list will contain a None element, with one or more keys afterwards.

To traverse the list of keys available:

import m3u8

m3u8_obj = m3u8.loads('#EXTM3U8 ... etc ...')
len(m3u8_obj.keys)  # => returns the number of keys available in the list (normally 1)
for key in m3u8_obj.keys:
   if key:  # First one could be None
      key.uri
      key.method
      key.iv

Getting segments encrypted with one key

There are cases where listing segments for a given key is important. It's possible to retrieve the list of segments encrypted with one key via by_key method in the segments list.

Example of getting the segments with no encryption:

import m3u8

m3u8_obj = m3u8.loads('#EXTM3U8 ... etc ...')
segmk1 = m3u8_obj.segments.by_key(None)

# Get the list of segments encrypted using last key
segm = m3u8_obj.segments.by_key( m3u8_obj.keys[-1] )

With this method, is now possible also to change the key from some of the segments programmatically:

import m3u8

m3u8_obj = m3u8.loads('#EXTM3U8 ... etc ...')

# Create a new Key and replace it
new_key = m3u8.Key("AES-128", "/encrypted/newkey.bin", None, iv="0xf123ad23f22e441098aa87ee")
for segment in m3u8_obj.segments.by_key( m3u8_obj.keys[-1] ):
    segment.key = new_key
# Remember to sync the key from the list as well
m3u8_obj.keys[-1] = new_key

Variant playlists (variable bitrates)

A playlist can have a list to other playlist files, this is used to represent multiple bitrates videos, and it's called variant streams. See an example here.

variant_m3u8 = m3u8.loads('#EXTM3U8 ... contains a variant stream ...')
variant_m3u8.is_variant    # in this case will be True

for playlist in variant_m3u8.playlists:
    playlist.uri
    playlist.stream_info.bandwidth

the playlist object used in the for loop above has a few attributes:

  • uri: the url to the stream
  • stream_info: a StreamInfo object (actually a namedtuple) with all the attributes available to #EXT-X-STREAM-INF
  • media: a list of related Media objects with all the attributes available to #EXT-X-MEDIA
  • playlist_type: the type of the playlist, which can be one of VOD (video on demand) or EVENT

NOTE: the following attributes are not implemented yet, follow issue 4 for updates

  • alternative_audios: its an empty list, unless it's a playlist with Alternative audio, in this case it's a list with Media objects with all the attributes available to #EXT-X-MEDIA
  • alternative_videos: same as alternative_audios

A variant playlist can also have links to I-frame playlists, which are used to specify where the I-frames are in a video. See Apple's documentation on this for more information. These I-frame playlists can be accessed in a similar way to regular playlists.

variant_m3u8 = m3u8.loads('#EXTM3U ... contains a variant stream ...')

for iframe_playlist in variant_m3u8.iframe_playlists:
    iframe_playlist.uri
    iframe_playlist.iframe_stream_info.bandwidth

The iframe_playlist object used in the for loop above has a few attributes:

  • uri: the url to the I-frame playlist
  • base_uri: the base uri of the variant playlist (if given)
  • iframe_stream_info: a StreamInfo object (same as a regular playlist)

Custom tags

Quoting the documentation:

Lines that start with the character '#' are either comments or tags.
Tags begin with #EXT.  They are case-sensitive.  All other lines that
begin with '#' are comments and SHOULD be ignored.

This library ignores all the non-standard tags by default. If you want them to be collected while loading the file content, you need to pass a function to the load/loads functions, following the example below:

import m3u8

def get_movie(line, lineno, data, state):
    if line.startswith('#MOVIE-NAME:'):
        custom_tag = line.split(':')
        data['movie'] = custom_tag[1].strip()

m3u8_obj = m3u8.load('http://videoserver.com/playlist.m3u8', custom_tags_parser=get_movie)
print(m3u8_obj.data['movie'])  #  million dollar baby

Also you are able to override parsing of existing standard tags. To achieve this your custom_tags_parser function have to return boolean True - it will mean that you fully implement parsing of current line therefore 'main parser' can go to next line.

import re
import m3u8
from m3u8 import protocol
from m3u8.parser import save_segment_custom_value


def parse_iptv_attributes(line, lineno, data, state):
    # Customize parsing #EXTINF
    if line.startswith(protocol.extinf):
        title = ''
        chunks = line.replace(protocol.extinf + ':', '').split(',', 1)
        if len(chunks) == 2:
            duration_and_props, title = chunks
        elif len(chunks) == 1:
            duration_and_props = chunks[0]

        additional_props = {}
        chunks = duration_and_props.strip().split(' ', 1)
        if len(chunks) == 2:
            duration, raw_props = chunks
            matched_props = re.finditer(r'([\w\-]+)="([^"]*)"', raw_props)
            for match in matched_props:
                additional_props[match.group(1)] = match.group(2)
        else:
            duration = duration_and_props

        if 'segment' not in state:
            state['segment'] = {}
        state['segment']['duration'] = float(duration)
        state['segment']['title'] = title

        # Helper function for saving custom values
        save_segment_custom_value(state, 'extinf_props', additional_props)

        # Tell 'main parser' that we expect an URL on next lines
        state['expect_segment'] = True

        # Tell 'main parser' that it can go to next line, we've parsed current fully.
        return True


if __name__ == '__main__':
    PLAYLIST = """#EXTM3U
    #EXTINF:-1 timeshift="0" catchup-days="7" catchup-type="flussonic" tvg-id="channel1" group-title="Group1",Channel1
    http://str00.iptv.domain/7331/mpegts?token=longtokenhere
    """

    parsed = m3u8.loads(PLAYLIST, custom_tags_parser=parse_iptv_attributes)

    first_segment_props = parsed.segments[0].custom_parser_values['extinf_props']
    print(first_segment_props['tvg-id'])  # 'channel1'
    print(first_segment_props['group-title'])  # 'Group1'
    print(first_segment_props['catchup-type'])  # 'flussonic'

Helper functions get_segment_custom_value() and save_segment_custom_value() are intended for getting/storing your parsed values per segment into Segment class. After that all custom values will be accessible via property custom_parser_values of Segment instance.

Using different HTTP clients

If you don't want to use urllib to download playlists, having more control on how objects are fetched over the internet, you can use your own client. requests is a well known Python HTTP library and it can be used with m3u8:

import m3u8
import requests

class RequestsClient():
    def download(self, uri, timeout=None, headers={}, verify_ssl=True):
        o = requests.get(uri, timeout=timeout, headers=headers)
        return o.text, o.url

playlist = m3u8.load('http://videoserver.com/playlist.m3u8', http_client=RequestsClient())
print(playlist.dumps())

The advantage of using a custom HTTP client is to refine SSL verification, proxies, performance, flexibility, etc.

Playlists behind proxies

In case you need to use a proxy but can't use a system wide proxy (HTTP/HTTPS proxy environment variables), you can pass your HTTP/HTTPS proxies as a dict to the load function.

import m3u8

proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}

http_client = m3u8.httpclient.DefaultHTTPClient(proxies)
playlist = m3u8.load('http://videoserver.com/playlist.m3u8', http_client=http_client)  # this could also be an absolute filename
print(playlist.dumps())

It works with the default client only. Custom HTTP clients must implement this feature.

Running Tests

$ ./runtests

Contributing

All contributions are welcome, but we will merge a pull request if, and only if, it

  • has tests
  • follows the code conventions

If you plan to implement a new feature or something that will take more than a few minutes, please open an issue to make sure we don't work on the same thing.

More Repositories

1

megadraft

Megadraft is a Rich Text editor built on top of Facebook's Draft.JS featuring a nice default base of components and extensibility
JavaScript
1,210
star
2

secDevLabs

A laboratory for learning secure web and mobile development in a practical manner.
PHP
894
star
3

huskyCI

Performing security tests inside your CI
Go
572
star
4

react-native-draftjs-render

React Native render for draft.js model
JavaScript
386
star
5

database-as-a-service

Database as a service (DBaaS) that allows users to quickly and easily deploy and manage database instances using cloud infrastructure
Python
368
star
6

gitlab-ci-monitor

A simple dashboard for monitoring GitLab CI builds. Alpha version.
JavaScript
177
star
7

share-bar

A pure JS plugin to generate a share bar for social media, used by Globo.com.
JavaScript
159
star
8

sawpf

Salve a web, por favor
JavaScript
154
star
9

GloboDNS

Api to manage Bind Name Server
Ruby
139
star
10

opensource

Conheça os projetos Open Source na Globo.com
JavaScript
133
star
11

slo-generator

Easy setup a service level objective using prometheus
Go
129
star
12

destaque

Destaque is a simple slideshow plugin with built-in parallax effect.
JavaScript
125
star
13

dojo

Dojos realizados na Globo.com ao longo dos últimos anos.
JavaScript
120
star
14

hlsclient

Python HLS Client
TypeScript
105
star
15

tornado-es

A tornado-powered python library that provides asynchronous access to elasticsearch
Python
96
star
16

GloboNetworkAPI

API to automate IP Networking management, resource allocation and provisioning.
Python
85
star
17

tapioca

Tapioca is a small and flexible micro-framework on top of Tornado. It provides a simpler way to create RESTful API's.
Python
77
star
18

prettylog

Logs for human beings
Go
65
star
19

redis-healthy

It retrieves metrics, periodically, from Redis (or sentinel) and send them to Logstash
Go
63
star
20

nautilus.js

[separated fork] Async JavaScript loader & dependency manager in ~600B [gziped]
JavaScript
59
star
21

loopback-jsonschema

Adds JSON Schema support to LoopBack
JavaScript
57
star
22

echo-prometheus

Middleware for echo v4 to instrument all handlers as metrics
Go
52
star
23

functions

DEPRECATED: An Open Source Serverless Platform
JavaScript
49
star
24

kong-plugin-proxy-cache

A Proxy Caching plugin for Kong makes it fast and easy to configure caching of responses and serving of those cached responses in Redis
Lua
46
star
25

content-gateway-ruby

An easy way to get external content with two cache levels. The first is a performance cache and second is the stale
Ruby
45
star
26

pluct

A JSON Hyper Schema client that allows hypermedia navigation and resource validation
Python
39
star
27

tornado-cors

Makes it easier to add CORS support to tornado apps.
Python
38
star
28

react-native-ua

React Native module for Urban Airship
Objective-C
37
star
29

alchemetrics

Elixir metrics reporter and collector
Elixir
35
star
30

responsive-hub

JavaScript goodies for Responsive Design
JavaScript
34
star
31

alf

Python OAuth2 Client
Python
32
star
32

derpconf

derpconf abstracts loading configuration files for your app.
Python
31
star
33

glog-cli

Python
28
star
34

gifv-player

Javascript library for playing video files with gif fallback
JavaScript
25
star
35

go-buffer

Asynchronous data buffer for Go applications
Go
24
star
36

gothumbor

Golang client for Thumbor Image Service
Go
23
star
37

huskyCI-dashboard

Frontend to display data from huskyCI analyses
JavaScript
23
star
38

go-redis-opentracing

go-redis hook to collect OpenTracing spans
Go
22
star
39

vault

Admin webapp for Openstack's Keystone and Swift
Python
22
star
40

gsenha

GSenha is a password manager designed to avoid information leakage in the case of a compromise.
JavaScript
22
star
41

go-redis-prometheus

go-redis hook to export Prometheus metrics
Go
21
star
42

letrilizar

Transforme citações em imagens e compartilhe!
JavaScript
21
star
43

gsh

GSH is an OpenID Connect-compatible authentication system for systems using OpenSSH servers
Go
21
star
44

galf

Go OAuth2 Client
Go
20
star
45

azkaban-cli

CLI for Azkaban 3 API access and flow upload.
Python
20
star
46

tornado-prometheus

HTTP metrics for a tornado application
Python
20
star
47

GloboNetworkAPI-WebUI

Web UI to GloboNetworkAPI
Python
20
star
48

gitlab-lint

An open source gitlab linting utility
Go
19
star
49

zabbix-scripts

A collection of scripts to ease Zabbix administration
Python
19
star
50

oauth2u

OAuth 2 server implementation
Python
18
star
51

view-port

Viewport-Android is a library that aims to track items from a Recycler View, which remain visible in a given region on the screen (view port), for a minimum time of 250 milliseconds.
Kotlin
18
star
52

mugshot

Ruby
18
star
53

gitlab-lint-react

An open source gitlab linting utility
JavaScript
17
star
54

tornado-alf

Tornado Oauth 2 client
Python
17
star
55

measures

Backstage Measure
Python
17
star
56

gcrypt

🔐 encryption for humans
JavaScript
17
star
57

pymigration

A generic tool for migrate in python
Python
17
star
58

rtmp2img

rtmp2img: Create images from rtmp urls
Python
17
star
59

gcloud-utils

Global package for Cloud Management in Python
Python
16
star
60

graylog-plugin-oauth2

Oauth2 plugin for graylog project
Java
16
star
61

stewie

System for anomaly detection in mass generic data
Clojure
16
star
62

GloboNetworkAPI-client-python

Python client for GloboNetworkAPI
Python
16
star
63

globomap-api

API abstract used to mapping of infrastructure, services and processes of Globo.com
Python
15
star
64

jquery-eventtracker

jquery.eventtracker is a jQuery plugin wrapper for Google Analytics custom event tracker
JavaScript
14
star
65

hacktoberfest

Globo Hacktoberfest project
TypeScript
14
star
66

go-openstack

Go packages for OpenStack APIs.
Go
14
star
67

simple-virtuoso-migrate

Ontology versioning and migration tool inspired by simple-db-migrate.
Python
14
star
68

stormdash

A unique dashboard to show simple alerts
JavaScript
14
star
69

iprange

IPRange - Redis as a storage for IP range
Lua
14
star
70

dash_timeline_validator

It parses and validate a given MPD about its errors of the timeline.
Ruby
14
star
71

functions-sandbox

Sandbox for Backstage Functions
JavaScript
13
star
72

megadraft-table-plugin

Table Plugin - Megadraft Plugin
JavaScript
13
star
73

docker-openvswitch

Docker image of Open vSwitch with ssh enabled running over supervisord
13
star
74

zookeeper-centos-6

zookeeper RPM package for CentOS 6.
13
star
75

httpclient

A HTTP client in Golang.
Go
13
star
76

enforcement

Project focused on the implementation of policies in Kubernetes clusters through GitOps.
Python
13
star
77

reliable-request

A golang opinionated library to provide reliable request using hystrix-go, go-cache, and go-resiliency.
Go
13
star
78

tdi

Test Driven Infrastructure. Automates validation of deployed servers.
Ruby
12
star
79

configurable-http-proxy-redis-backend

Redis Backend for Jupyter's Configurable Proxy
JavaScript
12
star
80

dojo.globo

Dojo na Globo.com
Python
12
star
81

lig4

Lig4 is a board game brought to the web
JavaScript
11
star
82

globomap-ui

Web Interface to explore Globomap API
JavaScript
11
star
83

benchmark-python-wsgi

Benchmark of Python WSGI Servers
Python
10
star
84

container-broker

Run any Docker-based task in a simple and distributed way
Ruby
10
star
85

megadraft-related-articles-plugin

Related articles plugin for Megadraft text editor
JavaScript
10
star
86

alchemetrics_web

Collect and report key metrics for a typical web application based on Phoenix and Ecto.
Elixir
10
star
87

generator-megadraft-plugin

Plugin generator for the Megadraft Editor
JavaScript
10
star
88

auror-core

Azkaban Auror core for flow creation
Python
10
star
89

dbaas-cloudstack

A cloudstack adapter to DBaaS
Python
10
star
90

gsenha-api

GSenha-API is a password manager. Its architecture was designed to avoid information leakage in the case of a compromise
Python
9
star
91

url-pinger

URL healthcheck
Python
9
star
92

generic_cache

A Python utility / library to facilitate caching functions results'
Python
9
star
93

alchemetrics_tesla

Tesla middleware to report external call metrics.
Elixir
9
star
94

Globo-Live-Cache

Configuração de caching de vídeos ao vivo da Globo.com
Shell
9
star
95

mongo-go-prometheus

Monitors that export Prometheus metrics for the MongoDB Go driver
Go
9
star
96

directional-navigation

Directional navigation that filters elements via frustum and ranks by distance
JavaScript
8
star
97

zabbix2odbc

Zabbix macro sync for ODBC configuration with support MySQL and Oracle databases.
Python
8
star
98

memcachedapi

memcached service API for tsuru.
Python
8
star
99

dbaas-credentials

A credential manager for dbaas integrations
Python
8
star
100

tornado-stale-client

An async http client for tornado with stale cache support
Python
7
star