gofeed
is a powerful and flexible library designed for parsing RSS, Atom, and JSON feeds across various formats and versions. It effectively manages non-standard elements and known extensions, and demonstrates resilience against common feed issues.
- RSS (0.90 to 2.0)
- Atom (0.3, 1.0)
- JSON (1.0, 1.1)
gofeed
takes a best-effort approach to deal with broken or invalid XML feeds, capable of handling issues like:
- Unescaped markup
- Undeclared namespace prefixes
- Missing or illegal tags
- Incorrect date formats
- ...and more.
gofeed
treats elements outside the feed's default namespace as extensions, storing them in tree-like structures under Feed.Extensions and Item.Extensions. This feature allows you to access custom extension elements easily.
Built-In Support for Popular Extensions For added convenience, gofeed includes native support for parsing certain well-known extensions into dedicated structs. Currently, it supports:
- Dublin Core: Accessible via
Feed.DublinCoreExt
andItem.DublinCoreExt
- Apple iTunes: Accessible via
Feed.ITunesExt
andItem.ITunesExt
In gofeed
, you have two primary choices for feed parsing: a universal parser for handling multiple feed types seamlessly, and specialized parsers for more granular control over individual feed types.
The universal gofeed.Parser
is designed to make it easy to work with various types of feeds—RSS, Atom, JSON—by converting them into a unified gofeed.Feed
model. This is especially useful when you're dealing with multiple feed formats and you want to treat them the same way.
The universal parser uses built-in translators like DefaultRSSTranslator
, DefaultAtomTranslator
, and DefaultJSONTranslator
to convert between the specific feed types and the universal feed. Not happy with the defaults? Implement your own gofeed.Translator
to tailor the translation process to your needs.
Alternatively, if your focus is on a single feed type, then using a specialized parser offers advantages in terms of performance and granularity. For example, if you're interested solely in RSS feeds, you can use rss.Parser
directly. These feed-specific parsers map fields to their corresponding models, ensuring names and structures that match the feed type exactly.
Here's how to parse feeds using gofeed.Parser
:
fp := gofeed.NewParser()
feed, _ := fp.ParseURL("http://feeds.twit.tv/twit.xml")
fmt.Println(feed.Title)
feedData := `<rss version="2.0">
<channel>
<title>Sample Feed</title>
</channel>
</rss>`
fp := gofeed.NewParser()
feed, _ := fp.ParseString(feedData)
fmt.Println(feed.Title)
file, _ := os.Open("/path/to/a/file.xml")
defer file.Close()
fp := gofeed.NewParser()
feed, _ := fp.Parse(file)
fmt.Println(feed.Title)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
fp := gofeed.NewParser()
feed, _ := fp.ParseURLWithContext("http://feeds.twit.tv/twit.xml", ctx)
fmt.Println(feed.Title)
fp := gofeed.NewParser()
fp.UserAgent = "MyCustomAgent 1.0"
feed, _ := fp.ParseURL("http://feeds.twit.tv/twit.xml")
fmt.Println(feed.Title)
If you have a usage scenario that requires a specialized parser:
feedData := `<rss version="2.0">
<channel>
<webMaster>[email protected] (Example Name)</webMaster>
</channel>
</rss>`
fp := rss.Parser{}
rssFeed, _ := fp.Parse(strings.NewReader(feedData))
fmt.Println(rssFeed.WebMaster)
feedData := `<feed xmlns="http://www.w3.org/2005/Atom">
<subtitle>Example Atom</subtitle>
</feed>`
fp := atom.Parser{}
atomFeed, _ := fp.Parse(strings.NewReader(feedData))
fmt.Println(atomFeed.Subtitle)
feedData := `{"version":"1.0", "home_page_url": "https://daringfireball.net"}`
fp := json.Parser{}
jsonFeed, _ := fp.Parse(strings.NewReader(feedData))
fmt.Println(jsonFeed.HomePageURL)
fp := gofeed.NewParser()
fp.AuthConfig = &gofeed.Auth{
Username: "foo",
Password: "bar",
}
If you need more control over how fields are parsed and prioritized, you can specify your own custom translator. Below is an example that shows how to create a custom translator to give the /rss/channel/itunes:author
field higher precedence than the /rss/channel/managingEditor
field in RSS feeds.
First, we'll create a new type that embeds the default RSS translator provided by the library. We'll override its Translate method to implement our custom logic.
type MyCustomTranslator struct {
defaultTranslator *gofeed.DefaultRSSTranslator
}
func NewMyCustomTranslator() *MyCustomTranslator {
t := &MyCustomTranslator{}
t.defaultTranslator = &gofeed.DefaultRSSTranslator{}
return t
}
func (ct *MyCustomTranslator) Translate(feed interface{}) (*gofeed.Feed, error) {
rss, found := feed.(*rss.Feed)
if !found {
return nil, fmt.Errorf("Feed did not match expected type of *rss.Feed")
}
f, err := ct.defaultTranslator.Translate(rss)
if err != nil {
return nil, err
}
// Custom logic to prioritize iTunes Author over Managing Editor
if rss.ITunesExt != nil && rss.ITunesExt.Author != "" {
f.Author = rss.ITunesExt.Author
} else {
f.Author = rss.ManagingEditor
}
return f, nil
}
Once your custom translator is defined, you can tell gofeed.Parser to use it instead of the default one.
feedData := `<rss version="2.0">
<channel>
<managingEditor>Ender Wiggin</managingEditor>
<itunes:author>Valentine Wiggin</itunes:author>
</channel>
</rss>`
fp := gofeed.NewParser()
fp.RSSTranslator = NewMyCustomTranslator()
feed, _ := fp.ParseString(feedData)
fmt.Println(feed.Author) // Valentine Wiggin
- goxpp - XML Pull Parser
- goquery - Go jQuery-like interface
- testify - Unit test enhancements
- jsoniter - Faster JSON Parsing
This project is licensed under the MIT License
- cristoper for his work on implementing xml:base relative URI handling.
- Mark Pilgrim and Kurt McKee for their work on the excellent Universal Feed Parser Python library. This library was the inspiration for the
gofeed
library. - Dan MacTough for his work on node-feedparser. It provided inspiration for the set of fields that should be covered in the hybrid
gofeed.Feed
model. - Matt Jibson for his date parsing function in the goread project.
- Jim Teeuwen for his method of representing arbitrary feed extensions in the go-pkg-rss library.
- Sudhanshu Raheja for supporting JSON Feed parser