Downie and Supported Sites

Downie now supports almost 1300 sites (as of May 8, 2017). You can access the list via Preferences > Supported Sites. The important thing to keep in mind is that this list is a list of explicitly supported sites – Downie can extract videos from thousands and thousands other sites. Within this article, I will try to shed a little bit of light onto how Downie processes the links and what does it mean that a site is on the list of supported sites.

Each site on the list of supported sites is represented by a source code file that defines a link pattern that it can extract videos from and how to extract them. For example, in case of YouTube, it’s a link that contains “youtube.com/watch?v=” and then has a video ID that’s several letters/numbers/underscore. When you drop a link onto Downie, it will look at YouTube’s integration, see that it can handle the link – no, so let’s see if there’s another integration that can, etc.

It is important to keep in mind that if Downie is missing support for a particular site, you can always try the User-Guided Extraction (UGE). You can learn more about the UGE from Help > Downie Help in the menu bar. This allows to extract videos from sites that are not on the supported list, or the site is known to use various encryption techniques that may change on a daily or weekly basis as it loads the webpage in a browser and allows the site to perform all the tasks that it needs to perform.

This leads us to a question if I can add support for every site out there – the answer is unfortunately no. For some sites that you report, I may respond that the UGE should be used. While some may dismiss this as my laziness, there are several reasons behind this. As I’ve mentioned, each site on the list is defined by a link pattern. Now imagine if I added support for each site that gets reported to me (and I get dozens of reports each day) – there’d be thousands of sites on the list, most of them not eventually used by anyone, or used by 2-3 users (as past has shown).

While it may seem like it’s not an issue, it is – each link that gets dropped onto Downie needs to be checked against all the sites in order to find which one can handle it. Having e.g. 4 thousand sites with explicit support would mean that the extraction would be incredibly slow and Downie’s memory footprint would rise as these patterns get compiled (regular expression) and cached in memory – as otherwise, the extractions would be painfully slow now already.

Also, each internal change can pretty much affect all the integrations, which means that it would take me several times longer to make new features or improve the existing ones. There’s one more aspect – future compatibility. Downie integrates with online webpages and they tend to change from time to time, some on a yearly basis, some on a monthly basis and some even on weekly basis. This would place an incredible upkeep that I simply couldn’t handle.

Which is why I filter sites that get reported to me and add support for only those that I find that are more popular/visited and more users would benefit from them. The User-Guided Extraction, however, is always there and will work in a majority of cases. Downie 3 which on its way will give the User-Guided Extraction a major facelift as well as new features that will make working with it much more pleasant and easy. I will be also posting a post about how to use the User-Guided Extraction with several tips quite soon.

  • twitter
  • facebook
  • linkedin

Comments

  1. good stuff. Thanks for taking the time to write this. Very informative to know what goes on behind the scenes.

  2. Thanks for your countless work, very appreciated !…

Comments are Closed.

Next «
Previous »