Skip to content

Commit

Permalink
fix: Handle HLS audio only request edge case (#903)
Browse files Browse the repository at this point in the history 
* Handle HLS audio only request edge case

* Tests for the HLS-related implemented logic

* Simplify overall HLS logic

* Add HLS related comments

* ninja-changes

* commenting

* switch to better? approch

* preserves the sorting for videos of same quality (e.g. based on encoding and bitrate) for highestaudio and vice versa for highestvideo

Co-authored-by: TimeForANinja <[email protected]>
  • Loading branch information
@Svallinn @TimeForANinja
Svallinn and TimeForANinja authored May 13, 2021
1 parent 374bbd8 commit 6f4907d
Show file tree
Hide file tree
Showing 3 changed files with 176 additions and 7 deletions.
7 changes: 4 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ ytdl('http://www.youtube.com/watch?v=aqz-KE-bpKQ')
# API
### ytdl(url, [options])

Attempts to download a video from the given url. Returns a [readable stream](https://nodejs.org/api/stream.html#stream_class_stream_readable). `options` can have the following, in addition to any [`getInfo()` option](#async-ytdl.getinfo(url%2C-%5Boptions%5D)) and [`chooseFormat()` option](#ytdl.downloadfrominfo(info%2C-options)).
Attempts to download a video from the given url. Returns a [readable stream](https://nodejs.org/api/stream.html#stream_class_stream_readable). `options` can have the following, in addition to any [`getInfo()` option](#async-ytdlgetinfourl-options) and [`chooseFormat()` option](#ytdlchooseformatformats-options).

* `range` - A byte range in the form `{start: INT, end: INT}` that specifies part of the file to download, ie {start: 10355705, end: 12452856}. Not supported on segmented (DASH MPD, m3u8) formats.
* This downloads a portion of the file, and not a separately spliced video.
* `begin` - What time in the video to begin. Supports formats `00:00:00.000`, `0ms, 0s, 0m, 0h`, or number of milliseconds. Example: `1:30`, `05:10.123`, `10m30s`.
* For live videos, this also accepts a unix timestamp or Date object, and defaults to `Date.now()`.
* This option is not very reliable for non-live videos, see [#129](https://github.com/fent/node-ytdl-core/issues/129), [#219](https://github.com/fent/node-ytdl-core/issues/219).
* This option is not very reliable for non-live videos, see [#129](https://github.com/fent/node-ytdl-core/issues/129) and [#219](https://github.com/fent/node-ytdl-core/issues/219).
* `liveBuffer` - How much time buffer to use for live videos in milliseconds. Default is `20000`.
* `highWaterMark` - How much of the video download to buffer into memory. See [node's docs](https://nodejs.org/api/stream.html#stream_constructor_new_stream_writable_options) for more. Defaults to 512KB.
* `dlChunkSize` - When the chosen format is video only or audio only, the download is separated into multiple chunks to avoid throttling. This option specifies the size of each chunk in bytes. Setting it to 0 disables chunking. Defaults to 10MB.
Expand Down Expand Up @@ -81,7 +81,7 @@ Can be used if you'd like to choose a format yourself. Throws an Error if it fai

`options` can have the following

* `quality` - Video quality to download. Can be an [itag value](http://en.wikipedia.org/wiki/YouTube#Quality_and_formats), a list of itag values, or `highest`/`lowest`/`highestaudio`/`lowestaudio`/`highestvideo`/`lowestvideo`. `highestaudio`/`lowestaudio`/`highestvideo`/`lowestvideo` all prefer audio/video only respectively. Defaults to `highest`, which prefers formats with both video and audio.
* `quality` - Video quality to download. Can be an [itag value](http://en.wikipedia.org/wiki/YouTube#Quality_and_formats), a list of itag values, or one of these strings: `highest`/`lowest`/`highestaudio`/`lowestaudio`/`highestvideo`/`lowestvideo`. `highestaudio`/`lowestaudio` try to minimize video bitrate for equally good audio formats while `highestvideo`/`lowestvideo` try to minimize audio respectively. Defaults to `highest`, which prefers formats with both video and audio.

A typical video's formats will be sorted in the following way using `quality: 'highest'`
```
Expand Down Expand Up @@ -151,6 +151,7 @@ ytdl cannot download videos that fall into the following
* Private (if you have access, requires [cookies](example/cookies.js))
* Rentals (if you have access, requires [cookies](example/cookies.js))
* YouTube Premium content (if you have access, requires [cookies](example/cookies.js))
* Only [HLS Livestreams](https://en.wikipedia.org/wiki/HTTP_Live_Streaming) are currently supported. Other formats will get filtered out in ytdl.chooseFormats

Generated download links are valid for 6 hours, and may only be downloadable from the same IP address.

Expand Down
28 changes: 24 additions & 4 deletions lib/format-utils.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,12 @@ exports.chooseFormat = (formats, options) => {
formats = exports.filterFormats(formats, options.filter);
}

// We currently only support HLS-Formats for livestreams
// So we (now) remove all non-HLS streams
if (formats.some(fmt => fmt.isHLS)) {
formats = formats.filter(fmt => fmt.isHLS || !fmt.isLive);
}

let format;
const quality = options.quality || 'highest';
switch (quality) {
Expand All @@ -115,23 +121,37 @@ exports.chooseFormat = (formats, options) => {
format = formats[formats.length - 1];
break;

case 'highestaudio':
case 'highestaudio': {
formats = exports.filterFormats(formats, 'audio');
formats.sort(sortFormatsByAudio);
format = formats[0];
// Filter for only the best audio format
const bestAudioFormat = formats[0];
formats = formats.filter(f => sortFormatsByAudio(bestAudioFormat, f) === 0);
// Check for the worst video quality for the best audio quality and pick according
// This does not loose default sorting of video encoding and bitrate
const worstVideoQuality = formats.map(f => parseInt(f.qualityLabel) || 0).sort((a, b) => a - b)[0];
format = formats.find(f => (parseInt(f.qualityLabel) || 0) === worstVideoQuality);
break;
}

case 'lowestaudio':
formats = exports.filterFormats(formats, 'audio');
formats.sort(sortFormatsByAudio);
format = formats[formats.length - 1];
break;

case 'highestvideo':
case 'highestvideo': {
formats = exports.filterFormats(formats, 'video');
formats.sort(sortFormatsByVideo);
format = formats[0];
// Filter for only the best video format
const bestVideoFormat = formats[0];
formats = formats.filter(f => sortFormatsByVideo(bestVideoFormat, f) === 0);
// Check for the worst audio quality for the best video quality and pick according
// This does not loose default sorting of audio encoding and bitrate
const worstAudioQuality = formats.map(f => f.audioBitrate || 0).sort((a, b) => a - b)[0];
format = formats.find(f => (f.audioBitrate || 0) === worstAudioQuality);
break;
}

case 'lowestvideo':
formats = exports.filterFormats(formats, 'video');
Expand Down
148 changes: 148 additions & 0 deletions test/format-utils-test.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,100 @@ const formats = [
hasAudio: false,
},
];

const liveWithHLS = formats.filter(x => x.isLive).slice();
liveWithHLS.push(
{
itag: '96',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '1080p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 2500000,
audioBitrate: 256,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
{
itag: '96.worse.audio',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '1080p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 2500000,
audioBitrate: 128,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
{
itag: '95',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '720p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 1500000,
audioBitrate: 256,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
{
itag: '94',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '480p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 800000,
audioBitrate: 128,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
{
itag: '92',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '240p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 150000,
audioBitrate: 48,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
{
itag: '91',
mimeType: 'video/ts; codecs="H.264, aac"',
container: 'ts',
qualityLabel: '144p',
codecs: 'H.264, aac',
videoCodec: 'H.264',
audioCodec: 'aac',
bitrate: 100000,
audioBitrate: 48,
url: 'https://googlevideo.com/',
hasVideo: true,
hasAudio: true,
isHLS: true,
},
);
const getItags = format => format.itag;


Expand Down Expand Up @@ -191,27 +285,71 @@ describe('chooseFormat', () => {
const format = chooseFormat(formats, { quality: 'highestaudio' });
assert.strictEqual(format.itag, '43');
});

describe('and no formats passed', () => {
it('throws the regular no such format found error', () => {
assert.throws(() => {
chooseFormat([], { quality: 'highestaudio' });
}, /No such format found/);
});
});

describe('and HLS formats are present', () => {
it('Chooses highest audio itag', () => {
const format = chooseFormat(liveWithHLS, { quality: 'highestaudio' });
assert.strictEqual(format.itag, '95');
});
});
});

describe('With lowest audio quality wanted', () => {
it('Chooses lowest audio itag', () => {
const format = chooseFormat(formats, { quality: 'lowestaudio' });
assert.strictEqual(format.itag, '17');
});

describe('and HLS formats are present', () => {
it('Chooses lowest audio itag', () => {
const format = chooseFormat(liveWithHLS, { quality: 'lowestaudio' });
assert.strictEqual(format.itag, '91');
});
});
});

describe('With highest video quality wanted', () => {
it('Chooses highest video itag', () => {
const format = chooseFormat(formats, { quality: 'highestvideo' });
assert.strictEqual(format.itag, '18');
});

describe('and no formats passed', () => {
it('throws the regular no such format found error', () => {
assert.throws(() => {
chooseFormat([], { quality: 'highestvideo' });
}, /No such format found/);
});
});

describe('and HLS formats are present', () => {
it('Chooses highest video itag', () => {
const format = chooseFormat(liveWithHLS, { quality: 'highestvideo' });
assert.strictEqual(format.itag, '96.worse.audio');
});
});
});

describe('With lowest video quality wanted', () => {
it('Chooses lowest video itag', () => {
const format = chooseFormat(formats, { quality: 'lowestvideo' });
assert.strictEqual(format.itag, '17');
});

describe('and HLS formats are present', () => {
it('Chooses lowest audio itag', () => {
const format = chooseFormat(liveWithHLS, { quality: 'lowestvideo' });
assert.strictEqual(format.itag, '91');
});
});
});

describe('With itag given', () => {
Expand Down Expand Up @@ -261,6 +399,16 @@ describe('chooseFormat', () => {
});
});

describe('that matches audio only formats', () => {
describe('and only non-HLS-livestream would match', () => {
it('throws the no format found exception', () => {
assert.throws(() => {
chooseFormat(liveWithHLS, { quality: 'audioonly' });
}, /No such format found/);
});
});
});

describe('that does not match a format', () => {
it('Returns an error', () => {
assert.throws(() => {
Expand Down

0 comments on commit 6f4907d

Please sign in to comment.