Fix: differentiate shorts, lives and long videos #371
Conversation
There was a problem hiding this comment.
Thank you, I'm amazed by the intent of this first contribution.
Unfortunately, I still have many questions / doubts that need to be clarified before we can dive in code details:
- why are you sometimes speaking about normal videos and at other times about long videos? I'm a bit puzzled. And I don't get why we have only one very specific playlist here, where from my experience we often have many playlists of normal/long videos per channel in many cases
- why are you making new calls to retrieve specific playlist IDs (get_long_videos_playlist_id, ...) where
get_channel_playlists_jsonis already supposed to retrieve all playlists of the channel? - why are you using so much magic in
is_shortfunction where you have a magic cut-off date, a magic maximum duration ... this needs to be justified at least - why are you making a new API call in
is_shortinstead of just checking the playlist ID of current video starts with the magic string? We need to limit API calls at their strict minimum to avoid having throttling issues - why do you differentiate lives from long videos? Are they really so special they need special care? does it make any sense in an offline scenario? Do live videos stays in the live playlist once the live is over?
arjitcodes
changed the title
|
There was a problem hiding this comment.
Thank you !
I still don't get why you are making additional API calls. The playlists you need are already in the results of get_channel_playlists_json, why can't you simply use information already retrieved here? To me it looks like it is just a matter of finding the playlist with the ID matching the convention you are using.
And on top of that, regarding this convention (short playlist is "UUSH" + channel_id[2:], live playlist is "UULF" + channel_id[2:], video playlist is "UULV" + channel_id[2:]), where does it comes from? I have at least one example where this is purely not working. For CarbiDIY channel (ID UCuct2fNB1nSdS8Dl6XzdA1g), the short playlist ID is PLUo94RsjxOG82eLooMlRoCS2vWOqzB4_m, there are multiple long videos playlists (all these videos are visible in the "Videos" tab on youtube) and the live playlist is not listed in the playlists API endpoint. See below. Do I miss something?
Playlists of channel UCuct2fNB1nSdS8Dl6XzdA1g (CabriDIY)
{
"kind": "youtube#playlistListResponse",
"etag": "RqabC8kHsOhhJm-tqxTMpmM5RoM",
"pageInfo": {
"totalResults": 22,
"resultsPerPage": 25
},
"items": [
{
"kind": "youtube#playlist",
"etag": "vn_h-lgruI6dTb8FkcNMTKyxkfs",
"id": "PLUo94RsjxOG82eLooMlRoCS2vWOqzB4_m",
"snippet": {
"publishedAt": "2023-10-18T13:49:02Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Short",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/U84iQbF7ob8/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/U84iQbF7ob8/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/U84iQbF7ob8/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/U84iQbF7ob8/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/U84iQbF7ob8/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Short",
"description": ""
}
},
"contentDetails": {
"itemCount": 7
}
},
{
"kind": "youtube#playlist",
"etag": "G9NnDqOYdC61FlYVMF0NQPRRgBc",
"id": "PLUo94RsjxOG9XYbTrSBwaUBRz_dSzC4am",
"snippet": {
"publishedAt": "2023-06-17T13:55:16Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "TikTok",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/tgfpj-Q1m80/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/tgfpj-Q1m80/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/tgfpj-Q1m80/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/tgfpj-Q1m80/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/tgfpj-Q1m80/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "TikTok",
"description": ""
}
},
"contentDetails": {
"itemCount": 6
}
},
{
"kind": "youtube#playlist",
"etag": "4rPLddxHvgD-EVEOR3H2OOV0Mu0",
"id": "PLUo94RsjxOG_C3ybIF7YxiRfhxwLYoMKq",
"snippet": {
"publishedAt": "2022-03-20T15:51:41Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Marathon",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/tyMbsYPd0zo/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/tyMbsYPd0zo/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/tyMbsYPd0zo/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/tyMbsYPd0zo/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/tyMbsYPd0zo/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Marathon",
"description": ""
}
},
"contentDetails": {
"itemCount": 3
}
},
{
"kind": "youtube#playlist",
"etag": "AD5glo4vlrDp72umzslhUpFyH4s",
"id": "PLUo94RsjxOG_Yt5kh1iklXpeH8zGMo5gB",
"snippet": {
"publishedAt": "2021-12-16T22:05:32Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "PlayStation 1 miroir",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/1TeRgE25fjo/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/1TeRgE25fjo/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/1TeRgE25fjo/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/1TeRgE25fjo/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/1TeRgE25fjo/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "PlayStation 1 miroir",
"description": ""
}
},
"contentDetails": {
"itemCount": 6
}
},
{
"kind": "youtube#playlist",
"etag": "k-tihhlyFRaZwXxLFDDczkniMZQ",
"id": "PLUo94RsjxOG8FEjZBeAcoM1u3j7b8rqJj",
"snippet": {
"publishedAt": "2021-07-28T11:14:15Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Réparation de l'extrême",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/BDjmFoSoqEM/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/BDjmFoSoqEM/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/BDjmFoSoqEM/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/BDjmFoSoqEM/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/BDjmFoSoqEM/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Réparation de l'extrême",
"description": ""
}
},
"contentDetails": {
"itemCount": 2
}
},
{
"kind": "youtube#playlist",
"etag": "lfUx3NpnZfPx85tzXhAFM9XLeiM",
"id": "PLUo94RsjxOG_QCmJOaSICFWmmcl3YdJ_k",
"snippet": {
"publishedAt": "2021-06-23T16:08:48Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Présentation PPT",
"description": "Voici la playlist des différentes présentations de console réalisés en live sur Twitch. Certaines sont des VODs complètent mais commencent par la présentation de la console.",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/3JyY84OZQ5U/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/3JyY84OZQ5U/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/3JyY84OZQ5U/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/3JyY84OZQ5U/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/3JyY84OZQ5U/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Présentation PPT",
"description": "Voici la playlist des différentes présentations de console réalisés en live sur Twitch. Certaines sont des VODs complètent mais commencent par la présentation de la console."
}
},
"contentDetails": {
"itemCount": 8
}
},
{
"kind": "youtube#playlist",
"etag": "Hq0KyqZZij6lSxObn5zqURiX5C8",
"id": "PLUo94RsjxOG8e0lku3sOAY3rsg4N7o7CP",
"snippet": {
"publishedAt": "2020-11-04T20:43:01Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Free copyright music",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/kcUh5viJCAo/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/kcUh5viJCAo/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/kcUh5viJCAo/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/kcUh5viJCAo/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/kcUh5viJCAo/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Free copyright music",
"description": ""
}
},
"contentDetails": {
"itemCount": 4
}
},
{
"kind": "youtube#playlist",
"etag": "SxVJiqpTVvoCKFwrerSJlXYIcwo",
"id": "PLUo94RsjxOG8E7f5OaaX_iMPqC2UGizdT",
"snippet": {
"publishedAt": "2020-11-03T17:15:56Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Modernisation Radio 1950",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/00jJBwZoOC4/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/00jJBwZoOC4/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/00jJBwZoOC4/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/00jJBwZoOC4/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/00jJBwZoOC4/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Modernisation Radio 1950",
"description": ""
}
},
"contentDetails": {
"itemCount": 14
}
},
{
"kind": "youtube#playlist",
"etag": "aygeqh8XE7uiYTu6krKoU1OfjZM",
"id": "PLUo94RsjxOG_w4TLMGd-1wTNeCkuebmql",
"snippet": {
"publishedAt": "2020-10-02T20:41:08Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Soirée réact",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/SHlyzqfIh3o/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/SHlyzqfIh3o/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/SHlyzqfIh3o/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/SHlyzqfIh3o/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/SHlyzqfIh3o/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Soirée réact",
"description": ""
}
},
"contentDetails": {
"itemCount": 7
}
},
{
"kind": "youtube#playlist",
"etag": "jwBimkhzpWwpccSffbqFSgM9Psw",
"id": "PLUo94RsjxOG9EaKOdzy7q4D7ran5sA-nF",
"snippet": {
"publishedAt": "2020-09-14T18:38:11Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "PS4 portable V2",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/cuHzEORnE2g/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/cuHzEORnE2g/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/cuHzEORnE2g/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/cuHzEORnE2g/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/cuHzEORnE2g/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "PS4 portable V2",
"description": ""
}
},
"contentDetails": {
"itemCount": 20
}
},
{
"kind": "youtube#playlist",
"etag": "-fEslU_EX5frfIe8EyQnHE3C048",
"id": "PLUo94RsjxOG9wv2-UzGNgaEz2szI1_lpX",
"snippet": {
"publishedAt": "2020-09-11T07:00:11Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "GB Studio",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/omdEc4cE46o/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/omdEc4cE46o/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/omdEc4cE46o/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/omdEc4cE46o/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/omdEc4cE46o/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "GB Studio",
"description": ""
}
},
"contentDetails": {
"itemCount": 4
}
},
{
"kind": "youtube#playlist",
"etag": "9M3m8iQCh67ApXf1YppOGDt6YWU",
"id": "PLUo94RsjxOG_hgit0aNBiKFqm1E9Di6Xc",
"snippet": {
"publishedAt": "2020-07-23T04:57:38Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Forza",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/ULrOv_IRvJ0/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/ULrOv_IRvJ0/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/ULrOv_IRvJ0/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/ULrOv_IRvJ0/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/ULrOv_IRvJ0/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Forza",
"description": ""
}
},
"contentDetails": {
"itemCount": 1
}
},
{
"kind": "youtube#playlist",
"etag": "NlK9rgt3E4dLiGAne4BJmSRBgGo",
"id": "PLUo94RsjxOG8RkLZrgkY3vwZaqLWTuySa",
"snippet": {
"publishedAt": "2020-07-16T20:18:45Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Warzone",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/dwuM0LNJxOg/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/dwuM0LNJxOg/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/dwuM0LNJxOg/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/dwuM0LNJxOg/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/dwuM0LNJxOg/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Warzone",
"description": ""
}
},
"contentDetails": {
"itemCount": 7
}
},
{
"kind": "youtube#playlist",
"etag": "EJ7Sv9i2EXYwkOKv9KWhD99IRQc",
"id": "PLUo94RsjxOG9b9DvyewIF-9UwLCQX50mW",
"snippet": {
"publishedAt": "2020-07-06T20:59:55Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "PS4 portable",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/a_3tiiofN2k/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/a_3tiiofN2k/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/a_3tiiofN2k/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/a_3tiiofN2k/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/a_3tiiofN2k/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "PS4 portable",
"description": ""
}
},
"contentDetails": {
"itemCount": 7
}
},
{
"kind": "youtube#playlist",
"etag": "-jinXOHlEOQt_m7Bqy16cPwEJ2s",
"id": "PLUo94RsjxOG_9LT_F-L7diQfTdVjROpl-",
"snippet": {
"publishedAt": "2020-04-20T17:54:00Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "PlayStation 2 Minitel",
"description": "Cette playlist contient les rediffusions de mes lives ou j'ai créé une Minitel Playstation 2. Donc mettre une vraie carte mère de PS2 dans un Minitel ainsi que rendre portable le tout avec une batterie tout en essayant de dénaturer le moins possible la forme originelle du Minitel.\n\nBon visionnage :)",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/0DW-fvakHto/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/0DW-fvakHto/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/0DW-fvakHto/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/0DW-fvakHto/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/0DW-fvakHto/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "PlayStation 2 Minitel",
"description": "Cette playlist contient les rediffusions de mes lives ou j'ai créé une Minitel Playstation 2. Donc mettre une vraie carte mère de PS2 dans un Minitel ainsi que rendre portable le tout avec une batterie tout en essayant de dénaturer le moins possible la forme originelle du Minitel.\n\nBon visionnage :)"
}
},
"contentDetails": {
"itemCount": 13
}
},
{
"kind": "youtube#playlist",
"etag": "61bs7IQg4EwDv8LFgxBihsmZedo",
"id": "PLUo94RsjxOG-pCzntzqcqSC0lFUqjBsHW",
"snippet": {
"publishedAt": "2020-03-31T10:07:28Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Unity",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/JaBpqoWtUlo/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/JaBpqoWtUlo/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/JaBpqoWtUlo/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/JaBpqoWtUlo/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/JaBpqoWtUlo/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Unity",
"description": ""
}
},
"contentDetails": {
"itemCount": 5
}
},
{
"kind": "youtube#playlist",
"etag": "YpXl-nmnBGR1-SIkEmkolQAHS54",
"id": "PLUo94RsjxOG83s148IMG7sWiQKKVYw9GO",
"snippet": {
"publishedAt": "2020-02-10T23:52:42Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Soirée Découverte",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/dPyVmhXx5zI/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/dPyVmhXx5zI/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/dPyVmhXx5zI/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/dPyVmhXx5zI/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/dPyVmhXx5zI/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Soirée Découverte",
"description": ""
}
},
"contentDetails": {
"itemCount": 38
}
},
{
"kind": "youtube#playlist",
"etag": "OcFw5SvGmFG-zUhM7RzO3vMQr7o",
"id": "PLUo94RsjxOG88Cdl01ZowabzfVfywKx17",
"snippet": {
"publishedAt": "2019-12-10T09:08:38Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Tutoriel",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/FEMwcdcnsHs/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/FEMwcdcnsHs/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/FEMwcdcnsHs/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/FEMwcdcnsHs/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/FEMwcdcnsHs/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Tutoriel",
"description": ""
}
},
"contentDetails": {
"itemCount": 4
}
},
{
"kind": "youtube#playlist",
"etag": "qE17gYd5af5SmRmJuujH5Mr3pdE",
"id": "PLUo94RsjxOG_ri33kAW5MrPhgrTGqkSxz",
"snippet": {
"publishedAt": "2019-09-12T10:46:28Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Vlog",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/X3gJKZnU1Xs/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/X3gJKZnU1Xs/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/X3gJKZnU1Xs/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Vlog",
"description": ""
}
},
"contentDetails": {
"itemCount": 2
}
},
{
"kind": "youtube#playlist",
"etag": "flZzgqMsz3FdJ47xYc-4qvK2cRg",
"id": "PLUo94RsjxOG_EUKSlM0VRocmsmkD2zYIn",
"snippet": {
"publishedAt": "2019-03-18T20:53:38Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Rétro Découverte",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/G_tNO82T2Pw/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/G_tNO82T2Pw/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/G_tNO82T2Pw/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/G_tNO82T2Pw/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/G_tNO82T2Pw/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Rétro Découverte",
"description": ""
}
},
"contentDetails": {
"itemCount": 25
}
},
{
"kind": "youtube#playlist",
"etag": "cSZsHlZVokco2TEEyp0oKLmYA1o",
"id": "PLUo94RsjxOG_MUINY63Xc7bZk5FK5iEfE",
"snippet": {
"publishedAt": "2018-06-20T18:00:02Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Montage",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/SCUpyAwtFlU/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/SCUpyAwtFlU/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/SCUpyAwtFlU/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/SCUpyAwtFlU/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/SCUpyAwtFlU/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Montage",
"description": ""
}
},
"contentDetails": {
"itemCount": 11
}
},
{
"kind": "youtube#playlist",
"etag": "_xRRJyYLN5wYZBoN4zIheQUhdHk",
"id": "PLUo94RsjxOG-SvUixo4LLRZX8Fy7xTO2E",
"snippet": {
"publishedAt": "2018-05-23T17:48:31Z",
"channelId": "UCuct2fNB1nSdS8Dl6XzdA1g",
"title": "Rediffusion",
"description": "",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/3JiHpX1K-pg/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/3JiHpX1K-pg/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/3JiHpX1K-pg/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/3JiHpX1K-pg/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/3JiHpX1K-pg/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "CabriDIY",
"localized": {
"title": "Rediffusion",
"description": ""
}
},
"contentDetails": {
"itemCount": 943
}
}
]
}
|
Thank you for your feedback! The additional API calls are necessary because In most cases, channels don't manually create a 'shorts' playlist, as YouTube generates this automatically by using an ID beginning with 'UUSH' (replacing 'UC' in the channel ID) for the 'Shorts' tab. This UUSH ID is the easiest and most consistent way to retrieve a channel's uploaded shorts; however, it’s not included in the
|
|
OK but then, if we run the scraper on CabriDIY channel, we will have no Lives tab, no Videos tab, and no Shorts tab. And we are far from moving close to Youtube UI. And #367 is not solved at all. Same for Do you have a sample channel where we have these "automatically created" playlists? It looks to me they are pretty rare (didn't found on in three channels I tested). And regarding #366, it looks like your CSS changes are only targeting the player, and they not very aligned with Youtube UI. The ratio of preview pictures in "Shorts" tab is still 4:3 (or something like this) instead being 3:4 (or whatever ratio Youtube is using). And the player still exhibits black areas where it would be way better if it matches the video ratio. |
|
"Automatically created" playlists are not in the result of
|
|
I think you didn't got me correctly. What I'm saying, is that by executing the code of this PR, I think that I will have a ZIM without any Videos, Live or Shorts tabs, despite your new code and the fact that these are displayed in Youtube online. I didn't tested from end-to-end, I just called your new method on my sample channel and got |
|
I understand the concern, and I’ve thoroughly tested the code on the CarbiDIY channel and other sample channels, ensuring that the Videos, Live, and Shorts tabs appear correctly. When calling the new methods, they retrieve the expected playlists, so it should work as intended for displaying these tabs offline as well. It’s possible there could be specific cases or conditions affecting playlist retrieval. I'd be happy to go through any additional channels or scenarios you’re concerned about to make sure the PR is fully robust. do you have any suggestions ? For CabriDIY, Videos Tab: For JamyEpicurieux, |
There was a problem hiding this comment.
OK, got it, I've probably simply made a typo when testing your code, sorry for that.
See inline comments for required changes.
Other than that, you need to have properly run invoke lintall (inside scraper folder) to lint your code and invoke checkall to type check your code. This will run in the CI, but this would avoid the CI fails.
And please find a way to not duplicate all tabs and grid components in Vue.JS, I'm pretty sure 90% of the code is identical, and duplicating all these files is only going to be painful to maintain (I can already very hardly tell what the difference is between these tabs and grids codes).
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #371 +/- ##
========================================
- Coverage 1.25% 1.24% -0.01%
========================================
Files 11 11
Lines 1120 1124 +4
Branches 165 164 -1
========================================
Hits 14 14
- Misses 1106 1110 +4 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
There was a problem hiding this comment.
Thanks a lot for your commitment, not an easy PR at all! But we are definitely getting much closer to a merge, I like it.
I've made some inline comments.
In addition to that, I tested the code on our test channel (ID UC8elThf5TGMpQfQc_VE917Q) https://www.youtube.com/@openZIM_testing/ and I've found (only) three additional issues.
First, we get two identical playlists : UULF8elThf5TGMpQfQc_VE917Q and UU8elThf5TGMpQfQc_VE917Q. The first one is the special Videos playlist you've added, the other one has always been returned by Youtube API when listing playlists of a channel. It seems to be a bit special as well given its ID. We probably need to have only one of these two. Don't really know how to do it properly. I think the safest solution (because I have no idea what can be found in the wild, at least I don't) is that when we have both UULF<magic_channel_suffix> and UU<magic_channel_suffix> playlists in the list, then we start a routine to compare contents of both playlists. If both lists are identical (which should be the case, at least it is on our test channel, but who knows ...) then we drop UU<magic_channel_suffix> from the list of playlists. Or maybe you have more info on this.
Second, I think that since special playlists have their dedicated tabs, they should not be listed on the Playlists tab anymore, just like on Youtube online.
Third, should we move special playlists to be the first displayed on the ChannelHomeListTab.vue? It seems to be the case in general on Youtube, but this is maybe customizable by channel owner, not sure. At least it is weird to me to have the Videos playlist at the end of the list.
|
Since we have special playlists for long videos, shorts, and live streams, the
This ensures Let me know if this approach works or if you have a better suggestion! |
|
Thank you! Mostly OK, slight adaptations and few missed points detailed below
I would prefer that we do it conditionally, only if
I forgot about this detail, good point! Looking further into the codebase, I see only two usage of this property:
I hence propose something slightly different, because I think we need a nicer fallback to always have a
Since special playlists have their dedicated tabs, they should not be listed on the Playlists tab anymore, just like on Youtube online.
Special playlists must be the first to be displayed on the
|
Handling Identical PlaylistsSince we have three special playlists (long videos, shorts, and live) and one main playlist (UU), the only case where we would encounter two identical playlists is if one of the special playlists exists while the others are missing. This would result in the main playlist (UU) and one of the special playlists being identical. Solution:
This solution allows us to detect the duplicate and remove it efficiently without needing to compare the contents of the playlists. By using the presence of the special playlists, we can determine if the |
Where does this statement comes from? I don't get it |
|
The statement
is based on the structure of the playlists. The UU (user upload) playlist contains all types of uploaded content, regardless of their form (shorts, long videos, or lives). On the other hand, the three special playlists each contain only a specific type of content:
Therefore, the UU playlist can only be identical to one of the special playlists if the channel (or user) has only one type of content: either only shorts, only long videos, or only live streams. This is the case with the zimtest channel, where only one type of content (only long videos) is present at a time. If multiple types of content were present, the Let me know if this clears things up! |
|
OK, so I finally got your point. And I agree that we should simply remove the Looking forward to see these last small changes implemented ^^ |
|
I've implemented the changes as discussed, including removing the UU playlists as per #371. Everything is ready for review now, and I'm excited to hear your thoughts! 😊 Looking forward to your feedback! |
benoit74
force-pushed
the
fix/differentiate-shorts-lives-normal-videos
branch
from
90d391b to
cb7f3ac
Compare
There was a problem hiding this comment.
@arjitcodes I've simplified your code further down, it was way too complex for probably nothing ; note that among other simplifications, I renamed mainPlaylist to firstPlaylist for clarity / disambiguation, and I completely removed the UU playlist since anyway we were adding this playlist manually
Can you please have a look and indicate what I could have missed? (not sure I've everything in mind, you spent more time than I did on this PR)
|
Nota: CI is failing due to secrets not being passed properly because you're an external contributor, I've opened #377 |
|
Thanks for simplifying the code—it looks great! I’ve reviewed the changes, and everything seems spot on. |
|
Superseeded by #378 (to ensure CI runs OK) |
Fixes #366 #367
Changes in scraper
userLongUploadsPlaylist,userShortUploadsPlaylist, anduserLivesPlaylistinchannel.json, which contain the user's uploaded long videos, shorts, and live videos, respectively, to mirror YouTube's organized experience, even offline, such as dedicated user uploads Videos (longs) tab, shorts tab, and lives tab.Changes in zimui
videosshortslivestabvideoscontains long videos uploaded by the channel.shortscontains shorts uploaded by the channellivescontains lives by the channelchannel-homeroute, which points to thehometab. The home tab was previously named thevideostab.channelHeaderfor dynamic tabs to display only available tabs, such asshorts,videos, andlivesonly if available in the channel.Visuals
Old Home
New Home
Old Player
New Player
