I'm retrieving videos of a playlist using youtube v3 API and getting 50 items without any problem with this link:-
But the video count is 100 and I'm only getting 50. How can I get the next 50 items? I tried start-index but it does not work for v3 API. Any help is appreciated.
YouTube Data API v3 results are paginated. So you need to get the next page of results for the others.
Basically in the response you have nextPageToken.
To get the remaining results, do the same exact call but setting pageToken into that token you received.
There are three tokes
and also you can set max page size using
maxResults=50 {allowed Values 1 to 50 }
if you are on page 1 you won't get prevPageToken
but you get nextPageToken
pass this token to next request's
pageToken = {nextPageToken get from last request}
this way you can navigate to next page Try it Your Self
Ok, for other scenarios
If you are on First Page then
If you are on neither the first nor last page then
@Manoj: you can find your answer below
if you are on the last page
Even the field may not be present.
This is a small example made in python using the Python Youtube Client Lib This also borrows boilerplate setup from the youtube API examples
""" Pull All Youtube Videos from a Playlist """
from apiclient.discovery import build
from apiclient.errors import HttpError
from oauth2client.tools import argparser
DEVELOPER_KEY = "YOURKEY HERE"
YOUTUBE_API_SERVICE_NAME = "youtube"
YOUTUBE_API_VERSION = "v3"
def fetch_all_youtube_videos(playlistId):
"""
Fetches a playlist of videos from youtube
We splice the results together in no particular order
Parameters:
parm1 - (string) playlistId
Returns:
playListItem Dict
"""
youtube = build(YOUTUBE_API_SERVICE_NAME,
YOUTUBE_API_VERSION,
developerKey=DEVELOPER_KEY)
res = youtube.playlistItems().list(
part="snippet",
playlistId=playlistId,
maxResults="50"
).execute()
nextPageToken = res.get('nextPageToken')
while ('nextPageToken' in res):
nextPage = youtube.playlistItems().list(
part="snippet",
playlistId=playlistId,
maxResults="50",
pageToken=nextPageToken
).execute()
res['items'] = res['items'] + nextPage['items']
if 'nextPageToken' not in nextPage:
res.pop('nextPageToken', None)
else:
nextPageToken = nextPage['nextPageToken']
return res
if __name__ == '__main__':
# comedy central playlist, has 332 video
# https://www.youtube.com/watch?v=tJDLdxYKh3k&list=PLD7nPL1U-R5rDpeH95XsK0qwJHLTS3tNT
videos = fetch_all_youtube_videos("PLD7nPL1U-R5rDpeH95XsK0qwJHLTS3tNT")
videos will be a list of all your videos concatenated to the first list. It will keep fetching until it has all the videos because of pagination by 50. A similar approach can be taken in other languages.
In the list there will be all the individual video meta data and order
In the JSON data sent to us by youtube data.pageInfo.totalResults is just the original total number of videos. After some months or years this number may be decreased because of video deleting, banning ... Actually youtube gives us only currently playable videos.
So we need to change the code to get the better one.
We shouldn’t use if(sum == data.pageInfo.toTalResults) as stop condition.
Instead let’s use the standard if(typeof nextPageToken == ‘undefined’).
In the case when the current total number is less than the original one. If we use old coding then the last function call is made with undefined nextPageToken. This cause Youtube to wrongly give us the first JSON page again. In the results, we have some duplicated videos.
Please try my new coding (it looks like the coding given by mister user3053204)
/* new coding. Please replace and use your api3 key */
function start(){
count = 0;
$('#area1').val('');
getVids();
}
function getVids(PageToken){
pid = $('#searchtext1').val();
$.get(
"https://www.googleapis.com/youtube/v3/playlistItems",{
part : 'snippet',
maxResults : 50,
playlistId : pid,
pageToken : PageToken,
key: 'AIz....................................zm4'
},
function(data){
myPlan(data);
}
);
}
count = 0;
str = '';
function myPlan(data){
nextPageToken=data.nextPageToken;
pageLen = data.items.length;
count += parseInt(pageLen);
for(i=0; i<pageLen; i++){
str += '"' + data.items[i].snippet.resourceId.videoId + '", ';
}
if(typeof nextPageToken == 'undefined'){
total = data.pageInfo.totalResults;
$('#all').html(count + '/' + total + ' videos');
ind = str.lastIndexOf(',');
str1 = str.substring(0, ind);
$('#area1').val('arr = [' + str1 + '];\n');
return;
}else{
getVids(nextPageToken);
}
}
<input type="text" value="PLTI6yRvQqlYq9KoU-NHu43uDmKON7Fsjv"
id="searchtext1" size="75">
<button onclick="start()">Get Items</button>
<br><br>
IDs for test: <br>
<br>
Ricky King playlist (92/103 videos):<br>
PLTI6yRvQqlYq9KoU-NHu43uDmKON7Fsjv<br>
Franck Pourcel playlist (425/425 videos):<br>
PLMGmDo5xNOgU7gerHMwEk6SmD_vbPyLe9<br>
Lobo playlist (25/41vids):<br>
PLFE095732AC64AD06
<br><br>
TOTAL: <span id="all"></span><br>
<textarea id="area1" style="width:600px;height:500px;;font-size:12pt"></textarea>
/* old coding - to be cleared */
sum = 0;
sumN = 1;
var nextPageToken;
function getVids(PageToken){
pid = $('#searchtext1').val();
$.get(
"https://www.googleapis.com/youtube/v3/playlistItems",{
part : 'snippet',
maxResults : 50,
playlistId : pid,
pageToken : PageToken,
key: 'YOUR API3 KEY'
},
function(data){
myPlan(data);
}
);
}
function myPlan(data){
total = data.pageInfo.totalResults;
nextPageToken=data.nextPageToken;
for(i=0;i<data.items.length;i++){
document.getElementById('area1').value +=
sumN + '-' + data.items[i].snippet.title+'\n'+
data.items[i].snippet.resourceId.videoId +'\n\n';
sum++ ; sumN++;
if(sum == (total-1) ){
sum = 0;
return;
}
}
if(sum <(total-1)){
getVids(nextPageToken);
}
}
function init(){
$('#area1').val('');
}<!- - old coding - to be cleared - ->
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<script type="text/javascript" src="http://code.jquery.com/jquery-latest.js"></script>
<body onload="$('#area1').val('')">
<input type="text" value="PLTI6yRvQqlYq9KoU-NHu43uDmKON7Fsjv"
id="searchtext1" size="75">
<button onclick="getVids()">Get Items</button>
<br><br>
IDs for test: <br>PLTI6yRvQqlYq9KoU-NHu43uDmKON7Fsjv<br>
PL32C69B40337EF920
<br><br>
<textarea id="area1" style="width:600px;height:500px">
</textarea>The easiest way to get all the videos,
DEVELOPER_KEY = "REPLACE_ME" YOUTUBE_API_SERVICE_NAME = "youtube" YOUTUBE_API_VERSION = "v3"
youtube = build("youtube", "v3", developerKey=api_key)
def get_videos_from_playlist(youtube, items, playlistID):
response = items.list(part="snippet", playlistId=playlistID)
while response:
playlistitems_list_response = response.execute()
for playlist_item in playlistitems_list_response["items"]:
# title = playlist_item["snippet"]["title"]
video_id = playlist_item["snippet"]["resourceId"]["videoId"]
yield video_id
response = youtube.playlistItems().list_next(
response, playlistitems_list_response)
Pass the playlist id,
items = youtube.playlistItems()
playlist = get_videos_from_playlist(youtube, items, 'PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z')
and then parse through the list:
for videoID in playlist:
print(videoID)
additionally, you can use pages to scrape through multiple pages use something like:
nextPageToken = res.get('nextPageToken')
while ('nextPageToken' in res):
nextPage = youtube.playlistItems().list(
part="snippet",
playlistId=playlistId,
maxResults="50",
pageToken=nextPageToken
).execute()
res['items'] = res['items'] + nextPage['items']
if 'nextPageToken' not in nextPage:
res.pop('nextPageToken', None)
else:
nextPageToken = nextPage['nextPageToken']
Another solution, using recursion:
$.fn.loadYoutubeResource = function(resource_request, resource_type, resource_id, resource_container, pageToken = null, callback = null){
$.ajax({
url: "https://www.googleapis.com/youtube/v3/" + resource_request,
type: 'get',
dataType: 'json',
data: {
part : 'snippet',
[resource_type]: resource_id,
maxResults : 50,
pageToken: pageToken,
key: '< API Key >',
},
success: function(data) {
console.log("New resource " + resource_type + " loaded:");
console.log(data);
for(var index = 0; index < data.items.length; index++){
var url = data.items[index]['snippet'].thumbnails.default.url;
var ytb_id = data.items[index]['id'];
jQuery('#' + resource_container).append('<img class="tube_thumbs" src="' + url + '" id="' + ytb_id
+ '" title="' + data.items[index]['snippet']['title'] + '" >');
}
if ( data.nextPageToken == null)
return callback();
$.fn.loadYoutubeResource(resource_request, resource_type, resource_id, resource_container, data.nextPageToken, callback);
}
});
}
And then call it as follows:
jQuery('body').append('<div id="ytb_container"></div>');
$.fn.loadYoutubeResource('playlistItems', 'playlistId', 'PLmwK57OwOvYVdedKc_vPPfbcsey_R0K8r', 'ytb_container', null, function(){ <callback code>});
here is my recursive function, maybe can help somebody:
First at all I created a button for the first call:
<button id="aux" class="btn btn-danger">Click Me</button>
Then in script section:
$(document).ready(function () {
function getVideos(t) {
var url = "https://www.googleapis.com/youtube/v3/search?part=snippet&key=YourAPIKey&channelId=YourChannelID&maxResults=50";
if (t != undefined) {
url = url + "&pageToken=" + t
}
$.ajax({
type: 'GET',
url: url,
dataType: 'json',
success: function (html) {
console.log(html.items);
if (html.nextPageToken != undefined) {
getVideos(html.nextPageToken);
}
}
});
};
//initial call
$("#aux").click(function () {
getVideos();
});
});
Regards
Another ways (no need of request format knowing and having api key): just use json loacated inside youtube playlist html file and use JSON.parse method
Unfortunate, this method can’t give more then 200 videos. In json I see some key codes, maybe for continuation, but can’t use them.
For getting html source code we can use jquery $.ajax method.
<meta charset = utf-8>
<body>
<div id="help" style="width:90%">
1- Open youtube.com and search for playlist (http://www.youtube.com/....&list=...)<br>
For example: click <a href="http://www.youtube.com/watch?v=sMyj9e2qOc0&list=PL33196EF7664597D5">paul mauriat playlist</a><br>
You will see in the adress bar: http://www.youtube.com/watch?v=sMyj9e2qOc0&list=PL33196EF7664597D5 <br>
Select all and copy this url.<br>
2- Open any online html source code viewer. For example<br>
<a href="https://www.duplichecker.com/page-snooper.php">
https://www.duplichecker.com/page-snooper.php</a><br>
Paste the url, click "Enter", wait several seconds, scroll down, click "Copy"<br>
3- Return to this page. Paste the code to the below input smal textarea, wait several seconds, click "GET VIDEO IDS".<br>
4- Click the button TEST JAVASCRIPT.
</div><br>
<textarea id = "input">
</textarea><br>
<button onclick="extract()">GET VIDEOS IDS</button><br>
Total: <span id='len'></span>
<br><button onclick="test()">TEST JAVASCRIPT</button><br>
COPY THIS TO CREATE A JAVASCRIPT FILE<br>
<textarea id="area2" style="width:90%;height:400px;font-size:14pt">
</textarea><br>
<script>
function extract(){
str = document.getElementById('input').value;
aa = str.indexOf('{"responseContext');
bb = str.indexOf('{"related', aa);
jsn = str.substring(aa, bb + 24);
obje = JSON.parse(jsn);
alert('Click OK to continue');
make();
}
glStr = '';
function make(){
len = obje.contents.singleColumnWatchNextResults.playlist.playlist.contents.length;
ss = '';
for(i=0; i < len; i++){
try{
ti = obje.contents.singleColumnWatchNextResults.playlist.playlist.contents[i].playlistPanelVideoRenderer.title.runs[0].text;
ti1 = ti.replace(/"/g,'\\"');
ide = obje.contents.singleColumnWatchNextResults.playlist.playlist.contents[i].playlistPanelVideoRenderer.navigationEndpoint.watchEndpoint.videoId;
}catch(e){
ide = obje.contents.singleColumnWatchNextResults.playlist.playlist.contents[i].playlistPanelVideoRenderer.videoId;
ti1 = '[Deleted Video]';
}
ss += '{vid:"' + ide + '",tit:"' + ti1 + '"}';
if(i != (len - 1)){
ss += ',\n';
}
}
glStr = 'obj = \n[' + ss + '\n];\n';
document.getElementById('area2').value = glStr;
document.getElementById('len').innerHTML = len + ' videos';
}
function test(){
var head = document.getElementsByTagName('head').item(0);
var script = document.createElement('script');
script.setAttribute('type', 'text/javascript');
script.innerHTML = glStr;
head.appendChild(script);
alert('obj.length = ' + obj.length);
alert('obj[5].vid = ' + obj[5].vid + '\n obj[5].tit = ' + obj[5].tit);
}
</script>
</body>
I did as follows, but i don't know this one is optimized or not any how i got the results as i expected.
Explanation: Will retrieve all videos from the playlist and push into an Array dynamically.
In Service:
getSnippets(token): any {
console.log('API_TOKEN:',token);
if(token){
return this.http.get('https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&playlistId=PLV_aspERmuCKxUPxu8UwUd3I52VLje0ra&maxResults=100&key=[API_KEY]='+token).toPromise();
}else{
return this.http.get('https://www.googleapis.com/youtube/v3/playlistItems?part=snippet&playlistId=PLV_aspERmuCKxUPxu8UwUd3I52VLje0ra&maxResults=100&key=[API_KEY]').toPromise();
}
}
To display data:
async getPlayList(){
try{
let pageToken;
this.finalAr = [];
// for(let t = 0; t <= 1; t++) {
const list = await this.service.getSnippets(pageToken);
pageToken = list.nextPageToken;
// let pageInfo = list.pageInfo;
let pageTot = list.pageInfo.totalResults;
let resultsPerPage = list.pageInfo.resultsPerPage;
let loopCnt = pageTot / resultsPerPage;
let finalCnt = Math.abs(Math.ceil(loopCnt)) - 1 ;
if(pageToken != undefined || pageToken != null){
let tempAr = list.items;
for(let i = 0; i < tempAr.length; i++){
this.finalAr.push({"vid_id" : tempAr[i].snippet.resourceId.videoId,
"vid_title": tempAr[i].snippet.title,
"vid_desc" : tempAr[i].snippet.description,
"vid_icon" : 'https://i.ytimg.com/vi/'+tempAr[i].snippet.resourceId.videoId+'/sddefault.jpg'
})
}
console.log('finalAr_1:',this.finalAr);
}
for(let a = 1; a <= finalCnt; a++){
const listF = await this.service.getSnippets(pageToken);
pageToken = listF.nextPageToken;
let tempAr = listF.items;
for(let i = 0; i < tempAr.length; i++){
this.finalAr.push({"vid_id" : tempAr[i].snippet.resourceId.videoId,
"vid_title": tempAr[i].snippet.title,
"vid_desc" : tempAr[i].snippet.description,
"vid_icon" : 'https://i.ytimg.com/vi/'+tempAr[i].snippet.resourceId.videoId+'/sddefault.jpg'
})
}
console.log('finalAr_2:',this.finalAr);
}
}catch (e){
console.log('ER:',e);
}
Did the job. But i want to retrieve all videos from multiple playlists if anyone came to know about this let me know.
