Notes tests, Scrapping, YouTube

[DocAElstein]

---

In support of this main Forum thread
https://eileenslounge.com/viewtopic....303644#p303644

First quick working attempt

Some brief notes of what I did, problems etc.
_ In the long play list I looked at it seems you only get a text file of all the stuff I want for a bit more than 75 videos at a time. This makes sense and ties up with the experience when you view manually in real time: The scroll box only goes up to on average a bit over the first 75.


Scrapping that, or rather to say, playing around with the text file from the page source text from this

 https://www.youtube.com/watch?v=rM-CtC6cklI&list=UULFwInqvNXb-GN0JHdtoul_9A    '  --  main play list link

,give links of this form
https://http://www.youtube.com/watch?v=rM-Ct...oul_9A&index=1
https://http://www.youtube.com/watch?v=YsnmN...oul_9A&index=2
https://http://www.youtube.com/watch?v=KIx_8...oul_9A&index=3
…….. up to about &index=79
If you want the next chunk of videos, and a new text file of it all, you have to click on a video towards the bottom. ( https://i.postimg.cc/65L3ydNF/Click-...t-next-lot.jpg ) I thought I would keep stuff in some organised order, so tried getting all the text in a text file from these 9 links, the ones ending with &index=1, &index=76, &index=151, &index=226 …. 301, 376,451,526,601
That sort of worked…. Eventually…
_ I end up with 9 big text files to play with So that is sort of Part 1. I got now all the info I need, somewhere I expect, in those files… https://i.postimg.cc/R06JWCxf/9-Big-...text-files.jpg


WieGehtsYouTubeServerChrome1.txt https://app.box.com/s/0r4fsgn31gjtzoep22e31378m719znh7
WieGehtsYouTubeServerChrome76.txt https://app.box.com/s/c2y7978m1o4qqzeia15vaz2ry6jygndo
WieGehtsYouTubeServerChrome151.txt https://app.box.com/s/aj1a0gdg45lhwu24nsykihz3ln3opj2z
WieGehtsYouTubeServerChrome226.txt https://app.box.com/s/or5vbv6abv2zb8mtnsz5z54u895fgn7e
WieGehtsYouTubeServerChrome301.txt https://app.box.com/s/j0cry0vh93w17g5m2mjtzvg0dcvb1437
WieGehtsYouTubeServerChrome376.txt https://app.box.com/s/d62s25tmv1mdfvyhxhxvcnvxf8bkde3q
WieGehtsYouTubeServerChrome451.txt https://app.box.com/s/uxt1secic6beh8ejh22g79pzpj61qox6
WieGehtsYouTubeServerChrome526.txt https://app.box.com/s/h5vakr7abi0r3edzhjdkrdcecanfxoh2
WieGehtsYouTubeServerChrome601.txt https://app.box.com/s/eftpuaxfnl8nrsvt6xbo0vn3n4klv2wb


_ a small snag: Previously using the main link, https://http://www.youtube.com/watch?v=rM-Ct...-GN0JHdtoul_9A , gets the first 79 links and with the index number, which is not essential but useful to have. But use a link with the extra &index=123 and I can’t find or get the index number from those 9 text files. Could be hidden there somewhere. I can’t see it initially. Maybe later.
No matter, not so important
_ ( I am actually using initially a hybrid Yasser/ SpeakEasy suggestion code to get those. So
Object "MSXML2.ServerXMLHTTP"
and the
.setRequestHeader "User-Agent", "Chrome".
Maybe that’s a sort of “belt and braces” approach? I don’t know. I have not had the time to look in great detail at the differences yet in the three files. The hybrid comes out the smallest of the three.
( https://i.postimg.cc/MK5Q4rYc/Hybrid...-text-file.jpg ) )



Coding to get those 9 text files

Sub WieGehtsYouTubeURLServerChromeHybridStep75()   '     https://eileenslounge.com/viewtopic.php?p=303644#p303644   https://excelfox.com/forum/showthread.php/2656-Automated-Search-Results-Returning-Nothing            https://excelfox.com/forum/showthread.php/973-Lookup-First-URL-From-Google-Search-Result-Using-VBA
 On Error GoTo Bed
    '_1 First section get the long text string of the HTML coding of the internet Page
    '_1(i) get the long single text string
        With CreateObject("MSXML2.ServerXMLHTTP")
         '                                                      .Open "GET", "https://www.youtube.com/watch?v=rM-CtC6cklI&list=UULFwInqvNXb-GN0JHdtoul_9A", False ' 'just preparing the request type, how and what type... "The True/False argument of the HTTP Request is the Asynchronous mode flag. If set False then control is immediately returns to VBA after Send is executed. If set True then control is returned to VBA after the server has sent back a response.
         '.Open "GET", "https://www.youtube.com/watch?v=4vcAvCLMyUY&list=UULFwInqvNXb-GN0JHdtoul_9A&index=1", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=NVaMcQcWLKc&list=UULFwInqvNXb-GN0JHdtoul_9A&index=76", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=8a0nYGk_DkE&list=UULFwInqvNXb-GN0JHdtoul_9A&index=151", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=4VreecmIQOY&list=UULFwInqvNXb-GN0JHdtoul_9A&index=226", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=WDCmlmylNm8&list=UULFwInqvNXb-GN0JHdtoul_9A&index=301", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=Pr2sS5p0wcE&list=UULFwInqvNXb-GN0JHdtoul_9A&index=376", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=ppJI61RNY0M&list=UULFwInqvNXb-GN0JHdtoul_9A&index=451", False ' '
         '.Open "GET", "https://www.youtube.com/watch?v=RgMdq3uQNuM&list=UULFwInqvNXb-GN0JHdtoul_9A&index=526", False ' '
         .Open "GET", "https://www.youtube.com/watch?v=YofVQq3VngI&list=UULFwInqvNXb-GN0JHdtoul_9A&index=601", False ' '
         '.Open "GET", "", False ' '
         '.Open "GET", "", False ' '
         '.Open "GET", "", False ' '
         'No extra info here for type GET
         '.setRequestHeader bstrheader:="Ploppy", bstrvalue:="PooH" ' YOU MAY NEED TO TAKE OUT THIS LINE
                                                                                    '.setRequestHeader bstrheader:="If-Modified-Since", bstrvalue:="Sat, 1 Jan 2000 00:00:00 GMT" '  https://www.autohotkey.com/boards/viewtopic.php?t=9554  ---   It will caching the contents of the URL page. Which means if you request the same URL more than once, you always get the same responseText even the website changes text every time. This line is a workaround : Set cache related headers.
         .setRequestHeader "User-Agent", "Chrome"  '  https://eileenslounge.com/viewtopic.php?p=303639#p303639
         .send ' varBody:= ' No extra info for type GET. .send actually makes the request
            While .readyState <> 4: DoEvents: Wend ' Allow other processes to run while the web page loads. Think this is part of the True option
        Dim PageSrc As String: Let PageSrc = .responseText ' Save the HTML code in the (Global) variable. ': Range("P1").Value = PageSrc 'For me for a print out copy to text file etc.    The responseText property returns the information requested by the Open method as a text string
        End With
    '_1(ii)  Optional secion  to put the text string into a text file , for ease of code developments
    Dim FileNum2 As Long: Let FileNum2 = FreeFile(0)                                  ' https://msdn.microsoft.com/en-us/vba/language-reference-vba/articles/freefile-function
    Dim PathAndFileName2 As String
     Let PathAndFileName2 = ThisWorkbook.Path & "\" & "WieGehtsYouTubeServerChrome601" & ".txt" ' "WieGehtsYouTubeServerChrome526" & ".txt" ' "WieGehtsYouTubeServerChrome451" & ".txt" '  "WieGehtsYouTubeServerChrome376" & ".txt" '  "WieGehtsYouTubeServerChrome301" & ".txt" '  "WieGehtsYouTubeServerChrome226" & ".txt" '  "WieGehtsYouTubeServerChrome151" & ".txt" '  "WieGehtsYouTubeServerChrome76" & ".txt"   '   "WieGehtsYouTubeServerChrome1" & ".txt"   '
    Open PathAndFileName2 For Output As #FileNum2 ' ' The text file will be made if not there, and if it is there and already contains data, then the data will be overwritten
     Print #FileNum2, PageSrc '
     Close #FileNum2
    
Exit Sub  '  Normal code error in the case of no errors
Bed:
 MsgBox prompt:=Err.Number & ":  " & Err.Description: Debug.Print Err.Number & ":  " & Err.Description
End Sub   ' Code end in the case of any error
'    Dim sTitle As String
'     Let sTitle = Split(Split(PageSrc, """title"":{""runs"":[{""text"":""")(1), """}]}")(0)
'
'    Dim sViews As String
'     Let sViews = Split(Split(PageSrc, """shortViewCount"":{""simpleText"":""")(1), """}}}")(0)