YouTube Scraping, XPath, and Google Sheets

APIs can give you much more power but they are often overkill for what people are trying to around here- lightweight social media Here’s a lightweight example of how you can use Google Sheets and the IMPORTXML function to grab quite a bit of data from various video pages with no API or technical skills. Straight off, we’re going to want the URL of the video. We’ll put that in column A and we’ll use it as a variable in all our other formulas. Getting the Paths to the Data =IMPORTXML(A2,”(//*[contains(@class, ‘watch-title’)])[1]”) So how’d that come to be? A2 is just asking what URL we want to go to. The XPATH stuff gets a little more interesting. It’s going to look for any class that is named watch-title. I found out the title was in that div by right clicking on the title and choosing inspect in Chrome. The appended [1] will only give us the first item that meets those qualifications. Otherwise the title shows up twice. The rest of the formulas are pretty much variations on that theme. =IMPORTXML(A2,”//*[contains(@class, ‘watch-view-count’)]”) – View count =IMPORTXML(A2,”(//*[contains(@class, ‘like-button-renderer-like-button’)])[1]”) – Likes count =IMPORTXML(A2,”(//*[contains(@class, ‘like-button-renderer-dislike-button’)])[1]”) – Dislikes count =IMPORTXML(A2,”(//*[contains(@class, ‘yt-user-info’)])[1]”) – User name Throw in a video of your own if you’d like.

Private Comments via XMLIMPORT

Making shareable (Sharing with a single person or specific group but not with the world.) comments on public writing is a fairly awkward spaaaaaace right now. There are things like AnnotateIt and Awesome Screenshot and the annotations in Diigo. So I’m looking around for other free options and brain storming odd ideas and not find a whole lot and I came up with the following . . . Note: I’m not saying this is a good idea, it may even be a bad idea but it might inspire someone to do something more interesting down the line.1 I at least found it mildly amusing. Here’s how you might pull an author feed from WordPress into Google Spreadsheets with separate cells each paragraph (for paragraph level commenting). The idea being that you can share the Google document with just that student and do the commenting via the GSS commenting feature. Google spreadsheets will import lots of things (xml, atom, rss). WordPress provides lots of specific feeds (author, tag, categories, combinations thereof). So step one is to get the author feed – for example http://rampages.us/fren330/author/sheehantm/feed/. You can then use the IMPORTXML formula in GSS to import that XML and do some XPATH parsing of the pieces. In this case I used =IMPORTXML(“http://rampages.us/fren330/author/sheehantm/”,”//p”) to pull out the paragraphs. I can then share the […]


Citation Workflow – Diigo/Pinterest to Google SS

Talking to Bud the other day he mentioned that generating the citation page for his digital stories was something of a pain. I’ve thought about it a bit since then and decided to try to simplify a workflow for this. Odd thing I learned – – CHAR(10) is the official way to get line breaks in Google Spreadsheet formulas. Flickr to Diigo to Google Spreadsheets Initially, I looked at the Flickr galleries because that’s the option that Bud normally uses. I saw that the gallery was in a standard HTML list format and I had some hope. Google spreadsheets lets you pull lists and tables like these in via the IMPORTHTML function. Martin Hawksey has some good instructions and examples over here. So that failed but I could import just about every other list on the page. So, I decided doing this through Diigo would make pretty decent sense for a number of people. Assuming you choose a unique tag for the images you plan to use- this example just uses “flickr”, I’d suggest something story/movie specific. So the basic Diigo URL you’d get is https://www.diigo.com/user/bionicteaching/flickr. Trying to make this really easy for people, I set up the first page to allow you to paste that URL in and our friendly formulas transform it into https://www.diigo.com/rss/user/bionicteaching/flickr. The example linked here […]

Google Forms to Exhibit Example (POC)

So, I’ve managed to create two quick websites for work that are driven by Google’s new form option for getting data into spreadsheets. I’ve put a quick example of a log here. Feel free to enter data etc. It’s up there to play around with and hopefully is simple enough to help people figure out how to do it. One thing I don’t like about the form option. I don’t like that changes I make to the submission form alter my spreadsheet. I might want the form to read “Your name here” while my spreadsheet says {name:text}. I don’t believe there’s any way to do that and it would be much nicer when using this with Exhibit. Instead I have to add another sheet and I use a formula to reference the data in. It’s just =sheet1!A2 in case anyone needs it. Then if I get my mouse in just the right place it turns into cross hairs and I can drag that formula dynamically so that it pastes as =sheet1!A3 and A4 etc. then I can drag it across to create =sheet1B2 etc. That is much better than typing all that in. In a perfect world I’d also be able to apply some css to it but that’s getting a little picky. So the key steps. create spreadsheet and […]