Discography to WordPress


flickr photo shared by Thomas Hawk under a Creative Commons ( BY-NC ) license

This is in response to something Adam Croom wrote two(?) days ago. I thought it’d be an interesting proof of concept and would let me figure out some things with a purpose. I also like to have a few projects going on at once so I have things to switch between when I get frustrated. I also see this kind of information pushing/pulling as broadly applicable. Some of this stuff is no doubt uglier than it had to be but I’ll try to show some intersections that happened to occur with other projects and how certain steps might be ignored entirely if you want to be all efficient and stuff. The final plugin is here and should be a decent start to any customized import you want to run against a CSV file.

Adam had information in Discogs. He wanted that information in WordPress where he could control it. I had never heard of the site, let alone seen its API. But it well documented and it took me a few minutes to realize I could get all the data I needed without even needing to authenticate.

The user data was associated with collections and appending 0 would get me the root level stuff.

With Adam’s user name, acroom, I could simply write the write URL structure https://api.discogs.com/users/acroom/collection/folders/0/releases to spit out the data and since there were only 29 records I didn’t have to worry about pagination. I have never taken json straight into WordPress but I have been playing with importing json into Google Sheets for a while now and so I figured I’d do a test run and see what I could do. The following Google Scripts works. Parsing out json gets a little complex at times but when I’m in doubt I append [0]. πŸ™‚

On the computational thinking side of things, the absolute key to my progress is getting in the habit of logging the variables at small intervals. That’s common sense to programmers but I don’t think I did it well until recently. In Google Scripts Logger.log() (var_dump in php) is your best friend as it shows you very quickly where you’re losing what you want. Without this, you’re a blindfolded drunk trying to hit a piΓ±ata which may or may not be a figment of your imagination.

function getThatData() {
  var url = 'https://api.discogs.com/users/acroom/collection/folders/0/releases'; //api endpoint
  
  var sheet = SpreadsheetApp.getActiveSheet();

  var response = UrlFetchApp.fetch(url); // get feed
  var json = response.getContentText(); //

  var data = JSON.parse(response);
  var counter = data.pagination.items;  
  
  for (var i = 0; i<= counter; i++) {
  var stats = [];
 	 
    stats.push(data.releases[i].instance_id);//works a
    stats.push(data.releases[i].date_added);//works b
    stats.push(data.releases[i].basic_information.labels[0].name);    //works c
    stats.push(data.releases[i].basic_information.labels[0].catno);        // work d
    stats.push(data.releases[i].basic_information.labels[0].resource_url);  // works e     
    stats.push(data.releases[i].basic_information.labels[0].entity_type_name); //workd f
    stats.push(data.releases[i].basic_information.formats[0].descriptions[0]); //stringify array?
    stats.push(data.releases[i].basic_information.formats[0].name);//works h
    stats.push(data.releases[i].basic_information.title); //works i
    stats.push(data.releases[i].basic_information.artists[0].name); //works j  
    stats.push(data.releases[i].basic_information.resource_url); //k
    stats.push(data.releases[i].basic_information.year); //l
    stats.push(data.releases[i].basic_information.id); //   m     
    
  Logger.log(stats);
  
	SpreadsheetApp.getActiveSpreadsheet().appendRow(stats);
  }
}

Now that I had the Discog data in a spreadsheet . . . I could start thinking about how to get it into WordPress. Granted, I could have done it direct from the API but I was doing some other work around parsing CSV files from Google Docs and I had the chance to re-use some of that code. I’m also relatively sure I could have used WP Ultimate CSV Importer out of the box but this was more an experiment for me.

Until I made my weekly bookmark post maker I’d never created posts in WordPress via PHP but I figured the pattern would be the same, I just needed it to loop through all of the entries and map the various elements to custom fields. Not knowing exactly what Adam’s goal for this was or how it fit within the big picture of his WordPress install it also seemed to make sense to make it a custom post type. From there, I thought for the most flexibility a plugin made the most sense. It also helped me avoid some confusion I have about how to trigger scripts in a multisite environment. I’m never quite sure on the security issues of having them out there naked and when I’m doing it myself it makes sense to make them custom page templates but that all felt overly burdensome when doing it for someone else.

I ended up using a slight variation of this method. It worked well as it was. I could trigger it via the url but . . . given I was still testing things I wanted to run it more than once and didn’t want the hassle of trying to reset things. So I just took if ( isset($_GET[‘run_my_script’]) && ! get_option(‘my_script_complete’) ) and made it if ( isset($_GET[‘run_my_script’]) ) . Long term, I like the idea of triggering scripts this way. I think it has interesting possibilities and would enable me to turn some pages into plugins which feels neater in my head.

Running the script this way also solved my runaway script problem- seemingly it was triggering repeatedly on init and so it would loop the loop. I’m not sure why but I was impressed with how quickly it could make posts. My testing site has 2000 or so records. πŸ™‚ I do all this development stuff in Vagrant now thanks to Mark. I don’t take full advantage of the environment (I need to figure out grunt/sass/watch files etc.) but it makes life far easier than the way I used to do things.

The CSV to post portion of the plugin is below. It’s commented up decently I think.

function makeRecords(){
	
 	    $file = 'https://docs.google.com/spreadsheets/d/1EHQXnOwBNiF6QbdFM3EYguLshn6-OwiApSeY16gR2lc/pub?output=csv';
            $csv = array_map('str_getcsv', file($file));
            array_walk($csv, function(&$a) use ($csv) {
            $a = array_combine($csv[0], $a);
                                    });
        foreach($csv as $item) { 	
        $title = $item['title']; //all these just line up a variable 
	$added = $item['added'];
	$label = $item['label'];			
	$label_cat = $item['label_cat_no'];			
	$entity = $item['entity_type'];			
	$format = $item['format'];
	$format_name = $item['format_name'];
	$artist = $item['artist'];
	$year = $item['year'];
	$body = $item['artist'];
//this next part starts to mash the variables into the post
		 	$my_post = array(
		    'post_title' => $title,
		    'post_content' => $body,
		    'post_status' => 'publish',
		    'post_author'   => 1,
		    'post_type' => 'records',
		);
		$the_post_id = wp_insert_post( $my_post ); //this very well may be overkill and I could have just gone with add_post_meta but ... 
		if ( ! add_post_meta( $the_post_id, 'added', $added, true ) ) { 
			   update_post_meta ( $the_post_id, 'added', $added );
			}
		if ( ! add_post_meta( $the_post_id, 'label', $label, true ) ) { 
			   update_post_meta ( $the_post_id, 'label', $label );
			}
		if ( ! add_post_meta( $the_post_id, 'label_cat', $label_cat, true ) ) { 
			   update_post_meta ( $the_post_id, 'label_cat', $label_cat );
			}		
		if ( ! add_post_meta( $the_post_id, 'entity-type', $entity, true ) ) { 
			   update_post_meta ( $the_post_id, 'entity-type', $entity );
			}
		if ( ! add_post_meta( $the_post_id, 'format', $format, true ) ) { 
			   update_post_meta ( $the_post_id, 'format', $format );
			}	
		if ( ! add_post_meta( $the_post_id, 'format-name', $format_name, true ) ) { 
			   update_post_meta ( $the_post_id, 'format-name', $format_name );
			}
		if ( ! add_post_meta( $the_post_id, 'artist', $artist, true ) ) { 
			   update_post_meta ( $the_post_id, 'artist', $artist );
			}	
		if ( ! add_post_meta( $the_post_id, 'year', $year, true ) ) { 
			   update_post_meta ( $the_post_id, 'year', $year );
			}
	}
	
}

Comments on this post

  1. easegill said on March 29, 2016 at 7:52 pm

    Pretty cool!
    I can get the app script to work with my Discogs a/c but it calls a halt at 50 records. There is some recognition that there are more records when I look at the debug
    [“{“pagination”: {“per_page”: 50, “items”: 361, “page”: 1, “urls”: {“last”: “https://api.discogs.com/users/easegill/collection/folders/0/releases?per_page=50&page=8”, “next”: “https://api.discogs.com…”]
    and from the urls section
    ({last:”https://api.discogs.com/users/easegill/collection/folders/0/releases?per_page=50&page=8″, next:”https://api.discogs.com/users/easegill/collection/folders/0/releases?per_page=50&page=2″}) “https://api.discogs.com/users/easegill/collection/folders/0/releases?per_page=50&page=8” “https://api.discogs.com/users/easegill/collection/folders/0/releases?per_page=50&page=2”

    Any ideas on how to get round the pagination hurdle? Caveat – I am not a coder πŸ™‚

  2. easegill said on March 30, 2016 at 12:32 am

    Awesome of Tom to throw me a solution and chat in the Google doc to explain what he was doing. Thanks!

Leave a Reply

Trackbacks and Pingbacks on this post

  1. The View from Here | bavatuesdays said on March 29, 2016 at 11:22 am

    […] Woodward wasn’t done yet (what a roll he is on)! Tom read Adam’s post, and figured it would be an interesting experiment for him to see how Adam might get his data out of Discogs and into something like […]

TrackBack URL