Also looks cool at night
Barely a week goes by when I don’t shop in Urban Outfitters. Or window-shop, or think about shopping there, or hear about something they’re doing. I ran through that conversion funnel, and am happily hanging out in the retention / referral / revenue zone. I got to thinking about who else was in there with me, and if that Instagram data I looked at last time could reveal who the top Urban Outfitters fans are.
In this blog post I build on the ideas in my previous Instagram campaign reporting piece, developing the code to capture more data, find the most engaged fans and produce an informative HTML report.
This is a much more technical post than part one. I focus on the architecture needed to get results and how to hack it together quickly, skipping over a lot of material at a fairly high level. If you’re learning how to write hacks like this, try reading each section and writing your own code to get the same results. Then look at my code and see how our approaches compare. If you just want the results, the code is all there for you to run.
Capturing all the Instagram data
In my earlier code I downloaded the Instagram data to a JSON file. It’s a trivial step from here to store it in a Mongo database, which gives us more powerful and flexible tools to analyse it with. If you’re on a Mac, I recommend installing Mongo with Homebrew. It’s easy and neat, and easy to keep up to date.
If you’re not, how did you come to be reading some hipster growth hacker’s blog post about tracking Urban Outfitters on Instagram? Didn’t I throw enough buzzwords and buzz-brands in there to put you off? I thought I was good at audience targeting, but you’ve thrown me. I don’t know what to say to you.
Getting set up
You can get everything you need with these commands:
ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
brew install mongo
git clone firstname.lastname@example.org:ollieglass/instagram-campaign-tracker.git
pip install -r requirements.txt
They will install Homebrew, install Mongo, clone my github repo and install the python dependencies, in that order. Now you’re good to go.
Downloading and saving to Mongo
The download_data.py script has two parameters at the top.
CAMPAIGN = "uostyle"
INSTAGRAM_API_ACCESS_TOKEN = "XXXXXXXXXXXXXX"
A quick hack to get an API key: visit Instagram’s web API console, change authentication to OAuth2 and make any request. Copy the access token from the request details in the left hand panel and paste them into the script. Quicker than registering for a key 😉
Get an access token from the Instagram API console
Setting the CAMPAIGN variable will change the hashtag that’s searched for and the Mongo collection the data is saved to, so you can collect data from many campaigns in your database.
The script downloads and stores all of the photo data it can get. You should see output like this as it paginates through the data:
Yeah, it just errors when it’s finished. It’s a hack, not some enterprise CRM solution!
Finding top users in Instagram data
find_top_fans.py. It will run through all of the photos in the “uostyle” collection, and record a count of each username that features in a like, comment, caption or as a photo creator. This script also has a CAMPAIGN variable you can change to analyse different campaigns.
Every username appearance is counted, and the top ten most frequently occuring are printed to the terminal.
This is just a quick script to see what kind of volume we have. Let’s do something a little more sophisticated and count each type of user engagement.
Recording user engagement
analyse_campaign_users.py script first records the details of every user into a new collection. Then it counts each engagement type – likes, photos, comments, captions and a total – and records them to a stats object, one per user.
The new collection name is the campaign name with “_users” appended, so uostyle’s users are in “uostyle_users”. You can view the contents of this collection in the database. Run
mongo to open a database query analyser, then
use instagram to switch to our database,
show collections to see each campaign’s collection and it’s users collections, and
db.uostyle_users.find.limit(1) to see a user.
Reporting the most engaged users
A modern web templating enthusiast, surely
Let’s get this data off the command line and into the browser. Here’s a minimal HTML report template that cycles through a dictionary of users, rendering basic information about them, a list of stats and a link to their Instagram profile. It’s written for Jinja2, in the handlebars syntax that all the cool templating libraries use these days. There’s some filters in there to sort and capitalise the user data.
All we need now is a script to collect the data from Mongo, feed it to the template and render it – enter
produce_top_fans_report.py. This does just that, and utf-8 encodes the result so it will print to the terminal without errors, letting you pipe the output to a file. Run it with
python produce_top_fans_report.py >; report.html to create your report. Click through on the image to see one I generated earlier.
Instagram campaign report in glorious HTML
You’ll notice there’s still issues with unicode names, but I didn’t want to get into unicode wrangling with Python. Sorry ⚓leannelimwalker⚓, maybe in the next version.
A quick trick for hosting flat HTML files is to put them in your public Dropbox folder. Visit the web interface, right-click and Dropbox will give you a public URL for the file.
Serving HTML with Dropbox
Reporting the most engaged users
So there you go. A bit more work has given me the foundations for a scalable, multi campaign reporting system. There’s more data in the API to segment and learn from, dates and locations would be good. With a bit more work I could also mine the data for insights like cliques of users and optimal posting times to get comments. It could also be linked to other APIs – I expect most people have the same username on Instagram and Twitter, so you can probably see where my hacker mind is going with that one.
I’d love to hear your thoughts. Please leave a comment, and feel free to use and develop the code.