Friday, April 29, 2016

Heroku and Wordpress: Afterthoughts

The last two postings were about my successful and happy migration of my personal Wordpress site from my home PC to Heroku. It went beautifully, but a little testing and a few days revealed some improvements. One could call these Pitfalls and Gotchas about Heroku, lessons for my next app.



Database Usage / Wordpress Revisions


My MySQL dump file was 2.3 MB, fitting quite well under the 5 MB mark. Wrong. After being loaded, it bloated to about 4.1 MB and I got a warning email from Heroku that I was nearing my capacity.

First off, thank you Heroku and ClearDB. Nice job on the notification before this became a problem.

Second, what's using my disk space? I did some queries (thank you, Stackoverflow) and found that wp_posts had 450 rows... for my 35 pages. Ah yes: Page Revisions. Wordpress keeps copies of every time I've clicked Update, so there are hundreds of extra pages in there.

I trawled the net a little bit for two pieces of advice that set this right:
  • Disabling revisions (or limiting them to 1 or 2 revisions, if you prefer), with a wp-config.php modification. Easy.
  • A plugin to clean out old revisions. I then uninstalled it since disabling revisions means there won't be more to clean up.
Now my table was down to 150 posts, which is right for 30ish pages and 120ish images. My dump file was under half the previous size.

To finish the job, I had to take a dump of the table and then reload it, so as to drop and recreate the table. (databases do that, keep the space for the future; it's normally a good idea, but not here and today) And now I'm down to 1.6 MB, one third of my free tier.

Sending Email


Heroku doesn't have a sendmail setup, not even sure they have a SMTP service. So Wordpress's email capabilities were not working.

Your basic options here are:
  • Sign up for SendGrid, since Heroku has an addon for that. They have a free tier of 12,000 messages per month, which is really more than my blog could ever generate.
  • Supply your own SMTP server.

I went for the latter since I use GMail and can definitely navigate the steps of a Project, the API keys, and the GMail SMTP plugin for Wordpress. The plugin's own instructions were somewhat outdated, but that wasn't a problem. And voila, Wordpress sending email.

For your own programming, you would want to use PHPMailer or some such so you can specify your SMTP settings. A lot of things presume that there's a local sendmail or local SMTP, but not here.



So yeah, that was it. Not too awful in the "gotchas" department. Not at all! I am suitably impressed with Heroku at virtually every level: price, performance, ease of deployment once you wrap your head around it, and notifications.


Wednesday, April 27, 2016

Heroku and Wordpress: The Migration Process

My post day-before-yesterday begun the story of my migration from a home-hosted website to Heroku. Here's part two: The Migration Process

Step: Move my Media Library content into S3


First off, the uploaded photos should not be under version control; so they shouldn't be in the repository and in the ephemeral filesystem.

This was a tedious process of a few hours since I had about 130 images.
  • I set up the S3 plugin I mentioned above, made a few tests and confirmed that it's sweet.
  • I SFTP'd into my Wordpress site's wp-content/uploads folder and grabbed everything.
  • Then deleted everything from my Media Library,
  • Then uploaded it all again and watched it load into S3 and leave my uploads folder empty.
  • Then went through every posting and replaced all of the images, which of course were now broken. Annoying, but fortunately I only had 35 pages with images and did it in about 2 hours.

Step: Init repo


I'm a fan of Gitlab. They offer unlimited private repositories for free, which is really excellent. I created the repository, then followed their simple instructions to load my Wordpress files into the repo and basically turn my site into a clone.

I also created a .gitignore file with these two entries for some folders I'll be creating in a little bit. The private is where I'll do some other work I don't want in version control, e.g. working files and database dumps that I want to keep close. The vendor would be generated by composer later on, trust me.
/private/
/vendor/

Step: Add Heroku as a Secondary Master


The trick in allowing git push to push to deploy to Heroku, is that you tell your repo clone to use your Heroku as a secondary master.

heroku git:remote -a your-server-name

As of now, when pushing you will need to distinguish between git push origin master and git push heroku master. One pushes into your git repository (Gitlab, Github) and the other would redeploy to Heroku.

Step: Wordpress Updates


I then noticed that some updates were available for some plugins and for Wordpress itself. So I ran those, and after each one noted that git status reported exactly what I would expect from each upgrade. So three commits later I had Wordpress and plugins all updated, with commit notes for each update. (I could have done this before the repo init, but why not do it under version control?)

Step: Add a Procfile and composer.json file


For Heroku compatibility, it's advisable to add a Procfile and a composer.json and composer.lock file, to indicate to heroku what PHP version to prefer, that you prefer Apache over Nginx, etc. You will want these in version control.

Procfile
web: vendor/bin/heroku-php-apache2

composer.json
{
  "require" : {
    "php": "^5.6.0"
  },
  "require-dev": {
    "heroku/heroku-buildpack-php": "*"
  }
}


composer.lock is generated from the composer.json with a command:
composer update --ignore-platform-reqs

Step: Push to Heroku


All set? Then here goes:
git push heroku master
And I visit my website. And it's a Wordpress error that it can't make the database connection. That's to be expected: my wp-config.php has the old credentials and I still need to upload the database content. But that's definitely Wordpress making the error, so a fine start.

Step: Database Configuration


First step was to scrub my database credentials from the wp-config.php file. Yeah, dummy move to forget to do that, but the credentials are wrong for Heroku so are useless, and my local MySQL is going away in an hour anyway. But yes... don't do what I did. ;)

When I scrubbed the database credentials, I replaced them with $_ENV variables like this:
define('DB_NAME', $_ENV['DATABASE_BASE']);
define('DB_USER', $_ENV['DATABASE_USER']);
define('DB_PASSWORD', $_ENV['DATABASE_PASS']);
define('DB_HOST', $_ENV['DATABASE_HOST']);
Add, commit, push. Site is still dead but it's ready for this next trick.


Run heroku config and it dumps the app's configuration variables back to me. I tease apart the database URL string, into my set of 4 environment variables for the 4 $_ENV items above.
heroku config:set DATABASE_BASE='heroku_XXX' DATABASE_USER='XXX' DATABASE_PASS='XXX' DATABASE_HOST='XXX'
My app/dyno reboots and... the site's up!

Step: Database Data


A simple mysqldump was all it took to take a backup of my database, then loading it was one more command.

mysqldump -h localhost -u olduser -p olddbname > private/dbdump.sql
mysql -h herokuhost -u herokuuser -p herokudbname < private/dbdump.sql

It doesn't get as lot easier than this.

And now the site is up! My data, on a new database on a new Heroku server. Shiny.


Step: Custom Domain


My site is www.fightingfantasyfan.info and not fightingfantasyfan.herokuapp.com Heroku calls this a custom domain. There are two steps in setting up the Heroku app to work properly with a custom domain.

  • I hit up Heroku's dashboard and the Settings for my site, and added two domains for it: www.fightingfantasyfan.info and fightingfantasyfan.info This allows it to respond to these alternate hostnames, should they point to this app.
  • I went to my domain registrar's domain control panel and set up a CNAME record, so that www.fightingfantasyfan.info is equivalent to fightingfantasyfan.herokuapp.com This took a little bit to propagate, but was done about the time I finished my cup of coffee.


And that really was it. Well, almost. More on this tomorrow.


Monday, April 25, 2016

Heroku and Wordpress

This is a bit off-topic for The Map Guy since it has nothing to do with maps, directly. But it's interesting and it's about the cloud, and about software deployment. It's about... Hosting a low-volume personal Wordpress website on Heroku for free.

Introduction


I have a couple of personal websites for hobbies. I'm almost ashamed to admit that I'm using Wordpress for them, being the hardcore, bad-ass hacker that I am... but in the evenings and with nothing at stake, the convenience of Wordpress works for me.

I had been hosting my site on a Raspberry Pi at home, but the reliability just wasn't there. The Pi would crash from time to time and require rebooting. After the second time it did this while I was away on a trip, I decided it was time for a change. Even better, this is an opportunity to say Hello To The Cloud with a real-world application with no stakes involved.

Enter Heroku, my hero.

Heroku is a service that spins up virtual machines (they call them "apps"), using as their filesystem your git repository. The basic idea is this:
  • You have a git repository which contains your website: HTML, PHP, Python, Ruby, etc.
  • You create an "app" via their dashboard, also called a "dyno", and it has an URL. You then attach "Add-Ons" such as a MySQL database, a PostgreSQL database, SMTP via SendGrid, and other such services that you'll need.
  • Using Heroku's command-line "toolbelt" you add to your clone of the repository, the fact that this clone is "connected" to that app/VPS. So your clone has two masters: git push origin master for saving your code to version control as usual, and git push heroku master to deploy it to the site.
  • When you push to heroku master, Heroku resets your VPS, loading up its filesystem from your repository content. Assuming that your code works, the site works too.
  • No permanent storage ("ephemeral filesystem"). When the VPS is load-balanced to another system, or put to rest, or you push again, it's reset from your repo. This is very important if your web app will be making filesystem modifications such as accepting uploads. More on this later.
So the effect is a build-from-components server, enabling the specific services you need.

For small personal sites, free tiers for the dyno and for the add-on services may prove sufficient to host the Wordpress code and the database entirely for free. The limitations of this free tier, as relevant to my personal Wordpress site, are:
  • The dyno (the VPS itself) will go to sleep if there's no activity, and the next incoming activity to wake it up does mean some delay. The dyno must be asleep at least 6 hours per day, so using Uptime Robot to keep it awake is a no-no. The next step up to get rid of the sleeping, is only $10 per month.
  • The ClearDB MySQL database is free only to 5 MB. This may not be enough for a long-running daily blog, but for a few postings per month maybe it's just what you need. The next step up is only $10 per month for 1 GB.

Since Heroku does support PHP, and a mysqldump of my database is only 2 MB in size, this sounds right on target for a Heroku freebie.


Ephemeral Filesystem


Let me reiterate that "permanent storage" comment above. Your Heroku filesystem is a git repository. Your dyno does have the ability to write files on disk,  e.g. an uploads folder, e.g. the Media Library component of Wordpress. But you won't like it. When your dyno resets, the filesystem is reinitialized and your modifications would be lost.

By "reset" is meant a reboot, it being load-balanced onto a new server, it going to sleep and waking back up, your next git push, or using heroku config:set to change environment variables. Bye bye, uploaded files.

The up side of this is that a hack of your website, assuming it didn't damage the database, can be solved by rebooting. The down side is that you need to find someplace else for long-term storage of any web-supplied files not in your repository (or else make a practice of manually downloading the files and adding to the repo? sounds awful)

In the case of Wordpress the easiest solution I found is the WP Offload S3 Lite plugin. When you upload to the Media Library, it goes to Amazon S3 instead, and media URLs are rewritten to point to their S3 version. Between Amazon's generous 5 GB free tier for a year and the real price being a paltry 3 pence per gigabyte per month, even an image-heavy website can get by on pocket change per month.

If you are writing your own ware, you'd want to code for your cloud storage of choice such as Amazon S3, Google Drive API, Dropbox API, etc. where you supply a file and get back a URL. I imagine you'd need to generate API keys, program the OAuth-style exchange of them, handle errors etc. and that sounds awful. For my own case here, though, the folks at Delicious Brains had generously done that heavy lifting for me.


Happy Ending


Spoiler time: My applications are all running on Heroku and the performance is quite acceptable.  And I learned a lot in the process. The rest of this story is how I got to this happy place.

So, now that I've covered the basics, tomorrow's story will be about the migration process!