The cool thing about technical howtos like this is the ability to share with other folks who get just as excited. Looking to save costs on our blog hosting, Amazon’s announcement of a 12 month free usage tier put us in the short list for migration. As Matthias did the bulk of the joyent public cloud & linode migrations, it was high-time I got my hands dirty. Without further ado, here’s how I migrated AWO to our first Amazon EC2 instance. Let’s get our geek on!
Choosing bitnami
My initial plan was to grab some Debian based Amazon Machine Image (AMI) and follow the various “howto config wordpress on lamp” posts out there, but Matthias asked if I’d had a look at bitnami. I’d never heard of it before and, truth be told, it actually sounded like some kind of BitTorrent client to me. Digging in a little deeper, I soon understood what all the hoopla was about and was soon sold on a nicely contained, pre-configured bitnami WordPress AMI. With Ubuntu and RedHat distros to choose from, you can even use an AMI which launches via Elastic Block Storage (EBS) – how’s that for easy?
Just in case you don’t know what EBS is good for – an Instance Store EC2 server doesn’t come with a disk for storing your data. Meaning if you plan on making changes to your AMI at runtime that you’d like to see after a reboot, you need to attach a disk (EBS in Amazon AWS speak).
Setting up a new EC2 instance on AWS
One of the catches for a 12 month, free EC2 instance is a new Amazon account, so be sure to register a new email address or you might get a surprise bill next month! And, yes, you will also need a valid credit card to open your account, because if your instance exceeds the free usage tier limits, you will be charged.
For those of you running multiple WordPress blogs, bitnami even offers preconfigured MultiSite AMIs!
You should definitely spend a few minutes considering the region in which to launch the instance. If you’re using Google Analytics, checkout your “Audience -> Geo -> Location” to see the regions sending your blog the most visits. Our primary traffic comes from the U.S. West Coast, but there are significant visits throughout the Continental U.S. and Europe.
We ultimately went with the mid-point “U.S. East” region.
Once you decide on a region and an instance type, just click on the desired bitnami WordPress AMI. You’ll be automatically forwarded to the AWS Console for launching that instance in the desired region:
The only change you’ll need to make to the default settings, is to add HTTP port 80 to the default security group. The Wizard will even give you the option of locking this down to your current IP address which I certainly recommend for the initial setup & config.
Click “Launch” and you’ll be prompted to create and/or download a public/private keypair in a .pem file (hereinafter referred to this howto as ‘my_aws_ec2.pem’) – this is your SSH keypair to get in the backdoor of your new instance:
After your instance launches, click on it in the AWS Management Console. The description tab below will list a “Public DNS” (hereinafter referred to in this howto as ‘public_dns.amazonaws.com’):
Point your browser to this URL and you’ll see the bitnami Welcome page with a few links including your new WordPress install. Not too shabby – a running blog on a new virtual server all in the space of about 5 minutes!
Last, but not least, make an EBS snapshot of your running instance so you have a known fallback before entering the next phase of the migration. Don’t forget about this capability – at every step where you feel you’ve accomplished an important step, back it up!
Migrating Your WordPress Blog
On your existing blog, dump your database and tarball your WordPress htdocs path.
$ mysqldump -uwordpress -p awo > awo_wp.sql $ tar -cjf awo_htdocs.bz2 /home/awo/public
Now, let’s get those dumps copied over to your new EC2 instance:
$ scp -i my_aws_ec2.pem awo_wp.sql awo_htdocs.bz2 bitnami@public_dns.amazonaws.com:/home/bitnami/originals/
And, don’t forget to copy over any releveant configurations and performance tweaks in your Apache (apache2.conf, httpd.conf) and MySQL (my.cnf, conf.d/*.cnf) setups.
Moving on to the IP address. Goto the Elastic IP’s entry in your AWS console. Allocate New Address and associate it with your newly created (and probably only) EC2 instance. While we’re waiting for the DNS to catch up, edit your local (meaning the computer you’re using to do the migration) /etc/hosts accordingly, overriding your blog domain with this new IP address.
$ sudo vi /etc/hosts ... www.agileweboperations.com
On the new instance do something like the following (remembering your EBS snapshot above ;)),
$ rm -Rf /opt/bitnami/apps/wordpress/htdocs/ $ tar -xjf ~/originals/awo_htdocs.bz2 /opt/bitnami/apps/wordpress $ mysql -uroot -pbitnami bitnami_wordpress < ~/originals/awo.sql
Note the location of the bitnami wordpress, and the MySQL root password and wordpress database schema.
Now, let’s setup a virtual host to point to your new blog:
$ sudo vi /opt/bitnami/apache2/conf/httpd.conf
...
# Virtual hosts
Include conf/extra/httpd-vhosts.conf
...
$ sudo vi /opt/bitnami/apache2/conf/extra/httpd-vhosts.conf
...
DocumentRoot "/opt/bitnami/apps/wordpress/htdocs"
ServerName agileweboperations.com
ServerAlias www.agileweboperations.com
ErrorLog "logs/awo-error_log"
CustomLog "logs/awo-access_log" common
...
Any special apache configuration (KeepAliveTimeout, ServerTokens, etc.) can be edited @ /opt/bitnami/apache2/conf/extra/httpd-default.conf. Be sure to include these edited files in the base apache conf:
$ sudo vi /opt/bitnami/apache2/conf/httpd.conf ... # Server-pool management (MPM specific) Include conf/extra/httpd-mpm.conf ... # Virtual hosts Include conf/extra/httpd-vhosts.conf ... # Various default settings Include conf/extra/httpd-default.conf
Finally, remember your canonical redirects! No duplicate content here:
$ vi /opt/bitnami/apps/wordpress/htdocs/.htaccess
RewriteEngine On
RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www.agileweboperations.com
RewriteRule (.*) https://www.agileweboperations.com/$1 [R=301,L]
Restarting processes with bitnami is accomplished using the ctlscript.sh. To restart Apache, just run sudo /opt/bitnami/ctlscript.sh restart apache.
MySQL my.cnf may be edited @ /opt/bitnami/mysql/my.cnf and the restart is accomplished in much the same way : sudo /opt/bitnami/ctlscript.sh restart mysql.
You can also use ctlscript.sh to check out your apache & mysql status:
$ sudo /opt/bitnami/ctlscript.sh status apache already running mysql already running
Gotchas & Caveats
Installing ec2-api-tools is not possible on a t1.micro Ubuntu instance. However, if you do need AWS CLI tools, checkout … well, there are AWS CLI tools which run on python right out of the box!
the “Public DNS” name will dynamically change upon restarting an instance. Don’t go hardcoding this into your /etc/hosts file and bookmarking it for all eternity!
For those spend-thrifty types, just starting an instance immediately costs you 1 hour of computing time, so don’t be surprised to see a bit more usage ($$$) than you’d expect if you do this a few dozen times over a weekend of experimentation.
YMMV, but the bitnami:daemon user:group which owned /opt/bitnami/apps/wordpress/htdocs was not sufficient for performing any “inline” wordpress upgrades/installs – meaning trying to upgrade/install a plugin asked for FTP access information. Performing a sudo chown -Rf daemon:daemon /opt/bitnami/apps/wordpress/htdocs did the trick, however. Anyone more familiar with bitnami, please feel free to step in and clarify.
MySQL died the first night after relaunch. A strange “Killed” showed up in the /opt/bitnami/mysql/data/mysqld.log:
101229 15:41:33 [Note] /opt/bitnami/mysql/bin/mysqld.bin: ready for connections. Version: '5.1.50' socket: '/opt/bitnami/mysql/tmp/mysql.sock' port: 3306 MySQL Community Server (GPL) Killed 101230 00:15:51 mysqld_safe Number of processes running now: 0 101230 00:15:51 mysqld_safe mysqld restarted 101230 0:15:53 [Note] Plugin 'FEDERATED' is disabled. Killed 101230 00:15:54 mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-10-112-91-108.pid ended 101230 07:32:02 mysqld_safe Starting mysqld.bin daemon with databases from /opt/bitnami/mysql/data ...
Turns out, the bitnami image makes use of Linux’s OOM Killer which takes a heavy-handed approach to potential Out of Memory issues before they over swap the server. The problem was I forgot to include the httpd-mpm.conf file I had altered so carefully above in the httpd.conf, so there were a bunch more simultaneous clients than this micro server could manage.
After you get an elastic IP setup, you may have to rerun the /opt/bitnami/scripts/init/updateip file. To manually set the wp_options values to your blog’s domain name:
$ mysql -uroot -p bitnami_wordpress
mysql> update wp_options set option_value = 'https://www.agileweboperations.com' where option_name in ('siteurl', 'home');
Query OK, 2 rows affected (0.00 sec)
Our “free” year worked out great for such little effort (~1 day) and was well worth the savings. Check out where the rubber meets the road – our Pingdom response times before and after the relaunch:
Professional Grade Virtual Computing for “Free”
-
In case you missed my other posts on acquiring, maintaining, backing-up and monitoring your virtual server, here’s the complete list:
- Performance : WordPress Caching For High Performance EC2 Micro Instances
- Backups : Poor Man’s Automated Snapshots for EC2
- Monitoring : Cloudkick Monitoring for EC2
Agile Web Operations 
Great article! Glad to hear that BitNami made your life easier – that’s certainly what we’re going for. Since you’re hosting your blog on Amazon, you may want to check out BitNami Cloud Hosting, which is in (free) beta: http://bitnami.org/bitnami_cloud_hosting
-Erica from BitNami.org
LikeLike
Thanks for taking the time to document the WordPress migration.
Is the instance that you started a micro instance? The last I heard that was the only one that was free for one year. If it is…how do you know that from the name?
LikeLike
The free instance is called a “micro” instance. Thanks, Barry, for pointing out I’d missed this – I’ve updated the post with a more helpful image!
LikeLike
That is so annoying how WordPress resets the option back to the public url. Any idea why? or a way to avoid it? I know how to fix it, but I find I have to restart the instance a lot during development.
LikeLike
Dan, thanks for the great post!
Chris, after you set a static ip you need to delete the /opt/bitnami/updateip file. I found it in the forums. http://bitnami.org/forums/forums/bitnami-cloud/topics/ec2-bitnami-wordpress-url-and-dns
LikeLike
Dan,
One probably “dumb” question…
Does the EC2 AMI “maintains” itself updated in the latest, stable build of WP?
Or do you have to fully account for all maintenance and updates after you’ve created the new instance under your AWS account?
Thanks a lot for sharing!!
Best,
Nicolas E.
LikeLike
Unfortunately, Nicolas, the AMI does not maintain itself. But I did configure the directory permissions in such a way that you can upgrade/maintain WP by clicking the appropriate button in wp-admin. Additionally, you’re responsible for keeping your Ubuntu instance patched via apt-get from the cmdline.
LikeLike
Hi Dan, Thanks for your post. Very informative. I have an additional question. After you have your EC2 instance ready and running, what is the easiest way of keeping the instance backed up in case the instance goes silent?
Thanks, Geo
LikeLike
Arrghh, I’m writing that post right now 😉 … I’ll give you a hint though : ‘ec2addsnap’ from the ec2 api tools. As soon as I get that post published, I’ll add a link here!
LikeLike
I ran into the issue you had where MySql died a couple of days after relaunch. I found the same log entries that you found in the /opt/bitnami/mysql/data/mysqld.log so my issue was also likely due to OOM Killer. I have also not included the httpd-mpm.conf in my httpd.conf. Did you use the httpd-mpm.conf file as-is from the Apache/config/extras directory or did you modify the settings in that file?
LikeLike
definitely modified the httpd-mpm.conf prefork module to :
StartServers 10 MinSpareServers 10 MaxSpareServers 10 MaxClients 20 MaxRequestsPerChild 0But we are also using MaxCDN to serve all our static resources.
LikeLike
How did you calculate these numbers based on a micro instance? Tips for how I would modify this for a small instance?
LikeLike
If you run
topand look for your Apache processes, you get an idea about how much memory each thread consumes. Multiply this times your MaxClients to get an idea of the max memory Apache could consume. Add this value to the max memory of your MySQL db (checkout out http://mysqltuner.pl) for an estimate of the total possible memory used by your LAMP stack.LikeLike
Got a different question, maybe you can help me out.
I’ve got a mysql database set in amazon, and i’m trying to dump one table back to my local machine (it’s a log table and i want to test some log processing applications on my live data)
I’ve looked and looked, but didn’t any way of doing this 😦
Any ideas ?
10x
Shan
LikeLike
Hi shan,
you should use
mysqldumpand scp the dump back to your machine. You can have a look at the Amazon RDS Customer Data Import Guide for MySQL, which explains the procedure if you use an RDS instance. Themysqldumppart is the same, if you have the DB on your own box.Good luck!
LikeLike
Aloha Dan-
I’m thinking of transferring a large website from a mediatemple DV, where I’ve learned way more than I truly wanted to about managing a server, over to a bitnami stack. The site I have is a wordpress based site that really utilizes it as a CMS whereby I have a lot of database querying going on. I’ve had to spend days tuning my MT-DV and still run into server hogs (also have cleaned the code over there to only use wp functions so it limits my bad querying). The site only gets about 1000 uniques a day but still is a database hog. I’m going to be using W3TC with an S3/Cloudfront CDN to offload some of the work, and will most likely be on a small instance (unless you convince me I could get a micro to work :).)
I’ve done a few installs of wordpress using bitnami stacks and plane jane Amazon AMI’s just to get familiar with things. But I’ve run into the same database kill issues you mentioned. I’m not going to claim to be a httpd.conf wizard by any means and was wondering if you could elaborate or point me in a direction as to why you list the values you do in httpd.mpm.conf and or if I run into issues, what method of adjusting I should follow to troubleshoot?
Any help would be great, but this tutorial has already helped tremendously, it was my starter to dive in a while back.
OK, back to hammering my test micro install with your httpd tweaks to see if I can get the database to shutdown.
LikeLike
Hi Erik,
definitely would not recommend a micro instance with a 1000 uniques a day (and a lot of db hits). The values in the httpd.conf are due to memory constraints on the micro instance which only gives you about 700MB. Each of our apache threads (client) uses about 15MB – 15*20 = 300MB. So max clients leaves a “bit” of room for Linux and MySQL (but things do get tight).
Another piece of advice, be careful how you cache your static resources at the CDN. Do it longer than 24 hrs (else, W3TC will recrawl your entire site everyday looking for new resources)!
Good luck (and have fun)!
LikeLike
Hey thanks Dan!
I’ve found a bunch of good resources on tuning apache at EC2, still getting the hang of that. Curious as to the base setting of 0 for MaxRequestsPerChild?
Also, i’ve set a couple up without enabling httpd-vhosts.conf and configuring virtual hosts. What’s the advantage of using virtual host?
Aloha,
Erik
LikeLike
MaxRequestsPerChild 0means we don’t care about killing the thread after x requests – allow it serve an infinite number of requests.Virtual hosts are helpful if you’re running more than one website (subdomain/domain name) from the same Apache server.
LikeLike
Awesome Dan, thanks a bunch again.
That was where my mind was headed, just thought I’d bounce it off someone.
If I ever write on my progress I’ll be sure to mention you and this great resource.
Aloha
LikeLike
Hey Dan,
I’ve been going through the various on-line tutorials on this for a couple weeks and can’t get it to work. I keep having issues with the re-directs. When I change my A Name to point to the Elastic IP, it works fine. Then when I go into httpd.conf to change docroot and directory to to point from bitnami/apache2/htdocs to bitnami/apps/wordpress/htdocs I lose the ability to access my wp-admin. Also, all the links on the default WP blog are pointing to the amazonurl/wordpress instead of myurl.
I tried using your virtual host method instead of httpd.conf and had the same issue.
Any ideas?
Thanks
LikeLike
Eric you can follow the instructions here: http://codex.wordpress.org/Changing_The_Site_URL
LikeLike
Hi Dan – MySQL crashes continuously (in less than a couple of hours). This is a test site (a clone of a production site – with no users except me). It’s WP + Buddypress + about 20 plugins. The DB is less than 2MB.
I did follow your instructions, I modified the httpd-mpm.conf and included
Include conf/extra/httpd-mpm.conf
My MySQL log shows:
111210 09:43:37 mysqld_safe Number of processes running now: 0
111210 09:43:37 mysqld_safe mysqld restarted
111210 9:43:37 [Note] Plugin ‘FEDERATED’ is disabled.
111210 9:43:37 InnoDB: The InnoDB memory heap is disabled
111210 9:43:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
111210 9:43:37 InnoDB: Compressed tables use zlib 1.2.3
111210 9:43:37 InnoDB: Using Linux native AIO
111210 9:43:37 InnoDB: Initializing buffer pool, size = 128.0M
InnoDB: mmap(137363456 bytes) failed; errno 12
111210 9:43:37 InnoDB: Completed initialization of buffer pool
111210 9:43:37 InnoDB: Fatal error: cannot allocate memory for the buffer pool
111210 9:43:37 [ERROR] Plugin ‘InnoDB’ init function returned error.
111210 9:43:37 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.
111210 9:43:37 [ERROR] Unknown/unsupported storage engine: InnoDB
111210 9:43:37 [ERROR] Aborting
111210 9:43:37 [Note] /opt/bitnami/mysql/bin/mysqld.bin: Shutdown complete
111210 09:43:37 mysqld_safe mysqld from pid file /opt/bitnami/mysql/data/ip-10-48-201-206.pid ended
Thanks!
LikeLike
First idea is that since this is a clone of your production site running on a test server, you might not have the same available memory on this machine. Double check the my.cnf file and how much innodb memory it’s trying to allocate – it’s probably way more than the available RAM.
LikeLike
Thanks Dan – there is no reference to innodb in my.conf. Do you know of a tutorial that shows how to add innodb to my.conf.
There are a bunch of example my.conf files on Bitnami (/opt/bitnami/mysql/support-files). These include a reference to innodb, but I have no clue how to add them to the original my.conf . Also, isn’t it strange that there is no innodb setting on my.conf?
The cloned site in question is running on micro ec2. The corresponding site runs on a shared hosting account, together with 4 other WP sites. I thought that a micro ec2 would have much more memory and run faster. And, it actually runs much faster – that is, till MySQL crashes.
Thanks for your help!
LikeLike
Dan – do you have a sample my.cnf file for an ec2 micro instance or can you recommend a configuration?
Thanks!
LikeLike
This is so great. Thanks so much!
LikeLike
This is the best tutorial I’ve found on using the Bitnami WordPress stack on EC2. Thanks a million.
One question: Have you actually tried running more than one site on a micro instance, either with separate vhost entries, or with a WordPress Multi-Site install? (I’d like to transfer a few low-traffic sites from a shared hosting account when my current subscription period runs out.)
LikeLike
Hi Van,
We’ve not tried it, but I don’t see any issues using Multisite.
Good luck!
LikeLike
Even though this article is about five years old now, I was able to follow most of it to get my site moved over. Thanks!
I do have one question however, when I ran the “sudo chown -Rf daemon:daemon /opt/bitnami/apps/wordpress/htdocs” command for inline updates, I can no longer browse the htdocs folder via SFTP. I’m still learning Linux, do you have any suggestions on how to fix this?
LikeLike
Great to hear that the article is still helpful.
The command changes the ownership of the htdocs folder to the ‘daemon’ user.
Running
ls -lon the ‘htdocs’ directory should show you who can access the directory. You need to make sure that your SFTP user has appropriate rights.Hope this helps. Good luck!
LikeLike
Don’t forget that you also have other options like Cloudways and SiteGround if you want to host WordPress on AWS. I have used SiteGround, Cloudways (http://www.cloudways.com/blog/host-wordpress-on-aws-cloud/ ) and Bitnami. Cloudways was the one I found really easy and straightforward. I particularly loved the UI of their console.
LikeLike