Mar 112012
 

For my own record and maybe for your convenience I wrote down the steps to install a Debian/Ubuntu VPS as web server. This setup is optimized for low memory usage (128-512 MB). This means nginx instead of Apache, no DNS server and no e-mail server (only outgoing mail) and some MySQL, PHP and nginx tuning.

This setup guide has been tested for a VPS on OpenVZ, Xen and KVM and for Debian 6.0 (Squeeze), Debian 7 (Wheezy), Ubuntu 10.04 (Lucid Lynx) and 11.10 (Oneiric Ocelot). My favorite combination so far is OpenVZ and Debian 7.0 with the Dotdeb repository (fast virtualization, stable Linux and the latest server software).

This setup is handling > 60,000 page views per day (> 100,000 hits) for a dozen of sites on a dual core VPS (2 × 2.4 Ghz) with 512 MB memory with ease (little CPU usage, load average 0.1-0.2, and almost no swapping).

I will update this guide each time I learn something new. Currently I am using a larger VPS (2GB memory) and a more recent OS (Debian Wheezy), but this guide is still valid and very useful to me each time a setup a new VPS.

I use Hurricane Electric Free DNS Management, because I like the fast web interface, the possibility to set the TTL and because it is free (up to 50 domains), but be aware wildcard domains are not allowed (anymore). List of free DNS providers.

has been so kind to provide a VPS to test this guide.

Cheap, reliable VPS providers:

ServerMania is my favorite VPS provider, for the simple reason their support is excellent, so good I am even placing a banner by exception.
Server Mania - Hosting Empowered

Fora:

Index

Setup VPS

From the hosting control panel:
  • Configure the host name
  • Configure rDNS and SPF (for reliable e-mail)
    • Check: dig -x <IP>
    • Check: dig TXT <domain>
    • SPF wizard
  • Point a domain name to the VPS
  • Install a recent version of Debian or Ubuntu

Setup security

  • Login to the VPS:
    • ssh root@domain
  • Set new root password:
    • passwd
  • Fix the hostname when needed:
    • hostname <name>
    • nano /etc/hosts
  • apt-get update
  • apt-get upgrade
  • apt-get install nano sudo
  • Sometimes needed:
    • locale-gen en_US en_US.UTF-8
    • dpkg-reconfigure locales
    • dpkg-reconfigure tzdata
  • When IPv6 doesn’t work: nano /etc/gai.conf
precedence ::ffff:0:0/96  100
  • If you want more recent package versions, use Dotdeb with PHP 5.5
    • Don’t change to Dotdeb afterwards, because you will run into dependency problems!
  • mkdir /root/.ssh
  • chmod 700 /root/.ssh
  • nano /root/.ssh/authorized_keys
    • paste key from local computer
    • cat ~/.ssh/id_dsa.pub
  • ssh-keygen -t dsa
  • nano /etc/ssh/sshd_config
Port 22022
ListenAddress 0.0.0.0
PasswordAuthentication no
ClientAliveInterval 120
ClientAliveCountMax 600
#Subsystem sftp /usr/lib/openssh/sftp-server
Subsystem sftp internal-sftp
Match Group users
ChrootDirectory /home
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp
  • service ssh restart
  • chown root:root /home/*
  • apt-get install iptables
  • nano /etc/resolv.conf
nameserver 8.8.8.8
nameserver 8.8.4.4
  • mkdir /etc/fw
  • Download FirewallBuilder
    • use web server template
    • allow https in
    • allow port 22022 in (ssh)
    • allow port 587 out (ssmtp)
    • allow http/https out (for updates)
    • allow ntp out
    • allow ftp out
    • Install
  • nano /etc/rc.local
/etc/fw/firewall.fw
echo 1 >/proc/sys/net/ipv6/conf/eth0/disable_ipv6
sysctl net.ipv6.conf.all.disable_ipv6=1

You may way to use the harding rules from here.

  • nano /etc/default/useradd
SHELL=/bin/false
  • apt-get install fail2ban
  • service fail2ban stop
  • nano /etc/default/fail2ban
FAIL2BAN_OPTS="-x"
  • nano /etc/fail2ban/jail.local
[DEFAULT]
ignoreip = 127.0.0.1/8 <your home/office IP>
action = %(action_mwl)s
destemail = <your e-mail address>

[ssh]
port     = 22022
  • service fail2ban start

Setup time

  • apt-get install ntpdate
  • nano /root/ntpdate.sh
#!/bin/sh
/usr/sbin/ntpdate pool.ntp.org
  • chmod 755 /root/ntpdate.sh
  • crontab -e
0 9 * * * /root/ntpdate.sh >>/root/ntpdate.log 2>&1

Not needed/possible when shared Linux kernel (for example OpenVZ)

Setup servers

Remove pre-installed stuff:
  • apt-get purge sendmail* exim4* apache2* bind9 samba xinetd
Install new stuff:
  • apt-get install nginx php5-fpm php5-cli php5-curl php5-gd php5-mcrypt php5-intl php5-mysqlnd mysql-server
Debian:
  • Use Dotdeb repository with PHP 5.5 (see before)

Setup MySQL

  • nano /etc/mysql/my.cnf
[mysqld]
skip-innodb
#skip-external-locking
skip-networking
query_cache_size = 64M
query_cache_type = 1
key_buffer = 64M
table_cache = 1024
  • Debian:
default-storage-engine=MyISAM
  • service mysql restart
  • Debian 7: nano /etc/mysql/conf.d/extra.cnf
[mysqld]
skip-networking
query_cache_size = 128M
query_cache_limit = 4M
query_cache_type = 1
key_buffer_size = 64M
table_open_cache = 1024

innodb = OFF
skip-innodb
default-storage-engine = myisam
default-tmp-storage-engine = myisam
  • Piwik: nano config/config.ini.php
[database]
adapter = PDO_MYSQL
type = MyIsam

Note that some software doesn’t work correctly without InnoDB or MyIsam.

Setup PHP

  • nano /etc/php5/fpm/pool.d/www.conf
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
pm = dynamic
pm.max_children = 7
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.status_path = /fpm_status
pm.max_requests = 500
request_terminate_timeout = 60s
  • nano /etc/php5/fpm/php.ini
cgi.fix_pathinfo = 0;
memory_limit = 256M;
user_ini.filename = ".user.ini"
upload_max_filesize = 4M
open_basedir = /home:/tmp
allow_url_fopen = Off
mail.add_x_header = Off
max_execution_time = 60

[PATH=/path/to/folder]
open_basedir = ...
  • Debian:
date.timezone = "Europe/Amsterdam"
  • nano /etc/php5/fpm/conf.d/05-opcache.ini
zend_extension=opcache.so
opcache.memory_consumption=512
opcache.max_accelerated_files=50000
  • Get maximum size: sysctl kernel.shmmax
  • nano /etc/sysctl.conf
kernel.shmmax=134217728
  • service php5-fpm restart

Setup nginx

  • nano /etc/nginx/nginx.conf
worker_processes 2;
server_tokens off;
client_max_body_size 4M;

The number of worker processes should reflect the number of CPUs.

  • Enabled GZIP:
#gzip on;
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 1;
gzip_min_length 1100;
gzip_proxied any;
gzip_types text/plain text/css application/json application/x-javascript text/x$
gzip_buffers 16 8k;
gzip_disable "MSIE [1-6].(?!.*SV1)";

(you might want to use extra config files in /etc/nginx/conf.d)

  • service nginx restart

If you want to migrate from one server to another, you can do this:

server {
        listen 80;
        location / {
                proxy_pass http://a.b.c.d;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $http_host;
                proxy_redirect off;
        }
}

server {
        listen 443 ssl spdy;
        ssl_certificate /etc/ssl/private/xxx.crt;
        ssl_certificate_key /etc/ssl/private/xxx.key;
        location / {
                proxy_pass https://a.b.c.d;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $http_host;
                proxy_redirect off;
        }
}

Setup WordPress

  • Segregate users:
    • useradd <username>
    • passwd <username>
    • usermod -G users <username>
  • File permissions
find /home -type d -name 'wp-content' -exec mkdir -p {}/../assets \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/gallery \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/uploads \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/upgrade \;

chown root:root /home
chown -R <username>:www-data <username>

find /home -type d -exec chmod 2750 {} \;
chmod 755 /home
find /home -type d -name '.ssh' -exec chmod -R 2700 {} \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/gallery \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/uploads \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/upgrade \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/upgrade \;
find /home/ -type f -exec chmod 0640 {} \;

Setup Piwik

9 * * * * su www-data -c "/usr/bin/php5 .../misc/cron/archive.php --url=http://example.org/"
>>.../piwik-archive.log 2>&1

Setup E-Mail

Root=user@example.org
MailHub=smtp.gmail.com:587
RewriteDomain=example.org
Hostname=example.org
FromLineOverride=YES
UseTLS=YES
UseSTARTTLS=YES
AuthUser=user@example.org
AuthPass=...
AuthMethod=LOGIN
  • nano /etc/ssmtp/revaliases
root:user@example.org:smtp.gmail.com:587
www-data:user@example.org:smtp.gmail.com:587

Note that sending an e-mail is not retried with this method.

An alternative is to use nullmailer, which has an outgoing message queue at the expense of using a little more memory.

  • apt-get install nullmailer
  • nano /etc/nullmailer/remotes
smtp.gmail.com smtp --port=587 --starttls --user=you@gmail.com --pass=secret
smtp.mandrillapp.com smtp --port=587 --starttls --user=you@example.org --pass=API-token

Nullmailer supports secure SMTP since version 1.10, so you might need to check the version number.

  • nano /etc/nullmailer/pausetime
900
  • nano /etc/nullmailer/adminaddr
you@gmail.com

For Mandrill or similar:

  • nano /etc/nullmailer/defaultdomain
example.org
  • nano /etc/mailname
example.org
  • service nullmailer restart

For Google mail send limits, see here.

  • nano /etc/Muttrc
unset record

Don’t forget to setup rDNS and SPF. It is wise to regularly use an Email blacklist check.

Possible alternative: Msmtp (not tested)

Setup FTP

  • apt-get install proftpd
    • standalone
  • nano /etc/proftpd/proftpd.conf
DefaultAddress 127.0.0.1
SocketBindTight on
RequireValidShell off
UseReverseDNS off
IdentLookups off
UseIPv6 off
  • service proftpd restart

(needed for WordPress updates)

Backup

  • Backup databases:
fn="/tmp/backup_mysql.sql.gz"
mysqldump -u... -p... --all-databases | gzip -9 >$fn
scp -q -P 22022 $fn user@domain:/path/to/folder/
  • Restore databases:
gunzip <test.gz >/tmp/dump.sql
mysql -u... -p... </tmp/dump.sql
mysqladmin -u... -p... flush-privileges
  • You might need to restore the debian-sys-maint password:
    • nano /etc/mysql/debian.cnf
  • Backup files with rsync (apt-get install rsync)
rsync -avz -e 'ssh -p 22022' /etc/ user@domain:/path/to/folder/
rsync -avz -e 'ssh -p 22022' /var/lib/ user@domain:/path/to/folder/
rsync -avz -e 'ssh -p 22022' --exclude some/folder /home/ user@domain:/path/to/folder/
  • crontab -e
0 1 * * * /root/backup.sh >>/root/backup_`date +\%F`.log 2>&1
duplicity --exclude-filelist=duplicity.exclude --full-if-older-than 1W --allow-source-mismatch / rsync://user@host:22022//path/to/backup/folder
duplicity remove-older-than 1M --force rsync://user@host:22022//path/to/backup/folder

Some typical excludes for duplicity:

- /dev
- /proc
- /sys
- /tmp
- /run
- /mnt
- /media
- /var/log
- /lost+found
- /var/lib/mysql
- /var/lib/mongodb

You might need a backport of duplicity on Debian.

(duplicity tutorial, Full system backup)

Monitoring

  • apt-get install munin munin-node sysstat libwww-perl libipc-sharelite-perl libcache-cache-perl
  • perl -MCPAN -eshell
  • install IPC::ShareLite
  • nano /etc/munin/munin-node.conf
host 127.0.0.1
  • nano /etc/munin/plugin-conf.d/munin-node
[nginx_*]
env.url http://localhost/nginx_status

[phpfpm_*]
env.url http://localhost/fpm_status
env.phpbin php-fpm
  • ln -s /usr/share/munin/plugins/nginx_* /etc/munin/plugins
  • ln -s /usr/share/munin/plugins/mysql_* /etc/munin/plugins
  • rm /etc/munin/plugins/mysql_innodb
  • rm /etc/munin/plugins/iostats_ios
  • service munin-node restart
  • munin-node-configure –suggest
  • PHP5-FPM
  • Other plugins

Tuning

  • nano /etc/sysctl.conf
vm.swappiness=10

(not possible on OpenVZ)

Tools

Be sure to lock these down behind a firewall / password.

Please let me know if you have any remarks or suggestions.