Sep 162012
 

Einde van de week gaan we drie weken het westen van Turkije verkennen. Als het goed is, is het nog prima weer in dit deel van de wereld. We vliegen naar de vijfduizend jaar oude, tweede grootste stad van Turkije, Izmir (ca. 2,5 miljoen inwoners). Waar we slapen en heen gaan, zien we wel, maar een bezoek aan Ephesus ligt voor de hand.

Amsterdam Izmir 21/9 14:05-18:35 PC298
Izmir Amsterdam 12/10 10:30-13:15 PC297

Pegasus Airlines

Jul 062012
 

Sometime ago I ported iodine to Android. It works, but it is not very stable and it is difficult to monitor if iodine is still running. MagicTunnel made it somewhat easier to manage, but didn’t solve the stability issue.

Since I found this unsatisfactory I developed my own DNS tunnel. The client is written in pure Java and the server is written in node.js. The client doesn’t even require root. My neighbour published the client application on Google Play and will take care of support and maintenance with me as second line. The source code is available on GitHub.

I am quite proud of this rather complex application!

DNS traffic is done through port 53 and iodine is element 53 of the periodic table.

Mar 112012
 

For my own record and maybe for your convenience I wrote down the steps to install a Debian/Ubuntu VPS as web server. This setup is optimized for low memory usage (128-512 MB). This means nginx instead of Apache, no DNS server and no e-mail server (only outgoing mail) and some MySQL, PHP and nginx tuning.

This setup guide has been tested for a VPS on OpenVZ, Xen and KVM and for Debian 6.0 (Squeeze), Debian 7 (Wheezy), Ubuntu 10.04 (Lucid Lynx) and 11.10 (Oneiric Ocelot). My favorite combination so far is OpenVZ and Debian 7.0 with the Dotdeb repository (fast virtualization, stable Linux and the latest server software).

This setup is handling > 60,000 page views per day (> 100,000 hits) for a dozen of sites on a dual core VPS (2 × 2.4 Ghz) with 512 MB memory with ease (little CPU usage, load average 0.1-0.2, and almost no swapping).

I will update this guide each time I learn something new. Currently I am using a larger VPS (2GB memory) and a more recent OS (Debian Wheezy), but this guide is still valid and very useful to me each time a setup a new VPS.

I use Hurricane Electric Free DNS Management, because I like the fast web interface, the possibility to set the TTL and because it is free (up to 50 domains), but be aware wildcard domains are not allowed (anymore). List of free DNS providers.

has been so kind to provide a VPS to test this guide.

Cheap, reliable VPS providers:

Fora:

Index

Setup VPS

From the hosting control panel:
  • Configure the host name
  • Configure rDNS and SPF (for reliable e-mail)
    • Check: dig -x <IP>
    • Check: dig TXT <domain>
    • SPF wizard
  • Point a domain name to the VPS
  • Install a recent version of Debian or Ubuntu

Setup security

  • Login to the VPS:
    • ssh root@domain
  • Set new root password:
    • passwd
  • Fix the hostname when needed:
    • hostname <name>
    • nano /etc/hosts
  • apt-get update
  • apt-get upgrade
  • apt-get install nano sudo
  • Sometimes needed:
    • locale-gen en_US en_US.UTF-8
    • dpkg-reconfigure locales
    • dpkg-reconfigure tzdata
  • When IPv6 doesn’t work: nano /etc/gai.conf
precedence ::ffff:0:0/96  100
  • If you want more recent package versions, use Dotdeb with PHP 5.5
    • Don’t change to Dotdeb afterwards, because you will run into dependency problems!
  • mkdir /root/.ssh
  • chmod 700 /root/.ssh
  • nano /root/.ssh/authorized_keys
    • paste key from local computer
    • cat ~/.ssh/id_dsa.pub
  • ssh-keygen -t dsa
  • nano /etc/ssh/sshd_config
Port 22022
ListenAddress 0.0.0.0
PasswordAuthentication no
ClientAliveInterval 120
ClientAliveCountMax 600
#Subsystem sftp /usr/lib/openssh/sftp-server
Subsystem sftp internal-sftp
Match Group users
ChrootDirectory /home
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp
  • service ssh restart
  • chown root:root /home/*
  • apt-get install iptables
  • nano /etc/resolv.conf
nameserver 8.8.8.8
nameserver 8.8.4.4
  • mkdir /etc/fw
  • Download FirewallBuilder
    • use web server template
    • allow https in
    • allow port 22022 in (ssh)
    • allow port 587 out (ssmtp)
    • allow http/https out (for updates)
    • allow ntp out
    • allow ftp out
    • Install
  • nano /etc/rc.local
/etc/fw/firewall.fw
echo 1 >/proc/sys/net/ipv6/conf/eth0/disable_ipv6
sysctl net.ipv6.conf.all.disable_ipv6=1

You may way to use the harding rules from here.

  • nano /etc/default/useradd
SHELL=/bin/false
  • apt-get install fail2ban
  • service fail2ban stop
  • nano /etc/default/fail2ban
FAIL2BAN_OPTS="-x"
  • nano /etc/fail2ban/jail.local
[DEFAULT]
ignoreip = 127.0.0.1/8 <your home/office IP>
action = %(action_mwl)s
destemail = <your e-mail address>

[ssh]
port     = 22022
  • service fail2ban start

Setup time

  • apt-get install ntpdate
  • nano /root/ntpdate.sh
#!/bin/sh
/usr/sbin/ntpdate pool.ntp.org
  • chmod 755 /root/ntpdate.sh
  • crontab -e
0 9 * * * /root/ntpdate.sh >>/root/ntpdate.log 2>&1

Not needed/possible when shared Linux kernel (for example OpenVZ)

Setup servers

Remove pre-installed stuff:
  • apt-get purge sendmail* exim4* apache2* bind9 samba xinetd
Install new stuff:
  • apt-get install nginx php5-fpm php5-cli php5-curl php5-gd php5-mcrypt php5-intl php5-mysqlnd mysql-server
Debian:
  • Use Dotdeb repository with PHP 5.5 (see before)

Setup MySQL

  • nano /etc/mysql/my.cnf
[mysqld]
skip-innodb
#skip-external-locking
skip-networking
query_cache_size = 64M
query_cache_type = 1
key_buffer = 64M
table_cache = 1024
  • Debian:
default-storage-engine=MyISAM
  • service mysql restart
  • Debian 7: nano /etc/mysql/conf.d/extra.cnf
[mysqld]
skip-networking
query_cache_size = 128M
key_buffer_size = 64M
table_open_cache = 1024

innodb = OFF
skip-innodb
default-storage-engine = myisam
default-tmp-storage-engine = myisam
  • Piwik: nano config/config.ini.php
[database]
adapter = PDO_MYSQL
type = MyIsam

Note that some software doesn’t work correctly without InnoDB or MyIsam.

Setup PHP

  • nano /etc/php5/fpm/pool.d/www.conf
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
pm = dynamic
pm.max_children = 7
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.status_path = /fpm_status
pm.max_requests = 500
request_terminate_timeout = 60s
  • nano /etc/php5/fpm/php.ini
cgi.fix_pathinfo = 0;
memory_limit = 256M;
user_ini.filename = ".user.ini"
upload_max_filesize = 4M
open_basedir = /home:/tmp
allow_url_fopen = Off
mail.add_x_header = Off
max_execution_time = 60

[PATH=/path/to/folder]
open_basedir = ...
  • Debian:
date.timezone = "Europe/Amsterdam"
  • nano /etc/php5/fpm/conf.d/05-opcache.ini
zend_extension=opcache.so
opcache.memory_consumption=512
opcache.max_accelerated_files=50000
  • Get maximum size: sysctl kernel.shmmax
  • nano /etc/sysctl.conf
kernel.shmmax=134217728
  • service php5-fpm restart

Setup nginx

  • nano /etc/nginx/nginx.conf
worker_processes 2;
server_tokens off;
client_max_body_size 4M;

The number of worker processes should reflect the number of CPUs.

  • Enabled GZIP:
#gzip on;
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 1;
gzip_min_length 1100;
gzip_proxied any;
gzip_types text/plain text/css application/json application/x-javascript text/x$
gzip_buffers 16 8k;
gzip_disable "MSIE [1-6].(?!.*SV1)";

(you might want to use extra config files in /etc/nginx/conf.d)

  • service nginx restart

If you want to migrate from one server to another, you can do this:

server {
        listen 80;
        location / {
                proxy_pass http://a.b.c.d;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $http_host;
                proxy_redirect off;
        }
}

server {
        listen 443 ssl spdy;
        ssl_certificate /etc/ssl/private/xxx.crt;
        ssl_certificate_key /etc/ssl/private/xxx.key;
        location / {
                proxy_pass https://a.b.c.d;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header Host $http_host;
                proxy_redirect off;
        }
}

Setup WordPress

  • Segregate users:
    • useradd <username>
    • passwd <username>
    • usermod -G users <username>
  • File permissions
find /home -type d -name 'wp-content' -exec mkdir -p {}/../assets \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/gallery \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/uploads \;
find /home -type d -name 'wp-content' -exec mkdir -p {}/upgrade \;

chown root:root /home
chown -R <username>:www-data <username>

find /home -type d -exec chmod 2750 {} \;
chmod 755 /home
find /home -type d -name '.ssh' -exec chmod -R 2700 {} \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/gallery \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/uploads \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/upgrade \;
find /home -type d -name 'wp-content' -exec chmod -R 2770 {}/upgrade \;
find /home/ -type f -exec chmod 0640 {} \;

Setup Piwik

9 * * * * su www-data -c "/usr/bin/php5 .../misc/cron/archive.php --url=http://example.org/"
>>.../piwik-archive.log 2>&1

Setup E-Mail

Root=user@example.org
MailHub=smtp.gmail.com:587
RewriteDomain=example.org
Hostname=example.org
FromLineOverride=YES
UseTLS=YES
UseSTARTTLS=YES
AuthUser=user@example.org
AuthPass=...
AuthMethod=LOGIN
  • nano /etc/ssmtp/revaliases
root:user@example.org:smtp.gmail.com:587
www-data:user@example.org:smtp.gmail.com:587

Note that sending an e-mail is not retried with this method.

An alternative is to use nullmailer, which has an outgoing message queue at the expense of using a little more memory.

  • apt-get install nullmailer
  • nano /etc/nullmailer/remotes
smtp.gmail.com smtp --port=587 --starttls --user=you@gmail.com --pass=secret
smtp.mandrillapp.com smtp --port=587 --starttls --user=you@example.org --pass=API-token

Nullmailer supports secure SMTP since version 1.10, so you might need to check the version number.

  • nano /etc/nullmailer/pausetime
900
  • nano /etc/nullmailer/adminaddr
you@gmail.com

For Mandrill or similar:

  • nano /etc/nullmailer/defaultdomain
example.org
  • nano /etc/mailname
example.org
  • service nullmailer restart

For Google mail send limits, see here.

  • nano /etc/Muttrc
unset record

Don’t forget to setup rDNS and SPF. It is wise to regularly use an Email blacklist check.

Possible alternative: Msmtp (not tested)

Setup FTP

  • apt-get install proftpd
    • standalone
  • nano /etc/proftpd/proftpd.conf
DefaultAddress 127.0.0.1
SocketBindTight on
RequireValidShell off
UseReverseDNS off
IdentLookups off
UseIPv6 off
  • service proftpd restart

(needed for WordPress updates)

Backup

  • Backup databases:
fn="/tmp/backup_mysql.sql.gz"
mysqldump -u... -p... --all-databases | gzip -9 >$fn
scp -q -P 22022 $fn user@domain:/path/to/folder/
  • Restore databases:
gunzip <test.gz >/tmp/dump.sql
mysql -u... -p... </tmp/dump.sql
mysqladmin -u... -p... flush-privileges
  • You might need to restore the debian-sys-maint password:
    • nano /etc/mysql/debian.cnf
  • Backup files with rsync (apt-get install rsync)
rsync -avz -e 'ssh -p 22022' /etc/ user@domain:/path/to/folder/
rsync -avz -e 'ssh -p 22022' /var/lib/ user@domain:/path/to/folder/
rsync -avz -e 'ssh -p 22022' --exclude some/folder /home/ user@domain:/path/to/folder/
  • crontab -e
0 1 * * * /root/backup.sh >>/root/backup_`date +\%F`.log 2>&1
duplicity --exclude-filelist=duplicity.exclude --full-if-older-than 1W --allow-source-mismatch / rsync://user@host:22022//path/to/backup/folder
duplicity remove-older-than 1M --force rsync://user@host:22022//path/to/backup/folder

Some typical excludes for duplicity:

- /dev
- /proc
- /sys
- /tmp
- /run
- /mnt
- /media
- /var/log
- /lost+found
- /var/lib/mysql
- /var/lib/mongodb

You might need a backport of duplicity on Debian.

(duplicity tutorial, Full system backup)

Monitoring

  • apt-get install munin munin-node sysstat libwww-perl libipc-sharelite-perl libcache-cache-perl
  • perl -MCPAN -eshell
  • install IPC::ShareLite
  • nano /etc/munin/munin-node.conf
host 127.0.0.1
  • nano /etc/munin/plugin-conf.d/munin-node
[nginx_*]
env.url http://localhost/nginx_status

[phpfpm_*]
env.url http://localhost/fpm_status
env.phpbin php-fpm
  • ln -s /usr/share/munin/plugins/nginx_* /etc/munin/plugins
  • ln -s /usr/share/munin/plugins/mysql_* /etc/munin/plugins
  • rm /etc/munin/plugins/mysql_innodb
  • rm /etc/munin/plugins/iostats_ios
  • service munin-node restart
  • munin-node-configure –suggest
  • PHP5-FPM
  • Other plugins

Tuning

  • nano /etc/sysctl.conf
vm.swappiness=10

(not possible on OpenVZ)

Tools

Be sure to lock these down behind a firewall / password.

Please let me know if you have any remarks or suggestions.

Sep 032011
 

We gaan terug naar één van de meest magische landen waar we tot nu toe geweest zijn: Peru. We gaan vijf weken het minder bezochte noordelijke deel van het land verkennen. Uitgestrekte, verlaten stranden, de mooiste bergen van de wereld en dichte oerwouden met mysterieuze geluiden.

Ik wil graag terug naar Huaraz, met in de nabijheid Huascarán (6768 meter), en naar TrujilloLa Ciudad de la Eterna Primavera (de stad van de eeuwige lente), met in de omgeving zeer interessante archeologische vindplaatsen, o.a. Chan Chan en las Huacas del Sol y de la Luna (de tempels van de zon en de maan). En als we het lef hebben, gaan we de brujos (tovenaars) en curanderos (genezers) in het afgelegen Huancabamba bezoeken.

Voor de eerste keer slapen we voor vertrek op Schiphol in Yotel. Op deze manier kunnen we wat vroeger vliegen en op tijd in Lima aankomen.

De eerste nacht slapen we in Pirwa Hostel Lima. Voor de veiligheid worden we op de luchthaven opgehaald.

21 sep 07:45 Amsterdam 10:30 Madrid IB3215 Airbus A321
13:10 Madrid 18:05 Lima IB6651 Airbus A340-600
26 okt 11:50 Lima 06:30+1 Madrid IB6658 Airbus A340-200
08:55+1 Madrid 11:25+1 Amsterdam IB3254 Airbus A321

Zo zullen we nog wel niet boarden …

Aug 122011
 

I am using the fast and light bbPress 1.x stand-alone forum software for support on some of my WordPress plugins.

To reduce spam I added the following code to bb-config.php to automatically close old topics:

// Auto close topics older than 10 days
mysql_connect(BBDB_HOST, BBDB_USER, BBDB_PASSWORD);
mysql_select_db(BBDB_NAME);
mysql_query('UPDATE bb_topics SET topic_open = 0 WHERE DATEDIFF(CURDATE(), topic_time) > 10');
mysql_close();
Jul 172011
 

BrowserID logoI welcome any initiative that could free us from logging in with a user name and password. I have created an account on countless websites and I have to remember all those user names and passwords. Except from being inconvenient, this is also not very safe.

The Identity Team of Mozilla Labs has recently launched BrowserID, an open source experiment to login with just two clicks. The idea is that your e-mail addresses represent your identity and that someone vouch for your ownership of it. You can read here how it works and you can try it here.

Because I like the idea and I wanted to support it, I wrote this WordPress plugin to allow logging in with BrowserID to any WordPress powered website (currently 50,899,997).

Install now

I am curious what you think about BrowserID.

SpitsScoren

 Werk  Comments Off
Jun 202011
 

SpitsScoren logoSinds vorige week doe ik mee met SpitsScoren. Simpel gezegd, krijg ik geld voor iedere keer dat ik de spits op de A15 mijd (5 euro voor de ochtendspits en 1,50 euro voor de avondspits). Voor iedere spits moet ik met een speciaal daarvoor bestemde smartphone, een HTC Wildfire S, aangeven of ik de spits ga mijden of niet. Dit kan bijvoorbeeld door thuis te werken of door te carpoolen. Met behulp van kentekencamera’s en de GPS-gegevens van de smartphone wordt gecontroleerd of mijn opgave klopt. Onlangs werd voor de 250.000ste keer de spits gemeden.

SpitsScoren wordt uitgevoerd door De Verkeersonderneming, een samenwerking tussen Stadsregio Rotterdam, Gemeente Rotterdam, Havenbedrijf Rotterdam en Rijkswaterstaat. Het doel is om het spitsverkeer op de A15, de belangrijkste toegangsweg tot de Rotterdamse haven, te verminderen.

Update 6/11: Filedruk op A15 slinkt aanzienlijk

Jun 142011
 

Sometimes guests write something on my weblog. I like guest posts to have a different background color. In the past I realized this with a little bit of PHP code and some CSS styling. To realize this in a nicer way and to make it available to more people I crafted another WordPress plugin for it. With the Author Color plugin each author can simply set the post background and border color using the personal option of his/her profile page.

Install now

Author Color - Profile personal optionsAuhor Color - Example post

Jun 122011
 

BackPackTrack for AndroidWhen I am traveling I like to display the route I have traveled so far for the people at home. To simplify this I wrote an open source Android application to keep track of my route automatically. To save batteries the application turns on the GPS of my Android phone periodically and tries to acquire an accurate location for some time. It is also possible to mark important locations manually (make waypoints), like an attraction or the hotel I am sleeping in. It is also possible to geocode an address (find the latitude/longitude for an address), for the case I forgot to make a waypoint. When I have internet access I can upload the route information to my weblog easily, maybe after I have reverse geocoded a few marked locations (find the address for a latitude/longitude). To make the upload to my weblog possible, I wrote a little WordPress plugin to extend the WordPress XML-RPC protocol. The route information is stored as a standard GPX file attached to a post which is automatically created at the first upload. The excellent XML Google Maps WordPress plugin can be used to display a map based on the GPX file (example).

Install now

QR code BackPackTrack for Android

May 172011
 

Just over three months ago I published the Add Link to Facebook WordPress plugin. I had never expected that the number of downloads would be over 100,000 today! Last week the number of ratings passed 300. The average rating is great (4 stars). The plugin is currently number 26 on the list of the most popular WordPress plugins. So far I have answered more than 1100 questions (!). To reduce the number of questions, I opened a forum, wrote an extensive FAQ and this week I added an User Guide. Since the first release I improved a lot of things and added quite a lot of features, such as showing the names of the people who liked the link, adding a Facebook like and send button and two-way comment integration. It is nice to see that something I wrote for myself is making a lot of other people happy too!

Update 20/6/2011: > 150,000 downloads, > 500 ratings, #19 most popular plugins, #131 top 1000 authors
Update 18/7/2011: > 200,000 downloads, > 650 ratings, #26 most popular plugins, #103 top 1000 authors
Update 24/8/2011: > 250,000 downloads, > 825 ratings, #26 most popular plugins, #86 top 1000 authors
Update 15/2/2012: > 500,000 downloads, > 600 ratings, #12 most popular plugins, #12 highest rated, #53 top 1000 authors
Update 15/12/2012: > 1,000,000 downloads, > 2120 ratings, #48 most popular plugins, #4 highest rated, #38 top 1000 authors