Archive for notes

Running Varnish 2.1.5 on a t1.micro

Now, I am not proclaiming that anyone should use Varnish on a t1.micro for anything more than experimentation. These instances are plagued by high CPU-steal, poor I/O, with little RAM. To get Varnish running without constant child reaping, I needed to tune a few parameters.

Jan 28 03:20:53 ip-10-204-49-11 varnishd[12262]: Child (12263) said Child starts Jan 28 03:20:53 ip-10-204-49-11 varnishd[12262]: Child (12263) said managed to mmap 268435456 bytes of 268435456 Jan 28 03:21:25 ip-10-204-49-11 varnishd[12406]: child (12407) Started Jan 28 03:21:25 ip-10-204-49-11 varnishd[12406]: Child (12407) said Jan 28 03:21:25 ip-10-204-49-11 varnishd[12406]: Child (12407) said Child starts Jan 28 03:21:25 ip-10-204-49-11 varnishd[12406]: Child (12407) said managed to mmap 268435456 bytes of 268435456 Jan 28 03:21:44 ip-10-204-49-11 varnishd[12539]: child (12540) Started Jan 28 03:21:44 ip-10-204-49-11 varnishd[12539]: Child (12540) said Jan 28 03:21:44 ip-10-204-49-11 varnishd[12539]: Child (12540) said Child starts Jan 28 03:21:44 ip-10-204-49-11 varnishd[12539]: Child (12540) said managed to mmap 268435456 bytes of 268435456

To ensure the master varnishd does not reap the child to quickly, try to change the timeout for a child ping response:

-p cli_timeout=30
. Since we’re operating in a virtualized environment with high steal, it’s frequent that the child might not respond in a timely maner.

Do not use the malloc allocator, since there is so little ram, unless you set the varnish cache size to something useless like 10M, it’ll probably trigger the oom-killer. Instead, I recommend using the {file} allocator. I had trouble using the allocator with less than 1G of cache space, which I presume has something to do with the way they are using mmap.

Lastly, make sure you have some swap space allocated (I allocate 4gb). By default, the t1.micro instances do not have any swap space, so your choices are to mount an additional EBS volume (preferred) or initialize a swapfile on the root filesystem. The t1.micro instances do not have any ephemeral disks available.

One-liner to Gzip / Compress Files in Parallel

Using xargs, it’s easy to run essentially any task in parallel:

find /mnt/ -type f -iname ‘all.txt*’ | xargs -n 1 --max-procs=8 gzip

Using MacPorts to Install CPAN Modules Not Included in the Ports System

Super simple instructions to create a Portfile from scratch. In this example, we’ll create one for “WWW::Mechanize::Cached”:

# Create a directory to store your Portfiles mkdir ~/MacPorts/ # Change directory cd ~/MacPorts/ # Get the “cpan2port” script used to create Portfiles curl http://trac.macports.org/export/69899/contrib/cpan2port/cpan2port > cpan2port # Set the executable permissions on the script chmod 755 cpan2port # Create the Portfile for the “WWW::Mechanize::Cached” module sudo ./cpan2port -t WWW::Mechanize::Cached; # Change directory to the new port cd perl/p5-www-mechanize-cached/ # Build the port (and dependencies) sudo port build # Install the port sudo port install

VARNISH String representation of ‘obj.ttl’ not implemented yet – FIX

Frustrated that I couldn’t display the remaining TTL for an object in response headers, I went about doing it the “VCL” way. Adding the following code to “vcl_deliver” will allow you to add an “X-TTL” header.

sub vcl_deliver { C{ double obj_ttl = VRT_r_obj_ttl(sp); if(obj_ttl > 0) { //char *str_obj_ttl = VRT_int_string(sp, VRT_r_obj_ttl(sp)); // Memory leak if we don’t free it using WS Release? char str_obj_ttl[255]; snprintf(str_obj_ttl, sizeof(str_obj_ttl), “%g”, VRT_r_obj_ttl(sp)); // \014 represents the octal representation of the length of the header including ‘:’ VRT_SetHdr(sp, HDR_RESP, “\014X-Cache-TTL:”, str_obj_ttl, vrt_magic_string_end); } }C

}

SVN switch –relocate complains about UUID mismatch?

Recently, I had a repo switched out from under me. The relocation of the repo wasn’t done in the proper way so as to preserve UUIDs (preventing a standard svn switch –relocate) and worse yet I had a bunch of uncommitted code. The solution I arrived at was to simply checkout the latest code, create a patch against HEAD on the new repo, then apply the patch. Shell script below. Since this uses “diff”, it will probably break if you have binary files in your checkout.

 

 

#!/bin/bash -x
# svn-relocate.sh [local copy] [new repo url]
#    Example: svn-relocate.sh /home/eosterman/project http://new-svn.host.com/project
#
dir=$1
repo=$2
mv $dir $dir.bak
svn co $2 $dir
diff –exclude=.svn -ruN $dir $dir.bak > /tmp/svn-relocate.patch
cd $dir && patch -p1 < /tmp/svn-relocate.patch

 

Automatic Canonical Link Tags in Symfony

 

<link rel=”canonical” href=”<?php echo url_for($sf_context->getRouting()->getCurrentInternalUri(true), true); ?>” />

 

More information about the new canonical link tags can be found http://ysearchblog.com/2009/02/12/fighting-duplication-adding-more-arrows-to-your-quiver/

SVN Commit Error: Can’t close activity db

Recently, I moved one of my SVN repositories onto a new server. Everything seemed fine until I tried to commit.

SVN complained:

svn commit . --message ‘….’ svn: Commit failed (details follow): svn: Can’t close activity db: Error string not specified yet

And the server logs show:

[Wed Dec 03 22:53:20 2008] [error] [client 207.7.129.212] Could not create activity /svn/!svn/act/cc8c14b8-904b-4438-872d-e129baae67bb.  [500, #0] [Wed Dec 03 22:53:20 2008] [error] [client 207.7.129.212] could not close dbm files.  [500, #89026] [Wed Dec 03 22:53:20 2008] [error] [client 207.7.129.212] Can’t close activity db: Error string not specified yet  [500, #89026]

Turns out, DAV keeps some cache files around in your SVN ‘dav’ directory. Removing all the ‘activ*’ files solved my problem.