2011

    Our Family - November 2011

    Thanks to Angie for taking the photo!

    Family of four posing outdoors on a lawn with evergreen trees behind them, two adults and two young children smiling at the camera

    Updating my offline backup disks. 4TB of data.

    Forget WordPress caching plugins if using nginx, just use fastcgi_cache_path. Same for MediaWiki.

    Solid Browsermob load test on my new Linode VPS with nginx.

    Count 404's in group of access logs

    I found myself needing to see all of the 404 errors in the access logs for all virtual hosts on my web server. I put all of my logs for a given application (in this case WordPress) in one place (/srv/www/wordpress/logs/$host-access.log). Logrotate kicks in to keep them segmented and compress them by day.

    A bunch of Unix magic later…

    zgrep " 404 " *-access.log* | \
    cut -d " " -f 1,7 | \
    sed s/\:.*\ /\ / | \
    sed s/\-access.*\ /\ / | \
    sort | \
    uniq -c | \
    sort -n -r | \
    head -20
    

    zgrep is just grep that handles both normal and gzipped files. Pipe that into cut to pull out just the data we want. The two sed commands pull out data that would mess up the aggregation (the IP address of the requester and part of the filename). Sort puts prepares the stream for uniq to do the counting. Then do a numeric sort in reverse and show the top 20 404’s in all log files.

    Output looks like

        380 thingelstad.com /wp-content/uploads/2011/09/cropped-20090816-101826-0200.jpg
        301 thingelstad.com /wp-content/uploads/2009/06/Peppa-Pig-Cold-Winter-Day-DVD-Cover.jpg
        300 thingelstad.com /wp-content/thingelstad/uploads/2011/10/Halloween-2011-1000x750.jpg
        264 thingelstad.com /wp-content/uploads/2007/12/guitar-hero-iii-cover-image.jpg
        130 thingelstad.com /apple-touch-icon.png
        129 thingelstad.com /apple-touch-icon-precomposed.png
        121 thingelstad.com /wp-content/uploads/import/o_nintendo-ds-lite.jpg
        114 thingelstad.com /wp-content/thingelstad/uploads/2011/10/Crusty-Tofu-1000x750.jpg
    

    Of course the next step would be to further the pipe into a curl --head command to see which 404’s are still problematic. That just makes me smile. 🙂

    As an aside, sort combined with uniq -c has to be one of the most deceptively powerful yet simple set of commands out there. I’m amazed at how often they give me exactly what I’m looking for.

    Tammy's Favorite Picture of Me

    Tammy went digging deep into our Lightroom library looking for photos for our new exercise room. She told me she had found her “favorite picture” of me. I was very curious what picture it was. I like it. This is from the finish of the 2004 Chequamegon Fat 40. The kid in the background is out of place though, a little Photoshop could deal with that.

    Mud-covered man sitting on the ground at a mountain bike race finish area, gloves in hand, bikes scattered around him, trees and a shed in the background.

    Move to Linode

    This website (and all the other websites I run) should all be running much faster now. This weekend I flipped everything from my previous VPS at Slicehost to a new instance at Linode.

    There is a ridiculous amount I could write about the move, and I’ll try to share what I think is most helpful to others. In general I can say that the Linode hosts are a lot faster than the Slicehost instance I had. Doing basic Linux stuff was 2-3 times faster on Linode than on Slicehost.

    Bar chart showing PHP cache hits at 99.9% (1588335) versus misses at 0.1% (2234), with green and brown bars respectively.

    My new setup is also a lot faster due to how I deployed WordPress and MediaWiki. I’m now running everything on nginx instead of Apache. I’m also serving all my PHP out of php5-cgi instead of mod_php. Perhaps even more importantly I got all of my wiki and blog instances running under one PHP install for MediaWiki and WordPress. As a result, the APC module for PHP can do its job right. I’m now getting 99.9% PHP cache hits.

    With all that said I fully expect I may have a thing or two not working right now. If you see anything broken a comment or email would be great.

    Bring on the Snow.

    Two red wooden sleds leaning against a large tree in a snow-covered backyard with a fence, slide, and small playhouse nearby.

    Great Minnestar board meeting this morning with strong plans for Minnebar and Minnedemo in 2012!

    iTunes Match Updates Ratings and Play Counts

    Apple launched iTunes Match today and having all my music in iTunes, with three different computers, an iPhone 4S, iPad 2 and three Apple TVs I figured I could benefit from using iTunes Match for my music. I enabled it and it is still in the process of doing its first update. I was very curious about how iTunes Match dealt with certain metadata.

    MacBook Pro displaying iTunes library with album artwork, a stack of CDs beside it, and a cloud graphic showing synced music albums above

    I’m a big user of Smart Playlists and many of them rely on song ratings, play counts and last played dates. I also often have Smart Playlists that reference other Smart Playlists. Some quick learnings from an hour of playing around.

    1. iTunes Match synchronizes ratings within 10-20 seconds between devices. I was playing a matched track on my iPad 2 and adjusted the rating. Within 20 seconds iTunes on my MacBook Pro reflected the new rating. I changed the rating in iTunes and within another 20 seconds it was updated on the iPad 2 without any action being taken.

    2. The play count for a track is updated promptly after the track is played.

    3. Unfortunately the last played date is not updated. This is annoying for me since I routinely use last play date to manage Smart Playlists.

    4. Smart Playlists that use other Smart Playlists as part of their criteria are not available in iTunes match. Simpler Smart Playlists are fine in iTunes Match.

← Newer Posts Older Posts →