Joel Collins

Useful tools: pt-archiver

Wed, 07 Oct 2015 00:00:00 +0000

If you have found yourself using mysql as a job queue, eventually you have to deal with the unpleasantness of removing old data from your database.

Don’t do this:

delete from tablename where created < 'whatever'

Depending on your database, writes may get locked out and if nothing else your DBA’s will thank you for not making the mysql slaves lag behind master.

A nice tool for jobs like this is Percona’s pt-archiver. Its a beautifully simple tool that does one thing and does it well. It can move data out of your database efficiently and without negatively impacting app performance.

I am using something like the following to archive data into a read only analytics database.

pt-archiver --source=... --dest=... --columns=just,the,ones,i,need \
 --limit 1000 --commit-each --no-check-columns --where "1=1"

Hope this helps someone find pt-archiver.

Vagrant Shared Folder (NFS / Rsync) Performance

Sat, 29 Nov 2014 00:00:00 +0000

Disclaimer… I’m still learning vagrant & virtualization.

tl;dr version: If you need a large codebase accessible via your host, rsync isn’t ready yet and nfs isn’t lighting fast

I’m spending the thanksgiving weekend trying to get a drupal vm setup using vagrant. After several past weekends failing to learn chef and docker I think I’m making some progress, but I’m getting stuck on sharing folders with the host. One of the major goals of this project was to allow developers to utilize IDEs on their host, so I wanted the bulk of the codebase to be accessible from the host. For the weekend, I’m on my macbook but ideally this setup would work on windows hosts as well. Unfortunately I think I’m stuck.

At least in virtualbox, native shared folder performance is not acceptable for hosting the drupal profile that I want hosted. (Depending on what is included, the codebase can be 50-100mb). NFS sems acceptable, but not great. Page load times are 5-10x slower than native. I haven’t been able to test smb yet but I presume its similar.

Rsync would seem like a good next step, as in theory the performance would be near native, but as of vagrant 1.6.5 there are major issues with the rsync-auto job. For now I’m considering either scrapping this for now or looking at some ability to toggle between the codebase living inside the virtual for performance or living on the host for debugging/ide capability. Another thing I wanted to explore was having the code live in the virtual but use nfs to mount it on the host.

Soylent

Mon, 29 Sep 2014 00:00:00 +0000

I finally got my 1 week supply of soylent from kickstarter last week. I think the biggest advantage for me is the relative ease of tracking macros that is afforded if you replace an entire meal with soylent. Unfortunatly the default soylent mixture is already too carb-heavy for my goals.

My current daily targets are 145g protein / 75g fat / 200g carbs. Soylent, assuming normal ration sizes, is 114g protein / 57g fat / 252g carbs. Soylent has a diy site that you can use to customize the mixture which looks interesting…

As far as taste goes… Its definitely not something to switch to just for the flavor. I’d rate it 3 or 4 out of 10. It basically tastes exactly what you would imagine an flavorless meal replacement to taste like.

A Jekyll Blog

Mon, 28 Jul 2014 15:00:41 +0000

About time I started to blog, mostly to force myself to learn things and write about what I learn. Jekyll seems pretty intuitive so far.

I wonder how tags work. I’m reading about canoeing right now, so I’m tagging this blog post with canoeing. But I don’t know how to link to it yet.

I’m reading Canoe Racing right now, its getting me pumped to learn how to make boats go fast.