Matt Hamer, founder of Attribyte, has been posting on mth·ology since 1999.
On the Road with the Asus Zenbook Prime
Sun, Sep 29, 2013
For the past two years my primary development system has been a Dell E6420. It runs Linux Mint “Maya”, the latest long-term support release, perfectly. Everything just works. It is portable, but no joy to carry on, or use, on a plane. Many corporate road-warriors carry this model; it is a laptop that you have to put identifying stickers on to make sure you pull yours off the x-ray belt. Aside: Waiting for ATL-MCI last week, I spied the absolute largest “laptop” I’ve ever seen waiting to board a plane. It must have been 11” x 18” and 3” thick. The owner was actually using it as a table while eating her slice of pizza.
I’ve been traveling a lot this year and needed something a bit more manageable than the Dell. So, I borrowed an 11” MacBook Air from Gawker.
I enjoyed my time with the Air. The hardware was solid and the form-factor perfect for vacations and short work-trips that didn’t involve intense coding. In theory, the 11” model doesn’t even have to be removed from luggage for security. Of course, like most things TSA, this wasn’t consistently applied. “The TSA website says I can leave it in my bag!” After the first few re-screenings and scoldings, I just started taking it out like a normal laptop. Anyway, Tom borrowed it (with its huge Riak sticker!) to cover the Tour de France. (Yes, Tom does it all at Gawker.) I decided to leave it with him. Why? I don’t mind OSX, but it was a real pain to synchronize and switch (mentally and physically) between the two environments when traveling.
Dilemma. Upcoming trip to the Budapest office and I’m really not looking forward to lugging the Dell with me.
How about the Dell XPS 13 Developer Edition? Two weeks to ship it? “Call for pricing?” Oh, Dell! You lost an easy sale. I read some positive reviews of the Asus Zenbook Prime and it seemed that it might be perfect. Would it play well with Mint? Yes, it seemed. I ordered the Asus Zenbook Prime (UX31A-SH71) from Amazon and had it two days later.
I installed the Mint LTS, applied most of the tweaks, and mirrored my normal development environment. Everything but (annoyingly) two-finger scroll was working and it fit nicely into my bag. It turns out that broken two-finger scroll was the least of my problems. The machine locked-up hard at least a dozen times while in Budapest. Not király!
I badly want this laptop to be my new travel buddy. Reluctantly, I’ve installed the bleeding edge: Mint 15, based on Ubuntu 13.04. I’m going to skip tweaking it for now because almost everything works out of the box! Two-finger scroll works, as do the monitor and keyboard backlight controls. Most importantly, I’ve experienced no lock-ups. So far, so good, but I’ll report back here after my next trip.
Bot Building with SPS
Thu, Aug 15, 2013
In 2003, I was writing lots of bots, and a pattern emerged. I thought, “There’s probably already a name for this” as I put the finishing touches on my new bot-building framework. “I dub thee: SPS — Source, Processors, Sink. Very catchy!” It turns out that I’d created an example of SEDA — Staged Event-driven Architecture.
SPS has three stages connected by two bounded queues. There’s almost always a single source that deposits items to be processed into the input queue. In the second stage, items in the input queue are consumed by a large number of processors. When a processor finishes its work, the result is put into the output queue. Finally, the sink (normally, just one) pulls results from the queue and sends them on to their final destination.
Monitoring the size of the queues tells us a lot about the state of the system. As Matt Welsh, the creator of SEDA, writes,
“The most important contribution of SEDA, I think, was the fact that we made load and resource bottlenecks explicit in the application programming model. Regardless of how one feels about threads vs. events vs. stages, I think this is an extremely important design principle for robust, well-behaved systems. SEDA accomplishes this through the event queues between stages, which allow the application to inspect, reorder, drop, or refactor requests as they flow through the service logic. Requests are never “stalled” somewhere under the covers — say, blocking on an I/O or waiting for a thread to be scheduled. You can always get at them and see where the bottlenecks are, just by looking at the queues.”
Let’s say the input queue is full; our source consistently times-out while offering items to the queue. Processing is a bottleneck. If more capacity exists, we can increase the number of processors. Otherwise, we’ll throttle input while we attempt to optimize the processor code. If this isn’t possible, we purchase more cores or grudgingly accept that we’ve discovered the maximum processing rate.
OK. The number of processors is tuned, things are working great, and now the output queue is full. Is the sink performing a blocking operation? Did something downstream fail or suddenly become slower? We at least know where to look.
This architecture is extremely flexible and testable. We’d like to build an image processor? The source supplies image URLs. A processor takes an image URL from the queue, attempts to download it and, if successful, creates a thumbnail and enqueues it. The sink writes thumbnails (somewhere, TBD). For performance testing, we can plug-in a dummy source that continuously keeps input the queue full with a test URL and a sink that extracts and discards results from the output queue. We can then adjust the number of processors to find the maximum processing rate. Now that we know the processor’s performance characteristics, we can replace the source and sink with functional ones. Maybe URLs come from a database and the sink writes thumbnails to Amazon S3?
Another nice property is that the only compile-time knowledge required to connect components is a shared set of messages. If components produce and consume the same objects, they are interchangeable. By carefully designing the shared message library, I can put together bots like Tinkertoys!™