Open Loop #7 - Turtle Turtle Turtle

Open Loop 7

2013-01-12

Transcript

Speaker-labelled transcript, machine transcribed and cleaned up with AI in October 2026. It may still have mistakes. Also as WebVTT and JSON.

Garrick: I’m looking at the revision history of the draft of the show notes for this episode.

Jamie: It goes back a ways?

Garrick: October.

Jamie: Well, you know, we got ahead of ourselves and drafted some, we drafted some that we didn’t end up, you know, that didn’t happen right away. Like

Garrick: Yeah.

Jamie: they didn’t, like not in October.

Garrick: It just goes to show how prepared we can be.

Jamie: Yes. Planning ahead. Do you have a beer?

Garrick: I do have a beer. And the reason I was in the show notes is I was adding it to to the notes. It is Sierra Nevada’s Bigfoot Ale. 2013. They vintage these, you know, because easily, easily one of the best beers I have had this year.

Jamie: I am drinking the Bakken Bock.

Garrick: You have had this one before.

Jamie: Have I had it on? I haven’t had it on our podcast, have I?

Garrick: Maybe not.

Jamie: Have I had

Garrick: You showed it to me one time.

Jamie: Yeah, I think I showed it to you. I’ve been scared of it. Because it’s a dark doppelbock. Yeah, but you know, I’ve been drinking all of the really hoppy IPAs lately, so I’ve been kind of wary of it and a little scared. And you know the fun thing about this beer, do you know why it’s called the Bakken Bock?

Garrick: Tell me.

Jamie: It’s named after the Bakken oil run in North Dakota. So they

Garrick: Ah, yes.

Jamie: saw this in Williston, North Dakota, where I hail from. My grandparents live up in that area. My relatives live up in that area. So this beer actually came back with me when I went to Lake Knight, North Dakota for a family thing, and I brought back a couple of these Bakken Bocks.

Garrick: Nice.

Jamie: Yeah, it’s got a picture of an oil rig on that, buffalo, bison in the background, and more oil rigs and then like hills. There’s no hills in North Dakota. I don’t know why those hills are there. I don’t. So I’m gonna try here. Pretty good. Actually, that’s pretty good. It’s by Bayern Brewery out of Missoula. And they don’t even list it on their website anymore. I think it was just like a short run, you know.

Garrick: Ah, nice.

Jamie: But it, yeah, that’s tasty.

Garrick: Doppelbocks are yummy.

Jamie: Do you remember, does Sam Adams still make that triple bock?

Garrick: Ah,

Jamie: That little blue bottle.

Garrick: Oh, that little blue bottle.

Jamie: You know what I mean? You could put it on pancakes.

Garrick: I know exactly what you mean. That little blue bottle is responsible for my beer journey.

Jamie: I thought you were going to say your third child.

Garrick: No, really,

Jamie: Really?

Garrick: my roommate in college brought it back one summer, you know, and said, hey, welcome back to school, I brought this little blue bottle for you. And you know, how can you go back to Schlitz after that? Yeah.

Jamie: I’m surprised you can even drink it. You know, I mean, because, I mean, that stuff, it was like syrup.

Garrick: Yeah, I have a homebrew recipe for it.

Jamie: Really.

Garrick: It’s remarkable because it’s not just a brew day process. There’s actually like, you know, every month you just put a little more wort into the fermenter. You just keep feeding the yeast and you keep feeding the water, and it’s like a process just to ferment the thing too. Yeah, and then you wait two years.

Jamie: You know, have they ever— did I tell you a story about when I went to the Sam Adams Brewery

Garrick: No.

Jamie: about this? So many years ago on a trip to Boston, I went to the brewery and took the brewery tour, and this was when the triple bock was available and they… And they were saying how there’s these three breweries or two breweries right, or sorry, bars right by the brewery, and they send things there to experiment and try. And they said that they sent the triple bock there, but they forgot to tell them what to do with it. And so they were serving it in pint glasses. Apparently they had some fairly drunk beer drinkers, because a pint of that stuff would have been— I mean, first of all, I’m amazed you could drink it. It’d be stuffed.

Garrick: Yes, yeah.

Jamie: Yeah, you wouldn’t finish

Garrick: I have these little— from my beer class I have these tiny what are they, they must be They might be — they’re 33-centiliter tulip glasses, so tiny little things.

Jamie: What do you— yeah.

Garrick: And I have found them to be fantastic for these

Jamie: For that kind of thing.

Garrick: bigger beers like this barley wine.

Jamie: Yeah.

Garrick: Because you’re like, yeah, when I finish this, I’m just about done.

Jamie: Hey, this is our first Open Loop since you became a certified beer judge, right?

Garrick: I need to yet figure out, I need to yet receive my final results on my tasting exam.

Jamie: Oh.

Garrick: Right. So I don’t really know exactly what my status is, and plus I need to get experience points.

Jamie: A badge.

Garrick: I need to get a badge. So I am recognized to some degree. And in an effort to get out in front of my sort of judging experience points needs, this Friday and Saturday, I’m judging at the big Upper Midwest, Upper Mississippi Mashout, the big two-day beer contest.

Jamie: Really cool. Yeah.

Garrick: Yes, so I’m judging two sessions on Friday and a session on Saturday, and then next month I’m going to Menominee to judge a beer contest there, and just hopefully by the time I get my results I’ll have enough points to actually have my merit badge.

Jamie: That’s cool. Is there a registry? I mean, like is there like a list?

Garrick: Yeah.

Jamie: Of course. One of the things that we talked about is that we kind of saturated Open Loop with Kubb.

Garrick: Right.

Jamie: But we do— so it’s gonna be— Open Loop’s not gonna be every minute of Open Loop is about Kubb. But we do have one Kubb thing to talk about in this Open Loop.

Garrick: So we’ll make a—

Jamie: The newest rage in Kubb.

Garrick: We’ll make it short.

Jamie: Tiny. A tiny

Garrick: Tiny

Jamie: Kubb. You know, so I think it’s hilarious. A tiny Kubb. It’s made by Becks, whose sets I often speak poorly of, but but their tiny sets are awesome. And you can get it on Amazon. It’s this little tiny, tiny Kubb set for playing on your desk. And it’s hilarious because you got it

Garrick: Yeah.

Jamie: for your birthday,

Garrick: My birthday, yeah.

Jamie: I got it for a Christmas present. I think we both got it on like the same day or like a day apart, right?

Garrick: Yeah, exactly.

Jamie: And I sent you like a picture and you’re like, oh yeah, I got that too. So here’s the clue for all of you listening. If you know somebody who’s into Kubb, just get him a tiny Kubb set. It’s like a guaranteed good gift. You can’t go wrong.

Garrick: It’s so much fun because… Becks, the packaging of it, they just shrank down the full-size box to

Jamie: Yeah.

Garrick: this, what, four-inch? three-inch

Jamie: Well,

Garrick: size

Jamie: yeah, if— yeah, not for three, but

Garrick: yeah, three-inch size. And so on the box, in the now one-point type, it says something like, game instructions inside. There’s no space inside for game instructions. No.

Jamie: Did yours come with a bag for the pieces? Well, ‘cause it says like includes a bag, and I was like, where’s the— because you need it, like, I don’t— you could lose them. And were you able to ever get them back in the box?

Garrick: Yeah, yeah, I got them back in the box, they fit

Jamie: I could—

Garrick: quite nicely.

Jamie: I could, but I couldn’t get them to fit like the way they came. I should have taken notes, I should have taken notes or something when I unpacked it. And it’s

Garrick: Oh, I still, every time I look at it I giggle, every time I play I giggle.

Jamie: So you played on the carpet.

Garrick: I played on the carpet, I played on tabletops.

Jamie: Yeah.

Garrick: I think the carpet is the best. The tabletops are weird.

Jamie: Tabletop bounces all over the place.

Garrick: Yeah, and you got to figure out, where do you put the pieces so you’ve got room and they’ve got room. And just go on the floor.

Jamie: But if the baton goes flying, it’ll end up in the cake or in the pie. Yeah. I mean, I played— my father-in-law and I, we played a couple games on the table at his house. Yeah, the stuff was flying everywhere.

Garrick: Stuff flying everywhere. I determined that the most sort of in-the-spirit-of-full-size-Kubb way to throw the batons and the kubbs is to fling them like a paper football. I’m not sure what you determine technique-wise.

Jamie: Well, I found that, you know, in the Minnesota flip—

Garrick: Really?

Jamie: Yeah, I couldn’t throw the way I normally throw. I was not getting good accuracy and things are going all over the place. So I just hold it by the top, just like some of the guys do when they play, you know, and did the flip. So I threw it from the top and flipped it, and that worked pretty well for me.

Garrick: Interesting.

Jamie: Yeah, how was your drilling?

Garrick: I just drilled the same way. I just,

Jamie: Did you actually spin it?

Garrick: No, I just set all the kubbs on the floor and just sort of flicked them like a paper football.

Jamie: Yeah.

Garrick: Yeah.

Jamie: Yeah. Stuff was flying everywhere.

Garrick: Still flying everywhere, and it is just, it’s silly.

Jamie: Yeah.

Garrick: It’s pure silliness.

Jamie: Totally silly. We should set up— It’d be funny at like nationals to set up a few tiny Kubb pitches.

Garrick: Yes. Yes. I’d also like to see, you know, how in football the coaches are looking over the plays and going over the play tape.

Jamie: Yeah. No, you could set it up. It’d be a perfect—

Garrick: You could set it up. Yeah.

Jamie: Exactly. Exactly. Oh man, yeah. You know, if Tad Kubbler is listening, set up a little table, and then when they conference for several minutes on end, they can do

Garrick: Yeah.

Jamie: a little what-ifs scenario planning.

Garrick: With the tiny Kubb.

Jamie: Yeah.

Garrick: Yeah. You know, you could easily put Velcro on the bottoms and have a little portable pitch.

Jamie: Oh yeah.

Garrick: That could do the, yeah, the little setup.

Jamie: Yeah. Oh, we’re getting ready for our first Kubb tournament of the year, two weeks in the frozen tundra on the lake, the Loppet tournament.

Garrick: Oh, so delicious. That’s right. That’s right. A couple weeks. Two weeks. Finally. Finally, after 18 months, some competitive winter Kubb properly done.

Jamie: After 18? Oh, because we didn’t… Yeah, last year we played in sweaters, so, man.

Garrick: I know.

Jamie: I don’t think it’s going to be like that this year.

Garrick: I think it’s

Jamie: It’s going to be very cold this year.

Garrick: But it’s shaping up to be quite a good tournament, I think.

Jamie: It’s going to be awesome.

Garrick: Yeah.

Jamie: Are you recruiting any teams?

Garrick: Am I recruiting any teams?

Jamie: Yeah, any new teams, are you dragging some new folks?

Garrick: Yeah, yeah, yeah, yeah. Got a new team, the guy I, or one of the guys from my beer class who is starting a new brewery, Jeremy Price from Price Brewing, is going to be in Uptown at some point in the next year or so, is fielding a team called Beer in Hand.

Jamie: Nice.

Garrick: Yeah.

Jamie: That’s great. I’m trying to get the Stillwater guys that I know to ante up, but I don’t think it’s going to happen. Yeah. That’s how it goes. Quote of Kubb. Hit.

Garrick: It was done.

Jamie: Done.

Garrick: So what I like about this Bigfoot, so the first time I cracked it open, there’s this wonderful, wonderful hop nose on it. Just. Oh, leafy and grassy and citrusy. It was so

Jamie: Wow.

Garrick: yummy. Immediately grabbed my style guide and wanted to see, because if you look at the style guide, the order of the beers indicates how accurate they are to the given style.

Jamie: Okay.

Garrick: So if you’re looking like, I want to get the most accurate doppelbock, you grab the first one on the list for

Jamie: Okay.

Garrick: examples, right?

Jamie: I understand.

Garrick: And it usually lists like five or six. This Bigfoot, first one on the list for American barleywine, and I’m like, yep, totally really deserves it. Yep, it had me at the, had me just at the nose, and it’s still there. It always, it’s so much amusing. You can look at this sort of goldish, amberish color and be like, well, yeah, it looks like a nice enough beer. And then you get it even anywhere near your nose, your mouth, and you’re like, oh my god. So unassuming. Anyway. You know, some places, some offices close between Christmas and the New Year. And historically, I’ve had a couple of clients that have closed during that time. Which I always thought was awesome, because generally, historically, I haven’t closed up shop during that time,

Jamie: Right.

Garrick: but that changes. This year was kind of weird because it was only three days, but historically I haven’t, which has been really good because during that time everything is usually so quiet that I can just work on one thing for a whole week and actually feel like I got something done, and I’m not worried about being distracted.

Jamie: Right.

Garrick: I can really focus on what I’m doing and really make the most of that quiet time. And I have a great fondness for that week of just being a discretionary week and being able to maximize that discretion.

Jamie: Yeah, I think that’s the, you know, this is the, for the last couple of years now I’ve taken that time off and have really tried to do some kind of things that push my skills, you know. And I think I really like this idea of actually putting that out there as a kind of this holiday project, because

Garrick: Right.

Jamie: I was thinking, is if you, and I don’t know what it would take to do it, but you know, you take that time, because there’s not a ton that does happen during that time between Christmas and New Year’s, and it’s not a huge block of time, and you really focus on doing a holiday project where you identify something, a new technology, something that you want to explore and go deep in. And I think it’d be a really interesting idea, particularly for a development team, for everybody to pick something you work on, that then when you come back in the new year, you do a show and tell, right? Everybody gets together and you’re like, hey, here’s what I did. It’s similar somewhat in concept to, like, you know, the 20 percent time, or like having these kind of you know, other things that you explore on your own. But the difference is that you know you got to fit it into essentially 20 hours or so, or 30 hours or so of time, and figure out how can you get to something that is at least a demo, if not like a functioning product.

Garrick: Right.

Jamie: And, you know, it’s a great way, I think, to explore new technologies, you know, to explore, you know, one of the things that I continue, I’ve seen this happen to myself and others for years, is that when you learn how other technologies integrate or execute things, or different patterns, that always is something that just goes into your toolbox and becomes something that you can leverage in other things that you’re doing. And yeah, I still remember back to some of the products that we built at MarketWatch and how they were directly inspired by some of the stuff that I was kind of playing with on the side and learning how, you know, some things happen on a JavaScript layer that was kind of different and novel, and then realizing, oh wow, we could use that for this, and then, you know, it had a really big impact. So yeah, I think it’s definitely a concept that should be explored, is, you know, this holiday project. You say like, okay, during this time, I was going to keep everything running and focus on, you know, if there’s priorities and urgent things you got to deal with, you got to deal with them, but let’s take that time and do something that kind of exercises the creative muscle and explores some curious space, and then share the outcome of that. Yeah, and I think…

Garrick: I like that a lot in a couple of different ways. The first way is that it’s longer than these popular hackathons, which I think are a day or maybe a weekend at the most.

Jamie: Yeah.

Garrick: I don’t know that that provides enough time to actually solve a meaningful problem in a meaningful way. But if you said, plus it’s more, if you said this is going to be more self-directed, I think you can pull out a great deal more uniqueness across the breadth of the participants.

Jamie: Yeah.

Garrick: Both of those things seem like wins to me.

Jamie: Yeah, no, I did one this year. I mean, I’ve done, I have a hard time remembering. I know that, I mean, last year you and I both explored this proxy. That was last year, wasn’t it?

Garrick: Yep. Yeah.

Jamie: How do we filter on a network layer with a proxy server, and in just a couple of days got something that could at least, you know, demonstrate the capability.

Garrick: That’s right, and then I rebuilt the thing in Node and realized why I don’t like Node,

Jamie: Exactly.

Garrick: which is perfect for that.

Jamie: And now, you know, like, hey, I don’t need to build projects in Node.

Garrick: Yeah, that’s right.

Jamie: So this year I, so I’ve been thinking about this for a long time, this whole, you know, as I’ve mentioned before in other Open Loops, you know, like, I’ve been very kind of lit on fire, if you will, by MediaWiki and Semantic MediaWiki, and what you can do with wikis. And so for a while I’ve been thinking like, wow, I really wish there was something that allowed me to administer and monitor and keep tabs on my own wikis better. And so, like, for a while now I actually used Munin, which I used to graph, you know, all the kind of standard stuff that you graph on a server. And I would actually graph certain statistics for the wikis that I ran. And that was fine, but it lacked a lot of, it just wasn’t all that useful. So it hit me just kind of like right before Christmas, like I had this bit of an epiphany about, and mostly the epiphany came about from looking at some of the new stuff that’s in Semantic MediaWiki as well as Semantic Result Formats. Notably, Semantic Result Formats in the newest version has added a number of new capabilities, including this charting library called Dygraph, D-Y-graph. And Dygraph is the same charting library, or it’s a derivative charting library, of what’s on like Google Finance and Google Trends. So you can handle very large data sets of time series data. And it struck me that, well gee, if I could just put these statistics somewhere, the Dygraph would be able to render it, and that should be pretty easy to do. And I know how to write a bot. So had this kind of like aha, and then needed a name, because you need a name, and connected with you and we explored names. And I think you came up with the idea of apiary, because we were playing around with the idea of, you know, MediaWikis are, the logo of a MediaWiki is a sunflower, and there’s all these bots, and the bots are kind of like bees, and the bees visit the visit the sunflower. And where do all the bees live? Well, they live in an apiary, or they live in a hive, and a collection of hives is an apiary. Which, hence the name WikiApiary. So put this thing together as a holiday project, and it’s been a ton of fun. So, you know, in, I think really in about three days, I was able to get it to the point where you could register a website. The website would then begin to collect data via a bot, and you could visualize that data through Dygraph, and then started to pull other data in. And what’s been really interesting about it is a couple of things. One, as a project itself, it’s interesting and has gotten so much interest and excitement in other people in the MediaWiki community. In fact, there’s a guy, Carsten, in Berlin who’s been hacking on it pretty much, you know, every day along with me. And, now I’ve got about five hundred wikis in there, and it’s pulling in data that is proving to be really, really interesting around the versions of extensions that are in use on different wikis, as well as how much edit activity is on a wiki. And it looks really nice, and it’s been a bit of an aha project for me on how to use Wikimedia, or sorry, how to use MediaWiki as a front end, kind of a user experience layer on top of other headless programs, and how you can integrate all those things together into a very unified user experience that is kind of significantly less complicated because you’re taking advantage of all of those capabilities.

Garrick: I like to think of, and you’re far deeper into it than I am, I really enjoy thinking of MediaWiki as a database that you can,

Jamie: A lost cause. Yeah.

Garrick: in much the same way that I like to think of CouchDB as this sort of mix of database plus view plus query. Right, if you absolutely need to, you can use this database as the presentation layer. But you don’t really need to.

Jamie: Right.

Garrick: The API is robust enough that you should probably use something else, because it’s kind of a pain in the butt to. But you could, and you could have all these three things together, and that’s really kind of nice.

Jamie: Right.

Garrick: One of the things, two of the things that I like about WikiApiary. One is it’s really remarkable how much performance data about about MediaWiki installs is just available publicly on their APIs by default.

Jamie: Yeah, isn’t it?

Garrick: Yeah.

Jamie: Yeah.

Garrick: And then the second thing is, you know, I’ve had a couple of questions for some wikis that I’m planning and have been like, I should ask Jamie if he’s ever seen — wait, I should just go to WikiApiary and check first.

Jamie: Yeah.

Garrick: Does this extension exist? Is it in use? Who’s using it? How are they using it? Can it be done? That sort of stuff. I can now go to WikiApiary and ask those questions, which I don’t think there’s been a resource, this sort of central resource before like this.

Jamie: No, there’s never been anything for extensions for sure. And there’s never been anything that answers the question like, what percentage of MediaWiki installs use extension X, now. Like the most popular extension with, you know, not surprisingly, is parser functions. It’s used on all but like 50 MediaWiki installs that are registered in WikiApiary. And so, you know, but interestingly enough, there’s about seven different versions of parser functions that are in active use across those wikis. And so, you know, knowing kind of what versions are out there is a very useful thing, and then for the extension authors to know, well, X percent of wikis use PHP version X, so if I want to move my minimum version up, it’s gonna cause a problem for these people. So no, it’s an amazing amount of data. And it’s interesting, when I first started doing it I really envis— I envisioned this as something that a wiki owner would sign up for, they’d register their wiki, they would use it to kind of monitor and see what was going on there, and that it would mostly provide value to that individual person, that one person. That’s how I envisioned myself benefiting from something like this. But it’s immediately obvious, as more and more wikis were added, that there’s a huge benefit to being able to cover a lot of the surface area of the MediaWiki ecosystem and know kind of globally what’s going on across all of those various wikis, to provide, you know, the kind of almost like discovery type information. Yeah, like, well, what’s an active wiki in this content area? Because it may be easy to find multiple wikis about, you know, about Minecraft, but if there’s four wikis about Minecraft, which one’s the most active?

Garrick: Right.

Jamie: And that’s easy to tell.

Garrick: And that was one of the questions I had earlier this week was, what does a wiki about a family history look like?

Jamie: Yeah.

Garrick: Oh, I could go to WikiApiary and scan the 500 wikis and find one that most likely will give me a template for how they solved the problem that I could use as a starting point.

Jamie: Yep. Yeah.

Garrick: Perfect.

Jamie: Well, and it also has highlighted the — what is it — the largest Semantic MediaWiki in there holds about 40 million records, which is a pretty big data set. So it also shows that, you know, you can scale this technology up pretty big. Yeah.

Garrick: MediaWiki scales.

Jamie: Yeah. Surprise, surprise. But it’s also been a fun — the other thing that’s been kind of interesting to me is that the way that WikiApiary was built, and why it was able to be done in a very, very short period of time, was that I’m leveraging it as kind of the control surface, right? So that code is always just such a pain to deal with. And usually if I have to do that, I’m not gonna do it. I’m not gonna do it, because I know I don’t have the time to do it, and if I have to start kind of at that layer, I’m just not gonna be able to get enough done to kind of encourage myself to keep on doing it. But this whole thing where, like, registering your website and validating and activating and all that being able to happen just in wiki forms, and then having very, very focused Python apps that then interact with the API to pull that data in, is a really interesting model. And it’s a model that would work for what I’m doing with this, but it’s also a model that would work for a variety of other verticals and content sets. You know, I mean, if you think about, well, what if I wanted to build a wiki that had weather, right? I

Garrick: Right.

Jamie: mean, it would be relatively trivial to register, here’s my personal, you know, like, there’s this open —

Garrick: The OpenWeather.

Jamie: OpenWeather, right.

Garrick: Yeah.

Jamie: And OpenWeather relies on you posting to their JSON feed, you know, with, hey, here’s my new information. And so you just post, post, post, post. You could also easily have it so that you had a wiki where you registered, here’s my weather station, pull the data using this specification, and then it could do a poll, you know, on a schedule. You just reverse the polarity of that, and you could imagine doing that for a variety of different contents — that’s, you know, sports, weather, financial information. All of a sudden it means that you could bring kind of this robotic capability into the contents, which is really something that I think has been thus far under-labored, like in corporate environments, you know. I think of the content management systems that many news organizations use. It’s not, you know, they all have collaboration and they all have workflow. I don’t know of any that significantly leverage robotic actions or, you know, that kind of thing. And that’s the kind of thing that this, like, API depends on. You know, the robots do all the work. You just tell them what to do via the wiki itself. It’s

Garrick: It’s really interesting to think of — and we’ve talked about this before, I think you mentioned it at some point — the number of robots that are continually working under Wikipedia or the other Wikimedia sites.

Jamie: Oh yeah, it’s tremendous.

Garrick: A freaking army of robots just to keep the place going and cleaned up and stuff. I like that image. We have these robots that are keeping the place not just clean and tidy, but also are the primary gatherers of data.

Jamie: Yeah.

Garrick: MediaWiki has this overlay that we as people can interact with to say, okay, this is crap, bad robot, but also clean it up and fix it and make it more usable, but we have somebody writing that first draft for us.

Jamie: Right. Yeah, when you look at what WikiApiary does, it collects information like the number of edits that have occurred on a wiki. And so the robot, thematically named Bumblebee, wakes up every 15 minutes to do that. He has no idea what wikis exist or what work he’s supposed to do. He just knows that he collects this data point. And then the first thing he does is query the wiki and say, well, what websites should I go query that for, you know? So as soon as a new site is added and validated and activated, he instantly goes to pick it up and start working on it.

Garrick: You have a number of robots that are waking up at different periods of time doing this work, and now you have six, and that number itself is even set in a wiki page.

Jamie: Yeah, isn’t that great? Yeah, so to pull the data, it struck me that obviously I’m gonna run into kind of some problems if there’s a large number of wikis to collect the data from. And a lot of the time is just spent in wait, right? It’s asking the remote web server for a JSON data block, and while it waits for that it’s just gonna sit in idle wait. So it had to be out of the serial process, ‘cause it was just gonna slow it down. So first I thought about doing it using an eventing library or a multi-threading library, which was really scary, ‘cause that’s, you know, I’m kind of over my skis if I get in that stuff. But then it hit me, well, wait a second, this set of websites that are supposed to be queried, they could just be segmented. So what if I segmented them into multiple parts? So I added a new property that says, you know, has bot segment, and has bot segment is automatically calculated off of the page ID, just by doing a mod of, you know, a number. I started with two, so you’ve got two segments. But then it hit me that, wait a second, actually six is where I should start. So what happened — it’s not running right now, but within a week or so — every five minutes, two Bumblebee instances will start. And so, you know, at zero minutes, segments one and two will run, at five after, three and four, and at ten after, five and six, and then on fifteen after it all repeats. And that way I get parallelization and I also segment the work. So

Garrick: And then they all come back and they do a dance for each other to tell us where the honey is.

Jamie: Exactly, exactly. But by doing that I figure it could, you know, deal with thousands and thousands of websites, because actually it’d be pretty easy to have up to 12, because you can run one per processor. But anyway, while I was doing this it turns out that to make all this work, the number of segments is a constant that you have to define.

Garrick: Right.

Jamie: And if that were in three or four different places, obviously that’s not good. And then it hit me, well, wait a second, I can just have a protected page in the wiki that only administrators can edit, that describes exactly what this does, and it establishes what the constant number of bot segments are, and then in all of the derivative templates I just reference that. And I use the very rarely used wiki syntax of onlyinclude, which is the transclusion that becomes very confusing, because people are familiar with noinclude and includeonly, but onlyinclude is this very, very unique transclusion where, when including the page, it will only pull what’s in onlyinclude, but onlyinclude will be included in the page itself. So literally this page has all this very nice descriptive English information, and then right around the number six it has onlyinclude. So all the other templates get that. And actually I built it with two, and then I went and changed that and I watched the property page dynamically readjust itself, and all the wikis rebalanced, and boom, within two minutes everything was rebalanced and ready to use six segments. Really cool.

Garrick: Oh, a long time ago I heard this great talk from an old Smalltalk developer who had gone to Ruby. And he was saying how the great thing about Smalltalk, or whatever the dialect of Smalltalk that he was most familiar with, was that it was Smalltalk all the way down, the entire stack. And he gets over to Ruby and he keeps bumping into these parts that aren’t Ruby. They turn out to be C or C++ or whatever, whatever, they’re not Ruby.

Jamie: Yeah.

Garrick: I think of that kind of like — I have this fondness for MediaWiki, of thinking of it like this guy talked of Smalltalk.

Jamie: That’s actually

Garrick: Turtle, turtle, turtle, turtle, turtle. Hmm, what should I put here? Oh, I know — turtle, turtle, turtle, turtle, turtle, turtle.

Jamie: That’s actually a great — you know, it’s funny that you mention that, because there have been a number of times, I mean, like a perfect example, when you’re hacking on a wiki. I’ve had the need — and I have a blog post about this — I’ll have a URL and I need to get the hostname. And, you know, one way to do that would be to just write a small helper extension that’d be in PHP that would take it, parse out the hostname and return it. Easy enough and probably faster. But I really do, I have this very, very strong bias that if it can be done in templating language and parser functions, I will. And that’s all because if it’s done there, anybody with an account in a browser can edit it. And this is this whole people source thing that I keep on kind of pondering about. It’s different than open. So like, there’s something about an environment where you can build and create and even code with nothing other than a web browser and an account. And so once you pass that barrier and you go into another layer, all of a sudden access is denied, and it feels very — you know, it’s just very different. So yeah, when I try everything I can to keep it in wiki space. A good example of that is in WikiApiary actually, the charts are far too specific and had to be far too unique that I couldn’t — I ended up not being able to use Semantic Result Formats. But I wanted to still use dygraphs. And I started thinking like, well, I’m going to have to write an extension for this, and that’s going to be terrible because then I’m the only one that can easily edit it, and this is something I really want other people to be able to engage in. But I was able to use the widget extension that allows you to just create, you know, any time you need to inject raw HTML, the widget extension is a great way to do that. And so I was able to do it in there, still keep it in wiki space, still have revision history, still have a talk page, but have this very, very robust functionality where I’m executing raw JavaScript. And it’s not pretty. You know, it’s implemented in Smarty, which is this templating language for PHP.

Garrick: Yes. Yes.

Jamie: It’s a bit ugly, but it works. And if there’s any motto for wikis, it’s ugly but it works. So yeah, I mean right now, like for example, I was trying to figure out a way to parse kind of abstract version strings into meaningful bits, right? You know, because you get everything from version 5.4.1 to 5.4 alpha to, you know, year-month-day, to really weird stuff. And I’m keeping that in parser functions, even though that may not be the most efficient. It’s the most accessible, and that’s the key. So I’m going to keep on hacking on this thing.

Garrick: Yeah.

Jamie: I mean, it’s not moving as fast as it as it was. And hey, we should also mention the Kernest wiki.

Garrick: Yes, coming along, coming along.

Jamie: Coming along nicely. I am really excited about the Kernest wiki. You should introduce it to people.

Garrick: Yeah, so long ago and far away, Kernest was an open source font directory. And a couple of years ago… my MySQL and Rails and my implementation of those two had a disagreement with me and told him to get the hell off. So I haven’t really had a directory since then, and I miss it. I miss it terribly. It’s been, it would have been really handy to have in the past year or so. And then you offered to turn it into a wiki, which I thought was a brilliant idea, totally in the same ethos of Kernest. And so I gathered up what remained of the database in the repository and we dumped it in over between Christmas and New Year’s or thereabouts, and then we’ve been going through it and tidying things up and tying things together. And I am real, real bullish on it. I’m very excited for what it can mean for open source typography and open source font directories. Again, I sort of think of open source fonts as this raw material that could get tweaked and updated by anybody, but today there’s not really a good place for anybody to say, you know what, I’ve just added a new character set, or I’ve adjusted the the hinting on this thing,

Jamie: Right.

Garrick: where do I put it now? Do I really list it as its own thing, or do I push it to another repo? Where is that repo? It’s just messy. And so to say that, well, here’s a wiki space for these things, and it’s got this other capability, and you know that it’s got some level of quality in it, or there’s some license, there’s some understanding around what’s in here, and you can help out. It seems real good.

Jamie: I love the idea of thinking of it as the typography equivalent of the Music Genome Project. I really, I think, you know, talking about things like the type classification or the contrast, or these like well-known and defined concepts, is fine and it’s necessary. Like, I think that there’s a really interesting kind of further extension to identify these kind of very, you know, layer deeper or two layers deeper

Garrick: Yep.

Jamie: characteristics

Garrick: Yep.

Jamie: and open it up to that kind of markup. I think that would be really, really interesting.

Garrick: I totally agree. And I think that’s where you can start to say, programmatically, what do these things have in common and how are they related?

Jamie: Yeah.

Garrick: I don’t think to date we haven’t had the technology or the accumulation of these assets in a centralized location to really look at them in a way and say, oh, hey, look, they all have this thing in common.

Jamie: Yeah, I mean, I tell you, I love it. Whenever I pull up the front page of the Kernest wiki, I just grin. I love that it’s just a mash of fonts, and you don’t see that. And I just, I really, I do see a really… a really great capability there that I think would be — that, you know. And the great thing about all these wiki projects is I just think of them as decade projects, right? I

Garrick: Exactly.

Jamie: mean, they don’t — there’s nobody’s got hair on fire. And you know what’s been also fun about the Kernest wiki is — and I’ve done a couple of other wikis like this — there’s a certain thing that you can do with wikis that you can’t do with many other technologies, because with a wiki you can essentially just kind of like come into the room, dump your toys on the floor and be like, there. Right?

Garrick: Yes. Yes.

Jamie: I mean, we didn’t import the Kernest data for months because

Garrick: I mean, in hindsight.

Jamie: hemming and hawing, oh, it’s not clean, it needs to be, we should bring it up in Excel and clean it all up before we — and then it’s like, you know, I think both of us were just annoyed and like, screw it, let’s just shove it in there. It’s a mess. Who cares?

Garrick: It’s so Mickey.

Jamie: Well, and the fact that you can run regular expressions across every page is an amazingly powerful thing. I was surprised that, you know, after running probably 20 regular expressions through all the pages, resulting in thousands and thousands of edits, the contents started to come together. And you can see right away, like, oh, wait, there’s something here. This is a…

Garrick: I am hearing from above that there is a screaming child. Yes. Who wasn’t feeling well for most of the day anyway.

Jamie: Shut it here?

Garrick: I think that might be the safest option right now.

Jamie: Yeah.

Garrick: Yeah. Just because I don’t know how long it might be.

Jamie: I think we got a good — I think we have over a half hour of content,

Garrick: For sure. Absolutely.

Jamie: so… All right, let’s

Garrick: Cool.

Jamie: shut it down and I’ll put the file up.

Garrick: Awesome. Thanks.

Jamie: All right.

Garrick: Bye.

Jamie: Bye.