2010-04-20

Learning Japanese with Sentences

Sentences

One of the major components of my Japanese study is sentences. I add sentences written in Japanese to my spaced repetition program, Anki. When I review a sentence card, my job is to read the sentence aloud (to confirm that I know the readings for each kanji) and to understand the sentence. [1] For simpler sentences which I know well, usually this is automatic and immediate. For new or complicated sentences, understanding sometimes demands translation into English. As I get more exposure to the language, my brain will get better at skipping that slow and lossy translation step. That's the plan anyway!

Reviewing a sentence

Reviewing a sentence card where the answer and translation have both been revealed. jNetHack is a great source of sentences for me because I know NetHack so well.

I always translate from Japanese into English. Never the other way around. For one, the comprehensible input hypothesis, which is my guiding light for language study, suggests that production is only a side-effect of language ability, which is developed only when input is received. The input hypothesis even goes so far as to suggest that premature production is harmful, because in doing so you are probably reinforcing incorrect usage. [2] Also, more pragmatically, because I am not a native Japanese speaker, or even close to fluent, it is difficult for me to judge whether two sentences have the same meaning, or even similar nuance. So I would have trouble grading myself. This problem doesn't occur when I am producing English, which I have a bit more experience with. For Japanese, I rely only upon correct sentences written by native speakers, instead of whatever nonsense I come up with. :)

I also avoid studying soulless [3] vocabulary lists. Briefly, consider the difference between a lethal injection and a mortal injection. Because you've never heard of a mortal injection before, [4] it sounds wrong, even though lethal and mortal both mean deadly.

If I want to learn a new word, I force myself to find a complete sentence which uses it. So where am I getting these sentences? It turns out the answer is "from all over the place." But before I can talk about that, I want to explain a bit more about Anki.

Facts vs Cards

Anki is great for separating facts from cards. A card is familiar; each has a front (the question) and a back (the answer). A fact, on the other hand, is a set of key/value pairs. You create a fact then generate one or more cards from it. For example, if you are studying world nations, you could have a fact for each country which includes Country Name, Capital, Languages, etc. Then you could generate many cards which test you on going from country to capital, country to language, capital to country, etc. Anki's pleasant design turns out to be a great timesaver since it frees you from repeating yourself. You only type the country's name once but it can be used in many cards.

Creating a new fact in Anki

This is fact creation screen, which you will see a lot.


Generating cards from facts in Anki

This is the card template screen, which you will rarely look at. [5]

We added only four facts, but from them we created twenty-eight cards, each of which tests a single piece of data. This way, your "language of Australia" card will be independently spaced from your "capital of Australia", which is good when you have trouble with one of them. If you already know "English" but "Canberra" just won't stick in your mind, that's okay. You won't have to review that the Australians use English nearly as often as you review their capital city.

Sources

One of the fields that my sentence facts have is 'Source'. Other than the sentence itself, Source is the only field that I require myself to fill in. Translation, readings, context, etc. are optional. Knowing where each sentence came from is useful because it lets me gauge the trustworthiness of each sentence. I always trust the correctness of sentences uttered in, say, film, whereas an offhand remark on Twitter could easily have a typo or unusually playful grammar. It's also useful to know whether I have accidentally incorporated feminine language into my lexicon.

My diligence in citing every sentence I learn also lets me geek out and analyze precisely how I am learning Japanese. I wrote a script to pull all the values for the Source field out of the database and categorize them according to rules like:

qr{The Matrix} => { speech, native, movie, source("The Matrix") },
qr{iPod USB/dock cable box} => {
    text, native,
    medium => 'misc',
    source("Miscellaneous"),
},
qr{IRC PM with (\w+)} => {
    student, correspondence, author(defer { $1 }),
},
qr{mt.endeworks.jp} => { blog('lestrrat') },
qr{Conversation at Ebisuya} => { convo, native },

My Input

Another component of my study is immersion. I try to always have Japanese music in my environment. I can't escape the deluge of Japanese tweets from those I follow. There's also movies, books, IRC, and when I'm lucky even in-person conversation. I expose myself to as much Japanese-meant-for-native-speakers as possible. Every now and then I get a sentence I almost understand, except for a word or a reading. For example, as for the sentence in the marked-up image at the top of this post, I already knew and understood everything except the reading for 門 (gate) which is もん (mon). So for each so-called i+1 sentence [6] , I look up the one piece I'm missing then add it to Anki. Repeat until fluent.

But this, for sanity's sake, is tempered with Japanese-as-a-second-language materials. It would be frustrating to listen to and read complete gibberish all day, so I begrudingly use a grammar guide. I also use various other second-language materials [7] , as long as they meet my impossibly high standards. [8]

Turns out my sentences are almost exactly evenly split between these two types. Eventually the number of sentences which are intended for native speakers will dwarf the sentences intended for students. I seem to be on the right track, as evidenced by this shiny chart [9] (the x-axis is of course time but the unit is unimportant).

A chart of the intended audience by week

It's also interesting to look at the ratio of added sentences which I have read (94%) versus sentences I have heard (6%).

A chart of sentences I have read versus those I have heard

There are a few reasons for this disparity. I am reluctant to add sentences from music. Though lyrics are usually grammatically correct, it's fair to say they probably utilize some iffy language in order to fit the structure of the song. Also there's the peculiarity of kanji and how I'm studying them. I do much better with words written in kanji than those in hiragana, which means I am better at reading than listening. Interestingly, this is the opposite of a child's language acquisition. Children become fluent in conversation well before they become literate.

Now, whose sentences am I stealing? If you speak Japanese and I'm following your Twitter feed or your blog, I probably have at least one sentence you've written in my head. Verbatim. So thank you. :) I'm not going to include most of the names in this chart since I don't want to embarrass or weird out my many teachers!

A chart of the authors of my sentences

I've taken sentences from 65 people. From the top person I've taken 34. For more sense of scale, from matz I've taken four sentences. This only counts Twitter, blogs, personal correspondence, etc. Movies, music, and other mass media are excluded.

先生、どうもありがとう!

A chart of the media (twitter, games, movies, etc) of my sentences

Maybe it's unfair to group together all the second-language materials I use, but as I said, they are used only begrudgingly, so unfairness is okay. I do love Twitter for sentence mining, since it's more casual and usually not a lot of context is needed (as opposed to a sentence in the middle of a book). The games from which I've mined most are Phantasy Star 4 and jNetHack (which I beat yesterday!). I'm not really into anime, but the sentences I have are from Fist of the North Star:

It's also worth noting that knowing the entire English script of The Matrix has been helpful for watching and learning from the Japanese dub. Just sayin'.

Truisms

I got this idea of learning from sentences using spaced repetition from the guys at Antimoon. They learned English to native levels while living in Poland.

In a year, I will rerun the script I used to generate these charts to see how they compare. In the meantime, I better get to back to studying!

Footnotes

  1. I too-often have to mark sentence cards incorrect for flubbing a reading even though I know what the sentence means. So I am sorely tempted to generate two separate cards for each sentence, one which tests readings and one which tests understanding. It would be easy to do, but I'm not entirely convinced that I should. (back)
  2. Though, of course, it's really fun to converse with speakers of your second language. The whole point of language is to communicate after all. So production there is fine. You just should not do it during review. (back)
  3. I've been using this word a lot lately. It's excellent. (back)
  4. Except when listening to language blowhards like me explain this point. (back)
  5. If you're familiar with Moose, then maybe a useful analogy is that the Add Facts screen is like use Moose, but the Card Template screen is like delving into Class::MOP. (back)
  6. Here, i is everything I understood about Japanese before adding the Gehennom sentence. +1 represents the reading of gate that I learned through SRSing this sentence. Now my i is a little bit higher, and I'm a little bit more prepared for the next sentence. (back)
  7. Such as various Japanese-English dictionaries. I haven't started using a monolingual dictionary yet. That is a scary - but necessary - step to take. Very soon! (back)
  8. I avoid learning sentences written or spoken by fellow learners - even those who are far better than me with the language. Sorry! Nothing personal, but my goal is to eventually confuse people into thinking my first language is Japanese. :) (back)
  9. The charts in this post were generated with Chart::Clicker, written by the inimitable Cory Watson. Any anti-Tuftean errors are mine, not the module's. (back)

2010-01-01

On Learning

At a party last night, I explained spaced repetition and Anki to two separate groups of people. [1] I'm really enthusiastic about Anki so I'm shooting for as many converts as I can in a twenty-four hour period.

Spaced Repetition

Spaced repetition is the principle that the best time to review a fact is just before you forget it.

If spaced repetition weren't so effective, I bet the advertising industry would be far smaller. Try completing these slogans:

  • I can't believe it's not butter
  • Melts in your mouth, not in your hand
  • Just do it
  • The breakfast of champions

You haven't explicitly memorized these slogans, but I bet you'd be hard-pressed to forget them. (Most of?) you know them effortlessly because you see and hear them every now and then. The same principle underlies spaced repetition.

A spaced repetition system (SRS) is basically a program that tests you using flash cards. The major departure from an ordinary flash card program is that an SRS has a scheduler which decides when to test you on each fact. The scheduling of a fact is influenced by how well you know that fact. That's the essence of an SRS; many SRS programs have all sorts of extra functionality on top of that since they're used (and developed) by people who are humorlessly serious about learning. Like me.

I really like spaced repetition because it is rarely boring. You are never bogged down reviewing the same dozen cards that you know. With traditional flash cards, facts you know very well and facts you don't have the same precedence. Spaced repetition software tests you on facts you know well far less often than facts you don't know well yet. Unlike people, not all facts are created equal. You don't need to review every day that Boston is the capital of Massachusetts, especially if that's where you live. On the other hand, you may need to be reminded often that Regina is the capital of Saskatchewan. I sure do. Who even lives there.

Ready for another example? The kanji 雑 (miscellaneous) is pretty new to me, so I'm likely to see it again in the next day or two. On the other hand, every time I've been tested on the kanji 副 (vice-) I've gotten it correct, so I'm not going to see it again for four months. And I bet I will remember it come May. If not, Anki will adjust the card's interval accordingly.

As of this writing I have 960 cards in my deck. It would take hours to review all of them every night. Not to mention that such reviews would be boring as hell, since I know most of the facts very well. [2] Thanks to spaced repetition knowing that I am capable of learning, I review only about 64 cards per day. That may still seem like a lot, but such a review takes only fifteen or twenty minutes a day. I'll gladly review 64 distinct cards a day over doing hundreds (thousands?) of repetitions of the same set of flash cards the night before an exam. Furthermore, SRS is designed to build memories that last longer than a semester.

Most spaced repetition systems will actually let you grade yourself based on how well you know the fact. There's obviously a large gap between struggling to recall a fact before eventually getting it and having the answer pop into your head before you even finish reading the question. You will see cards of the former type more often than cards of the latter type.

Anki

Anki is one of the best spaced repetition systems. I use it to learn Japanese. I trust Anki so much that I feel if my deck lacks a particular word, phrase, kanji, or proper name, then I do not know it.

I have recently begun to use Anki for more than just Japanese. I have a Miscellaneous deck into which I put whatever I feel is worth knowing. I learn state capitals, Canadian province capitals [3] , and words that I am not comfortable using in a sentence or whose definitions are vague. I should also start putting in different kinds of things like maps with one country's label erased and Perl's more unusual special variables. Maybe I'll even prepare for the CPAN drinking game. :) Getting more exotic, there are people who put art into the SRS to learn more about art history, and even quotes from personal development books so that the ideas are constantly in mind, which is more likely to effect change.

I am not in school any more, but if I were, I would definitely have an Anki deck for each of my classes.

I used Anki to learn the Japanese poem Iroha. I'm not very into poetry, but Iroha is worth knowing for several reasons. It's a pangram of the (circa 1000AD) Japanese syllabary, so it immediately has Cool Points. The order of the kana in Iroha is still used in modern Japan for labeling Go boards and theater seats. I use it to practice writing all the kana.

Iroha is far too long to use only one card, so the best way to learn it (as pointed out by Study Shack) is by using eight cards, one for each line of the poem. Each card has a different line blanked out and your job is to produce the missing line. Here's what Anki's review interface looks like (with the answer revealed in the lower half):

Anki has a convenient feature called 'cloze deletion' that makes creating eight similar cards like this take only a minute.

By the way seeing the other seven lines of the poem in every card is like even more spaced repetition, for free.

Design

Facts are not limited to a front and back. Each fact belongs to a model. The model lets you control which fields are available (including requiredness and uniqueness constraints on each field). Some models I use are Sentence (which has fields 日本語, Meaning, Readings, and Source) and Capital (Region, Capital). Models also control how cards of that model are displayed.

As you can see, you can also generate several cards from a single fact. Not only do I want to test myself to be able to produce the capital of a given region, but I also want to test myself on producing a region given just its capital city.

You can have as many fields in a model as you want, and you can generate many different types of cards for each fact. You could create eight cards based on ten fields if you were so inclined. This is powerful stuff! Anki lets you uphold the Don't Repeat Yourself principle dear to many programmers.

Interface

Anki has a lot of gears and widgets, but what matters most is the interface you see when you're reviewing a fact. It may not look like it, but Anki's review interface is actually a fully-featured web browser. With Javascript. It has, for example, alert(). Providing a full web browser gives Anki basically limitless flexibility!

Being able to use arbitrary fields in a model works out amazingly well when you have Javascript at your disposal. You can make one of the fields a YouTube video and show it if you want. [4]

Plugins

Anki is written in Python. Not only is its UI extensible, but the backend seems to be designed that way as well. There are a dozens of plugins that are easily available in File > Download > Shared Plugin. A lot of them offer incremental improvements, but there are two that have made Anki significantly better for me.

Learn Mode

I've noticed a nasty corner case of spaced repetition as implemented by Anki. If you get a (usually brand new) card wrong, you'll see it again very soon, sometime within the same review. You'll probably then remember it easily thanks to short-term memory (or perhaps very fleeting long-term memory). So you mark it honestly, as a three, which schedules it for review in two or three days. By the time you review it again you'll have forgotten it. Lame.

The Learn mode plugin alleviates this problem by performing a review with much shorter intervals. Cards are scheduled only seconds and minutes in the future, rather than days and weeks. When I add a batch of kanji, I go through them all in learn mode. You can stop whenever you're comfortable with each new fact, but I usually continue reviewing them in learn mode until there are no more cards for review (since they're all scheduled a few minutes into the future). This usually results in 10 or 15 reviews for each fact. Then I go back to the regular deck and everything runs smoothly, even weeks after the Learn session.

Cram mode

Cram mode is actually built into core Anki but it's so close to the Learn mode plugin that it makes sense to describe Cram alongside Learn. Like Learn, Cram shows you cards rapid-fire in a new, temporary deck. It's different in that it is designed for cramming right before an exam, whereas Learn is designed for reinforcement right after you add a new set of facts. The configuration is different for the two modes.

Since I personally don't take exams I don't use cram.

iAnki

I actually do most of my repetitions on my iPod Touch during my commute. This plugin facilitates using Anki on your iPod or iPhone. You sync with regular Anki using http over wifi; the plugin provides a web server. [5] There's another interface if your iPod/iPhone is jailbroken - Anki Mini - with more features but I had less luck in getting it to work correctly.

The iAnki interface is much simpler than Anki's. You can't add or edit cards, or even look at your card list. You can use iAnki only to review. Despite that, iAnki is still worth having. Maybe someday they (or I!) will add lots more functionality.

When learning kanji it's really important to practice writing them out. For a while I used the air or the back of my hand. Turns out I'm way too forgiving of errors when I do that. So I really wanted a canvas directly in iAnki that I could use to grade myself properly.

I found Web Canvas which turned out to be quite good in terms of functionality and ease-of-use. I had to patch it to accept my iPod's touch gestures (which was incorporated upstream) but after that it was pretty smooth sailing. (update: I've also submitted some patches to iAnki itself). Here's what my iAnki looks like for testing a kanji:

This is asking us to produce the kanji for "frost".


If I have trouble remembering the kanji, I can look at the primitive story of the kanji. Stories are a peculiarity of the method I am using to learn kanji which will be the topic of another post. The "(primitives)" link had a javascript onclick handler that replaced itself with the story. Hiding the story behind a link is handy because I usually do not want to see the story unless I am stuck. [6]


Now I've written out the kanji. As fun as touchscreens are, it's really hard to demonstrate good penmanship with them. This is actually embarrassingly bad. [7]


Finally I check the answer against what I wrote. Since I got this correct, and I recalled it effortlessly, I marked it as a four. Anki scheduled this kanji to appear again in about eight weeks.

Graphs

Most importantly, Anki lets you measure your worth with graphs and statistics! This is a graph of facts I've added over the past month. The spike on the fifth is when I played Phantasy Star IV [8] all day and added a bunch of sentences.

It also tells you how difficult each card is. So here are the cards that trouble me most. I should give them special treatment, such as cramming them for a while or something.

All of your cards are stored in a SQLite database so if Anki doesn't offer exactly the kind of information you want, you can still get at it. I used the database directly to produce a calendar of kanji I've learned. That was a little worrying; I didn't realize how few kanji I learned in November and December (compared to October). But for the same reason it was also motivating.


Anki is a wonderful program. I will use Anki and its successors for the rest of my life. It's almost too bad Anki is free software.. I would have liked to be able to retire on commission. ;) [9]

If I've sold you on SRS then download Anki then head on over to the Study Shack which talks about what kind of cards are good, and other SRS management tips.


Footnotes
  1. Yes I know I attend wild parties. :) (back)
  2. Similarly, writing this is boring as hell because I know all of this. But you don't. So you're welcome. As if you were even grateful to begin with... Just get out my face and click this: (back)
  3. Do you remember the capital of Saskatchewan? It's still Regina (back)
  4. I don't know. That's a useless example. I have a better one later. It lives between where you came from and where you are now. (back)
  5. The web server is WSGI-based. Also the Javascript bits of iAnki use Joose. We have these things in Perl F.Y.I.! (back)
  6. This is the better example I was talking about. Case closed. (back)
  7. In ten years I'm going to happen upon this image and cry. (back)
  8. which owns [10] (back)
  9. Quick! Capital of Saskatchewan! Go! Don't look at footnote 3 either. (back)
  10. Sorry about all the footnotes. I made it really easy to make them. :) (back)

2009-10-07

Jifty's Request Inspector

Jifty's RequestInspector plugin provides you with debugging information for HTTP requests. You can examine the SQL queries that were issued in servicing the request, inspect a Devel::NYTProf profile of the request, and more. The information is presented in an (admittedly ugly) admin panel. Here's what our Changelogger's request inspector page looks like:

If we click into one of the requests, we can get an overview of that request.

We can inspect in detail a particular part of a request with another click.

Backend

The frontend is good enough. It gets the job done. But what I am really interested in, and what I had far more fun creating, is the backend that powers this.

RequestInspector is a Jifty plugin. Its job is to manage all of the other plugins that inspect requests (such as SQLQueries, DumpRequestResponse, NYTProf, and Gladiator). RequestInspector informs the plugins of interesting events such as: a new request is beginning, that request finished.

Each plugin produces various kinds of data. That data is completely opaque to RequestInspector. The data can be an object, a number, a code reference, or whatever else. All RequestInspector does is keep track of that data, handing the data to the plugin when it needs to do something with the data.

When a new request begins, RequestInspector calls the inspect_before_request method on each plugin. When the request ends, inspect_after_request is called on each plugin. The interesting bit is that the opaque data is threaded between the return values and parameters of these methods. The return value of inspect_before_request is passed to that plugin's inspect_after_request. inspect_after_request's return value is stored for later rendering. (In what amounts to a global variable — sorry!)

When the user wants to see a request, RequestInspector calls the inspect_render_summary method on each plugin. When the user wants to see the detailed analysis by a particular plugin for a particular request, RequestInspector calls the inspect_render_analysis method on that plugin. It is completely up to the plugin to decide what to render. Both methods receive the data it returned from inspect_after_request for that request.

Benefits

You might wonder why this design is better than, say, a specific data format to which each plugin must adhere.

It is unclear what such a data format would look like. I don't think a single data format could usefully encompass the different kinds of data that these plugins produce. While SQLQueries's data is complex data structure with a lot of information, NYTProf's is basically just a random number used to track the on-disk profile directory.

The format would certainly evolve over time. This means that plugins would have to change as well just to keep up with RequestInspector. With my design, plugins really only need to change when they want to support newer RequestInspector features.

It's simpler for plugins to let them dictate how they want to store whatever they are tracking. It's simpler for the RequestInspector core too, since it is free from having to examine the plugins' data.

Finally, the RequestInspector plugin can get arbitrarily complex and the plugins do not have to care about it. The RequestInspector already lets you filter out URLs you know you will not want to inspect. The individual request inspector plugins do not have to implement that functionality.

In the future, I plan to add a feature to RequestInspector that lets you see which requests took the most amount of database time, or leaked the most amount of memory, etc. All the feature will require of plugins is that they implement an optional inspect_compare_requests method. The method will receive that plugin's data from two requests and returns -1, 0, or 1. So trivial that I should just do it now.. :)

Basically, RequestInspector leverages the Principle of Least Knowledge to keep things simple, easy, and flexible. Shriram Krishnamurthi's Moby Scheme Compiler talk was particularly influential on this design. He described a similar (though simpler) system that he uses to make programming fun and interesting for middle school kids.

2009-09-30

Google Wave mania

I've been on Google Wave since June 11th, thanks to a friend at Google. I haven't been using the account much due to other pursuits in my life. But tonight, since they're rolling to production (I'm in as sartak@googlewave.com), they gave those in the developer preview invitations. Since I know a lot of people are very excited about Google Wave I thought I would be fair and ask on Twitter...

I have google wave invites. DM me if you want one.

Almost immediately I was flooded with random people who have been scouring Twitter for idiots like me.

They also followed me on Twitter because that will help their chances I guess?? (edit: miyagawa pointed out that they did this so I could DM them their invitation)

Of course, friends who saw my tweet pestered me elsewhere too..

I lacked the foresight to turn off the DM-to-cell-phone Twitter notifications.

Dayum... Google really knows how to generate interest.

Update: Oh god it will never end

2009-09-17

YAPC::Asia 2009 Moose course

Moose course

I have received a lot of good feedback about my YAPC::Asia 2009 Moose course. Several people on IRC and Twitter thanked me. I got second-hand reports of students taking the course then immediately being able to write useful Moose code. That is nothing short of ideal. It makes me very happy that people not only enjoyed the course, but that it was practical.

Several people also wrote blog posts about the course's content and my presentation.

As expected, the students loved the exercises. I feel very strongly that the exercises are the best part of the course. They force you to make sure you understand what you have just learned. It's handy to have a Moose man there to help students when they get stuck. My only regret about this iteration of the course is that we had to skip the last two exercises for time constraints. To the students' credit, every single one did stay well over an hour past the scheduled end time.

I was pleasantly surprised that I received such nice remarks about my live coding. As these are not my slides, I sometimes had to dive into vim to explain a tangential point about Moose. For example, demonstrating the use of has after __PACKAGE__->meta->make_immutable (which led to a bug report and then a bug fix — all during the course!). I very much enjoy this style of presentation, but it is difficult to come up with real examples on the fly. It feels great when it all just works.

I had a whole lot of help in making this course happen. A very big thanks to Daisuke Maki and the Japan Perl Association for establishing and marketing the course (and of course running this excellent YAPC::Asia!). Thanks to Emerson Mills for working with me to make the course happen, and for translating the exercises. Thanks to Nathaniel for interpreting my presentation. I did not make it easy for him! Thanks to Kenichi Ishigaki (charsbar++) for helping out with translation, for buying (with his own money) my throat lots of tea, and for being generally awesome. Thanks to all of the students for asking very good, deep questions. It's very satisfying to know one's audience is attentive and thoughtful. Finally, thanks to Dave Rolsky for letting me use his course material yet again. Next week he at long last gets to teach his own course for the first time in Minneapolis.

I look forward to giving this course again. I really enjoy spreading the Moose love.

2009-06-28

Reflections on YAPC::NA 2009

I had a lot of fun at YAPC::NA this year. I met some forest friends that didn't make it last year, such as ilmari, Nick Perez, and Dylan Hardison. I also laughed until it hurt. A few times.

I was a hermit on Sunday and Monday nights because I had not finished my slides. That was very unfortunate, so I will have future talks ready well in advance. Jitting slides is so not my style.

Moose Course

The eight hour Moose course went very well. I was a passable Rolsky-substitute. The feedback I recieved was (nearly) unanimously good, so I'm thrilled that people enjoyed it. There were some minor complaints, so if I ever teach a course again I'll be sure to keep them in mind. This year, YAPC::NA is running surveys on talks and courses, so I expect I'll learn more ways that the class could be improved. If you attended the class, please do not worry about offending Dave or me if you disliked something.

I was under-prepared for the class, but the course material was very good. I got the impression that the exercises were extremely helpful. In particular, having a good test suite (powered by the metaobject protocol!) was very useful. The exercises were also useful for me because they let me sit down. Speaking on your feet for so many hours is tough.

Like the first version of anything, the course material contained some typos, inconsistencies, and bugs. It was pleasant to fix these and push them back to the source repository while people worked on the exercises. For some reason, we could not get public (i.e. noncommitter) access to the repository working, perhaps due to DNS issues. It would have been useful to have people pull my changes to fix the exercises. Instead, I settled for announcing what the problems were and how to fix them.

Context

Early in the course I had a vitriolic rant about Perl's notion of context. On the whole I think context is a nice solution to a certain class of problems, but it definitely has some edge cases where it really bites programmers. For example, propagating context is very fiddly. You must propagate context if you wrap a function to do work after it is called without affecting its return value.

sub method {
    my $self = shift;

    # call SUPER::method in the right context.
    # not handling void context is a BUG!
    my @ret;
    if (wantarray) {
        @ret = $self->SUPER::method(@_);
    } elsif (defined wantarray) {
        $ret[0] = $self->SUPER::method(@_);
    } else {
        $self->SUPER::method(@_);
    }

    $self->called_method($self->called_method + 1);

    # return what the caller asked for.
    # void context doesn't need to be explicitly handled here
    # .. or does it? maybe!
    return @ret if wantarray;
    return $ret[0];
}

I don't recall mentioning in the class that Moose does all of this for us in the after method modifier:

after method => sub {
    my $self = shift;
    $self->called_method($self->called_method + 1);
};

The other painful detail about context is in returning a list in scalar context. Most people know that an array in scalar context evaluates to the number of elements:

sub users {
    my $self = shift;
    my @users = $self->objects('users');
    return @users;
}

my $users = $game->users; # 51

However, a lot of people don't know that there is an irritating amount of subtlety in evaluating a list in scalar context:

sub administrators {
    my $self = shift;
    return ('Sartak', 'autarch');
}

my $admins = $game->administrators; # autarch

Instead of receiving the count of administrators, we get the last element of the list. The problem is actually the comma. Due to C's strong influence on Perl 5, the comma is not just a list separator. In scalar context it acts as a sequencing operator. We're evaluating the string 'Sartak', throwing it away, then evaluating the string 'autarch'. There is no list at all here!

Binary "," is the comma operator. In scalar context it evaluates its left argument, throws that value away, then evaluates its right argument and returns that value. This is just like C's comma operator.

perldoc perlop

Almost everywhere else in the language, arrays and lists are interchangeable. I wish this inconsistency could be excised. Thanks to do BLOCK, we do not need C's comma operator at all.

Extending Moose

On Tuesday morning I presented Extending Moose for Applications (keynote source). There were maybe 20 attendees. I blame the 8AM start time. Oh well.

I'm told that at one point someone walked out saying the talk got "too abstract". It certainly is a very abstract topic, but I hope the examples showed that there is value to be had in using the ideas I presented.

Some good questions were asked, and some people did get it. I conveyed the point I wanted to make, which is that the metaobject protocol uses concepts you already know well.

I encourage you to read the slides. There are notes which is basically what I spoke when I was presenting, so they should be pretty readable. Please let me know how I did. I've received very little feedback on this talk.

TAEB

One of my good friends Jesse Luehrs (doy) gave a lightning talk about our NetHack bot, TAEB.

The line "The code I'm working on isn't really a NetHack bot; it's more of a framework for writing NetHack bots" got quite a few laughs. It's completely true, too. The topic of the talk was writing a new AI for this bot framework.

The bitrotted web interface seemed to please people. It's too bad they saw how it was when I first wrote it. Since I took those screenshots, I improved it quite a bit, making it use NetHack's colors and providing an AJAXy interface. Oh well, I should really get that working again.

This lightning talk was a hit. Several people cheered! Great job, Jesse.

Others' Talks

My favorite talk, by far, was Brock Wilcox's CGI::Inspect talk. As I said during the talk, when I wrote Carp::REPL, I was actually aiming for exactly what CGI::Inspect is. And I meant it! About two years ago, it was mentioned to me how when there's an error in your Django application, it opens an AJAXy REPL for you. I figured out how to do some of that - REPL on error - but I never actually hooked it up to Jifty. Now I don't have to. Bravo, Brock!

Another talk I enjoyed was Ricardo Signes's Git is Easy talk. I knew most of the material presented, but since seeing that talk I have had a lot more confidence in using git. In particular, remotes now make sense to me. I also now use GitX to visualize where other developers are relative to me.

Scott Walter's Perl in Vegas talk was fascinating. He drew an interesting parallel between the incredible strictures of Vegas slot machines versus America's completely opaque and mostly unregulated voting machines. One strange requirement of the slot machines is that code must be compiled. Though Perl is an interpreted language, it provides a perlcc utility to "compile" the code.

Hans Dieter Pearcey's Dist::Zilla - Automating quality since 2008 reminded me of its excellent plugin-centric design:

This slide directly caused me to submit a talk about API Design to YAPC::Asia 2009.

Finally, miyagawa's Build a desktop application with Perl, HTTP::Engine, SQLite and jQuery talk was exciting. I've been working toward using his perl-app-builder for TAEB since he blogged about it. The talk showed how we can write standalone, web-based, GUI apps, in Perl, without inflicting CPAN on our users. Because Perl has been a web language ever since the web has had languages, the tools for generating HTML, serving pages, etc. are very good. The traditional GUI tools are mediocre at best. They are why my applications have been strictly web- and curses-based.

Thanks!

Thank you Robert Blackwell, Casey West, Tom Moertel, Dan Wright, and anyone else who helped! This YAPC was fantastic.

If you have never been to YAPC, you should seriously consider going to the next one!

2009-06-17

Shawn M Moose at YAPC

YAPC::NA 2009 begins Monday; it's not too late to sign up! Don't miss the affectionately-titled Moose track.

Extending Moose talk

I'm giving a 50 minute talk about Extending Moose for Applications. The content will be similar to the Moose's MOP series, though focusing more on how all the metaprogramming features work together in harmony. I've been writing the articles so I can learn how to teach the concept and utility of a metaobject protocol.

I'm still writing the talk, but you can have a look at what I have so far: [pdf] [keynote source]. Suggestions for improvement or clarification would be very welcome. I have the "script" included as notes at the bottom of each slide, so you can learn even without me there speaking at you. If you intend to attend the talk (or the Moose course on Sunday) you should probably not read it. I stick to the script, terrible jokes and all. When it is done and presented, I'll post about the final version.

Moose class

Due to health problems, Dave Rolsky will sadly not be attending YAPC this year. He was scheduled to give an eight hour class about Moose on Sunday. It would be a lot of hassle for organizers and disappointment for students to cancel the class, so I stepped up to present the bulk of it. Jon Rockway will be helping as well. I don't make great use of Moose's types, but he does, so we are forcing him to teach the types portion of the class.

I will also be presenting my Extending Moose talk at the end of the class (if we turn out to be short on time). This way you can see Matt Trout's Future of DBIx::Class or Christopher Nehren's CLI apps don't have to suck. Both will probably be Moose-heavy and certainly worth seeing. Or.. you can sleep in!

Please let me know if you have any questions about the talk or class. I've given a few talks before, but never an extended class, so it will certainly be interesting!

I look forward to seeing many of you next week. I'll try to squeeze in a post or eight about talks and all the other good stuff at YAPCs.