sardonick

Twittering provides just enough cognitive release that I haven’t been feeling the blogging itch. It’s so much lower-maintenance.

Spent this last week in NY, for a summit on Machine Learning for work. Had a wonderful time. Networking-wise I found the experience much better than academia (more possibility, ease for future collaboration with the folks I met).

Jetlag did a number on me (going to sleep at West Coast Time, then waking up in time for 8am EST talks…). Feh. Talks were worth it, though.

Met a bunch of really interesting people. It was great to talk to other folks that are doing classification at Google (including taxonomic classification like I’m doing!). Learned a lot (note to self: read up more on gibbs sampling, latent dirichlet allocation, and the RCV1 corpus). Also got to hang out with Chih-Jen Lin a bit, had great discussions about academia, publications (quality versus quantity of publications in China/Taiwan), and how to reform the academic publication system to give better signals as to what is readworthy. Right now, reviewers and conference organizers are the gatekeepers, and that doesn’t scale well. When you think of how many smart people read academic publications, the only way that they can give feedback is to publish something themselves. That’s such a high cost to communicate, it leads to stagnation and monoculturality in the community (echo chamber!). I wish I could easily see, for the profs I respect, a “Papers I read last year that I found really influential” list. Aggregate these and you get great, quantitative metrics of a paper’s worth. Also, “best in conference” awards for papers are so short-sighted; we really don’t know what’s good for a couple years. It would be great to have a “best 2 years ago” retrospective award for conferences and journals. компютриCiteULike starts to address these issues, but it isn’t in wide use and it’s not perfect.

The city itself

• So nice to be somewhere that prioritizes pedestrians over cars. By contrast, Santa Monica issues tickets if you start to walk in a crosswalk when the “walk” sign turns into a blnking red hand. Sigh.
• I’m reminded of Taipei every time I visit NY. There’s a semi-tangible energy in the air of both, lots of people crammed into a small area, everything walkable, and alive, all hours of the day and night. I would love to live here, at least before I have kids. Probably not going to happen, but it would be fun.
• One night, instead of going out, spent a chill 3 hours out on Google’s 8th story balcony overlooking the cityline, admiring the view and talking to co-workers. Amazing.
• Ate some good food there, but I have three regrets: (1) didn’t order any morning street bagels, (2) didn’t get any late-night pizza, (3) likewise for late-night street-gyros. Sigh, will have to come back again.
• Also, damnit, this is twice I’ve been to the city and haven’t yet heard any good jazz!

Conversations at work recently have turned again and again to consciousness and self-awareness (what, you thought “Android” was just a phone? ;) ). Now, I’m not going to belabor the point with discussions of artificial intelligence and yet another amateur’s resummarization of Searle’s Chinese Room[1]. Instead, I’ve been thinking about self-awareness in groups of humans.

A bullet-point braindump:

• As background, remember that short story in Godel Escher Bach, where the ant-eater communicated with the colony of ants (not the ants themselves, but the colony), and ate certain individual ants as a way to shape the colony into something that’s more intelligently connected?
• It’s a cliche’d remark that groups of humans begin to resemble organisms in their own right. Corporations seek after the good of the corporation rather than the good of any of its individuals. Cultures grow, intermingle, reproduce spawning new cultures. OK, so these macro-groups of humans are animals, that’s for sure. But are they self-aware Conscious? Would we recognize it if they were?
• It’s interesting when a group of people who’ve been meeting for a while realize that they are in fact behaving as a group, and in turn have a group identity. Is this awareness of group identity the same as self-awareness in the group? (answer: I don’t think so, this is something different).
• To extend the brain metaphor, imagine humans to be the neurons in a larger collective brain. Urgh, the speed of signal transition along axon-dendrite gap is horribly slow. What effect does this slowness have? Also, humans are damn intelligent signal processors compared to neurons. What effect would our individual intelligences have on the larger structure?
• Would such a self-aware “organism” think thoughts that are entirely separate and entirely transcendent above the thoughts of its constituents?
• Scale? Seems to be the general belief that intelligence is the emergent result of massive amounts of highly, highly interconnected neurons. How many people do you need in a group before it can be considered an organism? A self-aware organism? Is the interconnectedness of humans even on a large enough order of magnitude to support a functionally processing organism? What are such an organism’s inputs, outputs? Would human sub-organizations specialize into computational functional tools, similar to how neurons in the brain are specialized into groups like the PFC, the amygdala, etc?
• I imagine an extraterrestrial coming to the earth, and conversing with society as opposed to individuals. That would be an interesting story. But not the kind of sci-fi that would entertain a puny human mind, though, that’s for sure.

Hmm, I’ll have to think more about this… so many premature thoughts… And most of them the result of only 4 hours of sleep for the last couple days. My apologies, dear anonymous reader, for the unpolished words, the undeveloped concepts, the flaws. “Time past and time future / Allow but a little consciousness.”

[1] (In any case, I love Ben Goertzel’s take on the situation, which, to paraphrase: “When the time comes, and you’re actually arguing with the computer whether it is self-aware or not, then the point is already moot, isn’t it?”)

Two things this weekend:

• Port/migration/(possible downgrade) of my personal wiki to an earlier version of MoinMoin. I kept on getting a “CacheNeedsUpdating” error. Source diving and hack patching was no help. Fix was to go to “data/pages” directory, and run

  find . | grep "/cache$" | xargs rm -rf

  . Basically removed the outdated cache. Not sure if it was internal file format thing due to different version, or timestamp thing or what. But, it works now.

• Installed dd-wrt on my router (Belkin F5D7230-4, version 1010). There were a few bumps (the tftp copy wouldn’t take very well, I had to do it a couple times, and the router took a long while to reboot, which I mis-diagnosed as a crash). However, it all worked in the end, and I’m pretty happy with the results. Even the micro version of dd-wrt is pretty featureful, with traffic shaping, static IP address asignees for DHCP via MAC address identifiers, and other tasty bits. And a wonderful interface, compared with the factory-shipped firmware (this is, perhaps, the first instance in the history of the world when Free software has a better UI than closed software). Open source is a wonderful thing.

Also, this morning, via conversation with Hao-Chuan:
I need to think more about intellectual foraging. Metaphor of information tracking/consumption, based on food tracking/consumption.

• What are internal mental model & goal structures, instinctual to finding food gathering that we’ve now operator-overloaded to deal with information gathering, esp on the net?
• Peter Pirolli (PARK) : http://sigchi.org/chi2003/docs/t23.pdf, http://www.amazon.com/Information-Foraging-Theory-Interaction-Human-Technology/dp/0195173325
• How can we design information architecture, and web page architecture, to make information foraging more intuitive, more productive? (i.e. how can I format my stuff to be more tasty, digestible)
• Tangentially, what if we compare the “tastiness vs healthiness” of information. How do different sites (digg vs metafilter vs reddit vs slashdot vs my delicious feed vs my blog vs my reading habits) fit on this cartesian plot? Pages that are “naturally desirable” are not necessarily the best way to spend our foraging time.

(aside: It’s now August, last post in April… where has time gone? I think I was a lot more motivated to blog when i was back in Academia. The atmosphere back then was a bit… stagnating intellectually, so the internets became my vent. Now, here at Google, I’m in general more intellectually fulfilled, work around great people every day. This is so strange, I thought it was supposed to be the opposite (academia being the haven and nurturer of free thinking, and industry being the great pit of stagnation). Both, at least in my own microcosm, are anything but).

Migrating my life away from the ISI servers, as I don’t know how much longer I’ll have access to them. That means this blog needs a new home. And this is where it will stay, I guess, perhaps for the next decade at least…

My email, too. It’s now nick-at-motespacedotcom. Hosting everything myself, away from university hardware. The old email addresses I had will remain indefinitely, but I’m phasing them out. I suppose it’s good to change things up, but I’m going to miss fairuz, my old server that was sitting on a fat pipe out where ARPANET was birthed and, coincidentally, a couple floors below ICANN.

I suppose this is all a roundabout way of saying that my academic life is unfortunately on a bit of hiatus right now. I’m taking a one- or two-year leave of absence from USC, and am working for Google in the interim. When I come back, I’ll likely be transitioning away from Computer Aided Language Learning (that half-written thesis will be good for kindling next time I go camping, perhaps), and into the Ontology depths of Natural Language Processing.

• Ugh, circadians are all off today, what with daylight savings’ time. Yawn
• Circadian situation perhaps worsened by the weekend’s fun: On Saturday, got together with a 5 other Los Angelean tea-drinkers, and together we held a tasting of 12 different teas, drinking ~50 cups per person. The “theme” of the day was supposedly “light wuyi”, but we sampled a large range, from da hong pao to tgy to puerh to dancong. Will hopefully compile some more sensical tasting notes soon, as it was quite an experience. The thing that struck me most of all was how counter-culture it all felt—how does drinking traditional Chinese teas become a thing like punk rock, in the right cultural context?
• This also marks the second weekend of my decision to take a leave of absence from my studies at USC. I am totally enjoying a life where I can enjoy my weekends without guilt. Between tea on Saturday, doing a disc brake job on my old Civic Sunday, and cooking dinner with Mindy for my folks down in OC Sunday evening, it’s great to be able to enjoy life a bit. Being a grad student offers lots of time freedoms on the day-to-day scale (want to take a random morning off to run errands? want to take a random afternoon off to go to the beach? both not a problem), but it’s pretty draining on the macro level (want to enjoy a full weekend stress-free? good luck, with that impending thesis or conference paper floating over your head!). This new life is a nice change…

Went to see the Magritte exhibit at LACMA today. Thoughts:

• I’d never thought of the parallels between The Red Model and cyborgs/cybernetic enhancement before =).
• Likewise, I stared at The Human Condition for a good 20 minutes, and couldn’t help but thinking about AI agents and World Models
• Everything in the Treachery of Images section made me really appreciate the classes I’ve taken on semiotics/semantics. If I ever teach a linguistics course, I’d like to spend a class day looking at Magritte paintings, discussing the relationship between label, portrayal, and thing, and their respective roles in communication.

As a break from Artificial Intelligence ponderings, I was recently mailed a sampler of darjeeling oolong teas so that I could participate in an “on-line tasting”. Thanks to T-Ching, and Phyll Sheng for arranging the tasting. And, finally, a huge thanks to Lochan Tea for providing the leaves, and for pioneering this new form of tea production.

First, an explanatory note: Oolong teas are “partially fermented”, in that they’re halfway between unfermented green tea and completely fermented black tea. Typically, Oolongs are produced in Taiwan and Southern China. Darjeeling, India, by contrast, has traditionally produced black teas. A while ago, the enterprising and globalizing Lochan growers decided to try preparing their darjeeling leaves using traditional Chinese methods to produce Oolongs.

The three samples I tasted were quite exciting. Definitely a fusion of tastes, different from both typical Darjeeling and typical Oolong, but maintaining enough qualities of each that you can tell that it’s a mix of the two.

I brewed all three of these by Lochan’s provided recommendations (1 cup water with 1 teaspoon dry leaves, brewed for 3-4 minutes, 3 brewings) rather than typical Chinese style (higher leaf:water ratio, shorter brewing time, more brewings). Once I have a bit more time I’d like to go back and try brewing again, but using the Gong Fu method. Just curious.

All three were brewed using Glacier Springs water (good water is a requisite when tasting tea, and I like the flavors that a high mineral content water brings out). The teas were rated on a scale of 1(worst) to 5 (best), with a focus on flavor/smell/huigan rather than leaf appearance.

Here are my tasting notes:

nutty oolong

• glacier springs water, 100 degrees, 240ml, 1 teaspoon nutty oolong
• first brew (4 minutes)
  • leaves broken
  • smell very sweet (over-ripe fruit) with a little grassiness.
  • definitely tell it’s darjeeling, though not as astringent smelling.
  • definitely not “oolong” tasting (or, not like the formosan oolongs I drink). I wouldn’t call it oolong. it’s like darjeeling with that same fruity/flowery overtone
  • it’s quite smooth, but much too light for my liking (love the full body of darjeeling). will try increasing steeping time, see if i get more body with teh same smoothness.
  • astringent huigan with a flavor that lingers for half a minute, but not more
  • dirty gold color
  • score 3
• second brew (5 minutes)
  • maintains color, but despite longer brewing, even less flavor than before.
  • astringency more pronounced
  • honestly, not really enjoyable.
  • score 1
• third brew (4 minutes, 120ml)
  • more vibrant flavor—a little more of the fruitiness has returned (should have only used 2/3 cup water all along perhaps), but you can still tell that the leaves are “used”. astringent.
  • score 2

moonlight oolong

• glacier springs water, 100 degrees, 240ml, 1.5 teaspoon nutty oolong
• (after the prior tasting, decided to increase the amount of leaves per water)
• first brew
  • (4 minutes)
  • very light aroma, not as overtly fruity/fragrant as the nutty oolong. more subdued, more of a “classic oolong” smell.
  • lighter amber color with a touch of green.
  • unfortunately, misses on both the full-body of the darjeeling and the complexity of oolongs.
  • aftertaste is pleasant, pure darjeeling with a little longer huigan than darjeeling. nice. fades quickly.
  • like last, very dry mouthfeel.
  • score 2
• second brew
  • (4 minutes)
  • the leaves have “woken up” now. sweeter, richer, fuller taste. Darjeeling, with a little bit of fruity/nuttiness above it. Still very straightforward, though (not as complex as I’d hope a good oolong to be). Pleasant astringency like a good oolong. The aftertaste is still short and has lots of “darjeeling” flavor.
  • score 3.75
• third brew
  • score 2
  • wish I knew what the difference was the second time, the leaves are back to the banality of the first brew.
  • little bit of sweetness in the taste at the onset, but fades to very astringent, lightly colored water. You can tell the leaves are used up.
  • not too much aftertaste/huigan.
• first and third brewings are disappointing, but the second one was so good, I’d like to try this again using “gong fu” style and see what happens.

snow oolong

• glacier springs water, 100 degrees, 240ml, 1.5 teaspoon nutty oolong
• flowery/nutty smell rather than fruity
• first brew (3 minutes 20 seconds)
  • this is by far the best of the 3. glad I saved it for last.
  • starts with a rich flowery/nutty taste and finishes with classic darjeeling taste. complex and sweet.
  • slight huigan, which is the only thing really lacking in this tea. will try brewing it for 4 minutes the next time to attempt to elicit more.
  • dries the mouth, but pleasantly.
  • score 4.5 . Really enjoyed this one.
• second brew (4 minutes)
  • the 4 minutes did its work, the tea is dark.
  • floral taste still very strong at the onset, but the nuttiness and oolong tastes are faded quite a bit. mediocre finish.
  • likewise still a mediocre huigan (a bit richer than before, with the lingering sweetness from the beginning plus a bit of darjeeling flavor, but ddefinitely not as much aftertaste as I’d hope from a formosan oolong brewed for this long)
  • score 3 (still decent)
• third brew (3.5 minutes)
  • moderate floral taste, tea is obviously faded (perhaps the longer brewings took a toll on the leaves?). still smooth, not bitter. but very toned down compared to the first brewing. The aftertaste is a little better this time, a little more pleasantly permanent compared to the prior two brewings.
  • tea is ok. better than “restaurant tea” but definitely faded quickly compared to the latter brewings of formosan oolongs. that seems to be a trend with these darjeeling oolongs.
  • score 2.5

Thinking a bit about AI this weekend. 30 years ago, we tried to imagine what life would be like in 2010. Intelligent Agents, Strong AI, etc etc. It’s a bit disheartening that the pinnacle of AI that we have to show for our efforts are things like PageRank and phrase-based statistical machine translation.

Not to say that either of these algorithms are bad—on the contrary, they accomplish exactly what they set out to do, and they do it well. But, there’s no magic to them. No glimmer of human-like intelligence behind them. They show us that the major accomplishment of AI for these past few decades is one of statistics (treating measurable phenomena like the trust metric of a website, or how a word in one language corresponds to a word in another) rather than one of intelligence in the more generalizable sense.

I suppose this isn’t a bad thing—the things we build do what they were built to do, after all—but still, the idealistic part of me that grew up reading Asimov and Heinlein… that part of me can’t help but wish that submarines could swim.

I ran across Leonard Richardson’s Ultra Gleeper again yesterday. I hadn’t seen it for a year, and it’s been good looking at it with new eyes since I’ve begun hacking in earnest on machine learning problems and measuring “interestingness” of RSS posts.

The project is interesting because he aims to solve the same problem I do (automatically find interesting/relevant things on the net), but goes about it in a COMPLETELY different way–different in the “automatic finding” and also different in the “deriving interestingness”.

For automatic harvesting of possible material, Richardson looks to a number of resources (not only whatever is pointed at by his RSS feeds, but also whatever is pointed at by what his RSS feeds point at—basically, he wants to search not only his information sources, but also what his own information sources treat as sources themselves). He also harvests from technorati, some custom google queries, delicious (until Joshua got mad =) ), etc.

Good stuff. This harvesting technique definitely broadens the search space. If I look at it in natural language processing terms, I would say it increases “recall”: over the set of all possible interesting articles, looking at a larger set of articles on the whole increases our chances of finding interesting stuff that a more conservative algorithm wouldn’t find. However, if we take this approach then we’d have to have much stronger algorithms that can guess interestingness—otherwise precision will suffer. Now, I want to say that the RSS feeds that I already subscribe to are nearly guaranteed to point to “interesting” pages—otherwise I wouldn’t point to them.

In other words, first-degree information sources (what I treat as a source) are guaranteed high precision low recall. Second-degree information (what my sources treat as a source), by contrast, are much higher recall (quantity order n-squared vs n), but have a hit to precision.

With an order magnitude (or more!) more information, we’d need big changes–changes in GUI, changes in expectation from the program, changes in ranking algorithm.

Richardson also addresses the last of these points–change in ranking algorithm. To compute interestingness, he does something smart that’s almost like a “reverse pagerank”. He says “things are interesting if they point to interesting things” (contrast this with Google’s Pagerank, which says “things are interesting if they are pointed to by interesting things).

The good thing about this is that, once you have some initial seed data, it becomes a sort of passive goodness metric. It bootstraps off of the initial data you provide so that it can continue learning even after you stop providing it data. Unsupervised machine learning, in other words.

Lots of things to think on.