Monday, August 15, 2016

Pushing Python Performance With Parallelization

TL;DR: For certain types of programs, you can take advantage of idiosyncrasies in the Python interpreter and the host operating system to create real shared memory between processes and get some pretty good parallelization.

Premature optimization is the root of all evil.

As a developer, you've probably heard this before, and what it means basically is that you shouldn't waste time optimizing code unless it's already doing what you want it to do. We also live in an era of seemingly unlimited resources with AWS/Google Compute, and often the easiest way to get higher throughput in your programs or service is just to pay for more instances. But sometimes it's fun to see what sort of performance we can get on a simple laptop (and save some cash at the same time).

So anyway ... I've been working on this thing, and it took too damn long to run, and I needed to run it lots and lots of times ... so, it was time to optimize. Basic optimization has two main steps:

1) Profile, to find out where your program is slow.
2) Speed up the parts of your program that run the most.

Seems pretty obvious, but there are also a lot of different reasons programs can be slow. Are you I/O bound, meaning most of the time is spent waiting for reads/writes to and from your hard disk? Are you network bound, with most of the time waiting for network responses? Are you memory bound, meaning your program's performance is dependent on how much data you can fit in RAM? Are you CPU bound? That last one is kinda the best, since the CPU is the fastest part of your computer (not counting GPU and tasks specifically suited for GPU programming), meaning usually the CPU is underused because you can't shove data through it fast enough.

Let's talk about a simple type of program that is very common. It has a big chunk of data to process sequentially, and it spits out a result. Can you fit it all in RAM? Great. Do that, and you've just drastically improved the performance of your program. But modern CPUs have multiple cores, so technically if you could run multiple processes with the same memory, you could get better performance.

But ... Python. Python is not designed for performance. If you really want a performant program with shared memory, you should probably be coding it in C++. But I enjoy the ease and speed of development with Python, and wanted to see what sort of performance I could get out of it before rewriting in C++, so I started digging. First, there's a problem with Python parallelization called the Global Interpreter Lock, or GIL, meaning if you make a program multithreaded, only one thread is actually running at the same time (due to the GIL), so it won't take advantage of multiple CPUs/cores. The solution is the multiprocessing module, which gets around the GIL program by using full-fledged processes, each with their own instance of the interpreter, rather than threads (so you have the memory overhead of an interpreter instance for each process, but the advantage that they can run simultaneously). It's quite simple to use, but this is only really useful to us if we can create shared memory for our big data blob, and have each process read that same chunk of shared memory.

First, let's take a look at multiprocessing. Here's a simple example that creates a bunch of processes and waits for them all to finish. The nice thing is that it's all synchronized and automated - when a process finishes, the pool will start another process, until all the jobs are complete.

As you can see, it's super fast. Mainly because it's not doing anything.

For our second test, let's simulate having a big chunk of data, and passing that to our workers, since that's the type of program we're trying to optimize.

What the ... why is it suddenly so slow?? As you can see, the workers each run nearly instantly (looping over a million items and doing nothing with them is not very hard), but the total time ballooned to above seven seconds. The reason is that the one million object list is a local variable that's passed as a function parameter to the workers. What the Python interpreter will do internally is pickle the object on the calling side, then unpickle it in the worker. So not only do you get a copy of the object for each worker, you also get the expense of pickling and unpickling. No good.

(Quick side note: Python actually provides "shared" and "managed" objects for creating shared memory among processes. However, for what we're doing, they're extremely slow, since every access is proxied and made process-safe. They also take a very long time to create.)

Fortunately, it turns out we can do much better. On Unix-based operating systems (i.e., Linux, Mac OSX), we can take advantage of the operating system's "copy-on-write" forking behavior. This basically means that when the process forks, rather than making a copy of the program and its context, it will just use the same copy of memory unless values are changed, in which case it will create a copy of the changed part in each process (that's very simplified, and OS profs are probably cringing, but it's the basic idea). But how do we get our data into this "context" that's shared? Easy - use the dreaded "global variable":

As you can see, it runs fast again. Note that now our "data" object has 128 million ints rather than a million, so we can get some process time in the workers. And it's not creating a copy for the forked processes, because that would take a really really long time to pickle/unpickle (trust me). But, we introduced a new problem. Now we have a huge global variable, which is really poor form, since it will be initialized if our module gets imported. We also probably want our data to be created based on some program args, so it should be protected by a function or something. But we saw that if we pass the data to our workers after it's created in __main__, it gets copied. How can we solve this problem?

There's actually a simple solution. If we define our worker function in the same scope that we build our data object, then the worker function will have access to it, and the Python interpreter will put this in the "global" context of the process. Thus, it won't be copied like our local variable from our earlier attempt. Let's also increase the number of "jobs" from 10 to 100, because it takes a bit of time for parallelization to "warm up" (likely due to caching, which is a whole other topic for performance optimization).

As you can see, we're getting true parallelization now! With one worker, it takes about 20 seconds, and when we double to two, we get nearly twice the performance! Also note that my Macbook has a dual-core CPU, but four virtual cores ("hyperthreading"), so going from two workers to four only gets us a small increase in performance.

And that's it! Now you have a highly parallel data processor using shared memory in Python. The above examples were specifically run with Python 2.7 and PyPy on OS X 10.10.5. Note that PyPy and CPython behave somewhat differently, and will crash at different places in the automatic pickling/unpickling of objects. And also, apparently this won't work at all on Windows, because Windows will copy the entire process before forking. But if your program is constrained in the way described above, and you're running on 'Nix, you can actually get some pretty decent performance!

Happy parallelizing!

Friday, July 01, 2016

On Language and Getting from Here to There

The Atlantic recently published an interesting article regarding the difference in "efficiency" between languages. The basic idea is that some languages, such as Mandarin Chinese, are very efficient in conveying information and possess linguistic shortcuts such as eliminating gender and tenses and collapsing "he" and "she" into a single pronoun, whereas other languages like German are quite verbose and precise, having different articles (i.e., "the") for different gendered nouns along with plentiful verb conjugations.

They also touch very briefly on the idea of directional complements, which I've always found to be a fascinating difference between English and the Asian languages I've studied. Basically, directional complements are words attached to verbs that show the direction of movement of the actors. This sounds like an obvious thing, but it's actually a common pain point for English speakers learning Asian languages and vice versa. English also has directional complements, except we're maddeningly inconsistent with how we use them. For example, the verbs "to come" and "to go" have very clear intentions, yet we flip them around in a way that confuses the heck out of Asians. For instance, I can say, "I'll come home at 10pm", which means I'll go from where I currently am to home by 10pm, but what I really mean is "I'll go home at 10pm." Similarly, once I'm home, I can say, "I went home at 9:30", which is technically correct from the perspective of your past self (at 9:30, I began the process of "going" home from wherever I was), but from the perspective of your current self, you came home (more accurately, started coming home) at 9:30.

In Korean (and Chinese, for that matter), the verbs "to come" and "to go" are always relevant to your present location. If I'm at the office, I'm not going to "come home", I'm going to "go home". (Of course, every language has exceptions, and Korean has the strange and glaring exception of saying "거의 다 왔어요", which literally means "I've almost come all the way (there)", and is given from the perspective of your imminent future self rather than your present self. Language....) Furthermore, if you're "returning", you "(go and) return to" (돌아가다) if you returned to a place that you're no longer at (e.g., "언제 샌프란시스코에 돌아갔어?": When did you return to San Francisco? means I was originally in San Francisco, then I went somewhere else, then I went back to San Francisco, but now I'm somewhere else again), but you "return (and come back) to" (돌아오다) if you returned to a place that you're still at (e.g., "언제 한국에 돌아왔어?": When did you return to Korea? means I was in Korea, then I went somewhere else, then I came back to Korea, where I still am). It's a seemingly subtle distinction, but it makes all the difference for sounding natural. And if I started at place A, went somewhere else, and then came back to place A, there's actually a beautifully unambiguous grammatical form for just that: 갔다왔다. E.g., 일본에 갔다왔어 - I went to Japan (and then I came back). In English, I'd say, "I went to Japan", and leave it up to the listener to decide if that means I went and I'm still there (in Korean, "일본에 와 있어" - I came to Japan (and I'm still here)), or I went for a bit and then came back.

"Up" and "down" are also used along with "come" and "go" in a consistent and disambiguating way in Korean and Chinese, which is quite nice when you think about it, but again, confuses a lot of foreign learners. If I'm on the ground floor and my friend is in her apartment on the tenth floor, I'll tell her to "내려와" (Literally, "Come from a higher place down to a lower place", or "Come down"), and she'll say, "내려갈게" (Literally, "I will go from a higher place down to a lower place", or "I'm going down"). Yet the American on the tenth floor who says, "I'm coming right down" is again using the reverse of the proper direction from her present self's situation.

Anyway, the article in The Atlantic goes on to ask the question that neuroscientists and linguists have been asking for centuries - does language affect the way we perceive the world? This is the whole "Eskimos have a bazillion different words for snow [so clearly their perception of snow is fundamentally different than ours]" argument, and I'm glad that the author brushes off the differences in perception as being "true to a faint, flickering degree [that] a psychologist can detect in the artifice of experimental conditions." While I do think that language and perception are intimately linked, it often feels like experimental results in this area are grossly tinged with ethnocentrism. Scientists who speak a certain language (and let's be honest, it's typically Western scientists speaking English) love to make broad generalizations about foreigners who speak languages they don't quite understand. Eskimos do have a lot of different single word adjectives for snow, but does that mean they have a fundamentally different experience perceiving snow, or is it just that they happen to live in the snow? Chinese doesn't distinguish between "he" and "she" as a spoken pronoun, but does that mean Chinese people don't perceive them differently? The answer is obviously not - in fact, while the sound of the words might be the same, the characters for "he" and "she" (and even animals or "it") are different, and thus different sets of neurons are firing for each one.

So, is there any way to know what it's like to think and perceive the world in a foreign language? Well, actually there is, and it's pretty simple: just go ahead and learn that language, and then examine your own thoughts and perceptions while speaking it. Come on, waiting on you. ;)

Tuesday, March 15, 2016

AlphaGo and the AI Revolution: Is Natural Language Understanding Next?


Here's a guest post I wrote for a talented friend's linguistics blog, about AlphaGo and natural language understanding. Does the rise of AlphaGo mean that human-level machine translation is just around the corner? Or is language another beast altogether?

Enjoy!

http://saramariahasbun.com/2016/03/15/alphago-and-the-ai-revolution-is-natural-language-understanding-next/

Wednesday, February 24, 2016

Dumb AI Will Kill Us Before Superintelligent AI Has A Chance

Humans have been fascinated with the idea of a robot revolution since we first conceived of robots. Long before Skynet and the Terminator were imagined in the early 80's (?!?), Asimov wrote of the Three Laws of Robotics, warning us of the potential for intelligent robots to do us harm. Recently, with the rapid expanse of deep learning neural nets in the last few years, we hear more and more about how an artificial "superintelligence" (i.e., beyond human intelligence level) could wipe us out with as much difficulty as a human stepping on an ant. While I do think that superintelligence could potentially be an apocalyptic threat to humanity if it were ever to be created without proper safety built in, I am actually way more concerned about what I like to call "dumb AI". Which honestly can kill us right now if we're not careful.

What's "dumb AI"? Well, artificial intelligence, or AI, is a buzzword that spans many different technologies. The media tends to call any technology that does anything sufficiently complicated "artificial intelligence". In computer science circles, it means something different, but for our purposes, let's go with the popular view of "technology that does something complicated that probably used to be done by a human" (I am aware that this is incredibly broad). General human level AI would be a system that is as intelligent as an average human being. Superintelligent AI would be a system that is more intelligent than a human, perhaps by orders of magnitude. So "dumb AI" is what we have now - systems that are good at isolated tasks, potentially even better than humans, but that have no general intelligence.

Autonomous cars are an example of "dumb AI" - they're good at the single task of driving, which they handle by combining data from different sensors such as lidar and cameras, building a model of their environment, and using that model to follow the road and avoid obstacles. A difficult task, for sure, but your Model S is never going to become "self-aware", because the only thing it knows is how to drive. It has no opinions on current events, it can't make you a coffee (but one day it will probably drive itself to Starbucks to pick up a coffee for you), and it wouldn't hold up very well in a Turing test. It seems really "smart" that it can drive, but it is designed specifically for this one task, and has no general intelligence.

Teslas are amazing machines, but let's return for a moment to the fictional world of Terminator. In the first Terminator, a super strong, intelligent cyborg takes a hop in a time machine back to 1984 and starts kicking ass and taking names. This scenario is unlikely in the near future, obviously because of the time machine, but also because we're pretty far away in both robotics* and artificial intelligence. It would be cool if text-to-speech systems could do an Ahnold accent, though.

Terminator 2 was even more unbelievable - liquid metal time traveling robots, somehow with enough distributed intelligence in the molecules of metal that you could blow the thing to pieces and it would magically come back together. We're nowhere close with nanotech, sorry.

Which brings us to Terminator 3. A quaint little movie from 2003. Check out the start of the war:


An engineer reports Skynet "processing at 60 teraflops a second". Well, that was actually surpassed by a single supercomputer in the 2004 timeframe (http://www.top500.org/), and distributed systems have way more combined processing power, although processing power by itself isn't enough to incept intelligence. More important than the depiction of Skynet's self-awareness, check out a scene from a few minutes later showing a killer drone called a "hunter killer":


In 2003, this was pretty fantastic to see. A drone with heavy firepower and automatic targeting! Holy shit!

Well, guess what. All the technology needed to build that exists right now. It might have been Hollywood effects then, but we have it now. Quad copter drones with precise maneuvering capabilities? Check. Automatic targeting systems? Check. Slap some guns on a beefed up quad copter and give it a relatively dumb computer vision system, and you basically have the hunter killer from Terminator 3. The main difference is that there's no superintelligent Skynet behind it.

But actually, that's precisely my point. The technologies needed to accidentally start global world-ending conflicts are here, not tomorrow, not today, but yesterday. Check out this "automated turret" system from 2010, which South Korea wanted to put on the DMZ (not sure if they ended up installing it). http://www.gizmag.com/korea-dodamm-super-aegis-autonomos-robot-gun-turret/17198/ Here's the breathless videogame-like description from the article:

"The Super aEgis 2 is an automated gun tower that can find and lock on to a human-sized target in pitch darkness at a distance of up to 1.36 miles (2.2 kilometers). It uses a 35x zoom CCD camera with 'enhancement feature' for bad weather, in conjunction with a dual FOV, autofocus Infra-Red sensor, to pick out targets. 
Then it brings the pain, either with a standard 12.7mm caliber machine-gun, a 40mm automatic grenade launcher upgrade, or whatever other weapons system you want to bolt on to it, including surface-to-air missiles. A laser range finder helps to calibrate aim, and a gyroscopic stabilizer unit helps correct both the video system's aim and the direction of the guns after recoil pushes them off-target."

The BBC covered this company again last summer: http://www.bbc.com/future/story/20150715-killer-robots-the-soldiers-that-never-sleep. Supposedly, the autonomous firing mode has been turned off, but that doesn't make this tech any less terrifying. First, it can simply be turned back on if requested by the client (apparently, mostly friendly Middle Eastern countries...). Second, the things have a friggin' network connection, and Korean companies are not very well-known for their internet security (sorry, every Korean bank). Can you imagine an automated turret with a .50 caliber machine gun being hacked?? Additionally, the tech for this is honestly not very advanced. It's an infrared motion sensor combined with a turret. It has no way to distinguish friend from foe - it just shoots at humans. You know, like in Terminator. The "senior research engineer" doesn't inspire much confidence, either, stating,
“Within a decade I think we will be able to computationally identify the type of enemy based on their uniform.”
First of all, talk about lowering expectations! I could build a very accurate classifier today using open-source technology and my four year old laptop that would be able to accurately distinguish between South and North Korean soldiers based on uniform. Which is fine and dandy, but if slipping past an automated turret is as simple as stealing some South Korean army uniforms, then you can see why this is "dumb AI". What if this thing accidentally fires a bunch of RPGs over the DMZ due to a false positive in its vision system and starts WWIII?

Actually, buggy vision systems have nearly killed us already, many decades ago. Here's just one declassified incident from the Cold War that we know about: https://en.wikipedia.org/wiki/Stanislav_Petrov. The tl;dr version is that the Soviet nuclear missile early warning system erroneously detected a launch of ICBMs from the US, and Petrov decided that it was probably a mistake, and thus arguably prevented a "retaliatory" nuclear volley from the Soviets which would have turned the early 80's into a Mad Max apocalypse. The problem? The Soviet early warning system got confused by the clouds and some Soviet satellites and pegged them as incoming American ICBMs. Whoops.

So, while most people are worrying about their IoT coffee makers becoming self-aware and inciting grey goo scenarios, I'm worried about the tech we already have. Drones are getting better and better, cheaper and cheaper. Right now a motivated individual could build a Terminator 3 style hunter-killer with a modified quad copter, some light guns, and off-the-shelf vision systems. On the other end of the spectrum, "real" drones such as the Predator can run autonomously without too much difficulty, and those pack serious firepower. Auto-targeting turrets on the DMZ and other high tension areas. Increasing autonomy built into other parts of the military and supply chain. Autonomous systems acting on the basis of requests from autonomous satellites. Intelligence based on imperfect vision systems. We're putting way too much power in the hands of systems that are extremely precarious. And the consequences of a mistake could be devastating.

Artificial intelligence is going to change the world in fundamental ways. We can't even imagine how transformative some of these changes are going to be, and the next few decades are going to bring scientific and technological advances that were considered science fiction not long ago. I just hope dumb AI doesn't accidentally kill us before we have a chance to see it.

--------

* Of course while I was writing this, Boston Dynamics released a new video of their humanoid robot.


Slap some guns on it, along with today's best facial recognition system, and you basically have the T1. Give them a network connection and some goals (i.e., kill humans), and maybe we're not so far away from the Terminator vision after all.

Monday, July 20, 2015

Canterbury Tales Neural Network

I trained another RNN (multi-layer recurrent neural network), this time to generate poems in Middle English in the style of Canterbury Tales. It started generating interesting stuff after just a few minutes of training, but I let it finish anyway. I noticed in one of the samples that it had generated a title, so I seeded it with that title, and sure enough, it closed the poem eventually and started a new one. Kinda cool. The numbers are line numbers (you can see they're obviously not accurate), and it generates footnotes, too, since they were in the source text. The indentation and spacing is all generated by the neural net, too.

AUCCIATES TALE,

  This walmeth have,' quod Melibee, 'by see                   760
  For it was grave, and of my voys I may,
  To hir, with-outen fond to wedden she                         285
  How that a man unto pituk than;                              1160
  Til that were I his messaille aboghnis.
  For, in the sovereyntes ful marchant
  To me, night · ordeyned his peyne.
  This no-mune answerde, as in the mille!
  Ther ny spak, with many a special massthe.
  His body frelet mot I telle as fan?
  Ther was the bus he dide him in share;
  Lest is goddesse and his mulligat
  But he ne ben a fete wolde cosindyse,                         1890
  This fro the wrokes and his cours, and drinke,                  3820
  For men tolde love--bit he was of anhto                            1555
  Than scal, unto the north, wys '-bounder doute,
  Whan thoughte gon me in the bixardes,
  That to hem don to take now heer burde,
  And I remel for to mete for us,
  Out of my proverbe, 'as they, lokkes as,
  Of avento to your octileince
  Of whiche may the hawe of avoutry,'

    184. Hl. for; _rest_ thus.   140, 4. Cm. mak; Cp. Pt. Ln. Hl. maner.

    Lold carus, eft he mighte nat to yow hir lawe.                         1710
  Er he may wedded by his messes weyle!
  For later, she was fete · stinten it usave
  Reed right to him as fond in his care.
  And hydeth, was thy streng · turneth a claye aferve,
  That owne praye bounseth wery awey.'

    3684. Cm. Hl. Thansh.

  This Palamon he bitide, here up I woot?
  With whot his name turned to stroke,
  God wake and from obeyes also a best;
  Up werketh under burel in his tonge.                         610
  Ne made scondre for an eketh al
  And bad a marchal worthy and sent at daye
  Which never sinne, this is te biforn to ben
  With dwilkketh wel; Meclege answerde.                       10
  I yeve me yow to vysed and pleyn,
  That his gite is hadde an half, no violed ende.
  And forth wol I, than ther hanneth in a clere,
  And of Mavounses hadde a stoond fempcethe,               (720)
  Me verraunt on his hors, the Irw and shirke
  As, by the fleisons of a dignitee,                           140
  She wel ful of nature, and sone can,
  God save hom al that ech man y-comen noon,
  And sprikketh and therfore of place of bedde;
  The peple is in drank for to be my three!'

    3588. E. is; _rest_ lyf.   475. E.
    of; _rest_ Sumbiald (-now _is an est godesse                                615
  No brother him unturs he hath colessoun,                   165
  My ricersable, if ye had espyed me,                           95
  Up-on hory grete floor, wher was on yow preye;
  And I comth by biholde a litel wit.
  Ful ofte tyme, thurk he comen of hem be mighte,                 (390)
  Of silknesse my lord first caste throwe.
  Allas! the kin he bar to other wente,                        800
  Be comuned but ful often save, o writ
  That, if yow puccheth thus, his host at me.
  And leyne of his deeth, hir Boundes shape,
  Hir lady speke mete, and my turned al,
  Sow that, with-outen same the worthy finde;
  Therforthe why? faste yow y-woke,                        : 2950
  This lak may may be doun · of shewing bifore                     (311)
  That thilke, and smyle that I igive I;                     310

  This hand, ther is no croyn, if that my cave!
  In hering rancour wasship of twenty yong,
  The laste tokinge wonder yow yow wel,
  That him encheseth us a tecen or bath;                         (410)
  That I sin Penitence is so greet mercy.'
  For Nothewe menelly the book seith,
  'By procest'ded, 'is," quod this povreteon,'
  Somme be your wyf, bothe as wel lemed--
  Nat who-no coude castle youre foule;
  And in this siker Countel on his might
  The mighty siker, as it be desyre;
  Al were I here woot thus, he nolde secree!

    3. Ln, heer; E. on wompath, wink, un-toon, namidye.

[ OUDun
  Biforn that ben weren knightes Cristsaryd,                        1150
  'Biforen on him-self, her lyf was bindinge,               1145
  And vertuous and bisogeth him ful man,
  And hadded he so muche, or of my smoot!
  But with his song though he sat doun wisnesse;                    195
  Eut of the carage and of his dampnacioun.
  Dan him the eres on the livete or love,
  Another dyeth she on hir wille served.
  The Iange wyf, made also wolde hem;
  Yit in my cors of Attenaces
  Of feet at Almachius lerned by,
  His wyf hadde hong, on heven, by fleer lewe.
  Now wel enqueringe, by it, and for to fron,
  That he loked hid no mone his preyere,
  Ne feete, on his parfit up or mayde.

    445. E. Pt. barment; E. Ln. camully.   386. Cm. capede; _rest_
    allone.

  [424: T. 9095-90x7.]
  And al his sterës dyed, he made uttinge.

    1901. E. _omits_ a.
    1167. Cp. Pt. Pt; Hl. nought.

    'Thing we knowe it axe a parcour,                    1275
  That thus of lord I shal bigentes here.                         1680
  For which he hath no wight han I shal falle!                    735
  Under the goddes hous, and artow for no wymes,
  Werre is we demen hangoghtes to be.
  It never sette up-on the tendrely terke,                         3605
  Gan-coven his maistrie, with-outen crye,
  And elles by any corrymed to tweye
  Of an apgoterus of our lede,                                 1430
  For elles is ful ofte of peyne of noon,
  That they doon his gonsel and a belia blarde,
  Of good have fortuth, and leef and freends,                          830
  The hous wyse world, or wroght, out ofte, is swour.                  160
  [60: T. 3043-4394.]
  Both bything is best in this matere.                              1240
  Daunce, and thanke he iman with Gellige,
  Him thoughte his aventure he hadded it his shap,
  And whan men was as ever he put right wel;
  This litel of my wyf, it sholde breke bifil;                       1060
  Ne goode and only for swich a grace.

  O, Reder was y-presed, for the booghte,                      1525
  Reed giltly, and pouce, was on hir hewe.

    133, 1. Cm. goode.

    'Now, fyr a blake is vers muche hir governarnce;
  A mour, by this word of may reverich
  But-if that under troithe men also,
  Shul no maner mannes now in Cristet softe.                   1315
  This is a cartimwest, by-ooven save.
  For as your owene lord yet knowe he,                      OUT""    HERE ENDAT. AUCCIATES TALE.

       *       *       *       *       *

THE SEIPES THE MEMELIUM OF PISB.

HERE UDORES PERCUIS TIS ThiT BKERE.

      NEiBIGIEES TALE.

  By god, to daunge other in heven gentris.'                         4265
  And that they cart but o wol nat suffreth hone
  Withwomth an ydel purposour
and have this castel do.                      1550
  Ther shal me al my declared al this:                          1365
  [480: T. 9548-9753.]
  How we povre and al his aventure,
  If that oon on a mayde shewe aspecioun
  To slone in clot or to doon in vryde.
  And Whan they wolde y gebstable and blisdes,
  Whan that he bigilde me nat othing swate!

  Hir heigh but thou sholde have a knight
  [389: T. 1307-11200.]
  Lat thing, ne in your-self, far waries in the sinne.,
  Lenk in a partee in foryefelleden;
  Til he carled som · if they he loked me.
  God seile, umbarie, quod he, 'and fals so streng,                  2315
  Madler deer, gan vertu and of [Sa],
  And semblonwes and in he restenbed,
  Ne nat, now whan he neysed is me soot;                     (630)
  This Gamelyn is the ciftiesse of sonne,
  That stirme feendes weren as deed in bondes.                   1165
  With-outen hewed here meyne y-nough bityde,
  [88: T. 2904-2522.]
  Than litel hert be mawed alwey bigonne;
  I mighte men mene, as Envitatee                               595

  And seyde, 'it shal never falsly also;                               235

...

And so on.

Here's another, with no seed text:

Of Caunterboures deboname, and cloth,
  That hath hir housbond in the walles tale,
  His fruit I seye, his liste, lay, laughed                          4090
  Of his stories wroothly for to we;
  Which that hir mokle in this tresor foon,
  They looke y-nat usente I, foo dide or enfore
  Thou art to haven ful many a yere hond.                     (810)
  And up answakned and in his thoght!
  But natheler swete of this manere,
  Who, I luve me whul be mighte nat teille;                   (470)
  [404: T. 3148-9454.]
  To telle hir herm; me in his muncen save.
  For muche after any good of gyle,                    75
  Wol can, wrastthe thus, and my good rekne,
  And lyk to the game a that can paleme.
  With prively servaunt the sire,                              [T. 153.

    Leurne in this prove 'another coste,
  With wounes maken him herte any reve.
  O doketh frend His same swoken rorie,
  That half that he toweling of thilke lyf
  Purstaunce, came, thus no wight so geste him bothe?                 120
  Ful often man was fully of hye a knee,                   (625)
  An al be in, 'fer right anon atto,
  And that my daf to stille in sideraynesse.               (890)

  And al a fille aboute hir herte I quikke,
  For yeven maken herke, bothe gay the to see.                        1310
  And if thou me that gerle sone quat he;                 1155
  [201: T. 11211-12202.]
  Thy minstry, out of honour · trewely unlost,'
  'Who sente be faire, of his cherl no bere.             435
  This syde-ward she trusteth over he wente,                   (10)
  For in this curture riche; and the firste knight
  She schul I lyke gay be thoght and ay.
  And Auchaus han devaunce buk minde,
  That I was lye, and warisshe for to preite.
  This walken as the cladres wilcow attenly
  As he wentweth er, they tendren bisinesse
  Witnesse, avyseth to the hevenes sighte,
  Til colandre al this carpenteres fade,
  Yit nored_, as he was a greet dayes--
  'Adam,' quod she, 'E. manying in the rook was rede;
  As Plutage mayntaccioun heend pais.
  But this this cocauntilitee of Diendrye,
  And hoom was three, as repetse as hir therfonge,                      1250
  But soth the cherl-pirgrew wel he lay
  Bireveth man; and after his weddes say;
  With-inne his come a geyn with preching is,
  Dien in so fair, as laughed lecherye
  The dissembre it nosperie flasoun that malence
    Another beste, I saystom blessed and roune;
  In whiche they mo broon arrogeth you blak.'

...

Sunday, July 19, 2015

A neural network that writes Scalia dissents

I trained a recursive neural network (https://github.com/karpathy/char-rnn) on a bunch of Justice Scalia's dissents from the past few years. It spits out some amusing stuff, depending on the starter text and how "adventurous" you want the output. Since it's character-based and not word-based, it makes a bunch of spelling errors (unlike Justice Scalia), but is also able to create new words (just like Justice Scalia!). Here are some samples.

***

Starter text: "Justice SCALIA", random level: 0.8. Never would have expected this from a strict constructionist (check the first sentence). This one brings in same-sex marriage, constitutional interpretation, and the typical contempt-ridden air quotes.

"Justice SCALIA, dissenting. 
The Constitution is an opinion, and so views that "[t]he Court tait the structure relations (interneline) rejectly and weands is not categorical, while all this one inference to do be not applying a nample between the fairest regulation in the entire Regulate Called to his regulations." Ante, at 2-92. The own sittingt-assume industries are strigges not. See United States v. New Yorbo, Manyal, 508 U.S. 208, 368, 106 S.Ct. 2508, 41 L.Ed.2d 669 (1999)) (SCALIA, J. The Jouncis Col.Reverson (quoted Nationary 7, 49-585 (Mouns)). But the state exception that its pollutants so whatever his inoper affected by the United States does not avoid a bullenaby AEDPS?) the time of two applicable, however, such as the object of how question of its own interpretations must was "provided to achieve a federal strict system is the section was the Act's state reading of Fremed Read Iractal consequence of the statute in the regulations with the consequence was the present completed by the officials were reasonably every question—think if that attribution of the one would have dead the truth for an industrial rewilting records if it is arrested that the Federal Exchanges whom the speech at all strethed by the time-of-tax 1970, while so what the new same-sex marriage terms (whenevel is an internal assistabliness based on the State." The Act what the occusion said that he can considered to the offered foreign Congresss about predicted nor conturment like "scheme." 

"Justice SCALIA", 0.85. Talking about all sorts of stuff, including "pregnancy discrimination".

"Justice SCALIA, with whom THE Loore, would have into its retail to contain a significant constitutional jurisprudence. That is not maintenance of pregnancy discrimination to take insurance confessionary basis:
"We simply compels. 
How disablich that the Federal Government's approach a plafication was not only the pre many cases when they more than a statute at tax creates that "[t]he word "substantially unwilling" facts in the contents of the approach reductions. 
On the opinion has official before them. We have had standing. But that Congress might "grant on the [tore] later insurance basing plainness phrase." Ante, at 2506. But that is about a but used to timed with, for empless believe that "the amicisprisably rights in state law's detention with my, most. In Redels v. United States, 494 U.S. 479, 455, 96 S.Ct. 2767, 97 L.Ed.2d 837 (1996) (countench, the Court's business did not require relationship to may be same-free immunial procedural reality of the Commission that the release is impossible to the program based on the statement." Beaws that the tipster only another persuasions of evidence of Administrative United States v. Parrentes, 51 Can. Thie, Art. 607, 30, 102-11 (1896). I would all sense. 
The Court itself not a substantial to the contrary interpreting the requirements, identifying our precedents to be not soual into the society was unconstitutional for the deal services on the neighbout's interpretivens in their conclusion. Acrost-intitule to mentior informations is about establishment here. Does until Rule v. State Likeing § 6 believes that their liberty is not an effect to apply the number of each anyone it mottitung of "required).""

"Pretentious", random level: 0.95. I love how the second paragraph starts with such a forceful statement.

"Pretentious permits newly judges that the law constitutional holdings of national today. It is incriphic was so houselied herehplace. The conclusion that Maxwell's laffic between the common leow police the plaint's States as its highly holding see would have threatening substitute an assumate trade, see 22 U.S.C. § 931(b)(3). 
But that it is unquestionably. A New York has forenes present, the is not stend what prosecute results of hore, the issue at interpretation can be saying eyting "`arress botill to make matters" retain that, and funding or justification by 1986 that Congress extens its avoid participated billion, in view materely "subporablith" Medicaps to litigate influsts of nothing mandated to most compel else. Ante, at 11. So the Attorney General's claims questionable "based" in the Court's bearer warrantles objective and permit in the naturt is produced by reason on the trial court claim that rescre the statute state discussion of they asks not necesses 10 offer of Scwho Reaster, Felibe Haurt v. Zasking on their constitutional functions in the Antitory's defect the judiciary reading recorded by a new distinguisless laws in this theory to think the linef "Act] and it," for the data passpose. 
The Court of Appeals courts two bring the great registrues, error in a way to ensure the Syet or received or appropriate rule. That its respect to the Court does not just establishes constitutional: "colticial legal treat's powers to be relevant about settled regime to what matters would discorments and promotive with the Act's invalidative amserting an entry."

"What a parody", random level: 0.8:


"What a parody of an initient fact that Arizona affirmatively regulated: The Act be expressed that matter of according to say that he says no penalty discharges were meaning of the attorney error was related to far as a complaint that the Ninth Circuit, the Court underpresent a vigwated criteria state area, it was reflexted to no law that the context of a separate law benefit of loso shouse before uncondition upon those precisely the entire of all cosest as the correct according affected billiple because the Court had unintw what Congress was made by the Court's decisions of among the trial courts or tepporable terms of the Court declaring where "advised entirely tax removing the health production judicial state" requirement that a false of the unconstitutional facts of the context in power to reason to be uncorpective of a court may be unconstitutional." Consistence of People v. Weleso, 3 Cranch 23, 278, 488-131 (1998)(7) (SIV). As Congress of these vermison accepting in the law authorially does not entirely with the prohibition only to make of citizens than whether it meaning for the trial court of its action of context of a second of the substitution were standing a provisions that it means that when a way is an insurer that these whole testimony makes a has for other claims by Congress has validity, in the presumably justify the EPA, on the industrial hardly concluded the Court's) federal issues, and whereas many rule and debate. The failure search new theory is may well injury, who seek did not demonstrate whether the Act at at our certificate of an interpation was described the entire 1998."

"You would think", random level: 0.95. The first sentence up until the quote kinda makes sense.

"You would think the legitimacy thing prohibited in the distinction is simply criminal inquiry, "which this Court's acts a "persons whose pre-existing, and the election which retrostened with the DNA has alver: a contracts criminal criminal criminal courts, which excluder implication wasd delay anytorney federal gur to be saude that is own, it takes who leave the precedent of an initial provisions that equal-riscretion. Since forbidable to pre-enact work to the legislative powers against Grabe-special v. Palusheent, 184 U. S. 413, 96 (1968). The provisions of this case, never be marketur judiciary justify the purpose of a matter of U.S. Sovily representative with Magna Cartz, 3 Dyanchests)."

"Pure applesauce", random level: 0.85. My favorite new word in this is "decoking".

"Pure applesauces and other whole must be an bendailing the prohibition my say allowing responsible for the entire of the concept or continue to cannot be cherd, nuble of a precedent or recognized it even any failure in a criminal statute that establishing for records the State courts a habeas called the property of the Supreme Court may is the decoking the same-treatment proceded such appointed by the enactment] affecting the his constitutional oright, whenever the entire Circuit's detention interest out of the courts or provision anything was referently, because it as textual regulatory justice avoid of the higcies jurisdiction remanding taxes in its percentage. It is not the thousate those whole employment-mates of ability to relevant the jurisdictional conduct and other factor of a meaning of a dignity claims activities (inexplicable criminal viltely." Ante, at 14-51, in v. Nownnephshor, 287 U. S. ___, ___ ("SC. . .).
II: This case that describes Congress had advire foreign all who eqhevel undermine the statute to the attention to pre-enactment of the opinion the decision scheme to do so."

"The overriding principle", random level: 0.85. Apparently the "overriding principle of democracy" is the substantial cost of a firearm. "Assing" the prescribed, eh? Clever verbiage.

"The overriding principle of democracy action this substantial cost of a firearm. Application would have extraordinary from a fireard health-insurance considerations in respectively undermined to include Assing the "prescribed" in the Court conceded against sovereign powers") (2012) and Storch "a man private parts of state law based on asquictions — and can adjudge deal was no State's reading operates." 

"Who ever thought", random level: 0.5. At this level of randomness, sentences sorta make more sense, but go on forever. Not that dissimilar from actual court opinions.

"Who ever thought the Court's employees would have destroyed in the entire of a prevailing the majority-minority districts. And that is an incremental proceeding is an industrial review counsel that the proceeding is not the exclusive assistance of the pre-emption and industrial country and powers that the prosecution that the Court's opinion requires the action when the Arizona should not have the federal musus credit and standing to enforce the Fourth Amendment stands it that a state control of securated coverage in the decision to the concurrence of the First Amendment is because the decision mandatory minimums through the provision and that we have no purport to have the Framers were asserting about the time areas wholesale relevant to foreign criminals in the context of the Fourteenth Amendment is a contrary restraint."

"Perhaps", random level: 0.85. States' rights, baby.

"Perhaps the law infringed by the Alabama Acts and rights from coverage in a State suggesting its cases like this case is other powers or achieved by the guarantee correctness to the States — the normal law as a complaint that the Ninth Circuit, the Court v. Norkes, each cause of the California driving search power is the electoral interest." 

"I suggest", random level: 0.88. You can just feel the contempt dripping from this one, especially due to the mistakes of the "personders of contrary" and the "delicitations" of the Ninth Circuit.

"I suggested changes to reject the cases of change holding) none of the capering the other personders of contrary, nor "delicitation" of the Ninth Circuit siteed held that the Federal Department precedent that rather than the relevant is equally "trup estor through the precese when "[t]o accept the property of the Transport Rule 18(b) which documents to the DOA disclose schools, "nonjudicination of Floridaps "convicted contract regarding about all when the law injudicial soupce, when the purpose of interpretation on the Appehedated Dectivenes of Jo.s 26 U. S. C. §209*20A in this criminal v. §37 espord or "manafestorize that a holding would "involve other warrantless" enacted comes to dismiss gives relief.
A claim relationship as ever the upheal possible that the prohibiting "[I]f which is acceptly choices (intenting governmental conflict would not] held that even pick form." 

"Same-sex marriage", random level: 0.4.

 "Same-sex marriage of the proceeding has not shown that the criterion of the Court's opinion spending and the Act did not establish which the caption of religious accidenty of the new same-sex marriage in factor principle that the Court says that the Federal Government can be relevant to the constitutional provisions of the natural relationships because the Court change the new district court's decision to the problems on the Court does not present an unconstitutional proceeding that the Court decides the United States v. Manyna, 537 U.S. 373, 513-534, 114 S.Ct. 2748, 129 L.Ed.2d 1305 (2010) (slip op., at 3) (internal quotation marks omitted). It is not the statute and compels a straight of the Act. The Court has no constitutional proceeding the court below the pre-emption circumstantly was an interest. Ante, at 1352. The Court standing what the Court may reasonably a pre-encorment is that the Act and of the present case is a contrary is an opinion relief in the entire of a mere respection of the present case for the Act when it intended."

"Same-sex marriage", random level: 0.6. It's all about the Fourth Amendment.

 "Same-sex marriage in the order to companies precedent. The Fourth Amendment is not a single the Act that the Act does not effect through other crimes by control of "case"); (emphasis added). The Court has no reason to what is why a destrict court mistaken would be inappellant majority was a defendant's opinion concurring in the Agency and the tax credits to give Federal program with the constitutional expenditure was prohibited from a question by the trial procedural default of the Federal Government cases a tax any department of the latter resolved the problem is so and set of the provision requiring the present context in Such a State context of the constitutional conductes between the three court."
"Same-sex marriage", random level: 0.8. An "injustice relationship".

"Same-sex marriage is eagible television would comply with the tax driver's required extens preemptive evidence on the court already related dependence of preferable cost-emimst scull injustice relationship, "month" provision is the election sakee seeking decision in the logicial justice a matter, regard to the government of the Freingly reasonable Commerce Clauses was not wholenature that is not docuper to draw only of [a] law meaning in the prosecutor in the fairnes" of the judges did not mean in a followings applicable to enact criminal action and retreoting state Patition does not disparance of a provision of the statute." Id., at 54." 

"Same-sex marriage', random level: 0.85. This actually made me laugh.

"Same-sex marriage is eaple the ACA's evintory cannot present an erroneous collaterations plainness — whereas criminal stance in our canon case. Or the statute is not the market spot that the Court fources "toor "justify" the Court conduct that according decision in candod requires in 14 millooloyly, Floring engals that the Demott v. Relo, 308 U.S. 332, 410, 111 S.Ct. 9300, 160 S.Ct. 1409, 131 S.Ct. 1545. Crowing it be applied to do warred the majority may be correcion, which canon a proceeding by entire regulation, and so health insurance complaint from the First Amendment here is not the court penalty and inference to the hinds of the Act's my view, it must anticill from a wantonering any defendant that as the statute exercise of this case regulates." 

"Same-sex marriage", 0.9.

"Same-sex marriage is earlarn to buy intrusion held." Robest anlike Report Addines 39 (in). The Ninton Creeching upon a state context, declaration of the remanded role worked with this is,nind States foreign "remotable to say that the citizen'n or neither can be included to be reason," L.J, the Stonk v. Renavated State, 3 W. Wallawoop, 547 U.S. 393, 242, 84 S.Ct. 909 (quoting Commstanit and Ingensures. Killian zothers Magharca Code And. 476 (1986) (know Milloan, individuals provides enactment of the Court creates principle than authorize points would have meant there.
The Court leaves the Court insists that a preemption of the court directly and it can every substantial role at an agency" precedent under in which the rule same-sex marriage as to reny us for correct after the States to the present district-specific proceedings." 

Friday, June 05, 2015

Stuttering in Korea

I had given up on English. It's my native language, but I figured after 30 some-odd years of disfluent speech, it was time to try something else. So I signed up for language classes in Korean, rationalizing that if I was going to try to teach myself how to speak, I might as well learn a new language along the way.

This might seem completely insane, but when the prevailing theme of your conscious thoughts for multiple decades is some variant of "Why can't I say what I want to say?", you come up with lots of crazy ideas.

For background, I've been a person who stutters for my entire life. I wrote about it on this blog a few years ago, so I think it's time for a followup. I've learned a lot since then, about myself and about stuttering, but in this post I simply want to give some insight into what it's actually like to stutter, and how my speech has changed over time.

After the last stuttering post, the predominant reaction I got from friends was either "I never noticed", or more assuredly, "You don't stutter". I'm what's known as a "covert stutterer", which means I'm pretty good at hiding my stutter via word choices or other tricks, but at the core, it's basically the same as the sound-repetition s-s-s-stuttering that most people are familiar with. So when people tell me definitively "You don't stutter", it's actually kinda maddening! Yes, thank you for so obviously knowing me better than me!

Now, for the 99% of you who have never stuttered, you might be curious what it actually feels like. Well, the best that I can explain it is the following: Have you ever had a nightmare where you were being chased, or there was some other impending danger, and no matter how hard you tried to scream, no sound came out? Stuttering is exactly like that, except in real life, and potentially hundreds of times per day. It's not simply "not being able to say what you want to say", it's closer to "having a mini panic attack that prevents you from saying what you want to say, hundreds of times per day". A lot of people will try to empathize with stutterers with statements like, "Oh yeah, when I'm nervous, I stutter, too", but fundamentally they're completely different experiences: when a "normal" person stumbles over some words due to nerves, it's a one-off event, and when the stressor dissipates, so does the disfluency. But when a person who stutters experiences a disfluency, it's a complicated, systemic symptom of a chronic problem.

So, there's something wrong with the connection between your speech center and your vocal cords, right? Just fix the connection and the problem disappears, right? So simple. Except no, it's not nearly that easy. Stuttering (and conversely, fluency) comes and goes in waves, both minor and major, and is also situational. Stutterers usually have no problem talking to themselves, but often trip up when asked their name. Some stutterers do fine in one-on-one conversations but can barely get a word out if forced to speak in front of a group. Others, like me, actually speak more fluently in a leadership role (presenter, coach, etc), but trip up in small conversations. I used to dread group dinners at college (where you randomly sit at a table with 6-8 of your dorm mates), because I could barely say what I wanted to say, yet I loved giving lectures as a teaching assistant. And curiously, nearly all stutterers can sing and even rap with complete fluency. But it's not merely a situational occurrence, either - there are always certain sounds that are notoriously difficult for each individual stutterer, sometimes regardless of situation.

Anyway, back to the present day. Just the other day, nearly immediately after replying to an email about an upcoming lecture I'm giving, I got hit by an unusually common moment of disfluency. I hit "Send" on the mail, then went up to the barista at Starbucks to ask for more hot water for my tea: "뜨거운 물 더 주세요" (More hot water, please). This is a phrase I've spoken countless times, yet it's always a difficult one for me, because I tend to trip up on the starting "D" sound. Yet I felt pretty good about this one since I've gotten it out fluently the last couple dozen times. But the barista caught me off guard by saying "안녕하세요" (Hello) rather than the expected "필요한 게 있으세요?" (Is there something you need?). Something broke in the chain of fluency, and the response turned into, "Um um um um um ... 뜨거운 물 주세요". The classic "um loop". In terms of what's going through your mind, though, it's something like, "Um (shit I said um) um (uh oh, why am I saying um) um (damn it, you had this) um (the barista looks worried) um (getting super embarrassed, feels like entire world is focussed on you) 뜨거운 물 주세요 (finally. thank god. man, that sucked. i totally had that!! what happened? argh. just take the water). The barista's expression briefly changed to the "foreigners suck at Korean" expression, and then she gave me my water and that was that.

Imagine knowing exactly what you want to say, yet not being able to say it. Imagine how frustrating that would be. Now imagine that frustration repeated many times per day, day after day, year after year, all the while feeling like the person that you're showing the world is not who you truly are. The thought consumes your waking hours, as literally every time you speak, you have meta thoughts about your speech (which of course makes your speech worse). You celebrate the victories, reveling in that time you had your friend in stitches with the bad date story you relayed as deftly as the best standup comic, yet you beat yourself up over the defeats, like that time you couldn't ask for water in Japanese despite knowing exactly how to say it. You wince when your friend teases, "Remember that time when you were so drunk you forgot your name?", knowing that 1) Obviously no one ever gets that drunk, and 2) Of course I remember the time, because it was incredibly embarrassing having a guy you just met make fun of you in front of a dozen people for "not knowing your own name". You build up this thing inside you, this intangible yet all-too-real foe, which sometimes disappears for months at a time, yet always finds a way to make an unwelcome return at the most inopportune of times. And you understand completely that moment in The King's Speech where King George VI and his wife are elated over his success in giving a radio speech, despite the fact that the speech was literally to lead Britain into war against the Nazis. Imagine that! You've just led your nation into war against Nazi Germany, but your overriding emotion is, "Thank god I didn't stutter"!

That is what it's like to stutter. That's how important it is to people, and how heartbreaking and frustrating it is to not be able to say what you want.

Conventional wisdom says that if you stutter into adulthood, you will be a lifelong stutterer. It's supposedly binary, and if you fix your speech as a kid, you're fixed for good, but if you don't, you're supposed to just give up hope and accept your fate. Conventional wisdom says you can learn to "control" your stutter, but not to "cure" it (I think this is one of the fundamental problems with modern stuttering therapy, but that's a different topic altogether). When I left high school, I was pretty resigned to this fate. In college, I think my speech actually got worse, and interviewing for jobs as a grad student was incredibly terrifying for me. So I was confused and delighted when, a few years after college, my speech suddenly started getting better. At one point, I actually thought I was miraculously "cured", and even volunteered myself as a research subject for an audio-linguistics lab (they told me over the phone I wasn't a stutterer so couldn't participate - ironic!). I started to speak up at work meetings, started to do better at group dinners, and just started talking more in general. I felt like I was on top of the world, and more importantly on top of my stuttering, and with just a little more effort, I would be the "normal" fluent speaker I always aspired to be. But life doesn't work like that, and although my speech had gotten remarkably better, I never actually "grew out" of my stutter. It was tamed, but it was still very clearly there, always right under the surface.

So with that as background, a few years ago I signed up for my first language class in Korea, and I was scared shitless, mainly because I knew I would be forced to speak out loud in class. Somehow I made it through high school and college speaking as little as possible in class and dreading the times when I didn't have a choice and couldn't get out of speaking. I still remember as clearly as if it were yesterday the day a kid came up to me after English class senior year of high school and asked me, with a sneer, "Why do you talk like that, with so many pauses? Can't you talk normal?" Moments like that leave an indelible wound on your psyche, as real as ending up on the losing side of a schoolyard fight. And as you grow older, you realize that most of the world never actually left high school - assholes from high school grow up into older assholes, bullies tend to stay bullies, and nervous, shameful, wounded kids grow up into nervous, shameful, wounded adults. And that wounded kid had grown so frustrated with his inability to speak his native language fluently that he decided he would re-learn how to speak in Korean.

Well, was I able to leave my stutter behind in a new language? Obviously not :). In a completely unsurprising turn of events, I stutter way worse in Korean than English. Korea also seems to be what I like to call a "hyperfluent" country - the incidence of stuttering seems to be way, way lower in Korea than most other countries (it's typically between 0.5%-1%, but anecdotally feels much lower in Korea). A potential side effect of this is that Koreans tend to give you approximately 20ms in which to initiate a response, and any delay is taken to mean that you must not know how to speak Korean (time pressure makes most stutterers stutter worse). So Korean is tough, but as I've learned more of the language, I've gotten better at putting together longer and longer phrases and sentences, sometimes even surprising myself. In fact, every language I speak has its own distinct stutter, which fits nicely with neuroscientific theories about language compartmentalization and clearly hints towards complex system-wide root causes of stuttering. Korean intonation is particularly difficult for me due to unwanted pauses I make (i.e., stuttering) during sentences, whereas my Chinese vocabulary is awful, yet since the tonal sounds are unaffected by stuttering, I've been told my Chinese is better than my Korean (it's not). My Japanese pronunciation is supposedly good, yet I don't even know enough words to effectively stutter. And compared to them all, now I feel basically fluent in English. At least until I "forget" my name or have ESL speakers feed me English words (Thanks, random guy I just met! Without your help, I may have never known the proper English word to finish that sentence.)

If you don't stutter, and especially if you know me and how I like to tell long, sometimes rambling stories, you still are probably thinking, "It's really not that big a deal". The point is that for people who stutter, it is the biggest deal. Not merely because you have a lot of "oh that's embarrassing" moments, but more broadly, for me it's because I feel like when I stutter, I'm giving people a false impression of who I am. The guy who speaks fluently and passionately about whatever - that is the "real" me, and in those moments when the "real" me disappears, I feel like a fraud to the world. Don't misunderstand, though: this post is not about wanting sympathy or pity or anything like that, because I definitely don't. It's just a brief explanation of a fascinating psycholinguistic phenomenon that I happen to be a primary observer of. It's about starting a conversation (har har), and hopefully shedding some light on a condition that affects around 70 million people across the world. And it's a reminder that the mind is a fascinating thing that we're not even close to understanding.