TL;DR: For certain types of programs, you can take advantage of idiosyncrasies in the Python interpreter and the host operating system to create real shared memory between processes and get some pretty good parallelization. Premature optimization is the root of all evil. As a developer, you've probably heard this before, and what it means basically is that you shouldn't waste time optimizing code unless it's already doing what you want it to do. We also live in an era of seemingly unlimited resources with AWS/Google Compute, and often the easiest way to get higher throughput in your programs or service is just to pay for more instances. But sometimes it's fun to see what sort of performance we can get on a simple laptop (and save some cash at the same time). So anyway ... I've been working on this thing, and it took too damn long to run, and I needed to run it lots and lots of times ... so, it was time to optimize. Basic optimization has two main steps: 1) P...
The Atlantic recently published an interesting article regarding the difference in "efficiency" between languages . The basic idea is that some languages, such as Mandarin Chinese, are very efficient in conveying information and possess linguistic shortcuts such as eliminating gender and tenses and collapsing "he" and "she" into a single pronoun, whereas other languages like German are quite verbose and precise, having different articles (i.e., "the") for different gendered nouns along with plentiful verb conjugations. They also touch very briefly on the idea of directional complements, which I've always found to be a fascinating difference between English and the Asian languages I've studied. Basically, directional complements are words attached to verbs that show the direction of movement of the actors. This sounds like an obvious thing, but it's actually a common pain point for English speakers learning Asian languages and vice ...