Vinod Kurup

Hospitalist/programmer in search of the meaning of life

NSBrief Gets the Best Guests

I’m not an OS X programmer, but I’ve really enjoyed 2 recent episodes of the NSBrief podcast. First was Mark Dalrymple’s interview. Here was my 140 character description:

Today, I listened to Mark Aufflick’s interview where he talks eloquently about starting your own business. I love the perspective he had on his first business and how shutting it down gave him the necessary (though painful) experience to make his current business work. He also talked about the importance of computing history especially given the availability of interesting subjects. He ended with great advice on the importance of professionalism. Paraphrased: “Make sure your team is the most professional side of any relationship.”

I know Mark and Mark from the OpenACS glory days and it’s great to hear how well they’re doing. Computing history was a prominent part of both talks. I think this is important. Old ideas are always new again. Some of the ideas that these pioneers had were only limited by the horsepower we had to work with at the time. Now thanks to Moore’s law, those limitations are gone, so we should re-investigate those ideas. Or maybe we’re all just getting old and like to talk about the good-old-days. :-) It’s nice to see that they both mentioned the OpenACS period of their programming lives as important. It certainly was for me. It was also great to hear Mark Aufflick’s voice for the first time, even though I feel like he’s been a friend for many years.

Emacs for Python Programming

(Note: See the bottom of this post for updates)

I was honored to to give my “Emacs for Python Programming” talk at the inaugural PyCarolinas conference. The conference was a huge success in every way, thanks to the efforts of Calvin Spealman, et. al. I promised that I would post my Emacs setup, so here it is.

Step 0: Prerequisites

This may work with other setups, but here’s what I tested:

Step 1: Clear out any previous customization

As evidenced by the numerous blog posts documenting how to set this up, it’s not straightforward. I recommend starting from scratch to make sure everything is set up properly. Once you have it working, then you can customize it. Make sure to start these commands in your home directory.

1
2
vinod:~$ mkdir old-emacs
vinod:~$ mv .emacs .emacs.d old-emacs/

Step 2: Set up initial Emacs configuration

Create an empty .emacs.d directory and a subdirectory named with your username. My username is vinod, so we’ll create a directory structure called .emacs.d/vinod/. The emacs-starter-kit tells emacs to automatically load any elisp files from that special directory, so you’ll always have a place to drop custom elisp files.

1
vinod:~$ mkdir -p ~/.emacs.d/$USER

Create a file named .emacs.d/init.el with the following contents:

my-init.ellink
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
(require 'package)
(add-to-list 'package-archives '("marmalade" . "http://marmalade-repo.org/packages/"))
(add-to-list 'package-archives '("melpa" . "http://melpa.milkbox.net/packages/"))
(package-initialize)

(when (not package-archive-contents)
  (package-refresh-contents))

(defvar my-packages '(starter-kit
                      starter-kit-bindings
                      starter-kit-js
                      autopair
                      yasnippet
                      auto-complete
                      fuzzy)
  "A list of packages to ensure are installed at launch.")

(dolist (p my-packages)
  (when (not (package-installed-p p))
    (package-install p)))

;; autopair and yas in all modes
(autopair-global-mode)
(yas-global-mode 1)

;; autocomplete
(require 'auto-complete-config)
(setq ac-dictionary-files (list (concat user-emacs-directory ".dict")))
(ac-config-default)
;; hack to fix ac-sources after pycomplete.el breaks it
(add-hook 'python-mode-hook
          '(lambda ()
             (setq ac-sources '(ac-source-pycomplete
                                ac-source-abbrev
                                ac-source-dictionary
                                ac-source-words-in-same-mode-buffers))))

;; Set up python-mode
(setq py-install-directory (concat esk-user-dir "/python-mode.el-6.0.12/"))
(add-to-list 'load-path py-install-directory)
;; this will show method signatures while typing
(setq py-set-complete-keymap-p t)
(require 'python-mode)
;; activate the virtualenv where Pymacs is located
(virtualenv-workon "default/")

(defun load-pycomplete ()
  "Load and initialize pycomplete."
  (interactive)
  (let* ((pyshell (py-choose-shell))
         (path (getenv "PYTHONPATH")))
    (setenv "PYTHONPATH" (concat
                          (expand-file-name py-install-directory) "completion"
                          (if path (concat path-separator path))))
    (if (py-install-directory-check)
        (progn
          (setenv "PYMACS_PYTHON" (if (string-match "IP" pyshell)
                                      "python"
                                    pyshell))
          (autoload 'pymacs-apply "pymacs")
          (autoload 'pymacs-call "pymacs")
          (autoload 'pymacs-eval "pymacs")
          (autoload 'pymacs-exec "pymacs")
          (autoload 'pymacs-load "pymacs")
          (load (concat py-install-directory "completion/pycomplete.el") nil t)
          (add-hook 'python-mode-hook 'py-complete-initialize))
      (error "`py-install-directory' not set, see INSTALL"))))
(eval-after-load 'pymacs '(load-pycomplete))

;; pyflakes flymake integration
;; http://stackoverflow.com/a/1257306/347942
(when (load "flymake" t)
  (defun flymake-pyflakes-init ()
    (let* ((temp-file (flymake-init-create-temp-buffer-copy
                       'flymake-create-temp-inplace))
           (local-file (file-relative-name
                        temp-file
                        (file-name-directory buffer-file-name))))
      (list "pycheckers" (list local-file))))
  (add-to-list 'flymake-allowed-file-name-masks
               '("\\.py\\'" flymake-pyflakes-init)))
(add-hook 'python-mode-hook 'flymake-mode)

;; menu bar is useful when getting started
(menu-bar-mode)
(setq-default default-tab-width 4)

Step 3: Create a python virtualenv

We’ll keep all the python-side customization in a virtualenv named default. It’s important that the name you choose is the same as the name in the (virtualenv-workon) command in line 45 of Step 2.

1
vinod:~$ mkvirtualenv -p python2 default

After it does its thing, your shell prompt should change, indicating that your new virtualenv is activated

1
(default)vinod:~$

Step 4: Install Pymacs

Pymacs is a really cool piece of software that sets up a 2 way communication between Emacs and Python, allowing you to control Emacs with python commands rather than elisp commands. It requires a python piece (Pymacs.py) and an Emacs piece (pymacs.el). Unfortunately, they’re not installable via pip or package.el, but installation is easy enough.

1
2
3
4
5
6
7
8
9
10
11
12
13
(default)vinod:~$ mkdir src
(default)vinod:~/src$ cd src
(default)vinod:~/src$ git clone git://github.com/pinard/Pymacs.git
(default)vinod:~/src$ cd Pymacs

# check to make sure tests pass
(default)vinod:~/src/Pymacs$ make check

# install it (Be sure you're inside your virtualenv!)
(default)vinod:~/src/Pymacs$ make install

# install the emacs extension
(default)vinod:~/src/Pymacs$ cp pymacs.el ~/.emacs.d/$USER/

Step 5: Install other python packages

These helper packages are easier to install:

1
(default)vinod:~$ pip install pyflakes pep8

The pyflakes and pep8 packages check your code as you type using Emacs’ flymake mode. Now, deactivate your virtualenv.

1
2
(default)vinod:~$ deactivate
vinod:~$

Step 6: Install pycheckers

Flymake is the part of Emacs that checks your code for errors on the fly. It calls a shell script called pycheckers, so you need to have a script by that name in your shell’s PATH. Here’s mine (~/bin/pycheckers):

pycheckers.shlink
1
2
3
4
5
#!/bin/bash

pyflakes "$1"
pep8 --repeat "$1"
true

Step 7: Install python-mode.el

As I mentioned in my talk, there are multiple Python modes available, but I recommend using the one named python-mode.el, which is developed at http://launchpad.net/python-mode. It does periodically get uploaded to Marmalade, but the auto-completion using pycomplete doesn’t work well on the version that is there now (6.0.10), so I recommend that you manually download and install the latest stable version (6.0.12). I’ll update this post once a stable working version gets uploaded to Marmalade or Melpa.

1
2
vinod:~$ cd ~/.emacs.d/$USER
vinod:~/.emacs.d/vinod$ curl -L https://launchpad.net/python-mode/trunk/6.0.12/+download/python-mode.el-6.0.12.tar.gz | tar xz

Step 8: Test it all out

Launch emacs and open a python file named test.py. Type the following:

import os
os.path.jo

Wait at this point. You should see auto-completion of os.path.join followed shortly by a yellow popup showing documentation of that method. Hitting return should accept the completion. Then type ( and you should see the method signature in the minibuffer.

Move the cursor over any letter in join. Hit ‘F1’. A window should popup with the docstring for os.path.join. Hit ‘F2’. A new window should be opened with the code for os.path.join. Hit ‘F3’. You’ll be prompted to enter the name of any python command, and Emacs will show you the docstring.

Type the string ‘blah’ and hit return. The string should be highlighted in pink and if you mouseover it, the minibuffer will say undefined name 'blah'. That’s flymake working for you.

Hit C-c C-c and the buffer should be sent to a Python interpreter and you’ll be dropped in the REPL after the code has been loaded. Any errors in the code will be reported.

If all of this works, then CONGRATULATIONS!!! If not, let me know and I’ll see if I can help debug.

Step 9: Customize

All of this is customizable in hundreds of ways. Try M-x customize-group RET python-mode RET to see how.

Step 10: How to create a new python project

Whenever you want to create a new Python project, you have do the following:

Create a new virtualenv

1
vinod:~$ mkvirtualenv newproject

Install pymacs into that virtualenv

1
2
(newproject)vinod:~$ cd ~/src/Pymacs
(newproject)vinod:~/src/Pymacs$ make install

Install the other pip modules

1
(newproject)vinod:~/src/Pymacs$ pip install pyflakes pep8

Switch to that virtualenv in emacs

M-x virtualenv-workon RET newproject/ RET

Issues

Changing virtualenvs doesn’t restart Pymacs

For the most part, this won’t affect much, but if you move from a Python2 project to a Python3 project, you may have problems. The workaround is to call M-x pymacs-terminate-services, then M-x virtualenv-workon to change your virtualenv, and then finally M-x load-pycomplete to restart pymacs and pycomplete.

Latest stable version of python-mode.el isn’t on Melpa

I’d much prefer to just load python-mode using the built-in Emacs package manager.

Pymacs installation is harder than it needs to be

I wish Pymacs could be installed by a 2 step process

  1. pip install Pymacs
  2. M-x package-install RET pymacs RET

Which pycheckers to use?

My script uses pep8 and pyflakes. There are a lot of other options out there and I do not know which is best. See this post for more details.

Not using ropemacs any more

In my talk, I used ropemacs. In this setup, I have decided against it. I was mostly using it for code-completion and documentation lookup, but I like pycomplete.el better for those purposes. It’s included in python-mode and it shows method signatures as you type. Ropemacs does have other features such as project management (easily jumping to files in the same project), and refactoring (changing variable names throughout project). I haven’t used those, so I decided not to install ropemacs.

Project management not included

I haven’t yet decided which package to use for this. I’ll probably try projectile, but there are many other options, some lightweight (ropemacs, eproject, pony-mode) and some heavyweight (ecb, CEDET).

References

I gleaned knowledge from all of the following, in no particular order.

Updates

2012-10-27

Thanks to @danpoirier and @gregnewman for testing. I initially recommended using the master branch of python-mode.el, but now I recommend using the latest stable release (as of now) which is version 6.0.12 to avoid problems loading Pymacs. There’s a bug on 6.0.12 with virtualenv-workon that requires you to put a trailing slash after the name of your virtualenv, so I’ve updated the init.el file above to account for those changes.

Emacs Autocomplete Stumbles on Yasnippet

Autocomplete Mode is an Emacs mode that tries to guess what word you’re typing and provide auto-completion. Recently, it started failing in simple scenarios.

It can use various ‘sources’ to try to figure out what you’re about to type. The simplest source is ac-source-dictionary which is a simple list of words. If you start typing any of those words, it will try to autocomplete them for you. Normally, you’d put those words in a file, with each word on a separate line, but for testing purposes, you can also add them manually using an elisp function. Here’s the test that failed for me:

(add-to-list 'ac-user-dictionary "[email protected]")

That adds the email address to the dictionary. Then, typing ‘foo’ in any buffer, should offer to autocomplete the entire email address. Unfortunately, it wasn’t working.

To debug, I typed ‘foo’ and then did M-x auto-complete, which forces auto-complete to look for a completion. This resulted in an error in the Minibuffer:

ac-yasnippet-candidates: Symbol's function definition is void: yas/current-snippet-table

This led me to check which ‘sources’ autocomplete was trying to use: C-h v ac-sources

(ac-source-features ac-source-functions ac-source-yasnippet
 ac-source-variables ac-source-symbols ac-source-abbrev
 ac-source-dictionary ac-source-words-in-same-mode-buffers)

So, autocomplete is looking through each one of those sources, trying to find possible words I might be typing. The third one is called ac-source-yasnippet, which seems to be where my error is coming from.

So, I looked into the yasnippet source code and found that, indeed, there was no function named yas/current-snippet-table. That function has been renamed to yas--get-snippets-table in version 0.8.0, but autocomplete is still relying on the old function name. (RANT: Don’t rename functions that others might be using without providing some kind of alias, at least temporarily) I looked at the code for autosnippet on github and noted that the latest version has been fixed to use the new code, but it hasn’t yet been released on MELPA. Until it is, the quick solution is to remove ac-source-yasnippet from ac-sources.

(delq 'ac-source-yasnippet ac-sources)

Then typing ‘foo’ autocompletes to ‘[email protected]’.

It seems that autocomplete should be more forgiving of errors in any specific ‘source’, passing it up to use other sources. Alternatively, it should fail loudly when there’s an error, so that I can know to fix or remove the offending ‘source’.

I’ve Gone to the Dark Side

I was a huge Apple fan growing up and IBM was Apple’s enemy. Buying an IBM computer would have been unthinkable.

Well, the unthinkable has happened. Of course IBM doesn’t make laptops anymore, but their computers live on under the Lenovo name. I am the proud owner of a new Lenovo Thinkpad T420S. I love this computer. It’s an ugly black boxy thing, but it’s light (4 lbs), quiet and speedy. And the keyboard is just pure awesome. That’s the main reason that I bought the T420S as opposed to the current T430 line. The new line has the chicklet keyboard that is being used on all laptops these days. The T420 is the last one with the old-fashioned clicky keyboard, which I love.

I also feel very lucky to get the deal that I got. I bought it from Lenovo’s Outlet store, which had about thirty T420s’s in stock when I was shopping. This one was about half the price of the others, by my estimation. It’s so underpriced that I’m wondering if I’m missing an obvious flaw, but I don’t see it. (Please let me know if you do!)

Here’s a comparison between the specs of my computer compared to one that was available at the Outlet store on October 4th, 2012:

  Mine Other one
Part Number 417153U 4170CTO
Processor Core i5-2540 Core i5-2540
OS Windows 7 Pro Windows 7 Home
Graphics Intel HD3000 Intel HD3000
Memory 4 GB 8 GB
Display 14” 1600x900 HD+ 14” 1600x900 HD+
HD 128GB SSD 128GB SSD
Optical DVD recordable DVD recordable
Battery 6-cell 6-cell
Network Intel Centrino Advanced Not specified
Bluetooth V3.0 Not specified
Fingerprint reader No No
Price $ 663.72 $ 1,355.39

The RAM is better on the other model, but Windows 7 Professional is an upgrade on my model. I don’t really understand why mine was so cheap, but I’m pretty happy about it!

I added another 4G of RAM, moved Windows to a VirtualBox image and installed Arch Linux with Xmonad. It takes about 10 seconds to get from the BIOS screen to a working computer. I’ll post some info on my setup in a future post.

Avik Arrives!

Anika, Kavi, Mala and I are so proud to announce the birth of healthy, happy baby boy!

His name is Avik Puri Kurup and he was born at 11:01 PM on September 3rd, 2012. He is 6lbs, 12oz and about 19 inches long.

A few pics:

Avik
Avik

And the obligatory pseudo-nostalgic photo:

Avik

Mom and Baby are doing great! Thanks for all the well wishes and support!

Thoughts on Free Will

Time for some pop philosophy. I’ve never thought too deeply about the concept of free will. Of course we have free will! I chose to live my life a certain way. I could choose to live a different way. It is my free will to choose how I live. I’ve never thought about it more deeply than that.

I recently read Walter Isaacson’s biography of Einstein and I was forced to think more deeply about free will. The biography is excellent. It gives more depth to Einstein, who, despite his flaws, is still a heroic figure. One day, I’ll do a proper review, but the piece that struck me the hardest was Einstein’s views on free will. He didn’t believe that free will existed, and not only that, he didn’t even to seem to think that this was a controversial position. It appears that Isaacson didn’t either, because he didn’t spend much time on that part of Einstein’s character. But, it seemed so strange to me, since I assumed that we all have free will.

So, I read Sam Harris’s e-book Free Will, in which he concisely explains why free will is a myth. Take any conscious decision that you might make. Let’s say you choose at this moment to go eat a pizza. Do you have the free will to make that choice? Well, you certainly can go eat pizza, but the question is if you had the free will to do it. What were the steps that led to making that choice. Somewhere in your brain, a certain set of neurons fired to trigger the idea in your mind to have a pizza. How did those neurons fire? Either there was a conscious decision to have that thought, or it was an unconscious decision that you didn’t control. And if it was a conscious decision, then you can take that back a step further and ask how that decision occurred. Eventually, you will have to start at some unconscious (or subconscious) event which triggered the series of events which eventually led to your decision to have a pizza. No matter how far back you can track the conscious trail that led to a pizza forming in your mind, eventually you are left with an unconscious starting point. Certainly all of these thoughts are heavily influenced by your life experiences. If you’ve never seen or heard of a pizza, there’s a zero percent chance that you’ll think of a eating a pizza. But all of those life experiences are themselves the result of conscious and unconscious decisions, all of which we ultimately have no specific control over.

Every thought you have is the only thought you could have at that moment. It is the result of all of the life experiences and thoughts that you have had up to that moment. If you consciously try to “create” an original thought, just to prove that you have free will, well… you can’t. Because that decision itself originated from somewhere in your subconscious, and is therefore something that you don’t have conscious control over.

In addition to this theoretical rationalization against the concept of free will, Harris describes multiple studies from neuroscience which show that neurons fire in our subconscious well before we develop the urge to act. Examples:

Bottom line: I’m convinced that I don’t have the free will that I thought I did. The question is how this should affect my life. It could be depressing. If every thought of mine originates in a way that I don’t control, then am I just being steered through life? It makes my accomplishments in life seem less “worthy”. On the other hand, it can be quite freeing. My path through life is determined. Perhaps I shouldn’t be worrying so much about every decision, because I know that they’re not really “my” decisions to make. It can also give you more empathy for others, knowing that they also have no will do anything but what they are doing.

It’s still a confusing topic, because I “feel” like I have options in life and I “feel” like taking different options will change my life. That part is true. The part that isn’t true is that I have any control over the range of available options, nor over the specific option that I choose. Even Harris seems to be a little conflicted here:

Losing a belief in free will has not made me fatalistic—in fact, it has increased my feelings of freedom. My hopes, fears, and neuroses seem less personal and indelible. There is no telling how much I might change in the future. Just as one wouldn’t draw a lasting conclusion about oneself on the basis of a brief experience of indigestion, one needn’t do so on the basis of how one has thought or behaved for vast stretches of time in the past. A creative change of inputs to the system—learning new skills, forming new relationships, adopting new habits of attention—may radically transform one’s life.

The question is how do you accomplish a ‘creative change of inputs’, if you don’t have the will to do so? I guess the idea is that while I don’t have any control over the thoughts that arise in my brain, I do have perceived control over my decisions. With whatever power my conscious brain has, I can choose things that will improve my life, with the hope that exposure to new experiences will ‘change the inputs’ to my subconscious brain. Ugh, just thinking about how this all works makes my brain hurt. I guess this is why I never studied philosophy before. :-)

Free Will is short and cheap and I highly recommend it.

Facebook Is Killing Blogging

Looks like August 18th was International-Prominent-Bloggers-Complain-About-Facebook-Killing-Blogging Day. I was just two weeks too early.

On a side note, I’m really enjoying Scott Hanselman’s podcast This Developer’s Life. It’s an homage to the awesome This American Life, but focused on programming issues. They interview interesting programmers, but they carefully edit the interviews to focus on the parts that are the most interesting, interspersing them with music and effects. It’s really well done.

A Great Moment

I woke up, got ready and went downstairs. I am not a morning person. I try to do as little thinking as possible, because thinking eventually leads to the rational conclusion that I should just go back to bed. I get to the coffee machine, grab my plastic mug and start pouring. Suddenly, I’m woken from my daze. I hear little footsteps pounding across the hallway above me. I look towards the bottom of the stairs and see a little figure race around the corner with a determined, slightly fearful look in her face. She’s afraid that she’s too late and that I’ve already left for work. She spies me and I must be smiling because her face melts into the biggest, happiest smile and her pace quickens. We run to meet each other in a big hug. “Daddy, can I watch the milk go in your coffee?”

Moments like that are so joyous, and I have underestimated how much they mean to me. I want to capture them and never let them go.

NASA Curiosity Lands

Like the rest of the geek world, we’ve been excited by Curiosity’s successful trip to Mars. Here’s a cool video which combines an animation of Curiosity’s landing mixed with the real reactions of the astronauts that got it there. (Found on thekidshouldseethis.com)

I’ve been disappointed that real space travel is unlikely to become a reality during my life. I honestly thought it would when I was a kid. But watching Curiosity gives me hope that we will be able to continue exploring our universe. This trip to Mars was much more of an achievement than the last one. Here’s an animation of the last trip (2003 Mars Exploration). Note that the final landing was achieved by an air bag dropped roughly from high up in the air. Not much precision required there.

The Curiosity rover is too large for the airbag approach (See image at right comparing it to previous rovers), so an alternate system was devised using a sky crane which gently lowered the rover onto the Martian surface. Dropping an airbag seems much easier than gently landing a rover without damage. The real advance in our lifetime is that our machines are getting more intelligent. And the exciting thing is that I can actually participate in that type of exploration. I think I just convinced myself to learn how to program a robotic car!

Blogging Again

I’m going to start blogging more. I know I’ve said that before, so take it for what it’s worth. I think there is value in blogging. I’ve been thinking about my life a lot lately, trying to find what makes me tick. One of those things that gets me excited is programming. I started thinking about that and remembered this meme that had passed around the blog world a few years back. It was a list of questions related to how you got started programming. I remember it especially because I remember reading MarkD’s version and thinking how similar it was to my own beginnings. Then, because I love going off on tangents, I googled for ‘How I got started in programming’ to read other responses. Many of the posts I found were on abandoned blogs. And now, for the ‘causal relationship not supported by data’ statement. This is because of Facebook and Twitter. OK, maybe it’s just the natural decay of blogs. But I am concerned about the fact that most people write in forums that are controlled by other people rather in their own forum. That was one of the cool promises of the web. A little place of your own that you controlled. Facebook and Twitter and Google+ are all cool services, but the more time I spend on them, the less time I spend here. I’m sure other people have written about this with more eloquence than I could, but the bottom line is that I want to keep writing. I want to learn more about myself, I want to teach others what little I know, and I want to learn from others. I’ve been journalling a lot lately and that has been useful, but the act of hitting ‘publish’ will make my writing better. That’s what this blog is for and I’m going to start using it again.

Maybe one of these days, I’ll even put up my own answers to ‘How I Got Started in Programming’, but in the meantime, go read MarkD’s