I got 2 Nest E thermostats because Duke Energy was having a sale around Black Friday. I didn’t have any strong motivation to have a Nest. The idea of a smart home is cool, but most ‘smart’ things nowadays have ended up being more work than benefit. Anyway, for $75 each, I thought it was worth a try. We have 2 thermostats so I decided to install the upstairs thermostat first. The process started off smooth and easy. I like that they provide a back plate, which I didn’t even notice until my trusty assistant, Kavi, pointed it out. My existing Honeywell thermostat is rectangular and if I just put the small circular Nest in its place, there would be a big chunk of unpainted real estate on the wall. Having the back plate prevented this from being a ‘replace thermostat, and learn how to patch and paint the wall’ project. The online Nest compatibility checker and the paper instruction manual (which includes stickers to label the existing wires) are also nice touches. That all gave me the (misguided?) confidence that even I could do this. A study confidence which lasted until I flipped the circuit breaker back on and …. nothing happened. According to the manual, the Nest was supposed to turn on, say Hi, and then walk me through the set up process. But it remained dead as a doorknob. I pulled it off the wall and plugged it in via USB (a nice troubleshooting feauture). It turned on and then when I placed it back on the wall, it complained that it was not getting enough power from the R wire. The error code was E195. When I looked at the ‘Technical Info’, it reported that the R wire was only providing 2-3 volts, while it needed 24. I chatted up Nest support, which took about 30 minutes to connect on a Friday night. The representative was responsive and useful. They eventually recommended trying to disconnect the C (common) wire. For a reason I still can’t explain, this brought the voltage on the R wire from 2-3 volts up to 35 volts, and my Nest was working.

Having tasted the sweet nectar of success, I then tried to swap my second Nest in for my downstairs thermostat. I made a huge mistake here. It had been easy to identify the circuit breaker for my upstairs furnance because it was labelled, “Up Furnace”. But there was nothing labelling the downstairs furnace. I switched off the circuit breaker that I thought was responsible for the downstairs HVAC, but in hindsight, I think I got the wrong one (and since it was warm out and the heater wasn’t running, I didn’t notice because nothing was running in the first place). Anyway, I walked through the rest of the instructions and again got to the point where the Nest was supposed to power on, but it didn’t. So I powered it on via USB and placed it back on the wall. Unfortunately, this time I was getting only 0.01 volts from the R line. Still, I was hoping that the ‘disconnect the C wire’ trick would work again. Unfortunately, disconnecting the C wire did nothing. I walked through things with Customer Service chat. Again, they were helpful (rather than just spouting cookbook-type basic info), but in the end, they didn’t have a recommendation that would fix things. They suggested that I get a voltmeter and test the voltage output at the various wires. I do not have a voltmeter and felt that I was getting in over my head, so instead I put in some calls to HVAC service companies, but since it was the weekend and this wasn’t urgent, I was resigned to having no heat over the weekend. I watched a few Youtube videos of HVAC repairs, some of which were quite entertaining. Eventually, I tried to reconnect my old Honeywell thermostat and was surprised and intrigued when this ALSO failed. It made me think that I messed something up somehow. The only thing I could think of was that since I hit the wrong circuit breaker, there was live current in the wires. While moving them from the Honeywell to the Nest, perhaps I touched 2 wires together causing the system to short circuit. I don’t remember noticing a spark or anything, but that was my best guess. I eventually watched enough videos to give myself confidence that I could open up the HVAC and find the electrical panel. I did that successfully (after turning off the CORRECT circuit breaker, and also pulling out the pull out switch next to the furnace). After cleaning out a ton of cobwebs, I found the LED light on the electrical panel was off (even after connecting the circuits temporarily). I searched the panel and found a 3-amp E fuse (like the kind I’ve seen in automobile fuseboxes). I pulled it out and sure enough, it was blown. It looked like the ‘bad fuse’ in this photo. Off to Home Depot, I purchased 2 replacements for 3 dollars, and then replaced the busted fuse with the new one. I flipped the circuit breaker and put the pull-in switch back in. The first encouraging sign was that the red LED light on the electrical panel turned on. I went inside and, joy of joys, the Nest was booting up and showed 35V via the R wire. So, I guess the bottom line is that I need to be more careful when I’m at the circuit breaker box.

Guess what, we’re composting now! I told you was going to be a gardener, so now you know I’m serious. For oh so many years, every Friday, we’d haul the garbage and recycling out and I’d think to myself, “Wow, we’d have like no garbage at all if we were composting.” It’s only taken a few years of thinking that, week in and week out, to get me to try. Actually, that’s not even true. It was still way down on my future to-do list until I got a cheap copy of “The Complete Idiot’s Guide to Composting”. Thank you, BookBub! (As if I needed any other incentive to buy and read more books). But honestly, the book was excellent in that it made me think that yes, even I could compost. So shortly after finishing it, I headed down to the Orange County Solid Waste Center and purchased an Earth Machine compost bin for fifty bucks, and made the kids set it up near the screen porch. We’ve been throwing all our copious kitchen scraps for the past 2 weeks. So far, all that has happened is that a bunch of fruit flies think we’re awesome neighbors and seem to have moved in for the long haul. But I’m hoping that some composting starts happening soon. It’s supposed to be easy as pie, but I’m secretly worried that I’m missing something, oh, like maybe skills or talent. But our garbage bin sure is lighter!

I’m going to be a gardener. Yup, just decided. Well, re-decided, I guess, since we already have a garden. It’s a strip of dirt on the side of our house and we’ve been trying to grow vegetables the past couple years. But it has some problems. It’s way out of the way, so it’s a pain to check on it (since I’m lazy). The ground just underneath the topsoil is hard clay. It’s shaded most of the day either by our house, or by the beautiful forest surrounding our house. And I haven’t figured out a way to keep the deer from eating the few vegetables that we are able to grow in there. And even despite all those problems we’ve been able to grow some zucchini, okra, tomatoes and basil, which has been super exciting, and makes us want more! Here’s a photo of the harvest that greeted us when we returned from a week-long trip to visit my parents in Dallas this summer:

But, in general, things end up getting eaten by the deer or overrun by weeds, so we’re trying a new approach now, based on the book, Square Foot Gardening. Kavi and Anika helped me build this raised bed, which is itself a huge accomplishment given how not-handy I am.

This was my first time using a drill since shop class in high school! We’re going to try using a fishing line fence to keep the deer away. I’m not very optimistic about that, but fingers-crossed! We can also see the garden from our living room, so maybe I can just shoo them away any time they saunter by. The garden is looking quite pathetic because we bought some seedlings from the Carrboro Farmer’s Market but then all our garden plans got sidetracked by Hurricane Florence (which fortunately treated us very kindly, all things considered). So, the seedlings wilted away in our screen porch before we finally got a chance to plant them this week. We’re hoping there’s still some life left in them. Anyway, our expectations are low, especially given how late in the summer we’re starting, but it would be nice to get one head of lettuce or something to grow. I’ll be sure to let you know!

I briefly mentioned finding the 2012 UCLA lecture series, ”History of Indian Civilization”, a couple days ago. I’m about halfway through and have found it thorougly illuminating. I learned a lot about India growing up, but never in a formal way, so it’s nice to have someone as skilled as Vinay Lal connect all the loose threads into a coherent history. One of the things that surprises me is that aside from the fact that classes are being streamed over the internet, there is nothing different than the lectures I skipped (I mean religiously attended!) about 20 years ago. It’s a large auditorium of students watching a very charismatic, knowledgeable speaker pacing at the front of the class, occasionally making mostly-legible markings on a chalkboard. Not that there’s anything wrong with that. I learned a lot back in college and I’m enjoying these lectures immensely. It just seems strange that so little has changed in the university clasroom (at least this one from 2012) when so much has changed in the world at large.

Actually, I find it reassuring that little has changed, at least in some classes. There was a time when the ability to download the audio of any university class was absolutely new. For me, that was around 2002 or so. I listened to CS61A (UC Berkley’s intro to CS course) and a few other similar courses back then. The limitations of internet bandwidth and storage made anything more than audio difficult back then. But now with those barriers crushed, I was afraid that everything would be “multimedia”, and not amenable to listening while I commute to work. So, it’s reassuring to find that audio-only dissemination of knowledge is still a thing.

OK, this is a debug session in progress, so don’t expect a nice solution at the end. We’re working on a project that does analysis of some public voter registration data. The DB is hosted on Amazon RDS and I’ve been perplexed by how poorly queries are performing there, despite the tables only have about 10 million rows. Simple queries are taking many minutes, which is orders of magnitude slower than my laptop.

Mark suggested running ‘VACUUM ANALYZE’, which I didn’t think would help because my understanding was that the autovacuum process in PostgreSQL would be taking care of that on a regular basis. These queries had been slow for days with no recent inserts or updates, so certainly autovacuum should have caught up to them by now. But, I tried it anyway and lo and behold:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

db=> select count(*) from voter_ncvoter;
  count
----------
 12336571
(1 row)
Time: 315777.051 ms
db=> vacuum analyze;
VACUUM
Time: 11377035.096 ms
db=> select count(*) from voter_ncvoter;
  count
----------
 12336571
(1 row)
Time: 4300.107 ms

Woah, that worked! Sure, it took 3+ hours to run ANALYZE, but wow. So, why isn’t autovacuum automatically doing this for us. (I mean it has the phrase ‘auto’ in its name!!!)

I’ve found this great article on autovacuum basics which led me to do this query:

1
2
3
4
5
6
7
8
9

db=> select relname, n_live_tup, last_autoanalyze from pg_stat_all_tables where relname like 'voter_%';
       relname       | n_live_tup |       last_autoanalyze
---------------------+------------+------------+-------------------------------
 voter_changetracker |  306689271 | 2018-05-05 04:59:08.503876+00
 voter_filetracker   |         41 | 2018-05-13 02:00:47.802633+00
 voter_ncvhis        |          0 |
 voter_ncvoter       |   12336616 | 2018-05-06 13:20:30.073426+00
 voter_badlinerange  |        404 | 2018-04-10 05:44:39.949193+00
(5 rows)

So those 2 large tables haven’t been ANALYZEd in weeks, despite the fact that we import a 10 million row CSV once every week. This is the end of my debugging road, for now. Hopefully, I’ll figure out what’s going on.

I love this post by Austin Kleon. As someone who has found very few things that I don’t find intriguing in some way, it is nice to know that there are others like me and that maybe there’s nothing wrong with that. I love being a doctor and I love being a programmer, and I love honing both of those crafts.

He also links to a 2012 New York Times article about a UCLA sophomore who spends 1 hour each day learning something new, mainly by taking online courses from other universities or watching how-to videos on YouTube. We live in a golden age, don’t we? I mean, sure, people will look back at our period of history and cringe quite a bit, but the idea that I can search just about any term, and find information about it on Wikipedia, and probably find a course on it by an actual college professor…. it’s just mind boggling. I’m lucky as it is, but I often think about if I was a kid growing up in this era. I’d be overwhelmed, sure, but I would totally be taking online courses left and right.

Actually, even if I weren’t a kid. I just googled the subject of that NYT article and found his twitter account. His most recent post is from 2016 but links to a YouTube series from UCLA on the history of India. I am TOTALLY watching that.

Let’s play ‘Spot the bug’. We’re building a simple system that shows photos. Each photo has a publish_date and we should only show photos that have been published (i.e. their publish_date is in the past).

1
2
3
4
5

class PhotoManager(models.Manager):
    def live(self, as_of=None):
        if as_of is None:
            as_of = timezone.now()
        return super().get_query_set().filter(publish_date__lte=as_of)

And the view to show those photos:

1
2

class ShowPhotosView(ListView):
    queryset = Hero.objects.live()

Can you spot the bug? I sure didn’t… until the client complained that newly published photos never showed up on the site. Restarting the server fixed the problem temporarily. The newly published photos would show up, but then any photos published after the server restart again failed to display.

The problem is that the ShowPhotosView class is instantiated when the server starts. ShowPhotosView.queryset gets set to the value returned by Hero.objects.live(). That, in turn, is a QuerySet, but it’s a QuerySet with as_of set to timezone.now() WHEN THE SERVER STARTS UP. That as_of value never gets updated, so newer photos never get captured in the query.

There’s probably multiple ways to fix this, but an easy one is:

1
2
3

class ShowPhotosView(ListView):
    def get_queryset(self):
        return Hero.objects.live()

Now, instead of the queryset being instantiated at server start-up, it’s instantiated only when ShowPhotosView.get_queryset() is called, which is when a request is made.

A few cool Emacs posts have flown across my radar, so I’m noting them here for that time in the future when I have time to play with them.

• Get unicode glyphs to show properly. (╯°□°）╯︵ ┻━┻
• Creating slides with org-mode and Reveal.js. Finally, a sane way to create a presentation :)
• Anzu: Improves incremental search. Anzu shows the number of matches found, and how many matches remain in the document.
• Emacs as a Django IDE: I tried this out briefly, but didn’t end up sticking with it. It seemed too heavyweight. I’m probably just too used to my own workflow. I still think it’s an interesting idea and would like to try it again sometime.
• Travel back and forth in git history: I’m addicted to magit mode, to the point that the only git command I use manually nowadays is git clone. I do everything else in magit. Magit has a magit-blame-mode which shows when each line in a file was changed and by whom. git-timemachine seems to take that a step further by visiting each revision, one by one. I can think of a few times where that would have been useful.
• Search StackOverflow from inside Emacs: Enough said.
• Emacs RSS client: Still using NewsBlur, but totally miss Google Reader. Maybe I need to try this one out.
• Sacha Chua’s minicourse on Elisp: Have to read through this someday!

I wrote my first blog post in a little while (ok, ok… 18 months) yesterday and when I tried to generate the post, it failed. Silently failed, which is the worst kind of failure. I’m still not sure why it was silent, but I eventually was able to force it to show me an error message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

/home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.4/lib/pygments/popen.rb:354:in `rescue in get_header': Failed to get header. (MentosError)
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.4/lib/pygments/popen.rb:335:in `get_header'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.4/lib/pygments/popen.rb:232:in `block in mentos'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.4/lib/pygments/popen.rb:206:in `mentos'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/pygments.rb-0.3.4/lib/pygments/popen.rb:189:in `highlight'
  from /home/vinod/dev/kurup.org/plugins/pygments_code.rb:24:in `pygments'
  from /home/vinod/dev/kurup.org/plugins/pygments_code.rb:14:in `highlight'
  from /home/vinod/dev/kurup.org/plugins/backtick_code_block.rb:37:in `block in render_code_block'
  from /home/vinod/dev/kurup.org/plugins/backtick_code_block.rb:13:in `gsub'
  from /home/vinod/dev/kurup.org/plugins/backtick_code_block.rb:13:in `render_code_block'
  from /home/vinod/dev/kurup.org/plugins/octopress_filters.rb:12:in `pre_filter'
  from /home/vinod/dev/kurup.org/plugins/octopress_filters.rb:28:in `pre_render'
  from /home/vinod/dev/kurup.org/plugins/post_filters.rb:112:in `block in pre_render'
  from /home/vinod/dev/kurup.org/plugins/post_filters.rb:111:in `each'
  from /home/vinod/dev/kurup.org/plugins/post_filters.rb:111:in `pre_render'
  from /home/vinod/dev/kurup.org/plugins/post_filters.rb:166:in `do_layout'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/lib/jekyll/post.rb:195:in `render'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/lib/jekyll/site.rb:200:in `block in render'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/lib/jekyll/site.rb:199:in `each'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/lib/jekyll/site.rb:199:in `render'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/lib/jekyll/site.rb:41:in `process'
  from /home/vinod/.rbenv/versions/1.9.3-p286/lib/ruby/gems/1.9.1/gems/jekyll-0.12.0/bin/jekyll:264:in `<top (required)>'
  from /home/vinod/.rbenv/versions/1.9.3-p286/bin/jekyll:23:in `load'
  from /home/vinod/.rbenv/versions/1.9.3-p286/bin/jekyll:23:in `<main>'

Professor Google tells me that this happens when you try to run the pygments.rb library in a Python 3 environment. (pygments.rb is a Ruby wrapper around the Python Pygments library). The fix is to run the code in a Python2 virtualenv. I guess the last time I updated my blog, Arch still had Python2 as the system default. No, I don’t want to check how long ago that was.

1
2

$ mkvirtualenv -p `which python2` my_blog
(my_blog)$ bundle exec rake generate

So now I’m running a Ruby command in a Ruby environment (rbenv) inside a Python 2 virtualenv. Maybe it’s time to switch blog tools again…

It’s occasionally useful to be able to create a Django model class in your unit test suite. Let’s say you’re building a library which creates an abstract model which your users will want to subclass. There’s no need for your library to subclass it, but your library should still test that you can create a subclass and test out its features. If you create that model in your models.py file, then Django will think that it is a real part of your library and load it whenever you (or your users) call syncdb. That’s bad.

The solution is to create it in a tests.py file within your Django app. If it’s not in models.py, Django won’t load it during syncdb.

1
2
3
4
5
6
7
8
9
10
11

from django.db import models
from django.test import TestCase

from .models import MyAbstractModel

class MyTestModel(MyAbstractModel):
    name = models.CharField(max_length=20)

class AbstractTest(TestCase):
    def test_my_test_model(self):
        self.assertTrue(MyTestModel.objects.create(name='foo'))

A problem with this solution is that I rarely use a single tests.py file. Instead we use multiple test files collected in a tests package. If you try to create a model in tests/test_foo.py, then this approach fails because Django tries to create the model in an application named tests, but there is no such app in INSTALLED_APPS. The solution is to set app_label to the name of your app in an inner Meta class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

from django.db import models
from django.test import TestCase

from ..models import MyAbstractModel

class MyTestModel(MyAbstractModel):
    name = models.CharField(max_length=20)

    class Meta:
        app_label = 'myappname'

class AbstractTest(TestCase):
    def test_my_test_model(self):
        self.assertTrue(MyTestModel.objects.create(name='foo'))

Oh, and I almost forgot… if you use South, this might not work, unless you set SOUTH_TESTS_MIGRATE to False in your settings file.

Comments and corrections welcome!