Recently in hacking Category

Almost a year ago, I created a web app that would generate Google Earth maps with electoral districts and present/past results. This same app was adapted for use on national television (my employer until recently) and again adapted for the provincial election in Quebec, two months after the federal one.

Recently, a new app created by well-known (small c) conservative blogger Stephen Taylor (see video here below) was brought to my attention. What Mr. Taylor brings is in fact a very important and interesting level of granularity to the results. Elections Canada raw data for polling stations has for a long time been made available to everyone. What wasn't originally available last year was geographical boundary of the territory covered by each polling station comprised in an electoral riding.

This data is now available along the electoral district data that I originally found on GeoGratis (a Canada Natural Resources website).

The format provided is Shapefile. My original system relied on a MySQL database that kept plain decimal coordinates of electoral districts. Now, I've learned more about spatial geography tools such as PostGIS (a PostgreSQL extension, and the premier way of storing purely geographical features) and geo libraries like GDAL. Using shp2pgsql, I imported the data of this Shapefile to my database. This is currently where I am at.

I think that from there I will be looking at ways to import the results data to my PostgreSQL db. Then, I'll try to use Google's libkml library (code in Python) for managing and creating KML data (which is just XML really).

Because the ridings data is so heavy, perhaps it will be difficult to make a web-based app with every single one of the 308 ridings.

Maybe if this project takes off, I will try to start my contributions project again, which has taken a rest for lack of editorial need.

I've started putting some time again this weekend on the project that I was talking about last week.

To tell the truth, I had previously made this particular project work. In vague terms (I'll be less vague when I get something solid down), I was taking some huge sets of data from one particular national agency and making it meaningful on a geographic support, namely Google Earth. I did it, but it was terribly inefficient (what about loading thousands of points on the same map?) and unreadable (what if those points were all focused in urban areas?).

The next step was to use a less custom approach, with a web framework named Django, and its GIS libraries bundled into what's called GeoDjango. This framework can possibly use flat files as backend, but to store geographical data, the best choice is PostGreSQL and its extension PostGIS that adds the ability to store geographical data, as well as make operations of the kind of "select these points within a certain perimeter".

This is the neat part that I didn't have in my original project with a plain MySQL database backend. MySQL also has spatial extensions but does not support nearly as many spatial lookup types as PostGIS.

Now that I have my data in a database, I'm looking forward to play with the presentation part. It'll also be interesting to work on making this a compelling user experience.

I told myself that I would just write about what I'm doing in order to keep notes. So, this is basically just to keep track and for later to look back.

I'm working on a particular mashup for Google Earth. I actually had a version on a computer at work in December, but it was generated using a PHP program with a MySQL program. Besides having something that worked, my objective was also to make something that worked _efficiently_. Therefore, I've been trying to learn about new tools to do it more efficiently.

I first looked into web servers. Apache's HTTP Server is the default server everyone uses (if not IIS, guh). However, already with my last project, I had been experimenting with Lighttpd, one of those next-gen super-light web servers. nginx is the other that people use and is gaining ground as the fourth most popular web server according to Netcraft this month. Since I started with lighttpd, I decided to keep using it.

Then, I tried looking for a new language to use besides PHP, just because. I don't know if it's being tired of the same language one uses all the time or perhaps the reputation that other languages have. So this language would be Python, one language which I worked with in university during the course of one semester but didn't touch since then. The more you program, the easier you find picking up a new language is.

Then, I was looking for a Web framework, which greatly helps when developing web-based apps, which is what I want my application to be. So, I went for Django, which is today one of the leading Web frameworks for Python out there. Because geographical information is such an important component of today's world of mobile phones equipped with GPSes, then naturally there are geographical extensions written for Django which are part of GeoDJango.

From there, it's pretty exciting. I still need to see how I can store my geographically mashed data and how I can make retrieval efficient in Google Earth or any other platform supported by GeoDjango.

Everything is run from my laptop computer, because it's the best machine around over which I have entire control. Now, I should learn tricks about "getting work done" since that is definitely the most important hurdle right now...

Enter the valid URL to an online media resource:

I will write this entry (after this paragraph) in French, because it concerns a French-speaking audience. On Linux, people typically use the mplayer browser plugin to read ASX playlists. Perhaps like other sites, when a playlist is read, the plugin doesn't take into account that it may point to other ASX playlists, thus stopping before it played the media clip of interest. That is the case on Radio-Canada's excellent audio-video zone, which unfortunately makes it virtually unaccessible to Linux users with a standard setup (there must be ways to use other plugins or hack your mplayer to bypass this). In fact, some clips are read correctly since a couple of weeks, because of a change in the syntax of ASX files, and they are the ones with only one ASX under the playlist.

Yup, I messed around with templates, OpenID auth, and inadvertedly destroyed my styles. So now showing off the MT 3.2 default stylesheet... Which isn't that bad after all, but I suppose if I went out there, I'll realize I'm not the only person using MT-3.2. :P

Moving on... So I was trying a few plugins for 3.2, including one that allows authentication using OpenID, and Livejournal IDs. But unfortunately, yours truly is not very methodical in figuring out the use of MT tags, therefore I'll leave this for another night. It's not like troops of people are, like, dying to comment on my blog using their Livejournal IDs.

In other news, I activated some "tagging" plugin, which I guess, should help me classify the mess that is my blog?