{"id":206,"date":"2009-01-17T13:47:11","date_gmt":"2009-01-17T17:47:11","guid":{"rendered":"http:\/\/metcaffeination.net\/weblog\/?p=206"},"modified":"2009-01-17T13:48:40","modified_gmt":"2009-01-17T17:48:40","slug":"red-street-blue-street","status":"publish","type":"post","link":"https:\/\/metcaffeination.net\/weblog\/2009\/01\/17\/red-street-blue-street\/","title":{"rendered":"Red Street, Blue Street"},"content":{"rendered":"

In the end, I went with the upgrade to Mathematica 7<\/a>. Of all the new features, the one that really hooked me–which is comparatively minor, compared to all the other new features–is the ability to import SHP files. The importation is not terribly well documented nor is there much additional support, but it was pretty easy to do a few nifty things with the DC Street Centerline<\/a> file.<\/p>\n

As you may know, there is a street in DC for every state in the union. Pennsylvania Avenue is probably the most famous of these; the White House sits at 1600 Pennsylvania Ave NW. I used to live on Massachusetts Avenue. So my first idea was to make a street map of DC in which the state-named streets were colored red-ish or blue-ish depending on their vote in the recent election.<\/p>\n

Here it is:
\n\"\"<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/map>\n

Read on to see how I made it:<\/p>\n

<\/p>\n

I imported election returns from Wikipedia<\/a>, but the process of picking the data I wanted out of the whole Import<\/code> was so ugly that I’m not going to step through it here. I was inspired, though, by this similar post<\/a> on the Wolfram blog. If anyone at Wolfram is listening: it’d be really great if we could import data from the web by, say, clicking on a table of a web page displayed in a browser, instead of picking out which item in some nested list is the one we want.<\/p>\n

From the raw vote data, I computed the (approximate) percentage of the vote that Obama got:1<\/a><\/sup>
\n
\nntrs[x_] := N[ToExpression[StringReplace[x, \",\" -> \"\"]]]
\npercentData = {ToUpperCase[#[[1]]], ntrs[#[[2]]]\/(ntrs[#[[2]]] + ntrs[#[[3]]])} & \/@ votedata
\n<\/code>
\nWhich gives a list of things like
\n
\n{\"CALIFORNIA\", 0.622799}<\/code>.
\nThe conversion to uppercase is to match the way the DC street names are recorded.<\/p>\n

I’ll need a list of states, in order to match the street names. I get this from Mathematica’s curated CountryData<\/code> data set, using a Regular Expression to insert a space any time there’s a lowercase letter followed by an uppercase letter, to change, for example, “NewJersey” into “New Jersey”:
\n
\nstates = StringReplace[#,\u00a0RegularExpression[\"([a-z])([A-Z])\"] -> \"$1 $2\"] & \/@\u00a0CountryData[\"United States\", \"Regions\"]
\n<\/code><\/p>\n

And the street datafile itself. When Mathematica imports SHP files, it returns a list of layers; with only one layer in this file, we get a (long) list with one element. So I specify in the Import<\/code> that I only want this first (and only) element of the list:
\n
\ndcstreets=Import[\"StreetSegmentsLn.shp\",\"Data\"][[1]];
\n<\/code>
\nAnd this gives a list of 4 replacement rules, some of which are very long:
\n
\nIn[]:=dcstreets[[1]]
\nOut[]:=LayerName -> StreetSegmentLn
\nIn[]:=dcstreets[[2]]
\nOut[]:=Geometry -> long list of graphics primitives defining the data elements<\/em>
\nIn[]:=dcstreets[[3]]
\nOut[]:=Labels -> list of the names of metadata fields<\/em>
\nIn[]:=dcstreets[[4]]
\nOut[]:=LabeledData -> list of (long) lists of metadata for each data element<\/em>
\n<\/code>
\nFor this data file, the data elements are segments of streets between intersections, so the graphics primitives are lines defined by two (or more, for curved streets) points. There are 25 metadata fields, including the name of each street, the address range covered by the segment, the quadrant, and the type of road (e.g. Local, Principal Arterial, etc.). The FullForm<\/code> of Mathematica replacement rule, such as lhs -> rhs<\/code>, is Rule[lhs,rhs]<\/code>, so we access the data in each of the four rules as the second element of another list. Thus we can get at, say, the graphics primitives for the 27th street segment as
\n
\nIn[]:=dcstreets[[2,2,27]]
\nOut[]:=Line[{{396233., 138045.}, {396233., 138132.}, {396233., 138142.}}]
\n<\/code>
\nFor ease in working with these data, I combined the geometry and metadata into one table for the 13414 street segments:
\n
\ndcstreetdata =\u00a0Transpose[Join[{dcstreets[[2, 2]]}, dcstreets[[4, 2, All, 2]]]];
\n<\/code>
\nI separate this into two groups: state-named streets, and the rest:
\n
\nstatestreets =\u00a0Select[dcstreetdata, MemberQ[ToUpperCase[states], #[[6]]] &];
\nnonstatestreets =\u00a0Select[dcstreetdata, ! MemberQ[ToUpperCase[states], #[[6]]] &];
\n<\/code>
\nThe 6th element of each entry in dcstreetdata<\/code> is the name of the street. From this, I create two graphics:
\n
\nnonstatemap = Graphics[{Gray, nonstatestreets[[All, 1]]}]<\/code><\/p>\n


\nstatemap =\u00a0Graphics[Function[x,\u00a0Tooltip[{Directive[Thick,\u00a0RGBColor[.5 + 2 (.5 - x[[2]]), 0, .5 + 2 (x[[2]] - .5)]],\u00a0Transpose[Select[statestreets, Function[y, y[[6]] == x[[1]]]]][[1]]}, x[[1]]]] \/@ percentData]
\n<\/code>
\nI’ve packed a lot into statemap, using nested pure functions: it makes a Tooltip with the state\/street name popping up upon mouseover (it works better in Mathematica than in the export HTML). For each state from the list of election returns in percentData<\/code>, it finds all the street segments named after that state and makes a graphic element out of the geometry of that street segment, of a color that ranges from pure blue to pure red. The red-blue divide is stretched so that pure blue corresponds to the percentage of Obama’s highest percentage (Hawaii, 72.99%) and pure red corresponds to Obama’s lowest percentage (Wyoming, 33.43%). The final map is created with Show<\/code>:
\n
\nvotemap = Show[statemap, nonstatemap]<\/code><\/p>\n

So far, so good. I’ll write more on Mathematica and DC GIS data soon.<\/p>\n

  1. really, it’s Obama’s percentage of the combined Obama\/ McCain vote, ignoring votes for other candidates [↩<\/a>]<\/li><\/ol>","protected":false},"excerpt":{"rendered":"

    In the end, I went with the upgrade to Mathematica 7. Of all the new features, the one that really hooked me–which is comparatively minor, compared to all the other new features–is the ability to import SHP files. The importation is not terribly well documented nor is there much additional support, but it was pretty […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,12,26],"tags":[],"class_list":["post-206","post","type-post","status-publish","format-standard","hentry","category-dc","category-doing-the-math","category-politics"],"_links":{"self":[{"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/posts\/206","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/comments?post=206"}],"version-history":[{"count":6,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/posts\/206\/revisions"}],"predecessor-version":[{"id":213,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/posts\/206\/revisions\/213"}],"wp:attachment":[{"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/media?parent=206"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/categories?post=206"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/metcaffeination.net\/weblog\/wp-json\/wp\/v2\/tags?post=206"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}