12/16/2011 Using Orthophotos

I recently purchased a data set of parcels, streets, elevation contours, and building footprints (shapefiles) for Manchester-by-the-Sea, Mass. The 2,366 parcels look GREAT!

I want them to be transparent, with a nice basemap showing through. When I started with Esri’s “Bing Maps Hybrid”, however, the features were not good enough – their Eaglehead Rd (light gray) does not really line up with the parcels/streets:

The map made with the Terrain basemap looks fun:

I was hoping for some type of satellite view. The type of photo is called an orthophoto. Any (initial) photo is taken through a single lens, and therefore what you see is an image coming together at a single focal point. In an orthophoto, the image has been corrected (“orthorectified”) so that each point on the photo appears as if you were directly above it (“ortho-“ is a word element meaning “straight”). Orthophotos are available from MassGIS (the Office of Geographic Information) http://www.mass.gov/mgis/colororthos2005.htm

By consulting their index, I downloaded the appropriate 6 zip files in Mr Sid format (“Contrast Stretched” looks best). After unzipping them, I opened them in ArcMap – it looks great!

The size of the orthophotos (1 file = 9.76 megs and covers an area 2.5 miles x 2.5 miles) make this impracticable for areas larger than a town or two, but for that level of analysis/display, they are a very nice layer. And you can’t beat the cost.

12/15/2011 Solutions – not Problems – With Projections

I felt that the maps in yesterday’s blog looked “squashed” and unrealistic (which, of course, is silly because only a handful of people have been far enough out in space to realistically see the Continental United States!).

“Projection” refers to how the three-dimensional data (our real-world coordinate system on our real-world globe) gets “projected” onto a flat piece of paper (or, in this case, a flat computer screen). I only worry about projections in two circumstances: 1) when the map “doesn’t look right” and 2) when historical boundary files do not line up with current boundary files.

First things first – the top map was created straight from the TIGER 2010 boundary files downloaded from the U.S. Census Bureau, and those files use GCS_North_American_1983 for their Geographic Coordinate System. The nice map below, however, was started by using a basemap from ArcGIS Online – the Light Gray Canvas, which has GCS_WGS_1984 for their Geographic Coordinate System.

By starting with a layer that has a nice projection (GCS_WGS_1984, in this case), ArcMap will overlay any additional geographic data files (after giving you an appropriate Warning):

The second case of projection problems happens when your data suppliers change their data-delivery projection. This happened to me in the mid-1990’s – prior to 1995, TIGER files used the North American Datum of 1927 (NAD27). Beginning in 1995 (and continuing through today), the TIGER files are referenced to the North American Datum of 1983 (NAD83); as a result, historical boundaries no longer lined up, and an entire set of Mapping Data had to be created anew (streets, Census Tract boundaries, Block Groups, Counties, Water, etc.).

In summary, if it looks good, you are fine. But if it looks bad, investigate your projection(s).

12/14/2011 NTSB recommended cell phone ban – How Safe is Your State?

Press Release: The NTSB voted unanimously today [December 13, 2011] for a nationwide ban on the nonemergency use of portable electronic devices (other than those designed to support the driving task) for all drivers.

This is certainly in response to the December 8, 2011 Press Release from the National Highway Traffic Safety Administration [NHTSA] stating that although highway deaths fell to 32,885 in 2010 (from 33,883 in 2009), 3,092 were in “distraction-affected crashes”.

The National Traffic Safety Board is an Independent United States Federal Government Agency charged with determining the probable cause of transportation accidents and promoting transportation safety.

I am wondering “how safe is Massachusetts” in regard to other states, at least as far as traffic accidents are concerned?

The NHTSA December 2011 issue of Traffic Safety Facts
link to the issue
lists Total Fatalities, 2009 and 2010, by State in Table 8.

The NHTSA works with the Fatality Analysis Reporting System (FARS) which provides yearly data regarding fatal injuries suffered in motor vehicle traffic crashes.

The Research and Innovative Technology Administration (RITA) coordinates the U.S. Department of Transportation’s research programs, and has a division called the Bureau of Transportation Statistics (BTS) which publishes Number of U.S. Vehicles – total Highway registered vehicles were 254,212,610 in 2009.

The Federal Highway Administration (another part of the U.S. Department of Transportation) issues Table MV-1 each year (data for 2009 was issued January 2011), which lists, by State, Motor Vehicle Registrations – this means that instead of saying, in 2009, “the whole US had 33,883 highway deaths, distributed over 254,212,610 registered vehicles”, we can show the distribution spatially, state by state:


This first map shows count of deaths in each state – California, Texas, and Florida have the highest counts, followed by Georgia, New York, North Carolina, and Pennsylvania.

But I am not really a fan of “raw count” data – how does it relate to the number of vehicles registered in each state? The ratio2009 numbers are deaths per state/registrations per state, multiplied by 10,000, and they show California, Texas, and Florida as “not that dangerous”:


But now stay away from Arkansas, Colorado, and Mississippi!

One wonderful aspect of GIS/Computer Cartography is that you can display two (or more) sets of data at once. Working from the ratio2009 map, I added the raw count of highway deaths (as a graduated circle):

Is Massachusetts safe? With 340 highway deaths, 17 other states have fewer fatalities. But, with 5,420,020 vehicles registered, and 340 highway deaths, Massachusetts actually has the lowest deaths-per-registered-vehicle number in the United States!

12/9/2011 Traffic Cams, part 2

A quick follow-up on yesterday’s “Traffic Cams” post. I am trying to keep my code-writing “skills” buried, so I like clicking/double-clicking/cut-and-paste. I made a spreadsheet of the four traffic cams, but I needed their Latitude/Longitude coordinates (or so I thought).

To get the Latitude/Longitude, I open ArcGIS Online
ArcGIS Online
(note: you do not need to Sign In for this feature)

Click Map, then change the BasemapBasemap to Bing Maps Road. Zoom-in on the area north of Washington, DC. Click the Measure button, then click the Location tool.

Position your cursor over the intersection of I-495 and River Rd and Latitude/Longitude appear on the fly (work off the Camera Locations map identified in yesterday’s blog):

Fill in the spreadsheet for the four cameras, and copy-and-paste their urls:

Highlight the 20 cells and Save As TrafficCams.csv
In ArcGIS Online, close the Measure box. Click the Add button, then click Add Layer from File:

Go choose the csv file, then click the Import Layer button. The file is uploaded, and the program determines the locations from the Latitude/Longitude fields, and zooms appropriately:

Change the symbols by clicking the little arrow in the Contents and going down and selecting Change Symbols:

Then click the Change Symbol button. In the Change Symbol popup window, I like the camera in the People Places font:

Select it, click Apply, click Done, and click the X to close the Change Symbols section:

When you click on a camera, you get its info window:


The program is smart enough to include the url field – click on More info and the traffic cam appears in a new tab (if you right-click on More info, you can select Open link in new window, allowing you to have multiple windows open at the same time).

To Share this map, you must Save this map. To Save this map, you must Login/Sign In to your account. After Signing In, I can Share this map with Everyone:

Here is the url link to map
Additionally, I can Embed in Website or Make a Web Application – sounds like tomorrow’s blog!

12/8/2011 Traffic Cams, part 1 (too much NCIS?)

Maybe I’ve been watching too much NCIS, but when I was looking at the Maryland Department of Transportation website for Coordinated Highways Action Response Team (their Interactive Mapping page)
link to Interactive Mapping page

I was fascinated to see a link to Live Traffic Cameras (what is the traffic – out there/rightnow!)

Ouch – what a boring list, and totally unhelpful for someone not familiar with the area. It says “To view cameras from a map interface please use Interactive Mapping.”

So I went back to the Interactive Mapping page, and clicked the Video/Camera tab:

Living in the Boston area, I have a natural desire to look at, but not be involved in, traffic. Zoom-in on the area north of Washington, DC, and select the Video Camera at I-495 & River Rd (MD 190)

Up pops a web-window with a live feed:

And, yes, that is a url
http://www.chart.state.md.us/video/video.asp?feed=c600bd2b00070075004d823633235daa
Click on it, and you get your own live-link to "I-495 - RIVER N"!
If you work off the Interactive Map, the single pop-up window gets replaced for each camera you click on. But if you grab each url, you can have multiple windows open!
I-495 W OF MD-97
I-495 AT BRADLEY BLVD
MD-185 - MD-410 N

At the Northeast Arc Users Group Conference in Saratoga Springs, NY a few weeks ago, I saw a presentation populating the info boxes on Google Maps with pictures and urls. Which means that I should be able to have these video feeds just appear on the maps! Sounds like a great project for tomorrow’s blog.

11/20/2011 Tornadoes!

I have recently become very interested in “GIS in Disaster Response”, and am investigating ways to serve our country in times of Disasters. Since more and more people are now living on a continually-unstable planet, “Disasters” are certainly not going to “just go away” – they happened in the past, they are happening now, and they will continue to occur in the future.

At a seminar I attended last week, one speaker spoke of his frustration at not having (geographic) information regarding “tornado damage” a few years ago in New Hampshire – the data had not been assembled, in one place, to allow the generation of $ damage estimates across multiple counties. (That situation has improved with the creation of the New Hampshire Mosaic Parcel Map)

This led me to think about a map showing tornadoes – their locations and their paths. Which led me to the Tornado History Project:

I selected Massachusetts (my home, but not a hotbed of tornado activity, or so I thought):

The map shows 152 tornados from 1951 through 2008, with 102 fatalities – nothing to make light of. At the bottom of the page, I click the link for the Storm Prediction Center’s (SPC) historical tornado data file, and get taken to the page where I can download csv files of Tornado/Hail/Damaging Wind:

This data page is a FABULOUS page! Scroll up, and there is a beautiful collection of heat maps, thematic maps, plot charts of tornado reports over time, small-multiple time series maps, monthly Summary Charts, and a map of Actual U.S. tornadoes for January through July, 2011:

Obviously I can not discuss all those visualization tools today – each will be a separate blog this coming week! But for today, I have discovered that, although each event is unique and tragic, tornadoes occur repeatedly over time, the historical data exist, and the visualization tools are already out there. From the data comes knowledge, and with knowledge we can prepare ourselves to respond when disasters (in this case, tornadoes) occur in the future.

11/18/2011 Are Maps Dangerous?

In the November 14-November 20, 2011 issue of Business Week, the article on the drug war in Mexico (“The Drug War in Mexico, Now on the Blogosphere”, page 45) states that “The editors at Nuevo Laredo en Vivo recently compiled [Twitter] reports to create a map of drug sale locations and suspected lookouts.”
article link
The map was published in a blog on September 25, 2011
blog link

It is good that the blog published this map, because it is now unavailable:
link to map page

I do not speak Spanish, and I have not contacted the website to ask why the map was taken down. All I do know is that “The site’s contributors included Maria Elizabeth Macias, the beheaded victim”. It seems that yes, maps are dangerous, and the results can be fatal.

Congressional Redistricting follow-up
Regarding my Nov 8, 2011 blog, there have been some updates/corrections. Unfortunately for Texas, on Sept 19 the U.S. Department of Justice said that their Congressional redistricting map does not comply with Section 5 of the Voting Rights Act, and a court challenge was filed on Sept 23. An excellent discussion of the Texas situation may be found here
blog link

In Arizona, on Tuesday Nov 1, the Chair of the Independent Redistricting Commission was impeached by Governor Jan Brewer and the Arizona Senate, because the Governor did not like the maps that the Commission was generating. Unfortunately for the Governor, on Thursday Nov 17, the Arizona Supreme Court reinstated Colleen Mathis as chair of the Commission, ruling that the impeachment was not lawful. This is fascinating, but a little like watching a train wreck in progress.

Further sources on redistricting:
Michael McDonald, a redistricting expert at George Mason University in Virginia
Jennifer Steen, Assistant Professor of Political Science, Arizona State University
Meredith McGehee, Project Director, Americans for Redistricting Reform
Keesha Gaskins, New York University’s Brennan Center for Justice

11/8/2011 new Congressional District maps

Changes following the 2010 census
On December 21, 2010 the U.S. Census Bureau released its official apportionment results for congressional representation. The changes will be in effect for the U.S. elections in 2012, provided no legal challenges to the results significantly change the results.
Gain four seats
Texas
Gain two seats
Florida
Gain one seat

Arizona Georgia Nevada South Carolina Utah Washington

Lose one seat

Illinois Iowa Louisiana Massachusetts Michigan Missouri New Jersey Pennsylvania

Lose two seats
New York
Ohio
link to wikipedia article

The lead story on today’s (Massachusetts) news was 1 – over 50 Massachusetts cities and towns have elections today (out of 351 cities and towns), and 2 – the new Congressional maps have been released, showing how Massachusetts went from ten Congressional Districts down to nine.

My gut sense has been that although the maps for the State House and Senate were released in late October (and Governor Patrick signed them Thursday, November 3, 2011), the new map for the nine Congressional Districts would be released “as late as possible” to avoid rancor/conflict/stalemate. I want to see if Massachusetts is “quick” on this process, “slow” on this process, or “somewhere in between”. It turns out that every state has its own specific timetable – for example, the Florida Legislature is specifically required to redistrict the State House and State Senate districts during the 2012 Legislative Session [Florida Constitution, Art III, s. 16 (a)], and their Congressional Redistricting Plan is scheduled for their Legislative approval between January 10 and March 9, 2012 (there are currently 152 publicly submitted Florida redistricting maps). For Massachusetts, the key restriction is that, according to the Massachusetts Constitution, “Every representative, for one year at least immediately preceding his election, shall have been an inhabitant of the district for which he is chosen” – since the next election for State Representatives is Tuesday, November 6, 2012, the map for the State House must be released before November 6, 2011 in order to allow for the “one year residency” restriction. I believe the sense of that restriction carries over to both the State Senate and Congressional District elections/maps. Glad to see that we are on track!

Well, we already know that Florida is nowhere near close to getting their “official” Congressional District map. What about some of the other states listed above? Note: any state, regardless of losing or gaining Congressional seats, can redraw Congressional District Boundaries (except the following states which only have one Congressional Representative: Alaska, Delaware, Montana, North Dakota, South Dakota, Vermont, and Wyoming).

The new map for Texas Congressional Districts (PLANC185) is currently available as a shapefile (Texas PLANC185). The creation data within the shapefile is June 15, 2011, although it looks like the actual legislative dates are:
June 22, 2011 signed in the State Senate
June 24, 2011 signed in the State House, and sent to the Governor
July 18, 2011 signed by the Governor

Although Arizona’s Independent Redistricting Commission has approved a Draft Congressional District map (Arizona Redistricting Map), the Commission only recently (Saturday, November 5, 2011) completed the 2nd round of public hearings. They still need to issue the “Final Map”. Note: after issuing their “Final Map”, all states must engage in the Department of Justice preclearance process before the maps are truly finalized. They seem to have no “official timetable”.

11/5/2011 The Display of Information – Print and Online

There were news articles this week about the increase of poverty in the United States (The New York Times article link). The primary source is the report released by the Brookings Institute (“The Re-Emergence of Concentrated Poverty: Metropolitan Trends in the 2000s”, released Thursday, November 3, 2011). I am interested in these types of reports/articles for three reasons:
- as an American, I want to be aware of the ever-changing economic and demographic profiles of our society,
- as a person, I want to know how our society will deal with its problems, including poverty, and
- as someone who deals with data, I want to understand data, and understand and implement the most effective methods to transform that data into information.

The New York Times article

The online New York Times article consists only of text, plus four hyperlinks (“the report”, “The Truly Disadvantaged”, “Center on Poverty, Inequality and Public Policy”, and “earned income tax credit”) [the print version, of course, does not have even the hyperlinks]. Although there are mentions of geographic places (“Midwestern: Toledo and Youngstown in Ohio, and Detroit”, and “Sun Belt areas like Cape Coral, Fla., and Fresno, Calif.”), there are no accompanying maps or visualizations.

Just as it is important to discuss regional differences (Europe, Japan, North America, etc), and not just the entire “global economy”, an important part of the story of Concentrated Poverty in America is regional differences within the United States.

I went to the Brookings Institute website to both view their presentation of the Report, and to download the Report itself. Brookings link

The Brookings Institute Report

Before discussing the website, let’s look at the pdf Report. The 35-page report includes bar charts, number tables, a nationwide map, detail maps for Detroit, Dallas, Chicago, and Atlanta, appendices with the raw data, and two pages of endnotes.

The nationwide map shows colored dots (“circles that are all the same size”) over a map of the states. The dots are colored into four ranges, depending upon their “Concentrated Poverty Rate, 2005-09”. The colors are a good intuitive blue-color ramp (darker = more), but I would like to see the “count of areas” next to each colored dot in the legend:

15%< [24 areas] means there are 24 metropolitan areas with Concentrated Poverty Rates Greater Than 15%

As for the “circles that are all the same size”, welcome to “Displaying Data 101” – sometimes there is too much data to display on a single (static) map. In this case, because of the great number of metro areas in the Northeast, you can only see them all if they are relatively-small symbols (circles, squares, diamonds). Using different symbols for size variables could be used, but it might just make the map too busy.

Examining the map makes me wonder about applying Spatial Statistics to the data – examining each set of colored dots just gives me a “random” feeling, but that can certainly be validated through examining the Spatial Distribution of the data (and each data set). Another blog for another time.

The Brookings Institute online article

The Brookings online article is longer than the New York Times article (606 words versus 488 words), and includes the (linked) ability to “find concentrated poverty statistics for your metropolitan area”, plus a video. There are also two very nice maps of the 100 largest “metropolitan areas” in the United States. Although the maps are static (there is no slider to show changing data-over-time), the white non-outline of the light gray states provides the perfect background for the multivariate data.

Map 1 shows
- latitude/longitude location,
- size of circles equals current counts of people living in extreme-poverty census tracts, and
- color of circles indicates increase/decrease change from Census2000 data.

Additionally, the webpage takes advantage of the user interface, and offers the user a “hover over” ability – when you hover over a metro area, a detail popup appears with statistics relating to that metro area:


Map 2 shows
- latitude/longitude location,
- size of circles equals current level of Concentrated Poverty Rate, and
- color of circles indicates increase/decrease change from Census2000 data.

My next blog will discuss taking the data from the pdf Report into an Excel spreadsheet, then into Tableau Software for visualization over time.

10/4/2011 Sinkholes in Florida

I am in the process of purchasing a house in Spring Hill, Florida (Hernando County). In today’s discussion with the insurance company, they said “Well, of course, you need insurance for sinkholes.”

WHAT!!!!

Of course I have heard of sinkholes, but I thought they were very far-and-few-between – the stuff of headlines:

GERMANY - A giant sinkhole under a residential street opened up on Monday in central Germany. ...
GERMAN SINKHOLE link

GYPSUM (AP) - Sinkholes have been popping up near long-abandoned mines along a busy Lake Erie highway, and the state plans to fill in some of the tunnels before the holes reach the road. ...
Sinkholes near Gypsum prompt monitoring link

but, living here in relatively-geologically-stable New England, I think that “sinkhole insurance” is about as necessary as “elephant-through-your-living-room insurance”. But maybe Florida is different – let’s use our GIS resources and knowledge to investigate!

Because the tool is there, and so easy to use, let’s “Google Map” the house: 34606 Parlow Avenue, Spring Hill, FL:

And let’s zoom out to see the major highways on the east (589) and the west (19):


Also because it is easy to use, let’s Google “database of sinkholes”. Maybe Google knows I’m a GIS guy, or maybe I’m just lucky, but the first hit looks promising:

FGS, Sinkholes in Florida
webpage link

It is the Sinkholes page for the Florida Department of Environmental Protection – certainly a trustworthy source! Scroll down to the bottom, and in the Sinkhole Resources section, download and open Florida’s Sinkholes (Poster No. 11) Contact FGS to purchase this 24” by 36” poster


Depending on the size of your computer monitor, this is a pretty scary poster! Especially when you realize that Spring Hill is on the west coast just north of Tampa Bay! Let’s try to find some Good News.

First, though, what county is Spring Hill, Fl in? Wikipedia says that “Spring Hill is a census-designated place (CDP) in Hernando County, Florida”. We will see below that is a good assumption.

Back on the Sinkholes page, scroll down to the bottom, and in the Sinkhole Resources section, download and open Sinkhole type, Development and Distribution in Florida Map (Map Series 110). This is a beautiful thematic map of Florida, produced by the USGS in 1985, with soils divided into four areas related to sinkhole type:

Zoom in on Hernando County:


And we see that Hernando County has three-of-the-four areas:
Yellow is Area I – “Sinkholes are few, …”
Green is Area II – “… Sinkholes are few, …”
Blue is Area III – “… Sinkholes are most numerous, …”

So where is our property in Hernando County, in relation to these yellow/green/blue areas???

Open MapInfo, and open a County file, and open a Streets file, and zoom-in on Hernando County:


I have put a Gold Star in the area of the house. Comparing it with the USGS map, it looks like the house is in the green area under the “H” in “Hernando”. That’s Good News!

Maybe I should leave well enough alone/let sleeping dogs lie, but let’s see if there is a database of actual sinkhole locations. Back on the Sinkholes page, the left column has a Data & Maps link. Click it to get to the Data and Maps window:

By now, I realize that “sinkholes” are also referred to as “subsidence incidents”, which sounds much nicer. Click on the link for Subsidence Incident Reports to get to the Subsidence Incident Reports page:

Scroll down, and download the ESRI ArcGIS compatible shapefile. Unzip the file and bring it up in the MapInfo screen you worked on before:

Um, I don’t feel so happy anymore – there are sinkholes (“subsidence incidents”) all around the new house! Maybe the USGS should update its map/geologic database. Let’s make a copy of the sinkhole file (only County = “HERNANDO”, and only the oyear column [original reported year]), bring it up, and color the dots by oyear. This will allow us to see if the sinkholes happened back in the Sixties or Seventies, or if they are more recent:

Well, most happened in the 1990’s. Zoom-in on the house:

Using a radius tool, we see that the closest reported sinkhole is purple 0.4 miles to the west, and the closest red one is 0.6 miles to the northeast. I guess it is time to call the insurance company and get some “sinkhole insurance”.