Human Transit

what makes a good "planning game"?

People learn from doing more than by being told, so public outreach is increasingly turning to various kinds of simulations, which can broadly be called planning games.  A planning game is any interactive exercise that allows people to play with the tools of planning, under some kind of budgetary constraint, and thus to experience the hard choices that arise from the material. 

You've probably played with online tools, often published by newspapers, that invite you to "balance the government budget yourself."  Planning games drill down to a more detailed level than that, but they share the same spirit:  They invite the citizen to express her views through grappling with the actual problem that government is facing.  When used as outreach tools, the rule of these games is: Listen and educate at the same time. 

Planning games can be group exercises in public meetings, but of course they can also happen on the web.  Web-based versions are better for allowing an individual to explore at her own pace, but they don't fully expose the user to the diversity of opinions and needs in a community.  Public workshops tend to be much better for that.

I'm inspired to return to this topic because of an intriguing style of planning game developed by Community Transit in suburban Snohomish County, Washington (north of Seattle).  I'll talk first about the kind of planning game I do -- and the one on which my network design course is based.  Then I'll talk about the very different approach that Community Transit used.

Geography-Based Planning Games

To build stakeholder consensus around a transit plan in a difficult area, TransLink in Vancouver BC uses a geographical planning game.  I helped them hone this tool for their South of Fraser area in 2006 and they have used it successfully in other areas since then.  It can be helpful in many other contexts.  This planning game technique, applied to a fictional city, is also the core of the interactive course in network design that I teach.

For a long-range infrastructure plan, stakeholders would be gathered in groups of about six around a map of their community, with a layer of clear acetate over it.  We'd give them some tools:  the red tape is an elevated metro, the blue tape is light rail, the green tape is frequent bus service.  Here, fellow citizens:  We have 24 kilometers of green tape that we can lay out.  You can trade five km of green tape for a km of blue, or ten km of green tape for a km of red.  Design your own system, but experience the process of making hard choices as you do.

A similar exercise can be done when doing a short-range bus service redesign.  Here, you're just dealing with frequencies of service rather than technologies, so the costs are more obvious.  Red tape is a bus every 15 minutes, blue is a bus every 30 minutes, green is a bus every 60 minutes, so a kilometer of red is worth two km of blue or four km of green.

 At the end of the work at tables, different groups of citizens would have come up with different networks for the same area.  So we'd put the resulting maps on the wall and I'd lead a discussion about how different groups had solved the problem differently, and get citizens talking to each other about why they approached the problem in various ways. 

At the end, we understood their views, but more important, they understood each other's views.  They also understood the underlying problem facing the transit agency, so they could form more useful and constructive ideas in the future.

In my course, I add a number of other features to this tool, including different ways to analyze the resulting networks, and also ways to humanize the issues.  I also emphasize how the geography of transit generates choices among competing values, which is why citizens and their elected officials ultimately need to make the decision. 

I advocate geographically-based games because the network design problem happens in geographical space, and you can't really see either the obstacles or the opportunities unless you look at a map.  But Community Transit did it differently ...

The Community Transit "Transit Values Exercise"

Like many US agencies, Community Transit is having to make a steep service cut this year, around 20% of their total service.  They wanted a way to engage citizens in thinking about the choices that this implies, so they invented their own "Transit Values Exercise."

Instead of talking about geography at all, they forged a set of 15 fictional people who represent slices of their market.  The question for participants was:  "Which of these people should we no longer serve?"

Each person is represented by a card:

The narrative on the left is Cathy's story, but we're to understand that there are many people "just" like her.  Services that meet the needs of these people have a total cost of 7 points (lower right) out of a total of 125 for all the cards.  Icons on the left tell us that Cathy doesn't have a car (or a bike, or a wheelchair) and gives a sense of the diversity of trip purposes that she uses transit for. 

On the right, the five bars describe the consequences, for the whole network, of serving Cathy and people like her.  Note that she lives in a rural community, so it's not surprising that (compared to a whole network that also serves a larger urban area) she tends to need services that have high cost to operate (first bar) and require high subsidy (second bar.)  Like all rural services, the routes Cathy rides cover a large area (high "Coverage") but have very, very low ridership by system standards.  The middle bar, "Efficiency," refers very narrowly to schedule efficiency: the amount of dead running required.

(Obviously there's a little redundancy among these five, since subsidy itself is a result of ridership and cost [as well as fare levels, not modelled here].   Logically, subsidy could have been omitted, but it does tend to pique the interest of more conservative participants.)

Here's another card:

You get the idea.  At the APTA Multimodal Operations Planning workshop on August 17, a roomful of professionals had a chance to play with this tool, choosing different people to "discard" (harsh, but that really is what we're doing) and then seeing how that choice effects the system's cost, subsidy, efficiency, coverage, and ridership.  For example, if you discarded only the patrons associated with low-ridership routes, you can cut almost 50% of the system's total coverage area while cutting only 20% of cost and barely 10% of ridership.  Rural coverage is a lot of area, and almost no riders.

It was clear that the Community Transit exercise would teach citizens about basic budgetary tradeoffs, and obviously it gives those tradeoffs a human face -- or at least a set of demographics with a name [though not an ethnicity, an annual income, or a political persuasion].  The bracing task of jettisoning human beings, like deciding who gets the lifeboats on a sinking ship, certainly must have impressed participants with the gravity of the problem, and perhaps, in aggregate over time, such tools could motivate more support for funding sources that could change the picture.

But it's still about geography ...

In the last stage of the Community Transit exercise, participants are finally given a map of the fictional community where these people live:

Knowing this, we were to make one final assessment about whom to discard.

For me, this part was a problem, because the geography revealed so much more than the game wanted us to think about.  Now, for example, it was apparent that Cathy lives on the way to where Pat lives, so if we "discarded" Cathy but kept Pat we knew Cathy would still have some service.  Yet there was no way for the game model to capture this obvious fact.  I would probably have introduced the map much earlier in the exercise, and thought about how to integrate its geographic information into the "human" discussion about which cards to discard.

Ordinary citizens often assume that a transit agency's service to them reflects an assessment of what they need or deserve.  In fact, this is only true if you're at the outermost end of a line, where the line is serving only you.  Everywhere else, transit efficiency lies in combining the needs of diverse ranges of people onto a single vehicle, and this relies, above all, on taking advantage of situations where multiple people, often with very different demographics, are travelling along the same reasonably direct path.  If you're on such a path you'll get better service than if you're not, even if you are exactly the same person in all other respects.

I wonder if, by steering us away from noticing the way transit combines diverse markets, the Community Transit game may have missed an opportunity to educate about how transit efficiency actually works.  By presenting us with people whose needs are cheap or expensive to serve, the exercise may confirm a widespread and false assumption that transit planning really is about assessing the merits of interest groups and communities. 

That's not what it is at all.  Your transit service isn't a judgment about who you are, but about where you are!  The barriers and opportunities presented by your location (and the location of places you go) are what determine how much service you can expect.  We need to put a human face on our work, and help people understand the human consequences of changes, but we also need to help customers focus on the geometric fact of life.  More than anything else about you, your location matters.

public input into ongoing projects

A frequent reader asks

In your experience, what are the most effective means of maintaining public input into an ongoing transit project? Assuming they are a possibility, are formal advisory committees the way to go? Informal contact with the project team? Public meetings? A project storefront? What do you do to ensure that public concerns have some weight as the concept is translated into perhaps a less-than-ideal reality? If you have citizens' committees, do you prevent the involvement of people interested in seeing the project fail? For all of these, I am interested in the perspective from both sides - the public and the professional - and in any tips you might have.

Professionals, please leave your thoughts in the comments, including links to good resources on this.  It's not my core speciality.

But in my experience there are three questions here:

• What media should be used for public communication?  On this, I think the best practice is "get the information out there in every possible medium, and invite comment in every possible medium."  Inclusion of non-techie people is important, which is why snailmail still matters.  Public meetings require so much effort from the participants that they tend to attract only people with strong views, leading to unedifying shouting matches.


• Are there inner and outer circles of "the public"?  One common strategy is to appoint a "project committee" or "stakeholder committee" of interested people, with the idea that these people will get to know the project better, debate it more deeply, and engage with the larger public about it.  This last bit is usually what's missing.  These committees really need to reflect stakeholder communities and participants must feel obliged to represent those communities, not just their own point of view.

But the hardest and most important question is "What is the public being asked?"

I think it's very common to ask the public very general "what do you think?" questions, on the assumption that this lets everyone express their view.  It does that, but the answers to such vague questions are almost impossible to use inside the study, and a good part of the public will sense that. 

That's why I try to use questions that ask the public to consider the real choices facing the city or transit system.  That requires a process that listens and educates at the same time, and in which project planners give the public information and a framing of the problem.  This post, despite a dead link, is a pretty good overview of that mode of thought.  My network design course is also based on "planning games" that allow stakeholders to experience the tradeoffs themselves.  It's the same idea.

social media's influence on public transit (guest post)

Guest Post by Daniela Baker

Daniela Baker is a social media advocate at CreditDonkey where she helps entrepreneurs find small business credit cards.  She grew up in Europe where public transportation plays a huge role in everybody's life.  Public transit "consumers" like Daniela are finding their voice and publishing it on the web for all to read. With that newfound confidence, they are discussing political issues, gaining support for changes to be made when it comes to public funding and increased public transit choices.

With more and more people on the road and gas prices on the rise, Americans are looking for a better alternative to commuting by car. Unfortunately, many transit systems can be hard for the newbie to decipher, or fall short in offering what citizens are after. Even more unfortunate, many people are unaware of the options their towns offer when it comes to transportation.

When consumers aren’t finding what they’re looking for when it comes to public transportation, they are turning to social media to help garner the influence they need to get the services they’re after. This has left many professionals scratching their heads, uncertain how social media can influence a public service.

Here are some answers as to why consumers are turning to social media when it comes to public transportation, as well as how consumers and cities are driving the social media efforts.

Americans’ sentiments toward public transportation

A poll released in March 2010 by Smart Growth America and Transportation for America found that Americans are craving more transportation options. The poll indicated that Americans would be open to doubling the amount of funding that is currently being funneled toward public transportation.

The poll found that Americans are frustrated with the transportation options that are currently available. In fact, 73 percent of respondents stated they have no options other than driving as much as they currently do. In the study, only 1 in 5 of those polled took public transportation during the previous month (this included walking) but indicated they would like to use it more; about 47 percent indicated public transportation is not an option in their area and 35 percent said the timing of routes did not work with their schedule.

These survey results were not only for metro areas but applied to suburban and rural areas as well, with respondents stating that rural areas would also benefit from increased transportation systems. In fact, 79 percent stated that in rural areas the U.S. would benefit from expansion and improvements made to both bus and rail systems. Eighty-two percent of suburbanites shared the same sentiment.

Why social media

In the past, when a consumer was looking for information, they would go to the local library to research an issue or visit town hall for information on public services. If they were unhappy about the services provided or felt that there was a need that was being missed, they would meet with their elected official.

Nowadays, citizens are going straight to the Internet to get answers and try to make change. With social media like blogs, Twitter and Facebook, and websites created by the common consumer, people are making real connections with people who they have never met in person.

Nielsen Research wanted to see just how large of an influence social media has on today’s consumers. This question was answered with a study they conducted in 2007. When asked what sources they trust, chat/discussion comments and blogs ranked two and three on the list, just behind other consumers. Other choices included brand websites, TV/magazines, radio, sponsorships, search ads and banner ads.

Citizens making public transportation more convenient

Tech savvy individuals have started to take action by creating social media outlets that help fellow commuters find the information they need to make public transportation options work with their commute.

PBS.org has profiled one such effort – IAmCaltrain.com, which is utilizing web technology to make regional commuter train schedules easier to decipher. The site allows commuters to type in their starting and ending destinations and it maps the closest train stations and shows when trains are scheduled to arrive.

As websites like IAmCaltrain.com and commuter-driven blogs continue to be created, they are encouraging others in their area to embrace the transportation systems that are available to them as well as voice their opinions about what could be improved.

Cities increasing awareness of public transportation

Cities across America have started to embrace social media to help encourage their citizens to take advantage of the public transportations that are available to them. June 16 was the 6^th Annual Dump the Pump Day, sponsored by the American Public Transportation Association (APTA). APTA partnered with cities across the U.S. to encourage citizens to park their cars and utilizing public transportation for the day.

As reported by StatePress.com, the City of Tempe was one of the public organizations that participated in APTA’s annual event. City employees turned to a method they knew would be effective to get the words out to Arizonians—they went to Twitter and Facebook to encourage residents to choose public transit, showing the cost savings available through using alternative modes of transportation (gas prices in the Phoenix area have been hovering between $3.50 and $4.00 per gallon since 2008).

Outside of government, citizens and the private sector are asserting greater control over transit information.  Google Transit is now the go-to source for routes and schedules over much of the world, and many transit agencies are increasingly deferring to Google rather than maintaining their own expensive systems. 

More importantly, Google’s standard public data formats let anyone access route, schedule, and even real-time location data.  As a result, entrepreneurs have developed countless web and phone applications to present and customize transit data, so that customers can increasingly choose the style and emphasis that they want in their own information.   Down at the grass roots, activists are even drawing new styles of map for their transit systems, promoting these through social media, and getting their transit agency’s attention as a result.

What Comes Next?  Innovation

Most leading transit agencies now have Facebook and Twitter accounts, and some put significant effort into both listening and communicating via these tools.  The trick, of course, is not just to listen for suggestions and opinions, but also to notice how the whole communication task is evolving as new tools are invented and new ways of using them arise.

The hardest challenge for transit professoinals is simply to be open to innovation arising from the social media sphere.  Sometimes, online innovators will do something better, and more cheaply, than a transit agency can do it.  As that happens, the transit agency's interest may lie in encouraging public innovation, not trying to control or limit it.

mapnificent breaking through?

As WalkScore's fine transit travel time tool languishes in extended beta, the alternative, Mapnificent, is getting some mainstream blog attention. 

I may still be alone in this, but I as I explored with WalkScore's Matt Lerner here, I believe this tool, whoever finally perfects it, has revolutionary potential.  It can easily be converted into a two-digit transit score which, unlike the WalkScore Transit Score, actually describes people's ability to get places.  But it's bigger potential is as an alternatives analysis tool.  When you city is facing a series of possible alternative transit projects, what if every citizen could use a tool like Mapnificent to see the exactly impact of each alternative on their mobility, and that of people and destinations they care about. 

A major problem in transit politics today is that negative impacts of a project are obvious but benefits are often described in terms of ridership and development outcomes -- things that don't matter to the selfish present-minded citizen.  We will always have selfish present-minded citizens, and I'd rather work with them than complain about them.  Until we help people see the way a proposed project will change their lives for the better, sensible transit projects will continue losing these debates.

what rivers teach about transit

Cartographer (and artist) Daniel Huffman is drawing river systems as subway maps.  He insists it's just for fun, but these maps push an important button for me.

Leigh Holcombe's email pointing to Huffman's work is fascinating timing, because I was just about to write a section of the book on how branching dissipates frequency.

Really, these fanciful maps are a bit reminiscent of some real transit maps, aren't they?

Metlink's map of the Melbourne train network has important things in common with a river system map.  First, both networks are radial, which means that all lines converge to flow to a single point (New Orleans for the Mississippi River, the City Loop for Melbourne's trains).  And necessarily, both maps are full of branching:

But branching always divides frequency.  The Melbourne map gives a superficial impression that Lilydale, Boronia and Ringwood all have the same kind of transit service.  They certainly all have train stations, but the branching means that Ringwood has to have more frequent service than either branch, and that may be the difference between a service that can be used spontaneously and one that requires you to build your life around a timetable. 

It's interesting to speculate how transit policy might change if everybody was trained to be suspicious whenever they see this ...

... because this always means one of three things.  Either (a) points beyond the branching point have less frequent service or (b) one of the branches operates as a shuttle, requiring a connection, or in a few rare cases (c) the train itself comes apart, with some cars proceeding along one branch and some along the other.  Geometrically, it has to mean one of those things.  And before you decide whether the service is useful to you, or whether you support a proposed transit project whose map looks like this, you should ask which of those it is.

I've always been partial to mapping styles where a branch is rendered as a wide line splitting into two narrow lines, such that the width of the two narrow lines adds up to the width of the main line.  In my presentation "A Field Guide to Transit Quarrels," I used that approach to clarify one of the possible branching patterns of BART's San Francisco Airport line (though this is not the pattern that runs today.)



Because in that respect, transit lines really are like rivers.  Just as two converging rivers combine their volume of flow, two converging transit branches combine their frequency.

online "map movies": useful?

Can animation help people understand their transit options?  The Rotherham Metro Borough Council in the UK has done some simple "map movies" that highlight the paths followed by buses and trains.  Here's a still:

Watch the actual animation here.

As they stand, they're limited in usefulness, as the icons move along the routes with no indication of frequency.  They certainly do advertise complexity, which is accurate; this looks like a very complicated network.

But it's easy to imagine taking this to the next step, showing by animation the scheduled paths of all the services in a transit system.  This would be especially helpful in helping citizens understand pulse systems, where the integrated scheduling pattern is an essential part of how the network gets you where your going.  Now that I think of it, I'm pretty sure this has been done, but I've never seen it on a public information website, which is the obvious next step.

beyond "transit scores": an exchange with matt lerner

Matt Lerner of WalkScore.com and I recently exchanged emails about WalkScore's "Transit Score" product, which provides a two-digit score supposedly capturing the usefulness of transit at any address in the USA.  It's designed on analogy to the successful (if still controversial) Walk Score, a similar tool for summarizing how friendly a place is to walking.  For example, 300 Turk Street, San Francisco is scored 100 (perfect) on both Walk Score and Transit Score.

Toward another extreme, here's 7000 Lake Mead Blvd in the far east of Las Vegas.  Walkscore is 52, Transit Score is 33.

Tools like these have huge potential relevance to the real estate industry, and more broadly to anyone who makes locational decisions about anything.  To the extent that we encourage people who value transit to locate where good transit is viable, everyone wins. 

Earlier, Matt's team had created a "transit travel time tool" which can be used to show you the actual area you can get to in a specified amount of time.  I used this tool as the core of my definition of mobility, in one of this blog's earliest posts.

So which is more useful, the simple two-digit Transit Score, or an actual map of where you can get to in a given time?

Here's what I wrote to Matt:

Dear Matt,

About a year ago I mentioned your transit [travel time] tool as a useful way to visualize mobility ...

The first draft of the book I'm writing ... praises this WalkScore tool in some detail, as a way for people to understand the degree of freedom that transit will offer them.  I think it may be a crucial tool for helping people see beyond modal fetishes to understand how transit actually works and how to determine if transit can actually get you where you're going.

...  Transit Score is useless to me because it encodes an intrinsic bias toward rail modes, as though rail is intrinsically faster, which is utterly misleading in a world of 6.5 mph streetcars and 60 mph busways.  [I'm referring to the Transit Score methodology's use of a "mode weight" defined as "(heavy/light rail is weighted 2X, ferry/cable car/other are 1.5X, and bus is 1X)".  Note that unlike the travel time tool, Transit Score lacks a way to capture how far you can get how fast, so it uses mode as a proxy for speed, fatally in my view.]

In short, whereas the earlier tool presented raw information in a compelling way that the user could use for her own purposes, Transit Score contains value judgments ("we know you'd rather ride a slow streetcar than an express bus") which the user may not share, and thus may be an obstacle to her ability to make transit meet her needs.

So I guess I will praise mapnificent.net instead.

This issue sits at the very foundation of my current book, so as an admirer of your work, I'd like a better understanding of why you abandoned the original accessbility tool. Was it a matter of the processing required to do what's basically a massive search of trip planners?  I can see that Transit Score is a much faster calculation, but wonder if that was the key issue.

All the best for '11,

Jarrett Walker

Matt replied:

HI Jarrett,

Good to hear from you.

You're right that we are promoting Transit Score over Transit Time Maps -- but we haven't abandoned the transit time maps.  We are working on new ways to use them and they are still available here: http://www.walkscore.com/transit-map.php).

Here's why we've been promoting Transit Score more heavily.  Our mission is to promote walkable/transit friendly neighborhoods and we think the best way to do this is to have Walk Score / Transit Score on real estate listings.

As we developed Transit Score we looked at a few methods of calculating a Transit Score.  The two finalists were our current method (detailed here) and a method where we summed the Walk Score under the area of the transit shed in our Transit Time Maps.  In practice, the scores were very similar between the two methods so we chose the current method which is much easier to scale (we show millions of scores per day).

We want to boil down transit access into a score so that it can appear on real estate listings and people can compare locations.  ZipRealty.com has added our Transit Score to millions of listings and we have some more partners on the way. 

We did not see a lot of consumer/partner interest in the transit time maps -- which is unfortunate because of course I love them.  One scenario I'm hoping to promote with our Transit Time Maps (we just need a partner) is to allow people on real estate sites to search by transit time.  E.G. find me an apartment within 30 minutes of work on transit.  I'd also like to integrate the transit sheds into the standard Walk Score experience.

I like the simplification Mapnificent made which was to not include walking time in their transit sheds -- this makes it much easier to compute.

So to sum up, I like your suggestion of avoiding any mode weight value judgments -- it just turns out that in practice our current method was very similar and easier to scale. Anyway, would love to hear your thoughts on this. 

Also, we're launching a beta of "Street Smart" Walk Score later this month and we'd love your feedback on that too.

Happy New Year!
Matt

To which I replied:

Matt

Thanks!  ... Suppose, hypothetically, that you had the processing power and datasets to run a purely mobility-based Transit Score.  Let's call it a Transit Mobility Score.  The algorithm would be something like:  Identify the area reachable in 30 minutes on transit from the selected point (I choose this because it seems to have a long history as an acceptable commute time).  Then, grab the MPO's database of population and jobs by small zone and calculate the percentage of the region's jobs and population that are in that 30 minute band. 

That also gives you a 2-digit number, but now it's a fact rather than a score.  You're saying: "If you locate here, you'll have convenient transit access to __% of the region's activity."  And that strikes me as something that a realtor could value, understand and explain.  How far are you (or someone) from being able to do that kind of algorithm?  Obviously you'd need to have the MPO on your side, so start with somewhere like Portland where they generally are.

Frankly, you'd probably want to do two such scores, one based on your mobility at 8:00 AM and another for your all-day mobility -- say, at 1:00 PM.  But I think a realtor could make sense of those too.  Everyone understands that transit in the peak is different from midday, and that both matter.  Suburban areas, especially those under the influence of commuter rail and commuter express buses would show quite a difference between the two.

I think this could be huge.  Because the output is a fact, not a judgment. ...

All the best, Jarrett

Matt replies that this "sounds like an awesome planning tool and one we could potentially pursue if we had a grant or something to fund it."

I still think this could be huge.  What if everyone making a locational decision could go to something like the WalkScore travel time tool or Mapnificent and see a map of where they'll be able to get to in 30 mintues on transit?  It would finally make our mobility visible. 

And if we could measure our mobility so accurately, for so many hypothetical cases, we just might value actual mobility more, and be less distracted by unreliable symbols of mobility -- like, say, whether there are rails in the street.  Technophiles shouldn't be too alarmed; many people will still have modal preferences.  But meanwhile, those of us who just want mobility would be able to measure it, fast, for anywhere that we might be locating something, including our homes.

can you balance california's budget?

Bravo to the Los Angeles Times for this California Budget Balancer "game".

I've long believed that the only way to nurture a civilized democracy is to give voters the opportunity to struggle with the real choices that government faces.  My own work has included developing games and decision tools that enable communities to think about their choices in transport and urban form. 

Budget balancers bring the same principle to the hard choices that are expressed in a government budget.  Everyone's balanced a household budget, or at least monitored their spending and seen its consequences. 

Yes, they contain a million assumptions and simplifications, but once you've played with them you may well ask:  "And why does the real thing need to be so complicated?"   Good question. 

chicago: asking the right question?

Alex Davies in Treehugger looks at a Chicago campaign seeking public input on what would make people use sustainable modes:

Over the last few weeks, Chicagoans have been asked a simple question: "What would encourage you to walk, bike and take CTA (Chicago Transit Authority) more often?" Ads posing the question in buses, subways and public spaces invite the city's residents to respond with their ideas, via text message. This mass call and response is part of Give a Minute, a campaign created by advocacy group CEOs for Cities and media design firm Local Projects, which looks to take public dialogue out of town meetings and into the streets. The idea is simple: by making participation as simple as sending a text message, Give a Minute will bring more people, and ideas, into the debate.

We've long known that this kind of outreach is needed to have a useful conversation about new transit projects.  The old solution -- inviting people to attend public meetings -- has always produced highly skewed responses, over-weighting the angriest feedback.  Anything that can broaden the conversation is useful.

But for reasons explored here, I generally advise against grand hypotheticals: "What would you do if ...?" or "What would encourage you...?"  Nobody can answer those questions accurately, so the answers are a poor guide to what investments would really pay off.

What's really needed is to engage the public in thinking about real tradeoffs, such as the choices built into budgets.  What are your priorities, this or that?  Those hard choices are the real work of elective government.  If we ask people to think about those questions, we not only get more reliable answers, we also encourage citizens to ask smarter questions themselves.   

public surveying: the quicksand of hypotheticals

A recent post looked at the challenge of surveying the public and identifying what mixture of taxes and fees they would be willing to pay to fund a widely desired infrastructure plan.  In the Sydney Morning Herald's Independent Inquiry into public transport in Sydney, we did exactly that, using a survey team from the University of Technology at Sydney's Centre for the Study of Choice.   One commenter caught the crucial point about why polling is so difficult, and why its results are often hard to trust:

There's always a difference between what people say they want, what they actually want and what they actually do.

Indeed there is.  In the surveying biz, "what people say they want" is called stated preference.  "What people actually do" is called revealed preference.  Everyone prefers revealed preference data.  We try to glean what we can from our current ridership data, for example.  But revealed preference data is all about the past, and conditions are different in the future for which we're planning.  In fact, one common sign that a transit agency is conceptually stuck is when they think and talk only about their present riders, not new ones they intend to attract.

Infrastructure projects are all based on assumptions about how people will behave in the future.  To talk about those, we can talk about revealed preference from other places.  For example, we can quote ridership on one transit line as a reason to expect ridership on another. 

But for a city like Sydney, which cannot dream of the kinds of Federal funding that US cities are used to, the real question is what people will tolerate paying -- in taxes, fees, fares, and road charges -- to fund a system that they support.  And for that, we have to ask them.  And we're asking them about a hypothetical:  "What would you be willing to pay if ...?"   We're stuck with stated preference data, and most of it is worthless.

You've probably done stated preference surveys.  In our business, they ask questions like "Would you ride transit more if ...?"  or "Would you support a tax increase of $x to fund this rail project?"   Most people have no idea.  The question may not accurately describe the factors that would really determine their actual response.  They may not understand what the funding sources (tolls, fares, etc) would mean in their lives.  So people guess.  They make stuff up.  They say what they think the surveyor wants to hear.  They give different answers depending on the sequence in which the questions are asked.  They generate mounds of meaningless data.

So we (or rather our survey experts at the University of Technology, Sydney) tried something a little more subtle, called a discrete choice experiment.  It turns out that if you aggregate a lot of stated preferences from the same person, they can add up to something like a revealed preference.  

We wanted to find a combination of funding sources that would attract majority support and be adequate to fund the investment program.  Rather than just ask people single hypothetical questions, we asked each person about a whole series of scenarios, each with different combinations of funding sources.  We compared a "high investment in public transport" scenario, which we already knew to be the most popular, with a "high investment in roads" scenario and a "low investment in both" scenario.  Respondents went through about 10 of these, and were asked to identify which scenario they preferred, but each time the specific funding sources and levels were different.  Here's what just one of the questions looked like.

(Obviously, this has to be done on the web rather than over a phone.  Our survey "panel" was 2400 people who are used to doing surveys on the web but who are, in all other respects, strictly representative of the entire population in terms of age, gender, income and all the other usual demographics.)

By asking a person to think about a range of different scenarios, the survey could observe the funding  levels that were the end of each person's tolerance.  If one respondent tended to support the high-public-transport scenario except in cases where the fare increase required was more than $1, we could conclude that this repondent was saying, in effect "I support fare increases up to but not beyond $1 to fund the high-public-transport program. 

The key idea of a discrete choice experiment is that instead of asking people what their highest tolerable rate would be, it observed them making the choice.  Essentially, discrete choices experiments create an environment in which a stated preference can emerge as a revealed preference, something we see in a person's choices but not something we need them to be able to explain.

The results were pretty cool. We did find a mix of funding sources that got bare majority support, and that were sufficient to fund the infrastructure program.  We actually found a range of them, each supported by a slightly different majority group.  I'll talk more about them in another post, or if you're really curious, you can download Chapter 1 of our report here.