A Product Team in Every US Transit Agency?

When it comes to in-house product development teams at transit agencies, the MBTA stands out, boasting a unique blend of size, longevity, and commitment to open-source software. Years ago, I approached Siobhan—who was then consulting for the MBTA’s Customer Technology Department (CTD)—about starting a non-profit together. The original idea was to help other government agencies build in-house teams like the MBTA had done. We codenamed this venture "GovDesign." It was just the two of us back then.

To prepare, I began researching CTD’s history, identifying the factors that propelled it from zero to one, one to ten, and ten to fifty; it became clear that our initial plan had a significant flaw. Much of CTD's earliest and most crucial successes had a single, prominent factor in common: luck. To make matters worse, CTD was navigating occasional existential threats, notably, each time the MBTA’s senior leadership changed—which is not uncommon. In good conscience, we couldn’t recommend that other agencies emulate a CTD-like model—there were some truly remarkable people and excellent practices—but they didn’t add up to a replicable model. So, instead, we pitched Paul on a different idea: making open source more sustainable for agencies, regardless of who was building, maintaining, or supporting it. And so we became TransitOPS.

Now, for the first time since Siobhan and I began collaborating, I'm genuinely excited about the potential for transit agencies to sustain in-house product teams. However, the key won't be replicating the MBTA's approach. Instead, it lies in adopting a model with broader applicability and a proven track record in the social impact sector. This model, combined with a product development approach, can make in-house product teams sustainable by aligning their impacts with the public's needs and agency leaders' strategic goals.

Technology decision-makers are often moved by the excitement of new technologies, the potential of innovation, and the promise of cutting-edge solutions. In government, we have to manage our enthusiasm for dazzling websites, apps, and dashboards to judge initiatives based on how they change the lives of the public.

In public transit, it's tempting to measure the success of a technology project with statistics like app downloads or the accuracy of arrival time predictions. Instead, we must evaluate how these outputs drive meaningful outcomes. We should ask ourselves and our teams: Does this project build trust in our public transit system? Does it encourage more people to choose transit over private vehicles? How exactly does it achieve this, and to what extent? What is its theory of change, and what specific outcomes can we measure to observe its impact?

Sometimes, we look at technology as a new frontier for transit agencies that are good at procuring tons of concrete but not good at sustaining digital transformation. This may be true, but government agencies have always wanted to use their limited funds on initiatives that solve problems. In this way, technology projects are not special and can be judged similarly to any initiative an agency may or may not invest in: based on outcomes, not outputs. Suppose we’re adding more lighting to a station. In that case, we want to know that riders are and feel safer at night—we are advancing safety outcomes—we don’t add lighting because of the arbitrary value of installing a thousand new fixtures. The same is true for a public website, internal tool, or a feature—unless it solves a problem, it is a problem.

As public servants, our duty is to prioritize projects that create real, observable changes in behavior. These projects strengthen our communities, make our cities more livable, and provide tangible benefits to those who rely most on public services.

In practical terms, this post describes how government technology teams can achieve measurable outcomes, making in-house transit tech an integrated and sustainable part of an agency’s fabric and not just a fad. To do this, we’ll explore the relationship between program and product management as intertwined systems that unambiguously prioritize outcomes.

Today, leaders in transit tech are enthusiastic about adopting a modern, product-oriented approach similar to tech companies. This enthusiasm can mistakenly elevate product orientation to an overarching technology strategy. However, orchestrating products over projects is just one component of a comprehensive transit tech operational strategy.

Jennifer Pahlka, in her book Recoding America, highlights that the government often approaches technology initiatives as fixed-duration projects with a misleading notion of completion. Projects get funded and “completed,” only to enter a cycle of neglect followed by high-effort rework rather than receiving continuous attention and adaptation. Over time, as Pahlka demonstrates, this pulsating project-based approach is more expensive and less flexible.

That said, the key differentiating factor between a product and a project is continuity, not “capital A” Agility. While products are assumed to be Agile and projects waterfall, this distinction is often inaccurate. Products can be built using waterfall approaches, and projects can be delivered incrementally with Agile principles. The true value of the product model lies in cost reduction and potential adaptability.

I fully support product orientation, but it’s not enough on its own. Product orientation guarantees consistent attention and lower long-term costs but does not ensure impact. Without direction, product development can lead to aimless, continuously supported digital outputs.

To achieve meaningful impact, product development outputs must be directed towards and measured through outcomes—outcomes defined and measured by a strategic program aimed at solving a problem. This program should have a theory of change and a sequence of near and long-term outcomes that tech team members, agency leaders, and the public can understand and support.

Before we delve into programs and products, let’s consider why a transit tech team might overlook more appropriate operating models and see product orientation alone as a viable approach. Tech teams in public transit often face an identity crisis, seeing themselves as internal consultants or embedded start-ups. In reality, they are neither.

Transit tech teams belong to the social impact sector, unified by a single public benefit goal: increasing the number of people who choose public transit. They are an integrated part of a transit agency, not something disconnected or unfamiliar—like a consulting firm or startup.

I can anticipate the objection: don’t tech teams need to impress internal business leaders who want to see innovation? They do not. The public, transit agency leaders, and tech teams must align on impact goals and be deeply committed to making progress through achieving outcomes. If a tech leader can’t convince agency leadership of the value of making public transit more vital by solving urgent problems, then they are the wrong person for the job. Inflating the value of novel tech outputs, akin to a consultant’s sales tactic, misdirects spending towards speculative gambles as if the government were a tech start-up. This approach benefits no one.

Let’s briefly consider two misfit models that capture the government tech leader’s imagination. Internal teams shouldn’t operate like external consultants because consultants are known to undermine government agencies, making them dependent on external help to maximize their own profits. So, we don’t want to emulate them. What about start-ups? Start-ups spend other people’s money on high-risk gambles aiming for rapid growth, often failing after a few years. This is far too risky.

Instead, we align more closely with social impact-focused non-profits like the United Way. The United Way strives to use donor-provided resources responsibly to achieve maximum social impact for communities. They face numerous investment options and must apply rigorous selection and prioritization to ensure high impact, avoiding waste on well-intended but ineffective activities. Sound familiar? This is much like the responsibility of an internal transit tech team. Both must make difficult choices and act responsibly, as public welfare is at stake.

More than thirty years ago, United Way and other impact-oriented organizations developed the Program Logic Model (PLM) as a strategic framework for selecting, empowering, and evaluating social change programs. Today, this approach is proven in social impact organizations worldwide to reliably achieve measurable outcomes, implementation guardrails, and program oversight.

Pairing PLM with product orientation ensures that transit tech teams’ efforts have a meaningful impact.

We all know what a project is: a temporary effort with a specific goal, a defined scope, and a clear start and finish. But what about a program? Think of it as a broader, longer-term initiative aimed at solving more complex problems through a series of outcomes achieved via various actions, policies, and initiatives, including technology.

The Program Logic Model simplifies program management by providing a clear framework. It begins with identifying a significant problem or need, which becomes the focal point of a new or existing program. From there, you develop a theory of change with specific short and long-term outcomes. With these outcomes in mind, you plan the necessary activities and outputs to achieve them, continually evaluating and adjusting as needed. This approach ensures that the program remains focused on its original purpose even as new technology opportunities arise, preventing the common pitfall of losing sight of the primary goal.

The most crucial aspect of PLM is distinguishing between outcomes and outputs.

Technology teams often confuse outputs with outcomes. For instance, “arrival time prediction accuracy” might seem like an outcome because it results from a complex process. However, it is a technical output. Outcomes are in the realm of human behavior and experience.

Let’s explore an outcome. Imagine a terminal station with poor departure time prediction accuracy. Suppose the theory of change suggests that improving this accuracy will lead to more riders going to the station. In that case, the long-term outcome is an increase in ridership, which can be measured to confirm that better predictions attract more riders. Alternatively, improving accuracy might not directly increase ridership but could enhance the public’s perception of transit reliability, which, combined with other factors, leads to increased ridership. In this case, research with riders could determine if improved prediction accuracy positively impacts the perception of transit reliability, serving as an incremental outcome toward increased ridership. If it turned out that the prediction accuracy at terminals was primarily bothering agency tech folks and improvements had no material impacts on the public, the agency would know to invest elsewhere.

Let’s explore how program management aligns with agency leadership and the public to create meaningful impact.

Program Management

1. Transit agency leaders and their tech team select a high-impact problem or need through community engagement and opportunity mapping.

2. A cross-functional program is established to address the problem.

3. Through credible research methods, the program determines a high-level intervention based on a theory of change regarding what it would take to impact the problem or need.

4. Specific to the theory of change, the program determines multiple outcomes in the near and long term.

The program then looks to a product team to perform planning, design, and engineering activities that produce shipped digital product outcomes. These are evaluated according to their ability to measurably impact program outcomes. It is important to note that in most cases, the program is not telling the product team what to do specifically (describing a desired solution); they are describing their desired outcomes and looking for the product team to provide contextual solutions.

Product Management

Product management means different things to different people, but fundamentally, it’s about creating viable solutions within constraints to provide value by achieving essential outcomes. Typically, product management is combined with UX discovery and Agile delivery practices, which focus on incrementally delivering features. This approach allows for early and continuous measurement of program and product outcomes.

Some organizations expect product teams to determine outcomes and deliver outputs. This structure lacks adequate accountability and overlooks the inherently solution-focused nature of product teams. Product teams are geared towards developing solutions, and when tasked with both outcomes and outputs, they may not reach their full impact. They often define outcomes as outputs with user-centric wording, like “enable 20% more users to see accurate arrival time predictions.” While this sounds good, it doesn’t describe a behavior change and thus isn’t an actual outcome.

To achieve meaningful impact, teams must distinguish between outcomes and outputs and ensure product teams focus on delivering solutions that drive fundamental behavioral changes.

With the program’s logic model, a cross-functional product team:

1. Determines solutions that are viable within the constraints of the organization and technology and valuable according to their potential to impact goals.

2. Splits their solution into high-level units called epics or simply features.

3. Applies the appropriate degree of research and investigation to inform a high-level plan.

4. Designers and engineers co-ideate on a high-level plan.

5. Designers and engineers co-deliver their work through a series of sprints.

It’s no secret that what’s easier to measure gets measured. Outputs are usually easier to track than outcomes. However, we must prioritize measuring outcomes first because the purpose of tracking outputs is to ensure they effectively contribute to achieving those outcomes.

For example, consider an agency that launched a website feature allowing riders to apply for lower-cost fares. The theory of change is to increase ridership by making public transit more affordable. Tracking applications is an output measurement that shows public engagement with the form. If we don’t see increased ridership at stations promoting reduced fares, we can use the application data to adjust our strategy. Perhaps the form is too complicated, so we simplify it. Merely measuring outputs doesn’t indicate program success. The goal isn’t just to create the form or get people to fill it out; it’s to make public transit more accessible. If many people receive reduced fares but don’t use them, the theory of change needs reassessment—a valuable insight.

From an oversight perspective, agency leaders must evaluate programs on their ability to meet predefined and evolving outcomes. These outcomes should be highly transparent to leaders and team members. When team members understand the desired outcomes, they can make more strategic decisions. Organizational decision-makers can then invest in or divest from programs based on their effectiveness in achieving outcomes. Agencies should redirect funds from ineffective programs to those that are successful.

It’s acceptable to start a program, realize it isn’t working, and make necessary changes. However, letting an ineffective program continue aimlessly for years is unethical.

All transit technology aims to increase ridership, help more people access opportunities, and boost agency revenues—a virtuous cycle.

Agencies move more people by:

• Encouraging existing riders to use transit more frequently

• Attracting new riders

These high-level outcomes are driven by the following:

• Improving service quality, performance, and reliability

• Enhancing communication about regular and disrupted services

• Increasing safety

• Making public transit easier to understand

• Making public transit easier to use

• Expanding or restructuring transit reach

• Raising awareness of public transit services

Technology has the potential to assist in many ways, but it’s only part of the solution, not the entire solution. Someone once told me, “We don’t measure ridership outcomes—that’s how you measure marketing.” This view is too narrow, suggesting that marketing is only about the quality of advertising, regardless of the product’s quality. Transit tech aims to improve both the service and the user experience, and it can also enhance marketing campaigns to increase their effectiveness.

The ideal scenario for any organization is for people to enjoy and recommend their product to others. Word-of-mouth marketing is the most effective and cost-efficient form of marketing because people trust recommendations from those they know more than from advertisers. However, for this to happen, people must use and like your product.

For agencies in the early planning stages of forming technology programs, viewing transit tech through the lens of customer service principles can be a valuable thought experiment. This approach helps in developing theories of change to address various challenges.

Here are the guiding principles of customer service, aiming to consistently meet and exceed customer expectations by delivering high-quality, responsive, and personalized support:

• Consistency: Providing a high level of service across all touchpoints to build customer trust and loyalty.

• Customer-Centric Culture: Embedding a customer-focused mindset throughout the organization, where every employee understands and values their role in delivering excellent service.

• Empowerment: Empowering employees to make decisions that benefit the customer, allowing for more personalized and effective service.

• Proactive Service: Anticipating customer needs and addressing potential issues before they become problems, showing customers you are attentive and care about their experience.

• Effective Communication: Clear, timely, and honest communication with customers to build strong relationships and ensure customer satisfaction.

• Feedback and Improvement: Actively seek customer feedback and use it to improve service processes and offerings continuously.

• Building Trust: Establishing and maintaining trust through reliability, transparency, and delivering on customer promises.

In a sense, transit tech often exists as a form of customer service because it focuses on delivering seamless, user-centered solutions that meet riders’ needs, often during customer distress. Like customer service, it's never done.

Here is a scenario where we combine the PLM, product orientation, and selected customer service principles to frame a problem, describe outcomes, and draft a high-level implementation plan.

Problem: (hypothetical!)

Riders take fewer trips on public transit after they experience a detour that complicates a transfer.

Relevant Principles of Customer Service

• Proactive Service

• Effective Communication

• Feedback and Improvement

• Building Trust

Specific Intervention

A digital tool will reduce the challenges of transferring during detours.

Theory of Change

Riders who use a digital tool that simplifies re-planning their current trip during a detour experience less frustration and appreciate good customer service during an unplanned disruption.

Outcomes

• Bus dispatchers upload detour information before or at an early stage of a detour

• Bus dispatchers describe the cause of a detour in concise, plain language

• Riders on a bus detour read a message explaining the reason for their current detour

• Riders on a bus re-plan their trip and select an alternate trip plan when applicable

• Riders rate their satisfaction with the options presented and indicate how often they ride public transit

• Riders act on the information provided

• Riders respond to a survey 30 days after experiencing the detour to indicate how often they currently ride public transit

High-Level Implementation Plan

• Detour detection: the agency website can use the phone’s location to determine their current detour

• Detour explanation: the riders see a brief explanation for the detour before clicking to continue to the trip re-planner

• Trip plan preparation: pre-populate the rider’s location and ask for their destination station

• Show alternate trip plan: provide trip plans that accurately reflect the impact of the current detour

• Plan selection: enable riders to indicate that they will take a specific re-plan

• Re-plan companion: indicate to the rider the steps they need to take to get to their destination stop, according to where they are in the plan

• Location verification: use the user’s location to confirm they took the alternative route

• Satisfaction survey: ask about their experience, get permission to send a survey

• Follow up: ask the user about their current utilization of public transit

Marty Cagan explains in Alternative to Product Managers that products require product management, but there are many ways to approach product management with or without dedicated product managers. Similarly, a program should have a logic model, whether or not an agency has program managers on staff. It's not uncommon for a founder, stakeholder, or designer to take on product management tasks. Depending on an organization’s composition, a service designer could play a crucial role in developing an effective program logic model due to the highly relevant practice of service design.

Civic service design improves government services by focusing on user experience, involving iterative testing and refinement to ensure services are accessible, usable, and effective.

As a practical example, let's consider a scenario where a service designer employs service blueprinting, one of many service design research methods, to inform a program logic model. In this scenario, the service designer describes how to leverage technology to help public transit riders navigate the challenge of using a shuttle bus during a temporary subway closure.

Service blueprinting offers a detailed visualization of the service process, mapping out all interactions between the user and the service, including both frontstage (visible to the user) and backstage (behind the scenes) activities. This method helps identify pain points, inefficiencies, and opportunities for improvement.

1. Define the Service Scenario

   • Focus on the specific challenge of a temporary subway closure and using shuttle buses as an alternative.

2. Identify User Actions

   • Pre-Journey: Receive notification about the subway closure and find shuttle bus information.

   • During the journey, navigate to the shuttle bus stop, board the shuttle, travel on the shuttle, and disembark.

   • Post-Journey: Arriving at the final destination, providing feedback.

3. Map Touchpoints and Interactions

   • Touchpoints: Digital notifications (app, SMS, email), physical signage, staff assistance, shuttle bus stops, and the shuttle bus itself.

   • Interactions: Steps taken by the user at each touchpoint, such as checking the app for shuttle times or asking staff for directions.

4. Frontstage Activities

   • Actions and interactions are visible to the user, like receiving a push notification or viewing shuttle bus schedules on a display.

5. Backstage Activities

   • Internal processes and technologies supporting the service, such as the systems generating notifications, GPS tracking of shuttle buses, and staff coordination.

6. Support Processes

   • Additional resources and infrastructure are needed to support the service, including IT systems, customer service training, and coordination between transit agencies.

Based on the service blueprint, it is now more straightforward to devise our sequence of problem-intervention-specific outcomes for our PLM, such as:

• Short-Term:

  • Increased rider awareness and understanding of shuttle bus services.

  • Reduced confusion and anxiety about the temporary closure.

  • Improved navigation to shuttle bus stops.

• Medium-Term:

  • Higher rider satisfaction with the shuttle bus service.

  • Decreased travel disruptions and delays.

• Long-Term:

  • Enhanced trust in the transit authority’s ability to manage service interruptions.

  • Increased ridership retention during and after service disruptions.

In the introduction, I described how luck was a significant driver of the success of the MBTA’s CTD; here are a few notes on that:

• Unprecedented Event: Although “Snowmageddon,” which revealed or emphasized severe flaws in the MBTA’s digital and operational systems, was not lucky for the agency or riders at the time, in the aftermath, it meant the agency would fund critical technology initiatives.

• Visionary and Capable Leaders: Monica Tibbits-Nutt was on the board in 2015 and helped establish a new technology team centered on rider experience. David Block-Schacter (now at Transit App), who had been at the MBTA in 2014 but left to join a transit tech start-up, returned, summing up his mission as “The whole goal is to make it incredibly easy to ride the T.” Soon, Gene Shkolnik joins to lead engineering, bring his product-oriented technology leadership developed at his time in leadership at Kayak and Paul Swartz joins as principal architect, leaving ZipCar and laying down the foundation for the MBTA’s reliable, low cost, low-latency, high-concurrency realtime data systems. I’m sure, on the planet, many folks want to put the public first in government and are unusually talented with complex real-time data and customer-facing digital systems. In 2015-16, four of the best in civic tech were all at the MBTA. Swapping any of these folks with someone who is merely highly competent but not extremely great could have resulted in CTD failing in its early days or never getting off the ground.

• Longevity: With the money and the leadership, the MBTA’s in-house product development team had the runway and capacity to build significant customer-critical and safety-impacting systems that the MBTA still relies on today. This doesn’t mean that the shape and size of this department won’t change in the future. Still, by building, managing, and supporting critical digital infrastructure, the team has a foothold that many civic tech teams can’t achieve with their focus on the edges of customer experience. While this is good for longevity, its long-term viability depends on the department’s ability to drive essential outcomes—as it should be in Massachusetts and everywhere else.