The Smarter Trams Program: Laying the foundations for an intelligent transport system

Yarra Trams operates the world's largest tram network, accommodating over 200 million trips annually. It's a complex network comprising over 500 trams, 73% of which share their routes with cars, buses, and cyclists. Such interactions can cause delays or disruptions to the planned tram journey times.

The Smarter Trams Program implements real-time and dead-reckoning GPS on 399 trams and real-time Automatic Passenger Counting on 240 trams. The data collected will significantly improve passenger information, timetable planning, and safety reviews. It also has the potential to trigger traffic lights to reduce road congestion, as per the Government's Smarter Roads initiative.

It was clear that Yarra Trams would need a technical design optimised for performance while handling large amounts of data and so a set of requirements were defined:

1. Scalable data management — The system generates 200 pieces of data per second, a number that is increasing daily.
2. Low data latency — Traffic light priority systems require data to be communicated in under 6 seconds to be effective.
3. Effective automation — The platform and processes need to be fully redundant and capable of automatic recovery.

Yarra Trams received GPS data from units installed in the trams, while passenger count data came from real-time sensors on the trams. The system then collated and merged this data, along with contextual scheduling data, such as whether the tram was on time and its expected arrival & departure times.

A key design decision was to pass the data through the Event Hub component offered by Azure. This allowed multiple stakeholders to subscribe to the data and provided a secure, scalable solution.

Data was then sent to various stakeholders, such as the Department of Transport & Planning Victoria, YarraTrams' tramTRACKER App, and YarraTrams' internal Data Lake. Each of these entities utilised the data for specific purposes, like traffic light control decisions, accurate tram arrival predictions, and data archiving and analysis, respectively.

The project requirement was for total transit time from tram to the Department of Transport & Planning Victoria to be under 5 seconds for thousands of messages per second.

Portable worked alongside Yarra Trams stakeholders to develop a solution architecture service to optimise capacity and production load. Our technical solution introduced horizontal scaling, which evenly balanced the data load across multiple servers in an automatic, transparent, and cost-effective manner.

The entire system had to be fully and automatically redundant. If an error occurred, the system needed to adjust itself to work as if nothing had failed, while notifying support staff about the non-functional server.

In the process of initially engineering the system for performance, we enabled the spin-up of additional server apps to handle throughput for potentially thousands of trams in the future. Interestingly, this allowed us to achieve redundancy without additional work. If one server fails, the remaining servers automatically rebalance the load, and the whole process functions as if nothing happened.

To assist with this, we designed a monitoring and alerting system that automatically opens and escalates cases in ServiceNow, with options for SMS and/or push notifications for severe incidents.

A significant part of the testing phase involved exploring edge cases to ensure a high level of accuracy.

The Portable team completed functional and load tests for 240-500 trams and undertook extensive security testing and management of the deployment pipeline and processes.

With new data available for the first time, an opportunity emerged to improve data visualisation.

Yarra Trams, our partner Dear Watson, and Portable worked together to roll out the primary visualisation platform in Microsoft Power BI. We also set up a pipeline that synthesised data from a data mart to feed the Power BI platform.

This analytics platform enable the Yarra Trams team to gain real-time insights for network planning.

These insights include:

• monitoring tram and route uptime
• identifying times and geographical locations of network inefficiencies and congestion
• and understanding passenger data attributes, trends, and patterns.
  

This information allows Yarra Trams to prioritise optimising tram routes in specific areas and times of the day that are inefficient and potentially disrupting other trams.

1. Allowing customers to see how crowded a tram is before it arrives and make an informed decision about planning their journey.

2. Enabling customers to see where the tram is currently located, through real-time GPS locations, which means transparency on tram arrival times, altered trips, or shortened routes.

3. Giving external organisations such as the Department of Transport & Planning Victoria the opportunity to use real-time data to inform traffic light prioritisation.

4. Allowing the Yarra Trams team to internally optimise points of congestion, inefficiencies, oversupply, and under-demand.

“Smarter Trams system is a way for Yarra Trams to crunch a huge amount of data, convert it into useful insights, and then take action from that data.

This was a challenging project but one which was executed well and which is sure to see Victoria retain its role as the global leader of suburban short-transit transportation.” 

James Zhang, Technology Lead — Portable