- The SKM Stage 3 Report recommended keeping the Johnsonville Line by fixing the BCRs to hide that its modelling showed the Bus-On-Street Scenario was the best performing option.
- The Bus-On-Street Scenario only performed so well because the model invalidly included billions of future transport project improvements meaning future roads would be congestion free.
- SKM never realistically modelled future roads to North Wellington and so any findings from the model are invalid … it proved nothing but that consultants will fix a PT business case as needed by their political masters.
This is the 4th in this series of posts on “How to Fix a Transport Business Case for Rail” and it is recommended that the introduction post, The North Wellington Public Transport Study (2006) and intermediate posts (2, 3) are read first to provide context. This post will focus on how the transport modelling used to evaluate the Scenarios was fixed to make the Busway Scenario appear worse than the Bus-On-Street Scenario.
The Bus-On-Street performed very well … better than the Rail Scenarios!
The fixing of the BCR figures outlined in previous posts (2, 3) shows the SKM Stage 3 Report hid that NWPTS analysis showed the Bus-on-Street Scenario was clearly superior to the Base Case or any other Scenarios. A major reason was because the Bus-on-Street Scenario costs (once adjusted for the fixed figures in the BCR calculations) are just 2/3rds of the price of the recommended “Do Minimum” Base Case.
That Bus-on-Street costs less than the Base Case is no surprise, but the biggest puzzle is why does the Bus-on-Street Scenario also have benefits higher than the Base Case and Rail Scenarios?
The NWPTS benefit cost analysis actually found replacing the Johnsonville Train Service with buses would be a better service and cost less to build and cost less to run … it literally said the rail service was a complete waste of money!
To see what happened in the analysis, we must dive into the Transport Modelling for Stage 3 and how it was setup.
NWPTS Public Consultation – The road network will become more congested
The main public consultation was based on the four Scenarios and was supported by the NWPTS Scenarios Report. This report told the public about the expected future of the North Wellington transport including repeated predictions that current congestion will increase and that will seriously impact bus services:
Travel time reliability for the 57% of peak period commuters who travel on bus services will worsen as traffic grows
[NWPTS Scenarios Report, Executive Summary comment on Enhanced Rail Reliability]
Travel times from Johnsonville to Wellington railway station under this scenario will average approximately 25 minutes compared to the existing train service which takes approximately 21 minutes. This additional length of journey time is due to the need for buses to contend with general traffic.
[NWPTS Scenarios Report Bus-on-Street Section 4.2.1.3 Frequency of services and travel time]
The busway will improve the journey time reliability for all these services, although travel time for services that currently use the Ngauranga Gorge will increase from 16 minutes to approximately 20 minutes {Footnote 15}.
{Footnote 15}These figures are travel time between Newlands overbridge and the Lambton bus interchange. Current schedules show travel time of 16 minutes, although during peak periods actual travel times are often significantly longer.
[NWPTS Scenarios Report Busway Section 4.3.1.1 Operation of busway and other bus services] (Bold added for emphasis)
Congestion in North Wellington even had its own section in the NWPTS Scenarios Report including the following:
The Ngauranga Gorge provides particular congestion problems for buses during the peak periods. This also affects buses in the contra-peak direction in the afternoon, which in order to follow the Hutt Road into the CBD must exit the Motorway via the same link as vehicles travelling to the Hutt (becomes congested with northbound traffic on SH2).
Congestion is likely to get worse in the future extending journey times for private vehicles and for buses and reducing journey time reliability. The average length of a trip from Newlands onramp to the Aotea off-ramp via the SH1 Ngauranga Gorge in the two hour morning peak is anticipated to increase by some 20% between 2001 and 2016 {Footnote 3}.
{Footnote 3}: WTSM model base forecasts
[NWPTS Scenarios Report Section A.3 Congestion] (Bold added for emphasis)
Note that WSTM = Wellington Strategic Transport Model. This is the GWRC Transport model at the time of the NWPTS Scenarios Report forecast increased congestion.
The SKM Stage 3 Report – the road network will become less congested!
SKM Stage 3 Report on Current Congestion and Bus Reliability
As outlined in the previous section, the NWPTS Scenarios Report did not include any information on public transport travel times or reliability. It relied on current timetable schedules and anecdotal evidence.
This was corrected for the SKM Stage 3 Report where the following information was reported:
GWRC recently undertook surveys of bus services between Johnsonville and Lambton Quay using the SH1 motorway down the Ngauranga Gorge and bus services from Khandallah to Lambton Interchange via the Ngaio Gorge and Onslow Road during the weeks of 19 and 26 July 2006. These surveys indicate that travel times in the AM peak from Johnsonville to the Lambton interchange are on average 6 minutes longer than those timetabled (22 minutes vs 16 minutes). This average reduces significantly down to 18 minutes when two days where incidents occurred resulting in very long travel times are excluded. There was a large degree of variability in the travel times on this route with individual travel times ranging between 11 minutes and 40 minutes.
Significant improvements to the road network are anticipated in the RLTS. This includes improvements such as additional lanes along the Aotea section of the motorway, the Petone – Grenada Link and tidal flow at the Terrace Tunnel. These improvements will reduce travel times and improve reliability on the road network and have been included in the networks used in the scenario modelling.
[SKM Stage 3 Report – Appendix D Section 6] (bold added for emphasis)
So, first paragraph confirmed that bus delay along the Hutt Road corridor was worse than estimated and also showed the bus travel times were very unreliable. But the very next paragraph then states the “Significant improvements to the road network are anticipated in the RLTS”, that is the Regional Land Transport Strategy.
Further, and this is the key sentence in the whole report, the SKM Stage 3 Report states “These improvements will reduce travel times and improve reliability on the road network and have been included in the networks used in the scenario modelling”. That’s correct, RLTS road and rail projects that then totaled $4 Billion were added to the modelling programme used to predict which option would perform the best!
Appendix J of the SKM Stage 3 Report helpfully then lists RLTP projects costing hundreds of millions that have been put into the Transport Model even though funding mostly of them were unfunded.
The forecast years and future networks and other inputs were discussed and agreed with GWRC. …
The 2016 network modeled specifically includes:
The draft 2006 Wellington Regional Land Transport Strategy notes the cost as:
the strategy proposes a total 10 year investment of $4120 million.
[Draft RLTS 2007 – 2016 Section 12.4]
SKM Stage 3 Report model show a future based on less congestion
Unsurprisingly, the SKM Stage 3 Report then predicts a very different future for North Wellington road congestion compared to the NWPTS Scenarios Report predictions:
The measures impact on travel, level of congestion and network reliability and access to PT (proximity) do not change significantly between the scenarios or against the base. This is due to the marginal impact on vehicle trips and the fact that the majority of the northern suburbs catchment is already catered for by public transport. The scenarios all perform very similarly under the measure vehicle speeds and travel times, with all having a minor positive impact.
[SKM Stage 3 Report Section 4.3 Objective 3: Improve Access, Mobility & Network Reliability] (Bold added for emphasis)
In contradiction to the NWPTS Scenarios Report, the SKM Stage 3 Report predicts a future with dramatically reduced road congestion. This is because they fixed the model to analyse the scenarios (and ‘Base Case’) WSTM assuming the completion of the $4 Billion investment program under the RLTS by 2016:
In general public transport journey times to key destination remain constant or improve (Table 5-14 and Figure 5-8, Figure 5-9). Vehicle travel times remained very similar to the base because of the marginal impact on vehicle trips. The PT journey times reduce in general for all the scenarios. Services in the busway scenarios have a longer distance to travel via the Johnsonville corridor compared to the bus on-street scenario which travels via the road network. Under the busway scenarios more services travel via the corridor. The travel time from Johnsonville to the CBD forecast by the model for 2016 was less via the road than via the busway. The road travel times were based on the assumed 2016 road network which included the improvements anticipated in the RLTS. These improvements improve conditions for travel from the northern suburbs into the CBD. Refer to Appendix J for details of the improvements included.
[SKM Stage 3 Report Section D.3 Travel Times (journey times)] (Bold added for emphasis)
The impact of North Wellington Congestion Assumptions
Fixing the transport model to assume a massive reduction in road congestion will, of course, have a major impact on the model’s predictions for future public transport usage. Under the Base Case and Rail Scenario options (as outlined by the solid red circles below) rail patronage remains static (at just over 1,000 AM Peak) and all the increase in predicted public transport usage is actually on bus services (from 1,900 to nearly 2,600 AM Peak)!:
In fact (as can be seen by the green circle) the Bus on Street Scenario attracts more PT commuters than the Base Case or any rail scenario! The Busway and Light Rail Scenarios are predicted to get only 4% more commuters compared to the Bus-on-Street Scenario even though both offer a seamless service to the CBD on a dedicated public transport corridor.
Is it correct to include RTLS improvements into the ‘Base Case’?
Including multiple unfunded future transport investments into the base case a benefit cost analysis is highly unusual. Best practice is to model the “Do Minimum” Base Case without assumed unfunded transport investments do proceed and, if these are assumed, add a sensitivity test to estimate their future impact.
Masking the true costs and benefits for each option
The purpose of the SKM Stage 3 Report was supposed to be:
Stage 3: Scenario Evaluation – Based on the feedback from the Stage 2 consultation and through discussions with stakeholders, four final scenarios were developed. An evaluation framework and performance measures were developed and used to evaluate the scenarios in detail. This evaluation is the subject of this report.
[SKM Stage 3 Report Section 1.3 Study Process]
However, the fixing of the Transport Model by the inclusion of all future transport projects into the Base Case “Do Minimum” hides the benefits of each Scenario are from the investment in that scenario or from some other RLTS project.
For example, the modelled bus travel times for all the options end up being very similar in the key measure “Improve Access, Mobility & Network Reliability” with the SKM Stage 3 Report concluding the modeling showing:
The measures impact on travel, level of congestion, network reliability and access to PT (proximity) do not change significantly between the scenarios or against the base.
[SKM Stage 3 Report Section 4.3 Objective 3: Improve Access, Mobility & Network Reliability (Better Connected / More Liveable)]
What the NZTA and the public cannot tell with whether this result is from the investment proposed under the NWPTS or is derived from, for example the proposed Grenada to Gracefield Link Road that was assumed to have been built? As the ‘Base Case’ and Bus-on-Street Scenarios have minimal investment for reducing bus congestion, it is highly likely they gain a similar level of bus service reliability as the Busway Scenario from other, as yet unconfirmed, roading projects which explains the transport model’s prediction that future PT growth would all be on bus service instead of rail service.
Equally, the assumed removal of congestion from Ngauranga Gorge also negates the benefits the Busway Scenario offers from investment in by-passing this congestion. As can be seen on the previous page, in the predicted patronage in 2016, the busway attracts about 1% more commuters than the Bus-on-Street Scenario. With the assumed roading expansion under which the future was modeled, the busway essentially becomes a more expensive version of the Bus-on-Street Scenario.
Modelling the NWPTS ‘Base Case’ and Scenarios WITH RLTS Planned roading improvements means the relative benefits and costs of the investments cannot be truly understood.
Problem 5 – Change the transport model to hide the best option
As outlined above, the NWPTS Scenarios Report informed the public that:
Congestion is likely to get worse in the future extending journey times for private vehicles and for buses and reducing journey time reliability. The average length of a trip . . . via the SH1 Ngauranga Gorge in the two hour morning peak is anticipated to increase by some 20% between 2001 and 2016
[NWPTS Scenarios Report Section A.3 Congestion] (Bold added for emphasis)
After informing the public of this important prediction and receiving submissions in response to this and other information in the NWPTS Scenarios Report, the SKM Stage 3 Report says:
In general public transport journey times to key destination remain constant or improve (Table 5-14 and Figure 5-8, Figure 5-9). Vehicle travel times remained very similar to the base because of the marginal impact on vehicle trips. The PT journey times reduce in general for all the scenarios. Services in the busway scenarios have a longer distance to travel via the Johnsonville corridor compared to the bus on-street scenario which travels via the road network. Under the busway scenarios more services travel via the corridor. The travel time from Johnsonville to the CBD forecast by the model for 2016 was less via the road than via the busway. The road travel times were based on the assumed 2016 road network which included the improvements anticipated in the RLTS. These improvements improve conditions for travel from the northern suburbs into the CBD. Refer to Appendix J for details of the improvements included.
[SKM Stage 3 Report Appendix D.3 Travel Times (journey times)] (Bold added for emphasis)]
Instead GWRC officers and SKM consultants fixed the modeling assumptions for the SKM Stage 3 Report to reflect a different future, a future where major investments in more roads and rail proceeded and so reduced road congestion.
Stage 3 of the NWPTS is in serious deficit from not informing the public that the recommended Base Case was based on the assumption that the major roading projects outlined the RLTS would proceed and that this was expected to address the road congestion issues.
Does the NWPTS Modelling comply with NZTA Funding Rules?
Even more important, NZTA funding requires all transport studies to comply with their option modelling rules. As outlined in the SKM Stage 3:
G.2 Consider Economic Efficiency
Economic efficiency was measured by way of a comparison of economic benefits, revenues and costs of the scenarios. The economic assessment was carried out in line with the Land Transport New Zealand Economic Evaluation Manual, Volume 2, as required for any scheme which may require government funding.
[SKM Stage 3 Report] (Bold added for emphasis)
But is this statement true? Included in this manual is the specification that all proposals for transport funding must be properly evaluated including comparison with the ‘Base Case’ (called a “Do Minimum”) Option. The characteristics of the “Do Minimum” option are outlined as follows:
5.5 Do minimum
| Definition | The do minimum for evaluation of transport services is usually considered as a continuation of the present transport networks, service levels and the existing road network in the study area.
The do minimum shall include any costs and resulting demand implications of committed road or service improvements. All committed investment plans that relate to the do minimum during the analysis period must be taken into account. Maintenance, replacement schedules and any planned service changes shall also be included. Improvements are committed if they have been evaluated in accordance with Land Transport NZ’s project evaluation procedures and have been approved for funding. Any investment plans that are not committed shall be included in the evaluation as options. Any changes to the road system that are committed must be included in the do minimum. |
| Scope of do minimum | It is extremely important to:
not overstate the scope of the do minimum only include, as part of the do minimum, work which will preserve a minimum acceptable level of service. In some cases, particularly with respect to the road network, the do minimum service level may be less than the existing level of service. |
[Economic evaluation manual, Vol 2. Land Transport NZ, Oct 2005, page 5-6] (Bold added for emphasis)
LTNZ re-enforces the Do Minimum scope statement in its specification of how Alternatives and Options are to be described:
5.9 Alternatives and options
| Introduction | In most situations, there is a variety of means (alternatives) of achieving the objective that a transport services proposal seeks to address.
Options may include different configurations of the proposal, different fare and service levels and possibly different modes. |
| Rules | Where feasible alternatives to the proposal exist, they must be described and included in the evaluation.
Where appropriate, different service levels shall be evaluated as options. All transport service evaluations shall consider the status quo road or road network, as appropriate, and any practical road improvement projects as alternatives or part of options. The status quo road or road network will normally be part of the do minimum. Where there is no existing road, a proposed road project may be considered as an alternative. All investment plans and proposed service changes that relate to the alternatives and options during the period of analysis shall be taken into account. |
[Economic evaluation manual, Vol 2. Land Transport NZ, Oct 2005, page 5-11] (Bold added for emphasis)
But the SKM Stage 3 Report ‘Base Case’ does not model “The status quo road or road network” that would “normally be part of the do minimum” and model includes hundreds of millions in “investment plans that are not committed”. It is clear that the SKM Stage 3 Report does comply with the requirements for “Do Minimum” under Land Transport New Zealand funding rules.
It must also be noted the report specifically stated “The forecast years and future networks and other inputs were discussed and agreed with GWRC” so the regional council is complicit in “fixing” the NWPTS Transport Model in favour of the Bus-on-Street Scenario even though this breaks the NZTA funding rules.
Why fix the Model in favour of the Bus-on-Street Option?
If the NWPTS transport model did not have the assumed major roading improvements then it would show much greater benefits to options that did not rely on roads … naming all options except the Bus-on-Street Option. But why fix the model to make Bus-on-Street come first?
Well, we can be sure that any realistic transport model would not show the Bus-on-Street Scenario coming first … indeed, even with it being fixed, Bus-on-Street still did not have the PT commuter uptake of either the Busway or Light Rail Scenarios.
We can also be sure that any realistic transport model would not show the rail Scenarios coming first because, if this was the result then there would be no need to have … how many is it now? … oh yes, FIVE business case fixes all of which were needed to make the rail Base Case come first.
The only conclusion one can come to is that the realistic transport model (and you can be sure they ran it) showed the Busway Scenario having the greatest benefits and the highest Benefit-Cost Ratio. The whole point of the SKM stage 3 Report is to not to identify the best Scenario but to hide that a Johnsonville Busway would be the best PT solution to support North Wellington public transport!
Instead, NWPTS consultants and GWRC fixed the NWPTS transport model in favour of a Bus-on Street Scenario and then fixed the BCR figures to make Bus-on-Street come second to the Base Case. That the SKM Stage 3 Report also rejected both the Enhanced Rail Scenarios did not really matter because of what happened in the following years …
Be First to Comment