This appendix describes in detail the methodology used by NYPIRG Straphangers Campaign to develop our profiles of New York City subway lines and our line ratings.

Essentially, we reviewed six measures of transit performance compiled by MTA New York City Transit, presented them in concise, one-page rider-friendly profiles, and then gave a Line Rating based on their overall performance. We chose these six measures—which are fully described below—for several reasons. This included their importance to riders, as reflected in New York City Transit’s polling of riders and in our own survey of 38 transit experts. Additional factors included the availability, reliability and comparability of the data.

The line rating was developed in two steps, explained more fully below. First, we decided how much weight to give each of the six measures of transit services in our profiles. Then we placed each line on a scale that permits fair and consistent comparisons. Under that formula, a line whose 2000 scores fell on average at the 95th percentile of 19 lines for all six performance measures in 1996 (our baseline year) would receive a line rating of $1.50.

1. Presenting New York City Transit Data to Riders

Our first step was to gather information on transit performance. We decided to report data in the form of concise one-page profiles for each subway line. That met our goal of presenting the information in a way that would be easily understandable to the riding public.

Below is a description of each of six measures of transit performance that we used. We have listed the published sources of the data; if no published source is listed, we received the data from MTA New York City Transit in diskette form. In 1997, New York City Transit officials reviewed a draft version of the profiles and concluded:

Although it could obviously be debated as to which indicators are most important to the transit customer, we feel that the measures that you selected for the profiles are a good barometer in generally representing a route’s performance characteristics. . . Further, the format of your profiles. . .is clear and should cause no difficulty in the way the public interprets the information.

An advance summary of the findings for the 2001 State of the Subways report was provided to MTA New York City Transit.

A. Scheduled headways

We measured amount of service based on the scheduled "headways" between trains for weekday morning rush, afternoon rush and midday hours. Headways are the number of minutes between train arrivals. For example, the 4 line is scheduled to arrive every four minutes during the weekday morning rush, which MTA New York City Transit defines as the period between 7 a.m. and 9:30 a.m. Because virtually all subway lines operate at the same interval—20 minutes—during late night hours, we did not include overnight headways in our analysis. This approach allowed us to include train lines which do not regularly operate during overnight hours or on weekends, like the 3 and Q lines.

For our profiles, we decided to have the morning and afternoon rush hour intervals each contribute 40% to the overall headways measurement; midday headways account for the remaining 20%. We felt that this reflected the relative use of service. For any line which has different scheduled intervals for northbound and southbound trains, the average headway was reported. (Source: Letter, April 27, 2001 to Gene Russianoff, staff attorney, Straphangers Campaign from Barbara Spencer, Executive Vice President, MTA New York City Transit.)

B. Service regularity

Service "regularity," measured by MTA New York City Transit between 6 a.m. and 9 p.m. on weekdays, is defined as "the percentage of intervals between two train trips departing from all scheduled timepoints, not including terminals, that are within plus or minus 50% of the scheduled interval (for all intervals less than ten minutes) or within plus or minus 5 minutes of the scheduled interval (for intervals of ten minutes or more)." A line with a low regularity, for example, would show either gaps in train service during some portion of the day, and/or train bunching at others. In our profile, service regularity was measured as an average of the percentage of regularity during the last six months of 2000. (Source: "Old Service Quality Indicators, Third Quarter 2000 and Fourth Quarter 2000," MTA New York City Transit Committee Agenda for May 2001.)

C. Mean distance between failures (MDBF)

MTA New York City Transit states that MDBF measures subway car reliability, and "is determined by dividing the total number of subway car miles traveled in revenue service by the total number of mechanical failures that result in train delays." In our profiles we report the average MDBF rate for the last six months of the year 2000. (Source: Department of Subways Division of Car Equipment Projects and Operations MDBF By-Line Report data for the year 2000.)

D. Chance of getting a seat

We developed a formula to calculate the chance of being able to get a seat at the most crowded point on each line. First, we identified each line’s "instance of greatest crowding" using New York City Transit’s 2000 Weekday Cordon Count. We did this by isolating the most crowded 15-minute interval for each line, at the most crowded point of the route listed. Then we divided the number of seats on cars in a line by the number of passengers at that 15-minute interval. For example, the 2 line was at its most crowded point at 72nd Street in Manhattan, heading downtown between 5:30 and 5:45 p.m. on the day the count was taken; the average number of passengers counted was 144.0 per car. Cars on the 2 line are of the class R-33 G.E., a 51-foot A division car with 40 seats. Thus the ratio of the number of seats to the total number of passengers per car would be 40/144, or 28%. This figure, 28%, represents the chance that a rider will be able to get a seat on a train at the 2 line’s most crowded point, during the most crowded 15-minute weekday rush hour interval. (Source: New York City Transit Rapid Transit Loading Guidelines, November 5, 2000; and "Year 2000 Subway—By Route Analyzed by 15 Minute Intervals," Supplement to Year 2000 Weekday Cordon Count, June 2001. In cases where more than one car class was used on a line, we evaluated crowding based on the seating guidelines for the predominant type of car used on the line.)

E. Passenger Environment Surveys: Cleanliness and Announcements

New York City Transit conducts a quarterly "Passenger Environment Survey" (PES) to measure the quality of the transit environment experienced by riders. It does this for subway cars, stations and buses. The PES is performed by "surveyors who are specifically trained for this function and who have no direct association with the departments affected by the survey evaluations. The surveying of. . .subway cars is conducted throughout each quarterly recording period to the extent necessary to depict a representative sample of NYC Transit’s vehicles." Our profiles represent the first time that PES findings have been presented to the public on a line-by-line basis. We included PES ratings on cleanliness and announcements because New York City Transit’s own surveys of rider opinion show both are of major concern to riders.

(1) Interior cleanliness

The PES includes a rating on the cleanliness of the interior of subway cars. It defines a car with a light degree of dirtiness as one with "occasional ‘ ground in’ spots, but generally clean." Interior cleanliness in our profile was measured as the average of the total percentage of cars with "no dirtiness of car floors and seats," or "a light degree of dirtiness of car floors and seats" during the last six months of 2000. (Source: MTA New York City Transit’s Department of Operations Planning—Systems Data and Research, Passenger Environment Survey data for the third and fourth quarter, 2000.)

(2) Adequacy of routine in-car announcements

In-car announcements are also monitored in the Passenger Environment Survey. Our profiles note the average percentage of cars with adequate "routine in-car announcements" for the first six months of 2000 . Through the second quarter of 2000, the PES rated announcements as adequate if they are "understandable" and "correct." Such announcements would include the following information:

• next station announced (while enroute or at the station);
• transfer options, if applicable;
• route designation announced (either letter or number corresponding to a train line);
• route destination announced (both borough and terminal);
• next station announced (while at or when leaving a station); and
• "stand clear of the closing doors" announced. (Source: Department of Operations Planning—Systems Data and Research, Passenger Environment Survey data for the first and second quarter, 2000.)

2. Developing a Straphangers Campaign Line Rating

We decided to include one overall rating for each of the 19 subway lines. The rating is intended as a shorthand tool to summarize all of the information reported in the profiles and to allow for comparisons among lines.

As described below, under the formula used, a line whose 2000 scores fell on average at the 50th percentile of 19 lines for all six 1996 performance measures would receive a line rating of 75¢. A line which matched the 95th percentile of this range would receive a line rating of $1.50. However, some lines which ranked high on three measures of performance may have received only an average line rating due to poor relative performance in other areas (see Figure 1).

Some riders may find this scale too generous, believing that performance levels should be far better than they are now. Other riders who value transit service over other ways to travel in New York City, may believe the subways and buses to be a bargain.

The line rating does not seek to make a subjective value judgment of the worth of subway service. It is not based on economic factors, such as the cost of providing service or comparisons to the costs of other modes of transportation. Instead, it is a yardstick that permits a simple and direct ranking of subway lines.

Most importantly, the line rating has allowed us to use the same formula for ranking service on subway lines in the future. As such, it has been a fair and objective barometer for gauging whether service has improved, stayed the same, or deteriorated over time.

A. Ranking Subway Performance Indicators

We used two sources of information to formulate a scale of the relative importance of various subway line performance indicators. First, the Straphangers Campaign conducted a poll of 38 transit experts, activists and members of the riding public. We asked them to rank eight indicators of subway performance that opinion polls indicated were of major concern to riders. Second, we examined MTA New York City Transit’s own rider opinion surveys. One performance indicator, crime, ranked high in both the Straphangers Campaign’s poll and in the MTA rider surveys, but could not be included in the profile project because applicable data was not available on a line-by-line basis. A second measurement, "enroute schedule adherence," (commonly referred to as "on-time performance") was dropped from consideration because New York City Transit does not record this for rush hour and midday trains.

Three lines—the Grand Central, the Franklin Avenue and the Rockaway Shuttle—were dropped from consideration because not all six measures of service were available for these lines. The G was included in the overall charts summarizing our findings, but did not, as noted above, receive a line rating.

The remaining 19 subway lines were evaluated on the basis of six indicators. All the indicators are regularly measured by New York City Transit on a line-by-line basis. Each measure was assigned a percentage weight based on the priorities expressed by those polled as follows; the measures are grouped by the type of indicator:

Amount of service  (total: 30%)
  Scheduled headways  30%

Dependability of service  (total: 35%)
  Service regularity  22.5%   Mean distance between failures  12.5%

Comfort/usability  (total: 35%)   Chance of getting a seat   15%   Interior cleanliness  10%   Adequacy of in-car announcements   10% (total = 100%)

Three of these indicators—the mean distance between failures, interior cleanliness and in-car announcements—have never before been released to the public on a line-by-line basis.

B. Calculating the Line Rating

The 19 lines were rated on a linear scale for each of the six measurements used. A line in 2000 equaling the 1996 system best would receive a score of 100 for that indicator, while a 2000 line matching the system worst would receive a score of 0. Thus most lines in 2000 received a score for each measurement between 0 and 100. However, in some cases a line was awarded a score outside of that range if it outperformed the best line in 1996 or performed worse than the worst line. The lines’ scores were then multiplied by the respective weights afforded each indicator. The six adjusted scores were then added up, as shown in Figure 1, which uses the 4 line as an example.

The summed totals were then placed on a logarithmic scale. This scale emphasizes the relative differences between scores nearest the top and bottom of the scale. This method reflects the odds, rather than the percentage chance, of any train on a given line meeting a basic level of satisfaction.

Finally, we converted the scale to a dollar-based line rating, to offer riders a simple basis for comparisons among lines. We calibrated this scale so that a line whose 2000 scores fell on average at the 50th percentile of 19 lines for all six performance measures in 1996 (the baseline year) would receive a rating of 75¢. A line which matched the 95th percentile of this range would be rated $1.50. Each figure was then rounded to the nearest 5¢. The actual scale used to convert summed raw scores to line ratings is shown on the following page, with the 4 train as an example.

C. Future Line Ratings

In May 2001, transit officials made major changes in how several of the indicators are derived. The Straphangers Campaign unsuccessfully urged New York City Transit to re-consider its new methodologies, because of our concerns about the fairness of these measures and the loss of comparability with past indicators. Since transit officials rejected our request to re-calculate measures back to 1996 in line with their adopted changes, some historical comparability may be lost in future State of the Subways reports.