The record in this proceeding convincingly demonstrates two facts. First, local telecommunications competition can and will develop when the requirements of the Telecommunications Act of 1996¹ and the Commission's rules are fully implemented, bringing substantial benefits to consumers in the form of lower prices, innovative services, and bundled products that consumers desire. Second, Bell Atlantic has completed most -- but not all -- of the actions needed to achieve a fully and irreversibly open market in New York.

Because of the vigorous leadership of the New York Public Service Commission ("NYPSC") and the extensive efforts of Bell Atlantic and numerous competing carriers, most of the necessary preconditions for local competition are in place in New York. The terms on which competitive local exchange carriers ("CLECs") can obtain interconnection with and access to Bell Atlantic facilities and services have been largely resolved, in a manner that appears to permit efficient CLEC entry. In addition, all parties have worked hard to resolve the critical operational details of implementing the agreed-upon arrangements. As a result of those efforts, the Department of Justice ("Department" or "DOJ") does not have substantial concerns about the ability of facilities-based carriers² and firms that wish to resell Bell Atlantic's retail services to enter the local telecommunications markets in New York.

There has also been great progress in opening the market to competition through the use of unbundled network elements, but in this area, a few significant problems remain. Bell Atlantic has not yet demonstrated that it can adequately provide access to unbundled local loops, either for traditional voice services or for digital subscriber line ("DSL") technology used to provide a variety of advanced services. Moreover, Bell Atlantic's systems for handling orders for the unbundled network element "platform" ("UNE-platform" or "UNE-P") rely to a disturbing extent on manual processes that are prone to error and delay. There remains significant doubt that Bell Atlantic has provided the stable and efficient electronic systems that will be needed to support a competitive market. These remaining problems are few in number, but they will impose a significant constraint on competition if they are not adequately resolved.

There is reason to believe that these remaining problems can be solved in a short time, and Bell Atlantic, commendably, appears to have taken or committed to take action to do so. But Bell Atlantic filed this application before those actions were completed and therefore before their hoped-for success can be demonstrated. The Department has worked extensively with Bell Atlantic and other participants in the Section 271 process to define the conditions that must be in place for us to conclude that markets are fully and irreversibly open to competition. We have done so because of our belief that there should not be an ever-receding finish line for meeting the requirements for entry into the long distance market. By the same token, it is important for Section 271 applicants to cross the finish line, not merely come within sight of it. Bell Atlantic should be required to remove the few but important obstacles to local competition that remain in New York before it enters the long distance market.

Because of these remaining problems, we conclude that the Commission properly could deny this application, but as we discuss further in Section VII of this Evaluation, we do not foreclose the possibility that the Commission may be able to approve Bell Atlantic's application at the culmination of these proceedings.

I.          Laying The Foundation For Competition

Over the past three years, the NYPSC has worked tirelessly to create an environment in which local telecommunications competition can develop in New York. It has established rates and other terms and conditions for interconnection agreements for resale, unbundled network elements, and interconnection. It has developed and implemented wholesale performance measures³ and pursued with vigor its examination of Bell Atlantic's draft application under Section 271 under the capable eyes of its administrative law judges.⁴ Midway through this review, the NYPSC negotiated a "Pre-Filing Statement" in which Bell Atlantic committed, inter alia, to pay for a comprehensive third-party test of its wholesale support systems and to develop a plan to ensure adequate continuing wholesale performance.⁵ NYPSC staff subsequently oversaw a third-party wholesale support systems test of unprecedented scale, undertook an extensive validation of Bell Atlantic's performance measures, developed two performance assurance plans with Bell Atlantic and established several series of collaborative meetings between Bell Atlantic and CLECs to address specific problems.

The third-party test of Bell Atlantic's wholesale support systems has been particularly valuable in opening the New York market. Under the supervision of the NYPSC, KPMG LLP ("KPMG") and Hewlett-Packard ("HP") conducted a broad, independent and robust test of Bell Atlantic's wholesale support systems. Placing themselves in the position of a market entrant, KPMG and HP reviewed the processes by which CLECs establish and maintain a wholesale relationship with Bell Atlantic, independently developed interfaces to Bell Atlantic's operations support systems ("OSS"), prepared test data, and submitted test transactions. KPMG's review of Bell Atlantic's documentation, software testing and change- management processes identified serious problems, which were addressed by Bell Atlantic through process improvements during the test period. Together, the NYPSC and KPMG created an open testing environment -- consulting with all interested parties, disclosing contacts with Bell Atlantic, issuing draft plans and reports, and reporting in detail on issues of serious concern. As a result of these factors, KPMG's test itself had a substantial and valuable market-opening effect in New York.

KPMG's exhaustively detailed final written report is an important part of the documentary record of this application. The KPMG test, however, was not designed to address all significant aspects of Section 271 compliance. Most significantly, the transactional aspects of KPMG's test focused primarily on Bell Atlantic's computer systems and did not comprehensively assess the manual processing and provisioning of orders, areas that are critical to our evaluation. Further, KPMG's test could not exactly replicate commercial use of Bell Atlantic's systems; for this reason, concurrent commercial use of these systems significantly enhances our knowledge about their strengths and capabilities.⁶ Additionally, KPMG did not examine the CLECs' ability to order DSL-capable loops or Bell Atlantic's ability to provision such loops, important issues that are discussed later in this evaluation.⁷

The NYPSC's development of comprehensive performance measures has also helped enormously to identify possible performance problems in some areas and to provide convincing evidence of adequate performance in others. The real-world experience of implementing and using these performance measures has revealed several limitations, as might be expected. Some metrics do not appear to be appropriately disaggregated.⁸ There are disputes about whether certain measures are appropriately defined and accurately measured.⁹ At the time of Bell Atlantic's application, data for a number of measures covered only one month or were not available because the measure was "under development." The NYPSC is continuing its efforts to refine these performance measures; but that process is still underway, and the measures currently provide only a starting point for analysis. Standing alone, the performance data may indicate both false positives and false negatives; that is, the measures in some cases may suggest problems when in fact the underlying performance is acceptable and in other cases may suggest acceptable performance when, on closer examination, there are significant performance problems.¹⁰

II.          Competition In Local Telecommunications Markets In The State of New York

The extensive efforts of the NYPSC and all carriers operating in New York have produced impressive results in creating an environment in which local competition has begun to develop. As the Department has previously explained, in-region interLATA entry by a Bell Operating Company ("BOC") should be permitted only when the local markets in a state have been fully and irreversibly opened to competition.¹¹ This standard seeks to determine whether barriers to competition that Congress sought to eliminate in the 1996 Act have in fact been fully eliminated and whether there are objective criteria to ensure that competing carriers will continue to have nondiscriminatory access to the facilities and services that they will need from the incumbent BOC.

In applying this standard, the Department determines whether all three entry paths contemplated by the 1996 Act -- facilities-based entry involving construction of new networks, the use of unbundled elements of the BOC's network, and resale of the BOC's services -- are fully and irreversibly open to competitive entry to serve both business and residential consumers. To do so, the Department looks first to the extent of actual local competition as the best evidence that local markets are open. The degree to which such existing competition is broad-based determines the weight the Department places on it as evidence.

In the absence of broad-based commercial entry involving all three entry paths, the Department examines whether new technical and operational arrangements are available and shown to be working to support all three entry modes and whether benchmarks to prevent backsliding by the incumbent have been established. The actual experience of competitors seeking to enter a market can provide highly probative evidence concerning the presence, or absence, of artificial barriers to entry.¹²

The state of New York provides unique competitive opportunities for carriers seeking to provide local telecommunications services. With more than 18 million inhabitants,¹³ the nation's third most populous state encompasses New York City, the largest, most densely concentrated metropolitan area in the United States.¹⁴ The state has seven Local Access Transport Areas ("LATAs"): metropolitan New York City, Albany, Syracuse, Rochester, Buffalo, Binghamton and Poughkeepsie.¹⁵ As of December 31, 1998, there were 16.4 million access lines statewide, including 12.7 million switched lines.¹⁶ Bell Atlantic served 90.4 percent, or nearly 11.5 million of these switched lines. As of the end of June 1999, CLECs served more than 1,100,000 access lines in New York,¹⁷ approximately 8.9 percent of the total, which is significantly larger than the national average of less than five percent.¹⁸

While an 8.9 percent share of total switched access lines represents significant CLEC entry, it is important to recognize differences in the particular modes and extent of entry among various segments of the state. By entry mode, approximately 59 percent of all CLEC access lines were facilities based; 28 percent were resold; and about 13 percent were provided as unbundled network elements. By state region, approximately 90 percent of CLEC access lines served customers in the New York metropolitan area while the rest served upstate customers. By customer type, 70 percent of CLEC access lines served business customers while the balance served residential customers.

A.          Facilities-Based Entry

Clearly, serving metropolitan New York business customers with facilities-based access lines represents the most common form of CLEC entry. Indeed, even before the 1996 Act, competitive access providers had built significant facilities to link large customers in New York directly to long distance carriers. Competitive entry has been concentrated in metropolitan areas, and in the New York City metropolitan area in particular, for two main reasons. First, business customers have typically been charged higher rates than residential customers so the average revenue per customer will typically be higher in business districts. Second, CLECs that provide, or plan to provide, facilities-based service can serve densely populated areas at a lower cost per customer because a denser concentration of customers reduces the network buildout necessary to serve those customers.¹⁹ Given the extent of facilities-based entry in metropolitan New York and other cities in upstate New York, we have no substantial concerns about the ability of facilities-based carriers to enter the market.²⁰

B.          Resale Entry

Actual entry through resale has occurred to a more limited extent than facilities-based entry. Statutory resale discounts²¹ limit resellers' profit margins, and, as Bell Atlantic recognizes, it appears that resale may principally serve as "a transitional tool on the way to facilities-based competition."²² Specifically, resale allows CLECs -- especially those that serve the more lucrative business market -- to build a customer base with minimal investment while they construct their own network facilities. Resale also allows those CLECs that cannot justify the cost of investing in their own network facilities, such as those serving the less lucrative residential market, the ability to offer local exchange service as part of a bundled package of telecommunications services that "one-stop shopping" customers demand. Thus, although resale alone is not likely to be a major avenue for competitive entry, particularly for serving the residential market, the number of resale lines in service in New York continues to grow.²³

For this reason, it remains important that resale be accessible to those competitors that rely on it. In New York, it appears that the principal barriers to resale competition (other than the limits inherent in the size of the resale discount) have been removed. While Bell Atlantic's wholesale performance to resellers has not been perfect, the Department does not believe that there are performance deficiencies that are significantly impeding entry by resellers.

C.          Unbundled Element Entry

To date, the least common path of entry in New York is entry through unbundled network elements. The use of unbundled network elements was viewed by Congress as one of the principal options for competitors created by the 1996 Act.²⁴ The availability of unbundled elements leased from Bell Atlantic is critical to fostering competition to serve three important classes of customers: small and medium-sized businesses using unbundled loops; residential customers using the UNE-platform; and data services customers using DSL capable UNE-loops. Currently, however, somewhere around 200,000 local lines, approximately 1.7 percent of total access lines, are provided through these forms of unbundled network element entry, but we expect growth in this mode of entry to increase significantly as competition expands in the residential and small- to-medium business market segments.

Consistent with the Department's standard for approval, limited actual entry based on the use of unbundled elements requires closer examination to determine whether Bell Atlantic has developed the technical and operational arrangements to support this mode of entry and whether benchmarks to prevent backsliding by Bell Atlantic have been established. Based on the current record, Bell Atlantic has not yet demonstrated that it provides wholesale services sufficient to support fully open competition based on the unbundled element mode of entry. In the remainder of this evaluation we will focus on our specific concerns about Bell Atlantic's current wholesale support services and the reasons they continue to pose barriers to entry for Bell Atlantic's competitors.

The NYPSC, of course, has examined these issues with considerable care and has concluded that Bell Atlantic has satisfied the competitive checklist requirements of Section 271. Our assessment of the facts regarding Bell Atlantic's wholesale performance is substantially consistent with the NYPSC's assessment.²⁵ There is also substantial agreement between the Department and the NYPSC on the need for Bell Atlantic to continue to improve its performance in the areas we discuss below. To the extent there is a difference between the Department's judgment and that of the NYPSC, it arises largely from the Department's conclusion that needed improvements should be achieved before Bell Atlantic is authorized to provide interLATA services in New York, rather than relying on post- 271 approval regulatory mechanisms to attempt to ensure such improvements. We address this issue in more detail in Section VI of this Evaluation.

III.           Bell Atlantic's Wholesale Performance In Providing Competitors With Unbundled Local Loops

Unbundled local loops ("UNE-L") can be purchased by a CLEC from the incumbent either as a newly provisioned loop or by physically disconnecting the customer's existing in-service loop from the incumbent's switch and reconnecting the loop to the CLEC's switch. The latter process is called a coordinated loop cutover, or "hot cut." The vast majority of current UNE-L orders require a hot cut, and the Commission has recognized that a BOC "must demonstrate that it can coordinate number portability with loop cutovers in a reasonable amount of time and with minimum service disruption."²⁶

Bell Atlantic's performance in processing orders for hot cuts of unbundled loops appears to suffer from a number of deficiencies which, collectively, impose significant costs on CLECs and degrade the quality of service they can offer to their customers. Because of these deficiencies, competition through this important mode of entry is seriously constrained. Bell Atlantic's application provides limited data concerning its hot-cut performance, and much of that information is disputed by other parties.²⁷ However, even relying principally on information provided by Bell Atlantic²⁸ and the NYPSC, there appear to be serious deficiencies in a number of the key performance measures relating to unbundled loops.

First, Bell Atlantic has had substantial problems in providing timely confirmations and rejections of hot-cut orders. Information provided to the Department by Bell Atlantic indicates that approximately 30 percent of both order confirmations and order rejections are late -- i.e., beyond the 24-hour standard established by the NYPSC.²⁹

Second, when Bell Atlantic does return order confirmations, a substantial portion of those confirmations are inaccurate. Bell Atlantic has acknowledged in NYPSC proceedings that as many as 30 to 40 percent of confirmations are inaccurate,³⁰ and CLECs have alleged that levels of inaccurate confirmations are in that range or even greater.³¹ Moreover, it appears that as Bell Atlantic struggles to improve its performance in returning manually processed order confirmations and rejections more quickly, its accuracy suffers significantly. In September, Bell Atlantic improved its combined UNE-P/UNE-L on-time performance for confirmations and rejections,³² but only 42 percent of manually processed orders were correctly submitted by Bell Atlantic personnel to Bell Atlantic's provisioning systems (significantly down from only 64 percent for August).³³

These problems with late and inaccurate order confirmations appear to be the result of a high degree of manual processing of hot-cut orders at the ordering stage.³⁴ In August, more than 83 percent of unbundled loop orders required manual processing of some kind by Bell Atlantic employees, and the problems with late or inaccurate confirmations and rejections appear to arise almost exclusively in connection with these manually processed orders.³⁵ In contrast, almost all of the small number of order rejections and confirmations that flowed through electronically appear to have been reasonably timely and accurate.³⁶

The high level of slow and inaccurate manual order processing imposes significant costs on CLECs, which must devote time, effort and expense to identifying and rectifying problems in order to ensure that orders ultimately are processed correctly.³⁷ Moreover, these problems may require the CLEC to reschedule the cutover of customers' service from Bell Atlantic to the CLEC, imposing inconvenience and delays on customers that choose to switch service providers.³⁸

Third, Bell Atlantic fails to complete scheduled hot cuts on time in a significant number of cases -- around 10 percent of orders, even under statistics most favorable to Bell Atlantic. Reliable performance in completing hot cuts correctly and at the time scheduled is extremely important because of the risk to the customer of losing dial tone for more than a brief period.³⁹ Bell Atlantic reported to the NYPSC that it completed 94 percent of hot cut orders in July,⁴⁰ but a detailed, order-by-order review conducted by the NYPSC indicated that Bell Atlantic actually provisioned only 88 percent of AT&T orders on time.⁴¹ But that number appears to overstate Bell Atlantic's on-time performance in large part because it reflects a definition of "on time" under which an order not completed at the initially scheduled time, but within a subsequently rescheduled time, is considered "on time," even if Bell Atlantic failures caused it to be rescheduled.⁴²

Fourth, when hot cuts are provisioned, there are a substantial number of instances, perhaps more than 10 percent, in which the customer's directory listings are dropped or delayed.⁴³ This problem is a particular concern for business customers that depend on directory listings so that their customers can reach them.⁴⁴ After the KPMG test identified these problems with directory listings, Bell Atlantic implemented a process improvement plan that was highly reliant on manual review.⁴⁵ KPMG reviewed the new process, but evidence subsequent to that review suggests that the process changes have not provided a sufficient solution to these problems.⁴⁶

It is difficult to assess the precise point at which poor performance on any single dimension of Bell Atlantic's wholesale performance begins to have a significant adverse effect on competition, and we certainly do not mean to suggest that a small deviation from any single standard established by the NYPSC should be dispositive in evaluating Bell Atlantic's application. However, it seems clear that, collectively, the number and magnitude of the deficiencies noted above are imposing a real constraint on competition through the use of unbundled loops and that significant improvement is needed in this area.⁴⁷

We are unpersuaded by Bell Atlantic's argument that these deficiencies should be disregarded because they affect only a small percentage of the lines ordered by CLECs to date.⁴⁸ That argument is unpersuasive for three reasons. First, it seems clear that CLECs would have ordered a much larger number of unbundled loops but for the problems created by Bell Atlantic's handling of such orders. Bell Atlantic's recent performance, while clearly much improved over its earlier performance, still reflects significant problems. Because of the very serious and well-documented problems that persisted until quite recently,a number of CLECs severely limited or completely postponed their attempts to provide service through unbundled loops. Thus, the number of hot-cut orders submitted to date is relatively low precisely because of Bell Atlantic's historically poor performance in handling such orders. Second, the economic significance of competition through unbundled loops is greater than would be suggested merely by assessing the percentage of total customer lines served. The customers predominantly served by unbundled loops tend to be heavy users of telecommunications services and therefore tend to be particularly profitable customers both for CLECs and for Bell Atlantic.⁴⁹ Unbundled loops may be one of the principal means for CLECs to serve small and medium-sized businesses -- a large and important market.⁵⁰ Third, as competition develops and matures, this mode of entry is likely to become more significant than it is today. Among other factors contributing to that trend, limits on the availability of unbundled switching (and, hence, the UNE-platform) can be expected to increase CLEC demand for unbundled loops to be connected to the CLEC's own switch.⁵¹ In sum, adequate wholesale performance in providing unbundled loops is important today and will become even more important in the future.

As noted above, Bell Atlantic's recent performance with regard to hot-cut orders, though still deficient in a number of ways, is considerably improved over its performance in the first half of 1999. The NYPSC expects further improvement to be forthcoming and notes that Bell Atlantic has now "put in place the procedures and training to maximize effective loop ordering and provisioning . . . and to minimize provisioning postponements and local service request confirmation delays and inaccuracies due to Bell Atlantic-NY process problems."⁵² The problems noted above do not appear to be insolvable, and the Department is hopeful that recently implemented changes will effectively correct these problems. However, Bell Atlantic filed this application before the results of those improvements could be assessed or demonstrated, and at this time there is no basis in the record to conclude that the problems have been resolved.⁵³

IV.          Bell Atlantic's Wholesale Performance In Providing Unbundled Elements To CLECs Wishing To Offer DSL High Speed Data Services

Residential demand for high speed digital services is growing very rapidly as consumers and telecommuters take advantage of attractive "broadband" applications on the Internet. Some forecasts of the demand for broadband services predict tens of millions of subscribers within five years.⁵³ For some time, Bell Atlantic has aggressively marketed its integrated services digital network ("ISDN") service for Internet access;⁵⁴ it is now in the process of rolling out a major deployment of asymmetrical DSL ("ADSL") services, both under the Bell Atlantic brand and in conjunction with major Internet service providers such as America Online.⁵⁵ Such services are expected to be marketed with long distance service when Bell Atlantic receives 271 authority. Clearly, an ability to offer high- speed Internet access will soon be a crucial requirement for all major carriers.⁵⁶

Although the expected demand for digital services has increased in recent years, it has been clear for some time that CLECs would seek access to unbundled loops in order to offer these services. As noted by the NYPSC, the FCC's 1996 Local Competition Order required incumbent LECs to provide access to unbundled loops, including specifically "two-wire and four-wire loops that are conditioned to transmit the digital signals needed to provide such services as ISDN, ADSL, HDSL [High bit rate DSL], and DS-1 level signals."⁵⁷ In its proceedings pursuant to Section 706 of the 1996 Act, the FCC reaffirmed its requirement that incumbents provide competitors with access to loops for the provision of digital services and ruled that the incumbents may not dictate the particular use that competitors may make of these facilities.⁵⁸

Bell Atlantic's Pre-Filing Statement did not address provisioning issues for DSL service, because CLECs had not begun to offer DSL services in New York at the time of that commitment, although the commitment did propose to establish a metric to measure performance regarding "premium" loops that Bell Atlantic had agreed to provide to DSL carriers.⁵⁹ After CLECs began to offer DSL services in mid-1998, they complained to the NYPSC and the FCC⁶⁰ that (1) they could not obtain needed preordering information, (2) they were not receiving timely firm order confirmations, (3) installations of loops were not completed at the committed date, and (4) the DSL metric reported by Bell Atlantic was not meaningful because it was not adjusted for loops that were installed incorrectly.⁶¹ It is still not clear that these problems have been resolved.⁶²

Access to preordering information is particularly important in connection with DSL services because of the special loop requirements for such services. CLECs need detailed information about available loops so that they can quickly determine whether a prospective customer can be served and what grade of service can be offered. CLEC comments make it clear that their inability to inform their customers promptly and reliably of service availability and installation times has damaged their ability to compete.⁶³

Bell Atlantic moved to address these preordering issues only after it introduced its own retail DSL service in June 1999. Through a tariff effective August 30, 1999, Bell Atlantic offered to provide one automated and two manual options for obtaining preordering information. Bell Atlantic asserts that the automated database will cover 90 percent of the lines by the end of the year and, in testimony filed on October 18, promises to begin including more relevant information in that database than is currently offered.⁶⁴ CLECs object, however, that the database does not work reliably⁶⁵ and that as a practical matter they will have to resort to Bell Atlantic's manual processes, causing added delay and substantial additional charges.

These issues, and others discussed below, are the subject of an ongoing collaborative proceeding before the NYPSC.⁶⁶ While we expect that the NYPSC will soon resolve many of the disputed issues in that proceeding, we cannot conclude on the current record that Bell Atlantic is currently providing adequate access to preordering information needed to provide DSL services.

There are also serious unresolved issues relating to DSL ordering and provisioning processes. At the present time, orders for DSL loops do not flow through Bell Atlantic's ordering systems, but must be manually processed before entry into the provisioning systems. The CLECs complain that these procedures have resulted in late and inaccurate order confirmations.⁶⁷ These concerns seem to be supported by the performance reports for August and September, which show that, in those months, Bell Atlantic confirmed only 59.37 percent and 55.4 percent of ADSL orders on time.⁶⁸ Because there is substantial reason to believe that demand for DSL service will quickly grow to much greater volumes than are currently being experienced, the Commission needs to be satisfied that Bell Atlantic will be capable of handling reasonably expected increases in DSL order volumes.

Bell Atlantic's record for provisioning DSL loops is also the subject of sharply conflicting allegations in the record. Bell Atlantic states in its application that its on-time performance in providing DSL loops is very good, and the September report shows missed appointments for ADSL at only 3.22 percent for 653 loops.⁶⁹ NorthPoint and other CLECs respond that these measurements are meaningless because "a substantial number of DSL loops tendered by Bell Atlantic to DSL CLECS . . . are defective, open, impaired, or in some significant manner wholly ‘incomplete.'"⁷⁰ Covad similarly complains that a substantial portion of the loops Bell Atlantic installs are defective as shown by its test equipment.⁷¹

The NYPSC has not undertaken a reconciliation of these conflicting claims; however, during the collaborative instituted by the NYPSC, Bell Atlantic agreed to the CLECs' outstanding requests to establish a cooperative installation protocol, which it began to implement in the middle of September 1999. In addition, the NYPSC's carrier-to-carrier proceeding has begun the process of establishing metrics to measure more accurately performance in providing DSL loops.⁷²

As to Bell Atlantic's historical performance in provisioning DSL loops, we are unable to conclude on the current record that Bell Atlantic has demonstrated an acceptable level of performance. It is possible, however, that the Commission may obtain information not currently available to the Department that would support such a conclusion. Whatever the record as to historical performance, we are hopeful that the new installation procedures adopted by Bell Atlantic in September 1999, and the improved performance measures that will be adopted by the NYPSC, will soon result in documented improved performance. But because Bell Atlantic filed this application before the results of those efforts can be seen, we cannot conclude that CLECs currently have access to DSL loops necessary for them to compete effectively.

V.           Bell Atlantic's Wholesale Performance In Providing Competitors With The UNE-Platform

The UNE-platform is likely to be the principal vehicle, at least in the short term, for competitors offering mass market services to residential and small business customers.⁷³ Several carriers have invested heavily in preparing to offer service on a large scale through the UNE-platform. These carriers are currently in a startup mode in which their marketing efforts have been limited, as they and Bell Atlantic identify and correct problems in ordering and provisioning service. The number of orders submitted to Bell Atlantic for processing during this startup phase has been substantial -- roughly 90,000 orders in August⁷⁴ -- and we expect that number to increase greatly over the next six months if there are no serious systems problems which constrain that growth.

Effective competition through the UNE-platform will require both CLECs and Bell Atlantic to have stable, robust, and efficient automated systems. Profit margins for serving the average residential customer are relatively modest; if CLECs are required to devote substantial resources to manual processing of orders, the costs of doing so may have a serious impact on those margins. In addition, heavy reliance on manual processes inevitably generates mistakes and delays in processing orders, which may seriously affect service quality. Customers may be wary of switching to CLECs if there is considerable uncertainty about the quality of service they offer.

Bell Atlantic has done much to develop and implement the types of automated systems that will be needed in this market environment. After serious and persistent startup problems, many of which were identified and corrected through the KPMG testing process, the systems have been developed and refined to the point that Bell Atlantic has demonstrated an acceptable level of performance in many areas.⁷⁵

Despite this substantial progress, however, two concerns remain. First, a large portion of UNE-platform orders still require some degree of manual processing. This heavy reliance on manual processing unnecessarily increases CLEC costs and creates a significant risk that there will be customer-affecting service problems when order volumes substantially increase. Second, the process of coordinating, testing, and implementing changes in Bell Atlantic's systems has generated significant problems; it is not clear that these issues have been adequately resolved.

A.          Processing Of UNE-Platform Orders

Our concerns about Bell Atlantic's wholesale support for UNE-platform orders start with the high number of rejected orders. Overall, one third of the UNE orders that CLECs submit are rejected by Bell Atlantic.⁷⁶ Many of these orders are undoubtedly rejected because of errors committed by CLECs, for which Bell Atlantic should not be held responsible.⁷⁷ But order rejections may also occur for reasons within Bell Atlantic's control. Some "CLEC" errors may occur because Bell Atlantic has not provided adequate documentation of the requirements for valid orders,⁷⁸ and there is some evidence that Bell Atlantic erroneously rejects a significant number of correct orders.⁷⁹ The Department does not have sufficient information at this time to determine the extent to which Bell Atlantic is or is not responsible for the high levels of order rejections. But it is likely that the high rejection rate has unfortunate repercussions. CLECs must put rejected orders back into the ordering queue, and that may extend the original service due date. CLECs have to reschedule with customers service dates that are extended, particularly orders for new lines.⁸⁰

Even more troubling is the high level of manual processing that is required for UNE platform orders, a phenomenon that is largely within Bell Atlantic's control. At present, service representatives in Bell Atlantic's ordering center manually process almost half of UNE-platform orders.⁸¹ Manually processed orders are processed much more slowly and with much higher numbers of mistakes⁸² than electronically processed orders.⁸³ For example, while an electronically processed order confirmation is returned to the CLEC in an average of 13 minutes, a manually processed confirmation is not returned on average for 15 hours.⁸⁴ And Bell Atlantic's service order representatives make mistakes on a significant number of the orders on which they work.⁸⁵ Bell Atlantic may be improving its "on- time" performance for order confirmations and rejects, although it is difficult to know based on one month of improved performance.⁸⁶ Moreover, it will always take much longer to process these notices manually than it would to process them electronically, and one would expect the current level of mistakes on manually processed orders to be reduced in an automated process.

Manual processing of orders and high reject rates increase CLEC processing costs because CLECs must devote additional resources to monitor the ordering and provisioning process and correct mistakes.⁸⁷ Those costs can be expected to increase as order volumes increase, and such costs may impair the competitive vitality of CLECs.

It does not appear that the manual processing is creating serious customer-affecting service problems at current volumes. If, however, order volumes increase rapidly and substantially, in accordance with CLECs' current marketing projections, there is a significant risk that customer- affecting service problems will develop, absent a reduction in the current level of manual processing. CLECs currently are giving Bell Atlantic more time to provision most UNE- platform orders than the period -- the "standard interval" -- that Bell Atlantic has told CLECs it needs to provision these orders.⁸⁸ As competition for residential customers increases, CLECs will need to compete more directly on the amount of time needed to install local service. The record suggests that Bell Atlantic is not finding it easy to provision UNE-platform service when CLECs request the standard interval,⁸⁹ and order processing delays engendered by heavy reliance on manual processing may exacerbate the problem.

B.          Bell Atlantic Has Not Shown That Its OSS Environment Is Stable And Predictable

The record also indicates reasons for concern relating to Bell Atlantic's record of providing the necessary support to enable CLECs to develop and maintain their interfaces with Bell Atlantic's systems. CLECs intending to mass market UNE-platform-based service will ultimately have to build their own computer software to connect their ordering systems to Bell Atlantic's order processing and provisioning systems. In prior evaluations, we highlighted the competitive importance of these "application-to-application" interfaces.⁹⁰ It appears to be difficult for CLECs to move from Bell Atlantic's proprietary web-based Graphical User Interface ("GUI") to application-to-application interfaces, such as Electronic Data Interchange ("EDI"), for gathering pre-ordering information and for submitting orders.⁹¹ While there are myriad explanations for this continued dependence on the GUI,⁹² we are concerned that Bell Atlantic's EDI documentation has been so unstable that it has impaired CLEC ability to develop these interfaces⁹³ and that Bell Atlantic has not yet demonstrated, through its change control performance reports, that it is able to provide CLECs with relatively stable and predictable documentation.⁹⁴

Once they build interfaces using Bell Atlantic's documentation, CLECs must make sure that these interfaces interact correctly with Bell Atlantic's systems. Bell Atlantic provides CLECs with a quality assurance testing environment that serves two important functions: It is the environment in which new CLECs get their software interfaces certified by Bell Atlantic, and it is where established CLECs test new releases of Bell Atlantic's interfaces. Such testing is necessary to prevent major service disruptions when Bell Atlantic makes changes in its side of the interface. KPMG found Bell Atlantic's software testing environment seriously deficient; this finding raises the concern that competitors will be unable to develop and maintain the computer connections necessary to order high volumes of UNE-platform from Bell Atlantic.⁹⁵          

Commendably, Bell Atlantic has recognized the importance of implementing improvements in these areas. On October 8, 1999, after filing this application, Bell Atlantic proposed a series of flow-through enhancements and presented the NYPSC with a three-phase plan to increase the percentage of UNE-platform orders processed electronically.⁹⁶ To improve its software documentation problems, Bell Atlantic developed a set of change management metrics designed, inter alia, to measure how often it provides CLECs with complete software documentation in a timely manner.⁹⁷ Bell Atlantic also undertook a two-phase plan to improve its quality assurance testing environment. The permanent phase of the improvement plan, a new separate testing environment, opened in late September 1999, just before Bell Atlantic filed this application.

We are hopeful that the flow-through enhancements will be successfully implemented, that Bell Atlantic is improving its ability to comply with its change management commitments and that the permanent test environment will meet CLEC testing needs. The results of these process improvements, however, do not appear in the current record.

VI.           Post-271 Entry Performance Commitments Should Not Be Relied Upon To Ensure Implementation Of The Process Improvements Necessary To Open the Market

Bell Atlantic argues that if its application is granted, it will still have strong incentives to improve its performance in the areas discussed above,⁹⁸ pointing in particular to performance assurance plans which were orally adopted by the NYPSC on October 27, 1999.⁹⁹ The Department does not believe it would be wise to rely solely on these plans, rather than the more powerful incentives created by Section 271, to ensure rapid completion of necessary market-opening measures.¹⁰⁰

The standard that the Department uses in evaluating Section 271 applications -- the requirement that local telecommunications markets be shown to be fully and irreversibly open to competition before the BOC may offer long distance services -- is based, in significant part, on the difficulty of securing rapid implementation of new and complex access arrangements through regulation alone.¹⁰¹ Regulators necessarily have much less information than the regulated firm with which to judge which types of new arrangements are feasible, how they may best be implemented, how long it will take to implement them, and how effective they will be in achieving the desired wholesale performance. Moreover, the legal processes that are required to prove inadequate performance and to levy sanctions may generate substantial delay and uncertainty about the ultimate outcome of the regulatory process, and the sanctions which regulators may impose are often too small to motivate the regulated firm to implement the new arrangements rapidly, when rapid implementation will result in the loss of market power. The use of an appropriate standard under Section 271 avoids these difficulties by ensuring that the BOC has powerful incentives (i.e., the ability to enter the long distance market) to cooperate to open its markets.¹⁰²

Our concerns about relative efficacy of regulation (as compared to the use of incentives under an appropriate Section 271 standard) can be illustrated by specific aspects of the performance assurance plans as proposed by Bell Atlantic. The effectiveness of those plans will depend on several important factors, including (i) clarity as to the precise level of performance that will be required, (ii) certainty that inadequate performance will be sanctioned, and (iii) adequate penalties that are large enough to create incentives for adequate performance.

Penalties under the performance assurance plans are triggered on the basis of performance that drops below defined statistical standards on specific performance measures. But at the present time, there are still-unresolved disputes concerning the precise definitions that are or should be used for key measures and the level of performance at which penalties would be imposed. Bell Atlantic has proposed that lower standards be applied to the special measures regarding UNE ordering performance and hot cut performance. In addition, in the parallel track metrics docket, Case 97-C-0139, Bell Atlantic requested (the day after filing the proposed amended plans) that one of the flow-through metrics included as a special measure be redefined.¹⁰³ If the NYPSC were to accept Bell Atlantic's proposed redefinition, Bell Atlantic would be unlikely to incur any penalties under the special flow-through measure even if it fails to increase its current level of flow-through.

Even after these matters are clarified, there will be opportunities for Bell Atlantic to argue that inadequate performance should not trigger penalties. Within 45 days from the end of a month showing inadequate performance, Bell Atlantic can request to have its performance results modified on three grounds: (i) clustering of data, (ii) unusual CLEC behavior (modifications if "spiked" or highly variable order volumes affects manually processed confirmation and reject times), and (iii) for absolute standards, "non-normal" operating conditions.¹⁰⁴ No procedures or time requirements for considering these waiver requests are proposed in the amended plans, and the manner in which these standards will be interpreted is unclear at this time. This creates the potential for litigation and delay in imposing penalties and uncertainty that inadequate performance will in fact be punished.¹⁰⁵

The size of any penalties that may be imposed on Bell Atlantic for specific failures is not at all clear to the Department at this time. Bell Atlantic emphasizes the total penalties which could be imposed, in theory, for poor performance -- $269 million in bill credits in the first year of the plan, and $235 million in following years. Because of the structural caps and allocations within the plan, the penalties for specific deficiencies (e.g., a failure to improve flow-through rates or to provision unbundled loops adequately) would be much smaller -- though we are unable to determine exactly how much smaller.¹⁰⁶ Moreover, there is no evidence in the application suggesting what, if any, amount of bill credits will provide sufficient incentives for Bell Atlantic to improve its current performance levels.

In offering these observations about the performance assurance plans, we do not mean to imply any criticism of the diligent efforts of the NYPSC to develop tools for assuring adequate wholesale performance. Our point, rather, is that even the best efforts to do so will have a limited degree of success because of inherent weaknesses of the regulatory process in this context. The appropriate use of Section 271 incentives will overcome some of these difficulties and, in our view, will be more effective in securing rapid and effective removal of the remaining barriers to competition in New York.¹⁰⁷

VII.          Conclusions And Recommendations

The current application demonstrates that Bell Atlantic has completed most of the steps needed to establish local telecommunications markets in New York that are fully and irreversibly open to competition. But the remaining obstacles to competition, though few in number, are significant. Effective access to unbundled loops, to provide both traditional voice and advanced data services, is a critical precondition to competition to serve important classes of customers. Competition to serve millions of residential customers through the UNE-platform will require robust and reliable electronic systems so that CLECs will have the ability to provide high quality service in an efficient manner. In both of these areas, Bell Atlantic has done a great deal to open its markets but has not completed (or demonstrated that it has completed) the process.

Because Bell Atlantic has come so far, and because of the importance of the remaining steps, this application requires careful judgments by the Commission. It is clear to the Department that Bell Atlantic should be required to demonstrate additional progress in solving the remaining problems before it is permitted to enter the long distance market. It is somewhat less clear precisely how the Commission should effectuate such a requirement.

We note, first, that some of our concerns relate to disputed factual issues, as to which, on the current record, the Department has concluded that Bell Atlantic has not made a sufficient showing. It is possible, however, that information from Reply Comments and ex parte submissions will provide additional support for Bell Atlantic's claims and justify a conclusion by the Commission different from that reached by the Department on the basis of the current record.

As to other issues, the Commission will need to make careful judgments concerning the most appropriate disposition of this application. The Department of Justice starts with a strong presumption -- based on the structure and terms of the statute, on the Commission's prior decisions under Section 271, and on the Department's own economic and competitive analyses -- that a BOC should be required to demonstrate that all important market opening measures have been completed before it may enter the long distance market. Moreover, given the procedural constraints arising from the 90-day review period for Section 271 applications, we strongly support the Commission's prior decisions limiting the ability of applicants to submit data concerning post-application performance in support of their application.

These considerations lead us to the conclusion that a BOC should not be permitted to offer in-region interLATA services as long as important constraints on local competition remain. It is, therefore, our judgment that Bell Atlantic should not be permitted to offer such services until it demonstrates that it has solved the existing problems in its provision of access to unbundled network elements.

The Commission could implement this judgment by denying Bell Atlantic's application in a manner which identifies as clearly as possible the steps that Bell Atlantic must take to secure approval in a subsequent re-application. In light of the limited nature of the remaining problems, the Commission could also consider, and make clear that it will provide, expedited review procedures for any subsequent application for New York.

As an alternative, the Commission might be able to approve this application subject to carefully crafted conditions consistent with the principles we have articulated, under which Bell Atlantic would be permitted to offer interLATA services only after taking specified steps and demonstrating that its performance has met appropriate requirements. In weighing this option, however, the Commission should (i) consider carefully the scope of its legal authority to impose conditions on its approval of a Section 271 application, as to which we express no view; (ii) provide mechanisms sufficient to enable it to reach an informed judgment and ensure full compliance with any conditions; and (iii) take care to avoid a precedent that would permit the requirements of Section 271 to be satisfied merely by promises of future compliance. We are concerned that such a conditional approval of this application might encourage future applications in states that are less open to competition than New York has been shown to be. Still, in light of the substantial record of progress in New York reflected in the record, we do not foreclose the possibility that the Commission may be able to approve this application at the culmination of these proceedings.

I hereby certify that I have caused a true and accurate copy of the foregoing Evaluation of the United States Department of Justice to be served on the persons indicated on the attached service list by first class mail, overnight mail, or hand delivery, on November 1, 1999.

¹ Pub. L. No. 104-104, 110 Stat. 56 (1996) (codified as amended in various sections of 47 U.S.C.) ("1996 Act").

² But see infra note 20.

³ Proceeding on Motion of the Commission to Review Service Quality Standards for Telephone Companies, NYPSC, Case No. 97-C-0139 ("carrier-to-carrier proceeding"). See, e.g., Bell Atlantic Performance Measures Compliance Filing. For complete citations to prior DOJ Evaluations and FCC Orders, filings related to this application, affidavits and declarations and attachments thereto, KPMG's report on systems testing, and attachments to this Evaluation, see the citation index at iv-x.

⁴ Petition of New York Telephone Company for Approval of Its Statement of Generally Applicable Terms and Conditions Pursuant to Section 252 of the Telecommunications Act of 1996, NYPSC, Docket No. 97-C-0271 (Feb. 13, 1997), attached to Bell Atlantic Brief as App. C, Vol. 1, Tab 1; Supplemental Petition of Bell Atlantic-New York, In re: Petition of New York Telephone Company for Approval of Its Statement of Generally Applicable Terms and Conditions Pursuant to Section 252 of the Telecommunications Act of 1996, NYPSC Docket No. 97-C-0271 (Nov. 6, 1997), attached to Bell Atlantic Brief as App. C, Vol. 10a-c, Tab 122.

⁵ Bell Atlantic Pre-Filing Statement at 34.

⁶ For example, while KPMG's orders flowed through Bell Atlantic's order processing systems at very high rates, the actual commercial flow-through rate is much lower. Compare Dowell/Canny Decl., Tab 3D (OR-5-01) with KPMG Final Report, POP7, IV-160 to IV-161, Table 4-7.10: POP-7 Flow-Through, Test Cross References P7-2 and P7-3. See also Miller/Jordan Decl. ¶ 61. See also infra notes 35 & 81 and accompanying text.

⁷ At the NYPSC's request, KPMG participated in a one-day observation at a DSL CLEC. DSL was not a component of the formal test plan, and KPMG's informal observations do not appear in the final report. 7/29/99 Technical Conference Transcript at 3669-3672.

⁸ It would be useful for Bell Atlantic to report disaggregated UNE-loop and UNE-platform ordering data. Currently, the statistics for the smaller volume of UNE-loop orders are obscured when combined with UNE-platform orders. See DOJ Ex. 5: DOJ Table of Processing Times at 1-2.

⁹ CLECs have raised concerns about the hot cut and DSL measures, discussed more fully below at notes 27 and 72 and accompanying text, while Bell Atlantic has raised concerns about the usefulness of the average provisioning intervals.

¹⁰See Dowell/Canny Decl., Tab 3D.

¹¹ This open market standard is explained more fully in the Affidavit and Supplemental Affidavit of Marius Schwartz and in our evaluation of SBC's Section 271 application in Oklahoma. See DOJ Ex. 1: Schwartz Aff. ¶¶ 149-192; DOJ Ex. 2: Schwartz Supp. Aff. ¶¶ 26-60; DOJ Oklahoma Evaluation at vi-vii, 36-51.

¹² As we have stated previously, the Department does not regard small market shares held by competitors, or even the absence of entry (either altogether or using a particular entry path), standing alone, as conclusive evidence that a market remains closed to competition or as a basis for denying an application under Section 271. See, e.g., Evaluation of the United States Department of Justice, In re: Second Application of BellSouth Corporation, BellSouth Telecommunications, Inc., and BellSouth Long Distance, Inc., for Provision of In-Region, InterLATA Services in Louisiana, CC Docket No. 98-121, at 2-3 (Aug. 19, 1998).

¹³ U.S. Census Bureau, State Rankings of Resident Population (July 1,1998) http://www.census.gov/statab/ranks/rank01.txt.

¹⁴ More than 7.4 million people are city residents, http://www.census.gov/population/estimates/metro-city/SC100K98-T1-DR.txt, and more than 8.6 million people live in the immediate in-state metropolitan area. The greater metropolitan area -- which includes northern New Jersey and parts of Connecticut and Pennsylvania and is not considered for purposes of this section -- contains almost 20 million people .

¹⁵ New York was the nation's fourth-largest state in long distance traffic in 1998, with 43,115,409 interLATA billed access minutes -- 6.3% of the nation's total. See Federal Communications Commission, Preliminary Statistics of Communications Common Carriers, at Table 2.6 (1998). Only California, Florida, and Texas had more billed access minutes than New York.

¹⁶ There were slightly more than 16 million total access lines in New York served by reporting LECs, including 12.3 million switched lines, id. at Table 2.5, and an additional 389,194 lines presubscribed to non-reporting LECs as of December 31, 1997. Id. at Table 2.3.

¹⁷ Taylor Decl., Attach A. ¶ 1 at Table 1 & Ex. 2.

¹⁸ President's Council of Economic Advisers, Progress Report on Growth and Competition in U.S. Telecommunications (1993-1998) http://www.ntia.doc.gov/ntiahome/press/ceafinalrpt.htm (CLECs "have captured between two and three percent of the local services market measured by lines."). See also Federal Communications Commission, FCC Local Competition: August 1999 Report, Press Release, at 2 (Aug. 1999) http://www.fcc.gov/Bureaus/Common_Carrier/Reports/FCC-State_Link/IAD/lcomp99-1.pdf, (CLECs' presence remains less than 5% of the local market in most areas.).

¹⁹See, e.g., Taylor Decl., Attach. A. ¶ 8 ("Manhattan is especially attractive to competitors" because "[b]usiness loops in Manhattan are ... over 2,000 times more dense than in upstate New York" and "[a] competitive switch in Manhattan can reach more potential customers than one placed anywhere else in the country.")

²⁰ We note, however, that a number of cementers have raised complaints that Bell Atlantic has failed adequately to provision interconnection trunks on a timely basis. According to the cementers, Bell Atlantic often delays CLECs for weeks or months before installing interconnection trunks. See, e.g., Teligent Comments at 6-7, 8-10; Allegiance Comments at 11-12; e.spire/Net 2000 Comments at 16-22; Prism Comments at 20-21; Focal Comments at 5-6; Omnipoint Comments at 7-13; NEXTLINK Comments at 3; ALTS Comments at 44-45. These allegations, if true, would be cause for serious concern. However, the allegations here were not raised or considered in the final phase of the New York state 271 proceedings, apparently because the cementers chose not to raise them at that stage. See NYPSC Eval. at 17-18 and n.1. We therefore have very little record evidence before us and have not had the opportunity to evaluate fully the facts or circumstances surrounding the allegations. Because the ability to obtain interconnection trunks on a reasonable and timely basis is critically important to CLECs that have their own network facilities, the Commission should consider these allegations carefully before reaching any final conclusion.

²¹ In New York, a reseller may purchase wholesale telephone service from Bell Atlantic at a 19.1% discount if a CLEC uses Bell Atlantic's operator services, and a 21.7% discount if a CLEC provides its own operator services. See Opinion and Order Determining Wholesale Discount, Opinion 96-30, In re: Petition of New York Telephone Company for Approval of Its Statement of Generally Available Terms and Conditions Pursuant to Section 252 of the Telecommunications Act of 1996 and Draft Filing of Petition for InterLATA Entry Pursuant to Section 271 of the Telecommunications Act of 1996, NYPSC,Case No. 97-C-0271, at 3-4 (Nov. 27, 1996), attached to Bell Atlantic Brief as App. G, Vol. 1, Tab 7, App. B.

²² Taylor Decl., Attach A. ¶ 43.

²³ New York resellers served 121,000 lines in December 1997 and now exceed 310,000 lines. See Federal Communications Commission, FCC February 1999 Trends Report, at Table 9.2 (Feb. 1999) http://www.fcc.gov/Bureaus/Common_Carrier/Reports/FCC-State_Link/IAD/trend299.pdf. UniDial and CTC have recently signed multiyear contracts to resell large blocks of lines throughout Bell Atlantic's service region at a significant additional discount. See 19 Communications Daily, Issue 92 (UniDial), May 13, 1999, available in 1999 WL 7579433; 19 Communications Daily, Issue 130 (CTC), July 8, 1999, available in 1999 WL 7579879.

²⁴ 47 U.S.C. § 251(c)(3).

²⁵ We have examined these facts to assess their impact on the development of competition in New York and have not, however, attempted to determine whether they establish compliance with the legal requirements of the competitive checklist or the Commission's rules, matters which we leave for the Commission's judgment.

²⁶FCC Louisiana II Order at 164.

²⁷ In June 1999, Bell Atlantic withdrew all hot-cut data submitted prior to June 18, 1999, from the New York state 271 process in the face of concerns and questions regarding these data. See AT&T Comments at 39 n.9; Meek Aff. ¶¶ 16, 107-08;Letter from Randal Milch, Associate General Counsel, Bell Atlantic-State Regulatory North, to Andrew Klein, Assistant Counsel, New York Public Service Commission (June 18, 1999), attached to Bell Atlantic Brief as App. C, Vol. 51, Tab 789. As a result, there are only thirteen weeks of hot-cut data on the basis of which to evaluate Bell Atlantic's performance.

²⁸ In October 1999, after filing its 271 application to the Commission, Bell Atlantic provided the Department with supplemental data disaggregating its UNE-L and UNE-P performance. To the Department's knowledge, these data have not been provided to the Commission, the NYPSC or the CLEC community for review. We have attached these disaggregated Bell Atlantic data to our Evaluation as Ex. 3 ("UNE-L Disaggregated Data") and Ex. 4 ("UNE-P Disaggregated Data").

²⁹ In August 1999, Bell Atlantic returned only 72% of order confirmations (Local Service Request Confirmations or "Lsrcs") and 68% of rejects within 24 hours, far below New York's 95% standard, and performance in June and July was even worse. DOJ Ex. 3: UNE-L Disaggregated Data at 4. Even using combined UNE-L/UNE-P carrier-to-carrier data, Bell Atlantic has still consistently fallen well below the New York standards for timely return of Lsrcs and rejects. See DOJ Ex. 5: Table of Processing Times at 1-2; Dowell/Canny Decl., Tab 3D at 78 (OR-1-04, OR-2-04); Pfau/Kalb Aff. ¶ 103. Bell Atlantic's explanation that it meets the New York standard "on average," Bell Atlantic Brief at 41, only underscores the need for appropriate disaggregation so that poor performance in one area is not masked by aggregation.

³⁰ NYPSC Eval. at 81 (citing Minutes of a Technical Conference, In re: Petition of New York Telephone Company for Approval of Its Statement of Generally Available Terms and Conditions Pursuant to Section 252 of the Telecommunications Act of 1996 and Draft Filing of Petition for InterLATA Entry Pursuant to Section 271 of the Telecommunications Act of 1996, NYPSC, Case 97-C-0271, at 3956 (July 30, 1999), attached to Bell Atlantic Brief as App. C, Vol. 59, Tab 890). Although not directly a measure of Lsrc accuracy, Bell Atlantic incorrectly input a significant number of manually processed CLEC orders into its service order systems. See Dowell/Canny Decl. ¶ 53 and Tab 3D at 102 (OR-6-01) and infra n.33. These data support CLEC accuracy complaints and cast doubt on Bell Atlantic's claim of more than 98% Lsrc accuracy for the last several months. Dowell/Canny, Tab 3D at 102 (OR-6-03).

³¹See, e.g., NYPSC Eval. at 81 & n.3 (CLECs estimate Lsrc inaccuracies of over 50%).

³² Although Bell Atlantic fell short of the New York standard for the sixth straight month, it finally exceeded 90% for September. DOJ Ex. 6: Aggregate September Performance Data at 7 (OR-1-04 = 92% Lsrcs within 24 hours; OR-2-04 = 91% rejects within 24 hours). These data, of course, only became available after Bell Atlantic filed its application. The Department received these September performance data shortly before filing this Evaluation and has thus been able to undertake only a cursory review of them.

³³See id. (OR-6-01, September Completed Service Order Accuracy = 42%); Dowell/Canny Decl. ¶ 53 and Tab 3D at 102 (OR-6-01, August Completed Service Order Accuracy = 64%).

34 Hot cut loop provisioning inevitably is a heavily manual process, but Bell Atlantic processes orders into its back end provisioning systems in two ways: either on a fully mechanized, "flow-through" basis or through manual input by Bell Atlantic employees.

³⁵ DOJ Ex. 3: UNE-L Disaggregated Data at 4 (UNE-L flow-through for August only 17%). Actual commercial experience is vastly different from that of KPMG, which found that 85% of loop orders were capable of flowing through electronically. KPMG Report at POP7, IV-160, Table IV-7.10: POP-7 Flow-Through, Test Cross Reference P7-3.

³⁶ Based on Bell Atlantic's disaggregated UNE-L performance data, Bell Atlantic on average returned mechanized order confirmations ("Lsrcs") and rejects within the New York two-hour standard more than 98% of the time. DOJ Ex. 3: UNE-L Disaggregated Data at 1-5. Even under aggregated carrier-to-carrier data, Bell Atlantic exceeded New York's standard (95% within two hours) for timely return of mechanized Lsrcs for June, July and August (OR-1-02) but fell slightly below the New York standard for timely return of mechanized rejects (OR-2-02). DOJ Ex. 5: Table of Processing Times at 1-2; Dowell/Canny Decl., Tab 3D at 78, 90, 102.

³⁷See, e.g., AT&T Comments at 35-37; Meek Aff. ¶¶ 36-41, 61; Aquilina Aff. ¶¶ 36-38.

³⁸See, e.g., AT&T Comments at 37; Meek Aff. ¶¶ 34, 61.

³⁹See AT&T Comments at 31-32; Mulligan Aff. ¶¶ 5, 28; Allegiance Comments at 10; Choice One Comments at 5. According to a survey conducted by the Competition Policy Insititute, "[t]he strongest impediment to switching [LECs] comes from concern about service interruptions during the change over." CPI Comments, Att A at 11.

⁴⁰ Bell Atlantic Brief at 18; Lacouture/Troy Decl. ¶ 72. Bell Atlantic also relies on KPMG's "test" of its hot-cut procedures, which found that Bell Atlantic's technicians followed the hot-cut procedures 97% of the time. Bell Atlantic Brief at 19; KPMG Final Report, POP3, IV-60 to IV-62, Test Cross References P3-22 and P3-24; Lacouture/Troy Decl. ¶ 73; Meek ¶¶ 121-122. However, KPMG had previously found significant problems with Bell Atlantic's ability to follow its hot-cut procedures and issued an "exception." See NYPSC Eval. at 89; Meek Aff. ¶¶ 121-122; KPMG, Exception ID 54 . KPMG closed the exception following a limited two-week "retest" in June 1999 during which KPMG observed the technicians performing their work on the due date. Bell Atlantic Brief at 19; Lacouture/Troy Decl. ¶ 73. KPMG did not check whether Bell Atlantic performed any of the required steps prior to the due date, such as the dialtone check on due-date minus-two, and KPMG did not test whether the hot cut was successful (i.e., working post-cutover). 7/29/99 Technical Conference Transcript at 3861-3866, 3889.

⁴¹ See Meek Aff. ¶¶ 124-125; see generally NYPSC Eval. at 85-87. The NYPSC did not conduct a review of non-AT&T UNE-L orders during this time period. AT&T Comments at 39; Meek Aff. ¶¶ 132-35. CLECs contended that Bell Atlantic's on-time hot-cut performance was significantly worse than reported. Choice One Comments at 4; AT&T Comments at 38. If Bell Atlantic reported all non-AT&T orders (accounting for slightly less than half of the total hot-cut orders) correctly, then Bell Atlantic provided on-time provisioning in this period for approximately 91% of hot cuts. See NYPSC Eval. at 85-87; AT&T Comments at 38-39; Meek Aff. ¶¶ 132-135.

⁴² See Meek Aff. ¶¶ 127-130. In this regard, the NYPSC's July 1999 data reconciliation used on-time "miss/make" definitions that were more favorable to Bell Atlantic than it will use in the future under the Amended Performance Assurance Plan. See NYPSC Eval. at 88-89; Meek Aff. ¶¶ 127-130. Even Bell Atlantic admits that it is at fault on 11% of hot cut delays, a factor not considered in current on-time performance metrics. See Bell Atlantic Brief at 19; Lacouture/Troy Decl. ¶ 73. CLECs allege that the percentage of hot-cut delays that are Bell Atlantic's fault is much higher. See, e.g., Allegiance Comments at 11 (20% caused by Bell Atlantic); AT&T Comments at 38; Choice One Comments at 5. It is noteworthy, also, that it appears to take Bell Atlantic significantly longer -- as many as two to three days longer -- to provision service for CLEC UNE-loop orders involving a dispatch than for its own retail service. Dowell/Canny Decl., Tab 3D at 80, 92, 104 (PR-2-03, PR-2-04, PR-2-05).

⁴³ See, e.g., Choice One Comments at 7-8; AT&T Comments at 42-44; NYPSC Eval. at 119-120; Mulligan Aff. ¶ 33.

⁴⁴ AT&T Comments at 43; Callahan/Connolly Aff. ¶¶ 9-10.

⁴⁵ NYPSC Eval. at 120-21.

⁴⁶ AT&T Comments at 42-44 & n.13; see also Callahan/Connolly Aff. ¶¶ 22-28 & Attach. 1.

⁴⁷ Indeed, Bell Atlantic's Performance Assurance Plan Reports state that Bell Atlantic would have paid the maximum penalty of $787,037 in June, 55% of the maximum penalty ($432,870) in July, and 65% of the maximum penalty ($511,574) in August for poor hot-cut performance had the plan been in effect. Bell Atlantic Ex Parte Filing on PAP at Sheets I (June, July, August data).

⁴⁸ See, e.g., Bell Atlantic Brief at 18 n.20.

⁴⁹ Mulligan Aff. ¶¶ 6-7.

⁵⁰ See NYAG Comments at 14-15; AT&T Comments at 30; Mulligan Aff. ¶¶ 11, 13-16. Commenters estimate that the small and medium-sized business market alone may account for 3.2 million lines in New York. AT&T Comments at 30; Mulligan Aff. ¶ 7.

⁵¹ The Commission's recent decision regarding the Supreme Court's remand of Rule 319 will significantly limit the availability of unbundled switching and the platform for business customers. Federal Communications Commission, FCC Promotes Local Telecommunications Competition: Adopts Rules on Unbundling of Network Elements, Press Release (Sept. 15, 1999) http://www.fcc.gov/Bureaus/Common_Carrier/News_Releases/1999/nrcc9066.html;. See also Mulligan Aff. ¶ 13. Bell Atlantic's platform offer has substantial limitations on its availability to serve business lines. Tariff P.S.C. No. 916 (issued Jan. 26, 1999) (revised Sept. 8, 1999 and Sept. 10, 1999) at 80, attached to Bell Atlantic Brief as App. H, Vol. 2, Tab 3.

⁵² NYPSC Eval. at 99.

⁵³ We believe that demonstrated, rather than promised, improvement is particularly important in this context. As noted above, most of the order processing problems appear to arise from the manual processing of orders; and although Bell Atlantic has recently submitted a detailed plan to improve flow-through processing of UNE-platform orders, Bell Atlantic has no near-term plans to increase flow-through for UNE-L orders.

⁵³ See Federal Communications Commission, Broadband Today, Cable Bureau Staff Report, at 32 ("Broadband Today") (Oct. 1999) http://www.fcc.gov/Bureaus/Cable/News_Releases/1999/nrcb9017.html. "Bell Atlantic plans to double the availability of its DSL products to 17 million telephone lines by year-end 1999." Id. at 28.

⁵⁴ Bell Atlantic has informed the DOJ that it currently has about 195,000 ISDN lines in New York.

⁵⁵ Broadband Today at 28; NorthPoint Comments, Attach. B at 2. Bell Atlantic is able to offer its customers ADSL service without installing another line because the data service uses only the high frequency portion of the loop's bandwidth and, thus, is compatible with analog phone service. Broadband Today at 20.

⁵⁶. See Aquilina Aff. at 6 n.1.

⁵⁷ First Report and Order, In re: Implementation of the Local Competition Provisions in the Telecommunications Act of 1996, CC Docket Nos. 96- 98 and 95-185, 11 FCC Rcd 15499 ¶ 380 (1996) ("FCC Local Competition Order"), aff'd in part, rev'd in part andremanded sub nom. AT&T v. Iowa Utilities Board, 525 U.S. 366, 119 S.Ct. 729 (1999); NYPSC Eval. at 76.

⁵⁸ Memorandum Opinion and Order, and Notice of Proposed Rulemaking, In Re:Deployment of Wireline Services Offering Advanced Telecommunications Capability, 13 FCC Rcd. 24012, ¶ 53 (1998).

⁵⁹ Bell Atlantic Pre-Filing Statement at 25-26.

⁶⁰ NorthPoint Comments at Attach. B.

⁶¹ NorthPoint Comments at 18.

⁶² Seegenerally NorthPoint Comments at 6, 10, 18; Rhythms Comments at 21-22; Covad Comments at 15-16; Cutcher/McChesney/Clancy Aff. ¶¶ 34, 61-66; NAS Comments at 7-8; Prism Comments at 8-10.

⁶³ Geis/Williams Aff. ¶¶ 38-39.

⁶⁴ Bell Atlantic Brief at 21; DOJ Ex. 7: Bell Atlantic DSL Panel Testimony at 21-22.

⁶⁵ NorthPoint has found that its queries are rejected even when it uses Bell Atlantic's address validation system. NorthPoint Comments at 10.

⁶⁶ NYPSC Eval. at 93.

⁶⁷ See, e.g., NorthPoint Comments at 13, 15.

⁶⁸ DOJ Ex. 8: ISDN/ADSL Performance Data at 1-2 (OR-1-04 & OR-1-06, Percent Orders Confirmed On Time).

⁶⁹ Bell Atlantic Brief at 20; DOJ Ex. 8: ISDN/ADSL Performance Data at 2 (PR-4-04 & PR-4-05, Percent Missed Appointments - BA).

⁷⁰ NorthPoint Comments at 18.

⁷¹ Covad Comments at 15.

⁷² NYPSC Eval. at 94-95 ("Recommendations to the NYPSC are expected in December for the adoption of DSL-specific metrics to ensure that these services can be separately monitored to ensure provisioning at a commercially reasonable level of quality and timeliness.").

⁷³ CLEC dependence on the UNE-platform to provide local service to residential customers derives in part from the fact that other service options have not proven competitively viable to serve large numbers of residential customers. The investment needed to provide these services on CLEC networks is too high, as is the investment needed to provide service leasing UNE-loops. The resale discount has been insufficient to keep major carriers such as AT&T and MCI WorldCom from abandoning their statewide resale residential service offerings in New York. Also, the UNE-platform permits CLECs to offer service options unavailable through resale, such as advanced intelligent network features. Z-Tel Comments at 5-6, 8-9.

⁷⁴ See Dowell/Canny Decl., Tab 3D at 102 (OR-3-01: Percentage of Rejected Orders; this percentage is calculated based on the total number of CLEC orders submitted to Bell Atlantic); see Bell Atlantic Performance Measures Compliance Filing at 25.

⁷⁵ See generally KPMG Final Report.

⁷⁶ Dowell/Canny Decl., Tab 3D at 79, 91, 102 (OR-3-01, Percent Rejected Orders: June (28.69%), July (34.01%) and August (33.65%)).

⁷⁷ Bell Atlantic Brief at 43.

⁷⁸ Crafton/Connolly Aff. ¶ 227; see also KPMG Final Report, POP5, IV-114, Test Cross Reference P5-13 (standard error messages on rejected orders not consistently clear and accurate).

⁷⁹ Crafton/Connolly Aff. at Attach. 18; Z-Tel Comments at 19.

⁸⁰ The majority of current UNE-platform orders may be for service migrations where a rescheduling might not be required, but UNE-platform orders that involve new lines for which customers must be home at installation are expected to increase as the market matures.

⁸¹ Dowell/Canny Decl., Tab 3D at 79, 91, 102 (OR-5-01, Percent Flow-Through Total: June (54.48%), July (54.36%), August (59.28%)). We note that this performance measure understates the amount of manual processing that actually takes place in Bell Atlantic's ordering centers because it reports the flow-through rate for orders that are provisioned but does not include rejected orders that are not provisioned or orders that are canceled before being provisioned. See Bell Atlantic Performance Measures Compliance Filing at 28. When all orders submitted by CLECs are taken into account, Bell Atlantic reports that 52% of UNE-platform orders flow through electronically. DOJ Ex. 9: Excerpt from Bell Atlantic Presentation to Assistant Attorney General Joel I. Klein at 8.

⁸² Dowell/Canny Decl. ¶ 53 & Tab 3D at 102 (OR-6-01, Order Accuracy: August (only 63.59% of electronically submitted orders correctly input by Bell Atlantic service representatives in the ordering center)); DOJ Ex. 6: Aggregate September Performance Data at 7 (OR-6-01: 42%).

⁸³ Compare Dowell/Canny Decl., Tab 3D at 78 (June), 92 (July), 102 (August) (OR-1-03, OR-1-04, OR-2-03, OR-2-04); DOJ Table of Processing Times at 1-2 with Dowell/Canny Decl., Tab 3D at 78 (June), 92 (July), 102 (August) (OR-1-01, OR-1-02, OR-2-01, OR-2-02); DOJ Table of Processing Times at 1-2.

⁸⁴ Dowell/Canny Decl., Tab 3D at 102 (OR-1-01 and OR-1-03).

⁸⁵ See supra note 82.

⁸⁶ See DOJ Ex. 5: DOJ Table of Processing Times (improvement from July to August on all disaggregated UNE-P metrics listed).

⁸⁷ See, e.g., Crafton/Connolly Aff. ¶¶ 24-29 & Confidential Attach. 2.

⁸⁸ Aquilina Aff. ¶ 35; Minutes of an Oral Argument, In re: Petition of New York Telephone Company for Approval of Its Statement of Generally Available Terms and Conditions Pursuant to Section 252 of the Telecommunications Act of 1996 and Draft Filing of Petition for InterLATA Entry Pursuant to Section 271 of the Telecommunications Act of 1996, NYPSC, Case 97-C-0271, at 4260 (Aug. 31, 1999), attached to Bell Atlantic Brief as App. C, Vol. 63, Tab 989.

⁸⁹ The data comparing the time it takes to provision wholesale UNE-platform orders with comparable retail orders are murky, but even Bell Atlantic's substitute analysis causes us some concern. In that analysis, Bell Atlantic appears unable to provision UNE-platform orders within the standard interval when CLECs request the standard interval. As calculated by Bell Atlantic's experts, it took Bell Atlantic on average half a day longer than the standard interval to provision UNE-platform orders in August. Gertner/Bamberger Decl. at 10, Table 4.

⁹⁰ See DOJ Oklahoma Evaluation, App. A at 71-76; DOJ South Carolina Evaluation, App. A at 10-14; see alsoFCC South Carolina Order ¶¶ 156-159, 166.

⁹¹ One hundred CLECs use the GUI for pre-ordering; only three CLECs use EDI. Bell Atlantic Brief at 37; Miller/Jordan Decl. ¶¶ 22-23. So far these CLECs are able only to retrieve customer service records, which is just one of several pre-order functions. See, e.g., Lichtenberg/Sivori Aff. ¶ 56. More than 100 CLECs use the GUI for submitting orders; only six CLECs use EDI. Bell Atlantic Brief at 39-40; Miller/Jordan Decl. ¶ 35.

⁹² For example, the cost of purchasing or creating the software necessary to build an application-to-application interface is high and may be out of reach for smaller CLECs. See Z-Tel Comments at 16.

⁹³ KPMG Final Report, RMI1, VII-8, Table VII-1.8: RMI1 Evaluation Criteria and Results, Test Cross Reference R1-6 (documentation of proposed changes untimely; finality of documentation uncertain).

⁹⁴ Dowell/Canny Decl., Tab 3D at 97-98 (PO-4-01, Percent Notices Sent On Time-Bell Atlantic Originated: August(only 75% of change notifications with 45-day intervals and 88% of change confirmations with 66-day intervals provided on time, during the period Bell Atlantic characterized to the Department as a "major" software change)); see also Bell Atlantic Performance Measures Compliance Filing at 12 (specifying that notifications have 45-day intervals and confirmations have 66-day intervals).

⁹⁵ KPMG Final Report, POP1, IV-18 to IV-19, Table IV-1.9: POP1 Evaluation Criteria and Results-EDI Certification Test, Test Cross Reference P1-2.

⁹⁶ Bell Atlantic plans to try to increase the percentage of flow-through orders from 52% to 62-67% by October 30, 1999, to 67-72% by December 18, 1999, and to 72-77% by June 2000. See Joint October Reply Affidavit of Stuart Miller, Sean J. Sullivan and Arthur Zanfini on Behalf of Bell Atlantic-New York, NYPSC, Case No. 97-C-0271, ¶¶ 11-15, attached to Crafton/Connolly Aff. as Attach. 3. Bell Atlantic intends to increase flow-through in phase 1 primarily by rejecting more CLEC orders. The next two phases of flow-through improvement will focus on systems enhancements: software changes that permit additional order types, accounts with contracts, and order cancellations to be electronically processed.

⁹⁷ These are metrics PO-4-01, PO-4-02, and PO-4-03 (Timeliness of Change Management Notice); PO-6-01 (Software Validation); PO-7-01, PO-7-02, PO-7-04 (Software Resolution Timeliness). Bell Atlantic Performance Measures Compliance Filing at 12 (PO-4 category), 14 (PO-6-01), 15 (PO-7 category).

⁹⁸ Bell Atlantic Brief at 67-71.

⁹⁹ Bell Atlantic filed two amended performance plans, the APAP and ACCAP, for approval by the NYPSC on September 24, 1999, less than one week before filing this application. The NYPSC orally adopted the APAP and the ACCAP at its October 27, 1999, session. A written order is expected on November 1, 1999. At this time, we do not know whether the NYPSC will order any modifications to the plans proposed by Bell Atlantic. A full analysis of the APAP must wait until the NYPSC redefines some of the performance measures on which the APAP is based. In particular, how the "Achieved Flow-Through" metric is defined will affect the efficacy of the special flow- through measure contained in section E.1 of the APAP. APAP at 11. The NYPSC expects to issue an order addressing these performance measures issues during the week of November 1, 1999. Both of these orders will be issued after this Evaluation is filed with the Commission.

¹⁰⁰ In contrast, in advising the Commission to approve Bell Atlantic's New York 271 application, the NYPSC assumes that Bell Atlantic's level of wholesale performance on a number of items will improve after Bell Atlantic has received authority to offer long distance service. As part of these promised improvements, Bell Atlantic will:

1. take steps to ensure that preorder response times remain adequate as order volumes increase, NYPSC Eval. at 40;

2. improve Lsrc and reject response times pursuant to additional monetary incentives in the APAP, id. at 43-44;

3. increase flow-through in a three-stage plan over the next several months, id. at 47;

4. improve "change control" compliance after long distance entry based on financial incentives in the ACCAP, id. at 57;

5. improve compliance with hot cut procedures after long distance entry by instituting a new measuring and reporting process, id. at 88-89;

6. disaggregate data relating to reported installation problems after long distance entry, id. at 90-91;

7. institute many process improvements for ordering and provisioning DSL loops in the ongoing collaborative process, id. at 92-94;

8. implement process improvements for repair of complex loops, id. at 99; and

9. provide unbundled "dark fiber" transport to CLECs, id. at 104.

¹⁰¹ DOJ Ex. 1: Schwartz Aff. ¶¶ 154-57.

¹⁰² Regulation has proved to be more effective at maintaining adequate wholesale performance once the necessary new access arrangements have been put in place and a benchmark of acceptable wholesale performance has been established. Id. ¶¶ 137-140.

¹⁰³ APAP at 11-13 & n.13.

¹⁰⁴ APAP at 15-17.

¹⁰⁵ This concern is not merely theoretical. The Attorney General of the State of New York states that Bell Atlantic has sought waivers for at least 17 months of data under its retail performance regulatory plan since it was instituted in September 1995. NYAG Comments at 34.

¹⁰⁶ According to the NYPSC, the APAP would have required Bell Atlantic to post about $5 million in bill credits out of $17.3 million in bill credits at stake during August had it been in place at that time. NYPSC Eval. at 7. See also Bell Atlantic Ex Parte Filing on PAP.

¹⁰⁷ We are concerned also about the precedential implications of relying on promises of future improvement as a basis for approving applications under Section 271. It would be unfortunate if future applicants were less committed to actually opening their markets because of the expectation that it would be sufficient for them to make such promises.