Statement of Harold Feld, Legal Dir., Public Knowledge; Before the FCC; National Broadband Plan Workshop: Benchmarking

Most recent activity:

This statement is also available in
PDF Format
.

September 2, 2009

Thank you for inviting me to speak today on the subject of necessary
benchmarks for the National Broadband Plan. Rather than focus
on specific benchmarks (although I intend to address some of these as
well), I would like to focus primarily on the nature of the benchmarks
required by ARRA[1]
for the National Broadband Plan and how the FCC can successfully monitor
achievement against the benchmarks. Briefly:

  1. ARRA requires an incredibly rich and complex set of benchmarks that
    include not merely traditional quantitative metrics such as deployment,
    but also qualitative metrics that ensure that all Americans receive the
    projected benefits from ubiquitous, affordable broadband access. In
    other words, the FCC should envision this not as a limited exercise in
    prodding the broadband marketplace, but an exercise in understanding
    and transforming the “broadband ecology” — by which I
    mean a recognition that broadband exists in a complex, interrelated
    system going well beyond producers and consumers, where these separate
    elements of the system all have different needs and responses, and
    where benchmarks must reflect this interrelation and complexity.

  2. The FCC must examine whether to set benchmarks dynamically and with
    reference to each other, as well as absolutely. For example, the FCC
    should consider whether to set both an absolute benchmark for last mile
    speed (e.g., “everyone to have access to 5 mbps down by
    2011”) or interim benchmarks for last mile speed measured as
    improvements over existing speeds regionally (e.g., “improvements
    of 20% over previously measured speed per year until ultimate goal is
    reached”) or both (e.g., a “floor” and benchmarks for
    continued improvement). This, in turn, would inform appropriate
    benchmarks for middle mile infrastructure to support this last mile
    goal. Similarly, if the FCC determines that Americans are not receiving
    the projected benefits of broadband, it must reexamine whether it has
    set appropriate goals and benchmarks for affordability, capacity and
    speed.

  3. If the FCC relies exclusively on traditional means to track these
    benchmarks, it will fail miserably. The burden of collecting the
    information alone would constitute a significant drain on existing FCC
    resources. On the other hand, attempting to limit data collected to
    something manageable ensures that the FCC will develop a picture of
    broadband deployment and the benefits of broadband so incomplete as to
    be positively misleading — especially as significant data on the
    impact of broadband access lies outside the FCC’s jurisdiction
    (e.g., impact on education, economic development).

  4. The FCC must therefore employ techniques for information collection
    that adequately distribute the burden of information collection and
    processing while remaining reliable. The FCC can achieve this goal by:

    • Distributing information collection to the general public. This
      includes not merely existing tools of “crowd sourcing,”
      but automated systems of tracking and reporting that volunteers could
      install on their own devices.

    • Automating reporting processes for mandatory real-time reports by
      providers and other stakeholders.

    • Distributing information collection among other federal agencies.

    • Close coordination with federal, state and local government to
      leverage existing data collection.

  5. The FCC should anticipate that it will need to revisit benchmarks
    periodically as the national broadband plan evolves, its experience
    with collection and analysis of data grows, and its systems for
    collecting data improve. While this should not be done lightly (the
    entire purpose of benchmarks is to provide a stable measure of
    progress), the FCC must also recognize its very limited knowledge at
    the moment — particularly on the question of how broadband will
    improve people’s lives. Specific benchmarks, especially those
    used as proxies for complicated effects, will need reexamination over
    time.

While this appears daunting, I observe that both the private sector and
other federal agencies engage in such activities every day. The Energy
Information Administration of the Department of Energy, for example,
compiles weekly and monthly reports and analysis tracking the complex
energy needs of the country. The Commerce Department requires
coordination among multiple agencies within it department to produce
regular reports on the state of the economy — demonstrating that
multiagency cooperation is both possible and beneficial. Indeed, it would
be a useful first step for the Commission to examine a wide variety of
agency data collection and reporting practices both to inform its own
efforts and to determine where questions relevant to the National
Broadband Plan could be inserted in existing reports and surveys.

Finally, while “crowd sourcing” and other methods of directly
involving the public in a systemic way — rather than simply
soliciting comments or complaints — would be new to the FCC, the
private sector has employed this practice for many years. To take one
example familiar to the Commission, such as Nielsen discovered more than
half a century ago that members of the public are not only able and
willing, but can be downright eager, to provide real time data on their
behavior under the right conditions. With modern tools and applications,
the FCC can tap into a wealth of volunteers to crowd source reliable
information collection — including the suggestion of relevant new
fields to measure. These tools include not only familiar tools for
interested individuals to actively self-report, such as wikis, text
messages, Twitter and so forth. They also include monitoring and testing
applications — voluntarily installed and expiring after a set time
— that would constantly test factors such as actual delivered
speeds or network congestion in real time and report back information
directly to the FCC.

Such suggestions naturally raise privacy concerns. I do not suggest the
FCC should download spyware into everyone’s laptop and iPhone.
Rather, working in consultation with privacy advocates and others, the
FCC can develop applications for volunteers that would provide an
extremely rich and useful set of reliable data that minimizes traditional
concerns about the accuracy and reliability of self-reporting. Similarly,
working with broadband access providers, the FCC can collect data without
creating burdensome reporting requirements while protecting proprietary
information.[2]

Finally, we all recognize that the FCC ultimately has limits on the data
it can collect and analyze no matter how efficiently it distributes the
burden, and that these efforts will inevitably fall short of the ideal
world. By adopting the methods proposed here, the FCC can maximize the
value of the benchmarks used for the National Broadband Plan, and thus
maximize its chance for success.

I.WHAT BENCHMARKS DOES ARRA REQUIRE?

A. Effective Benchmarks Include More Than Last Mile Measures.

The National Broadband Plan, like the broadband stimulus program
generally, marks a significant shift in the focus of federal policy from
merely encouraging deployment to ensuring that all Americans enjoy the
economic, educational, and civic benefits of broadband. Whereas Section
706 of the Telecommunications Act of 1996 charges the FCC to
“encourage the deployment on a reasonable and timely basis of
advanced telecommunications capability to all
Americans,”[3]
the ARRA requires much more than mere deployment. Under ARRA, the
National Broadband Plan:

shall seek to ensure that all people of the United States have access to
broadband capability and shall establish benchmarks for meeting that
goal. The plan shall also include—

(A) an analysis of the most effective and efficient mechanisms for
ensuring broadband access by all people of the United States;

(B) a detailed strategy for achieving affordability of such service and
maximum utilization of broadband infra-structure and service by the
public;

(C) an evaluation of the status of deployment of broadband service,
including progress of projects supported by the grants made pursuant to
this section; and

(D) a plan for use of broadband infrastructure and services in advancing
consumer welfare, civic participation, public safety and homeland
security, community development, health care delivery, energy
independence and efficiency, education, worker training, private sector
investment, entrepreneurial activity, job creation and economic growth,
and other national purposes.

This encompasses a scope well beyond mere deployment. Critically, it
requires the FCC to look beyond simply the number of connections, speed
and capacity. It requires the FCC to address and consider how the
universal availability of affordable broadband will genuinely improve our
lives with specificity, and with an ability to record these measurable
results. In other words, it moves us from a simple “broadband
marketplace,” focusing exclusively on producers and consumers, to a
“broadband ecology,” in which broadband deployment and
availability have wide reaching impacts on diverse sets of stakeholders
and are in turn impacted by developments in these different stakeholder
communities.

Such a monumental task, extending well beyond the FCC’s
jurisdiction, creates an enormous temptation to read the statute narrowly
and create benchmarks solely for the question of deployment. Even under
such a narrow reading, the task of creating benchmarks seems daunting.
But such a narrow reading would defeat the entire purpose of the statute.
To give meaning to questions such as capacity and speed, these benchmarks
must be made in reference to something. What
does it mean to be fast enough to facilitate “consumer welfare,
civic participation, public safety, etc.”? It is impossible to
answer these questions without also gathering information on whether the
broadband deployed is achieving these goals.

Similarly, an effective plan requires more than the measurement of last
mile connectivity. Does every American really have “access to
broadband” if a lack of middle mile infrastructure slows
information transmission or makes the cost of transporting information
effectively unaffordable? One could technically comply with the plain
language of the statute, but the result of such a report would be worse
than meaningless.

In other words, trying to limit data collection to the traditional areas
of speed (whether advertised or delivered), homes passed, price, and
other last mile metrics cannot truly satisfy Congress’ intent. The
benchmarks the Commission adopts and the information gathered to
determine whether these benchmarks are being met must include data from
all segments of the network, and must include data that verifies that
Americans receive the benefits of broadband.

B. Finding the Right Details To Measure.

To reduce this to something concrete and manageable, we should begin by
dividing the types of benchmarks into three categories. 1) Traditional
last-mile metrics, 2) other infrastructure metrics, and 3) qualitative
impact metrics.

1. Traditional Last Mile Metrics.

The FCC can and should continue to measure last mile metrics, although we
will want to refine these metrics. These metrics include availability
(both wireline and wireless), price, speed, and capacity. Ideally, the
FCC would collect information on a per household basis. In reality, of
course this level of granularity would be quite burdensome to achieve. As
described in more detail in Part II, the Commission should therefore
employ a mix of methods to collect the information.

It is important to recognize that the FCC should collect last mile
information from last mile consumers as well as
from providers. In addition, information should be collected at regular
intervals. Through automated reporting systems, collection of information
could occur in real time. Alternatively, the FCC could collect
information on a weekly or even a monthly basis. This would not only
improve the accuracy of the information, it would improve the relevance
of the benchmarks. Regular feedback allows for regular adjustments in
policy and provides a much clearer sense of the real world picture,
whereas an annual or even semi-annual “report card” has much
less utility for making adjustments to meet the National Broadband Plan
goals.

In addition, the Commission can and should examine last-mile spectrum
availability and spectrum use. Whether or not wireless broadband access
is a substitute for wireline, a compliment, or a replacement for wireline
where wireline is unavailable, it is clearly part of the National
Broadband Plan. But the FCC collects virtually no data on last mile use
of spectrum, and has no metrics to determine the efficiency of spectrum
use — despite consistent commitment to maximizing spectrum
efficiency as a means of enhancing the delivery of mobile services.

2. Middle Mile Metrics

Until now, the Commission has simply not examined the question of middle
mile transport. Any plan that seeks to ensure affordable universal last
mile broadband access must have benchmarks for necessary middle mile
infrastructure. Failure to include these metrics translates into an
inability to develop rational policy. It is the diagnostic equivalent to
treating circulation disorders without any knowledge of the body’s
circulatory system. Without a set of benchmarks to ensure steady growth
of a middle mile infrastructure capable of supporting ubiquitous,
affordable broadband access at the last mile, the National Broadband Plan
cannot realistically expect to succeed.

Unsurprisingly, many of the benchmarks for middle mile parallel the
benchmarks for last mile. Are middle mile services readily available? At
what capacity, what speed, and what price? Although the Commission has
sought information on these questions in to some degree in its special
access docket, it must expand and regularize the collection of this data
for middle mile to ensure that the middle mile infrastructure can support
the target level of last mile access.

Because middle mile impacts the ability of last mile providers to deploy
and provide new levels of service, the Commission must establish
benchmarks for build out of middle mile capacity with reference to
desired last mile capacity. The need to determine not merely existing
middle mile infrastructure, but capacity to support future last mile
deployment, requires that the FCC gather information and set benchmarks
with regard to two critical inputs: fiber and spectrum.

The FCC’s middle mile benchmarks should include not merely a
measurement of “lit” fiber, but also measurements of dark
fiber, plans for construction of fiber. The Commission will need regular
information on plans by middle mile providers to increase available
capacity to inform federal policy. In addition to measurements (and
benchmarks) for fiber in the ground, the Commission must also collect
information (and set benchmarks) for affordability and availability to
last mile providers. The Commission cannot develop a successful market
structure and plan for last mile deployment without an assessment of the
ability of last-mile competitors to secure affordable backhaul.

Similarly, the Commission must have an assessment of the availability of
spectrum for middle mile uses. Such an assessment would include access to
federal spectrum, as well as efficient use of commercially available
spectrum. As noted above, however, not much research supports how to
construct a useful spectrum access or spectrum efficiency metric.

3. Qualitative Impact Metrics.

I use the term “qualitative impact metrics” to mean measuring
the impact of broadband to confirm that the American people enjoy the
benefits of broadband envisioned by Section 6002(k)(2). This covers a
vast territory. It runs from the question of what people do with their
broadband access to whether the presence or absence of broadband access
creates jobs.

These metrics go to the heart of why Congress directed the Commission to
develop a national plan, and allocated $7.2 billion as part of the
national stimulus package. But while a number of studies have shown a
connection between broadband adoption and civic engagement, broadband
adoption and increased economic opportunity, and broadband adoption and
the development of social capital, and a host of anecdotal case studies,
there exists no systemic way to measure impact or set reasonable
benchmarks by which to judge the success of the National Broadband Plan.
For example, the E-Rate Program has existed for over 10 years, and nearly
every study of it confirms that it dramatically enhanced broadband access
in K-12 schools. But we have no systemic way of measuring the impact of
this on educational outcomes — or even the extent to which K-12
schools incorporate this access into their curricula. From the
perspective of the FCC, and therefore of the National Broadband Plan,
E-Rate is as likely to result in children trained for digital jobs as it
is to result in an empty classroom with no computers and no teachers to
show students how to actually use their broadband connection.

Even studies of user behavior based on surveys tend to yield very gross
data. For example, a recent study by the Pew Project for Internet and the
American Life recently found that 62% of internet users have watched a
video or movie online.[4] While the presumption is that this is for
entertainment purposes, how much video use is for educational purposes?
For business purposes (webinars and other training videos)? Social
networking (videos of family or friends)? Similarly, studies of social
networking sites tend to focus on gross metrics such as the number of
fans or followers rather than qualitative metrics on use as valued by the
user. To take another example, a recent study found that nearly 40% of
Twitter use was “pointless babble” while another 37% was
“conversational.” Viewed one way, Twitter seems a waste. But
its popularity demonstrates that a large number of users value the ease
with which it facilitates conversation — whether it is a
conversation about healthcare reform or a conversation about where to
meet in a food court.

As these two examples suggest, gathering meaningful metrics on the impact
of broadband is exceedingly difficult, and assigning them meaning within
the context of the goals of the ARRA often presents difficult value
judgments. Nevertheless, this information is arguably the most critical
for setting benchmarks for the national broadband plan. As noted, we care
about broadband access as a national priority because of the activity it
enables — from the creation of jobs to the simple and critical
function of allowing people to talk to each other more effectively. We
cannot tell if we have set appropriate goals for speed, capacity,
deployment or affordability unless we can tell how these things impact
use and facilitate these benefits.

To provide an essential starting point, I recommend that the FCC begin
with two sets of gross measurements to set initial benchmarks and refine
them over time. First, direct measure of the use determined by surveys or
other means of gathering direct use information that go to the categories
listed in Section 6002(k)(2)(D): consumer welfare, civic participation,
public safety and homeland security, community development, health care
delivery, energy independence and efficiency, education, worker training,
private sector investment, entrepreneurial activity, job creation and
economic growth. This would include either surveys or gathering real time
information (such as diaries) that would ask users to make short,
descriptive statements of their actual online activities. Some of this
activity might be conducted by other agencies, or by private sector
companies. This data would include federal data from other agencies
(e.g., HHS data on the use of electronic medical records, DOE data on
smart grid deployment), state and local government data, and other
sources.

At the same time, the FCC — with the cooperation of other agencies
— would also develop a set of proxies using existing data based on
projected benefits. For example, if broadband facilitates distance
learning, do we see a rise over time in the number of GEDs or other forms
of education certification in correlation with broadband penetration? A
recent study by the US Department of Agriculture correlating broadband
penetration with increased household income[5] demonstrates both the difficulties in devising
such studies and the value of conducting such studies in determining
whether the National Broadband Plan is delivering the expected benefits.

II. CHANGES IN INFORMATION GATHERING.

Traditionally, the FCC has gathered information on use in three ways.
First, through notices of inquiry, the FCC has sought voluntary
information from interested parties. Second, the FCC has compelled
reports under various statutory provisions — most notably the Form
477 Report. Finally, the FCC has engaged in some self-initiated research.

A. Why NOIs And Existing Form 477 Provide Little Useful Information For
Benchmarking Purposes.

The first two methods most relied upon by the Commission, NOIs and the
Form 477 Report, appear least likely to provide meaning full benchmark
information. To maximize effectiveness, the data collected to inform
benchmarks should be consistent in format, straightforward to process,
and contain indicia of reliability. Information gathered in response to
NOIs contains none of these. All information
obtained in NOIs (and I speak as someone who participates in many of
them) comes from self-selected, self-interested parties submitting
whatever information exists that furthers their policy objectives. It is,
in fact, designed as an adversarial process
with the FCC staff acting as the ultimate evaluator of the information
submitted.

Whatever the virtues of this approach for formulation of rules and
policies, it can serve almost no useful role in tracking the development
of the broadband plan over time. The inability to ensure regular
participation and standard inputs make it virtually impossible to do
meaningful comparisons. The need for FCC staff to evaluate all
information as a unique input and — ideally — examine its
veracity and usefulness represents both an enormous drain of resources
and an opportunity for political manipulation.

The Form 477 as currently constituted simply does not address relevant
areas in a timely fashion. The fact that broadband providers give
aggregate data covering a six month period — in contrast, for
example, to the EIA’s weekly and monthly reports on various energy
sectors — compromises the usefulness of the data as a matter of
timeliness and accuracy. From the perspective of the FCC, the Form 477
provides a sudden glut of very basic data — unverified by the FCC
for reliability — every 6 months.

B. Generation of Survey Data.

The next and most logical step for the FCC is the generation of survey
data. Numerous federal agencies use survey data to assess the current
state of various sectors of the economy, and even such intangibles as
consumer confidence. Surveys conducted by phone of end users will also
provide information from those not yet online. Given the importance of
facilitating adoption, such survey data promises to be extremely useful.

Perhaps more importantly, the FCC should work in coordination with other
agencies to include questions pertaining to the deployment and use of
broadband in surveys and reports conducted by other federal agencies. For
example, the Energy Information Administration conducts regular surveys
of the energy sector. The FCC could request that EIA include in its
surveys questions on the deployment and use of smart grid technology.
Similarly, the FCC could partner with the Department of Education and
other agencies to develop survey data on the use of broadband in the
classroom.

Participation in the survey pool could be tied to the receipt of federal
funds. For example, all recipients of E-RATE could be required to
participate in surveys on the actual use of broadband in the classroom
and the educational outcomes as a result. Because the pool of E-Rate
recipients is large, the actual burden on any single survey participant
can be minimized.

C. User Generated Data, Autoreporting, And Other New Tools.

By far, the most valuable data comes from users and network providers
themselves. By using existing tools and applications, the FCC can access
this data in a useful and organized manner that respects user privacy and
proprietary information.

1. User Diaries, Wikis and Other Manual Self-Reports

Users can potentially provide an enormous amount of information.
Traditional problems with user generated information, such as the problem
of making value judgment as to data (e.g., is something a “civic
use”), the difficulty of ensuring compliance with a reporting
protocol to ensure standard and reliable results, and the problem of
encouraging user participation, can now be overcome by providing tools
that make it simple and easy for volunteers to provide information and by
providing very basic guidance to users to simply describe activities
rather than try to categorize them.

Although it is possible to automate this, concerns for user privacy
— even among volunteers — should dictate that certain types
of information always be manually recorded by
users (if collected at all) rather than automatically. For example, the
FCC most emphatically should not try to track
user content through applications commonly referred to as
“spyware,” even if users consent. The possibility for abuse
is simply too great. But nothing would prevent the FCC from providing a
“diary app” which would allow users to essentially
“Tweet” their activities to the FCC in real time.

2. Autoreporting

On the other hand, certain sorts of tests and data collection lend
themselves very well to automated collection without raising privacy
concerns. Consider the simple question of delivered speed as opposed to
advertised speed. Imagine a distributed group of volunteers downloading
an FCC application that ran a standardized speed test at regular
intervals and reported the results back to the FCC (and the user, if the
user so desired). This would provide a constant flow of information in
real time which would allow the FCC to collect necessary information to
set realistic speed benchmarks and determine whether these benchmarks are
being met.

Similarly, the FCC can automate the process of gathering information from
providers and others by developing and providing tools that reside within
the network and report information back to the FCC. Again, protection of
user privacy and proprietary information would mandate against even
attempting to collect certain sorts of information voluntarily. But this
would not prevent, for example, immediate reporting back to the FCC when
a connection goes live or goes down, providing basic information on the
number of connections and general traffic data.

Other tools exist or can be developed with the same aim in mind, to
automate the process of information collection. The private sector
already uses these “information mining” technologies. Again,
while the FCC must take care to respect user privacy, the experience of
the private sector indicates that many users are willing to participate
in passive information gathering that does not impose on them additional
costs.

III. SETING USEFUL BENCHMARKS.

A critical problem in the development of the National Broadband Plan has
been the rush to provide definitive answers to complex interrelated
questions. “How fast is broadband” has the same semantic
value as “how high is up.” It only becomes meaningful when we
ask the question “how fast in order to do what?”

A. The Complex Broadband Ecology.

As I have suggested above, while most approaches visualize the problem at
hand as a “market failure” of the “broadband
market,” it will prove more helpful to conceptualize this as an
effort to develop a “broadband ecology.” This
conceptualization moves beyond the relatively simple visualization of
buyer, seller, producer and recognizes that the ARRA requires the
National Broadband Plan to consider the entire universe of stakeholders
for the purpose of promoting beneficial impact. As in a natural ecology,
impacts in one part of the system influence other parts of the system
both directly and indirectly. And, as in an ecology, if we allow a single
species to dominate it will naturally try to reshape the ecology to its
own advantage.

With the ecology framework in mind, the FCC can consider benchmarks for
their direct impact and for the system as a whole. Just as
environmentalist seeking to reclaim a polluted area consider a multitude
of factors and how they relate, so must the FCC in setting its
benchmarks.

As discussed above, the National Broadband Plan should include a middle
mile component capable of supporting he last mile component, which we
value to achieve the goals set forth in Section 6002(k)(2)(D). It is
clear then that the appropriate benchmarks for the middle mile component
depend on the benchmarks for the last mile. These goals, in turn, depend
on what the evidence demonstrates we need to achieve the desired outcomes
in Section 6002(k)(2)(D). We break out of this cycle of self-reference by
working backward from the desired outcomes, not starting in the
traditional place with last mile access on which we have focused for so
long.

For example, rather than pick a number for last mile speed based on
existing technology — the method used in the Section 706 Report
— the Commission would set a goal for speed based on the minimum
speed necessary to achieve significant measurable improvement in the
Section 6002(k)(2)(D) goals. This informs the goal for middle mile. From
these goals, the Commission establishes benchmarks that represent
realistic progress for last-mile and middle mile deployment, measured
against the benchmarks established for the Section 6002(k)(2)(D).

B. National Benchmarks, Regional Benchmarks, and Dynamic Benchmarks.

Certain goals are also relative to external factors and exhibit sharp
regional — or even local — variance. For example, in defining
“affordable,” federal agencies have often looked to such
factors as personal income. The Department of Housing and Urban
Development (HUD), for instance, defines “affordable housing”
as housing cost of 30% of income.[6] Trying to adopt such personal metrics for broadband
service — which is not priced on an individual basis — raises
difficulties. But this does suggest that the FCC should consider local
factors and consider local information — such as average income
within a census block — and not merely establish national
benchmarks.

In making this suggestion, I recognize that we have as strong national
commitment — for sound economic, historic, political and moral
reasons — to ensure that we do not have a double standard for
broadband in which one segment of society gets much better (however
defined) broadband than another. This should certainly be the case for
our ultimate goal of broadband accessibility. But the question of
benchmarks is different. Benchmarks are interim goals to measure progress
and enable changes to the plan as needed. The FCC can, and should,
measure our progress with benchmarks that recognize the tremendous
regional divergences in the existing broadband ecology.

Similarly, the FCC should not only report regularly on the progress
against the benchmarks, but should on occasion revisit them as our
understanding of the complex broadband ecology. Again, I recognize the
potential risk in this suggestion. Inviting reexamination of benchmarks
invites the FCC to redefine “success” and thus avoid taking
difficult measures. At the same time, everyone in this process must
recognize that we stand in the unfortunate position of developing a plan
and benchmarks simultaneously, in a complex and dynamic system, with
little understanding of all the relevant variables — let alone how
these variables relate to one another (a common problem in the case of
complex and dynamic system). But rather than refuse to act because the
Commission lacks perfect data and therefore fears it may do some
unexpected harm, it must recognize that knowledge and judgment will grow
only with experience. As a result, the FCC should set the best benchmarks
possible, recognizing that it will need to periodically revisit them as
our understanding of the broadband ecology grows.

I will conclude by observing that while the benchmarks to track progress
will need reevaluation over time, the ultimate goal should not. This
serves as the most important safeguard against efforts to redefine
failure as success by altering benchmarks. In 2004, President Bush
announced a national goal of affordable access to broadband for all
Americans by 2007. The Commerce Department declared this goal achieved by
the simple expedient of defining access to broadband to meet the existing
standards. That we sit here today, engaged in a far more extensive effort
to make this goal a meaningful reality, demonstrates the folly of trying
to define success after the fact by whatever we have achieved.

CONCLUSION

No one likes process discussions — especially on such subjects as
how to develop proper measurements for a complex task. It is an enormous
temptation to both limit the consideration of benchmarks to
infrastructure and to create simple benchmarks in the same manner the
Commission has always done. But the old methods of developing
measurements and gathering information will not serve for anything as
potentially transformative of every aspect of our society as the National
Broadband Plan. If the FCC intends to fulfill the goals of ARRA, it must
adopt both new measurements and new tools to keep the National Broadband
Plan on target.

[1] American
Recovery and Reinvestment Act, Section 6002(k)(2).

[2] I do not discuss
here details as to what level of disclosure of the data collected from
industry sources the FCC should disclose. There is an obvious tension
between what industry stakeholders would like to disclose and the value
of disclosing to the public generally as much information as possible to
enhance understanding of the market and trust in the reliability of the
FCC’s studies. But the discussion of disclosure goes beyond the
discussion of benchmarks and tracking these benchmarks by the FCC.

[3] 47 U.S.C. 157
nt.

[4] Mary Madden,
“Online Video,” July 25, 2009 available at http://www.pewinternet.org/Reports/2007/Online-Video.aspx

[5] Peter Stenberg,
Mitch Morehart, Stephen Vogel, John Cromartie, Vince Breneman, and Dennis
Brown, “Broadband Internet’s Value For Rural America,”
available at http://www.ers.usda.gov/publications/err78/.

[6] http://www.hud.gov/offices/cpd/affordablehousing/index.cfm