Peter Lyman, University of California at Berkeley
Following is an interview with Peter Lyman, professor in the School of Information Management and Systems, University of California at Berkeley. Lyman is the author of the essay "Archiving the World Wide Web." In August 2001, the Council on Library and Infornmation Resources commissioned six "environmental scans" at the request of the Library of Congress. The essays represented the informed views of the "state of the art" by recognized experts.
Your survey published in 2003 states that the surface Web in 2002 was 167 terabytes. In a March 2003 survey (not yours) it was stated that the "estimated annual production of materials in Web-ready formats by the year 2007 is projected to be too large to estimate by many analysts." Do you believe there is validity in the statement that we are approaching a point where it will not be possible to estimate the amount of information?
Yes, in principle. In practice there are some interesting issues. One is: What do we mean by "information"? A large amount of "information" that is stored is actually data, such as weather and space satellite pictures or video camera content, which cannot actually be used without rendering by software. Two is: We can always estimate how much storage technology is needed to hold it, because we have industry production numbers that are very accurate. This is what we did, but the problem is that it ignores the quality and utility of information, since all of Shakespeare online takes up much less space than a few pictures – but which is more important?
You estimate that the amount of new information stored on paper, film, magnetic and optical media doubled in 1999-2002. Do you see that increase getting even larger after 2002 or will we reach a plateau in the increase of information?
I believe it will continue to expand at this rate at least, doubling every three years. Remember that there was a global recession during that period, for one, and that the Third World is just beginning to produce much more information (look at cinema production for example) and to keep accurate numbers as well. Note also the growth of the Web, which shows that "the means of production" of information is in everyone's hands. The growth of phone calls through mobile technology will also increase the amount of exchange of information – if you count this as information (but it is usually not stored).