This research was sponsored by the MIT Center for Coordination Science, the MIT International Financial Services Research Center, and the Sloan Foundation. Portions of this paper are based on an article titled "The Productivity Paradox of Information Technology," which originally appeared in Communications of the ACM, in December, 1993. Special thanks to Lorin Hitt and Miriam Avins for numerous valuable comments.

In recent years, the relationship between information technology (IT) and productivity has become a source of debate. In the 1980s and early 1990s, empirical research generally did not significant productivity improvements associated with IT investments. More recently, as new data are identified and new methodologies are applied, several researchers have found evidence that IT is associated not only with improvements in productivity, but also in intermediate measures, consumer surplus, and economic growth. Nonetheless, new questions emerge even as old puzzles fade. This survey reviews the literature, identifies remaining questions, and concludes with recommendations for applications of traditional methodologies to new data sources, as well as alternative, broader metrics of welfare to assess and enhance the benefits of IT.

I. The "Productivity Paradox"-A Clash of Expectations and Statistics 1II. Research on Economy-wide Productivity and Information Worker Productivity 8III. Industry-Level Studies of Information Technology Productivity 13IV. Firm-Level Studies of Information Technology Productivity 17A. Service Sector Studies 17B. Studies of Manufacturing Sector and Cross-Sector Studies 21V. Contribution to Consumer Surplus and Economic Growth 25VI. Conclusion: Where Do We Go from Here? 30Figures 37Bibliography 44Index of TablesTable 1: Principal Empirical Studies of IT and Productivity. 7Table 2: Selected Investment Components in 1970 and 1993. 9Table 3: Industry-Level Studies. 16Table 4: Investment in Computers (OCAM) in the US economy. 17Table 5: Studies of Firms in the Service Sector 21Table 6: Studies of Manufacturing Firms and Cross-Sector Firms 24Table 7: Growth Rates of Aggregate Output and Contribution of Factors. 28

I. The "Productivity Paradox" - A Clash of Expectations and Statistics

Over the past decade, both academics and the business press have periodically revisited the so-called "productivity paradox" of computers: while delivered computing-power in the United States has increased by more than two orders of magnitude since the early 1970s (figure 1), productivity, especially in the service sector, seems to have stagnated (figure 2). Despite the enormous promise of information technology (IT) to effect "the biggest technological revolution men have known" [Snow, 1966], disillusionment and frustration with the technology are evident in headlines like "Computer Data Overload Limits Productivity Gains" [Zachary, 1991].

Interest in the "productivity paradox" has engendered a significant amount of research. Although researchers analyzed statistics extensively during the 1980s, they found little evidence that information technology significantly increased productivity. As Robert Solow quipped, "you can see the computer age everywhere but in the productivity statistics."

Now, after some researchers found firm-level evidence that IT investments earned hefty returns, the media pendulum has swung in the opposite direction. Businessweek's proclamation of "the productivity surge" due to "information technology", and Fortune magazine's headline heralding the arrival of "technology payoff" represent the latest trend. A growing number of academic studies also report positive effects of information technology on various measures of economic performance.

Just as the business media's premature announcement of a "productivity paradox" was out of proportion to the more carefully worded academic research, the current cover stories on "productivity payoff" are often overblown. A consensus on the relationship between IT investment and economic performance is still elusive. More than a decade ago, one of the earliest surveys concluded that we still had much to learn about measuring the effects of computers on organizations [Attewell and Rule, 1984]. A more recent survey also reports a "sobering conclusion: our understanding of how information technology affects productivity either at the level of the firm or for the economy as a whole is extremely limited" [Wilson, 1995].

As more research is conducted, we are gradually developing a clearer picture of the relationship between IT and productivity. However, productivity measurement isn't an exact science; the tools are blunt, and the conclusions are not definitive. Thus, while one study shows a negative correlation between total factor productivity and high share of high-tech capital formation during 1968-1986 period [Berndt and Morrison, 1995], another study suggests that computer capital contributes to growth more than ordinary capital [Jorgenson and Stiroh, 1995]. More recently, Brynjolfsson and Hitt [1996] report positive effects of IT based on firm-level evidence.

This paper seeks to summarize what we know; distinguish the central issues from peripheral ones; and clarify the questions that future research should explore. Results and implications of different studies should be interpreted in the context of specific research questions. The question of aggregate economic performance differs from the question of firm-level economic performance. Data sources, and performance measures may also depend on the level of aggregation. Even within one level of aggregation, results may depend on the measure of performance or research method. It is hoped that the process of reviewing studies of the productivity mystery will serve as a useful springboard for examining other methodologies and the broader issues involved.

As a prelude to the literature survey, it is useful to define some of the terms used and to highlight some of the basic trends in the economics of IT.

Definitions:

"Information technology" can be defined in various ways. In terms of capital, among the most common is the BEA's (U.S. Bureau of Economic Analysis) category "Office, Computing and Accounting Machinery (OCAM) which consists primarily of computers. Some researchers look specifically at computer capital, while others consider the BEA's broader category, "Information Processing Equipment (IPE)." IPE includes communications equipment, scientific and engineering instruments, photocopiers and related equipment. Besides, software and related services are sometimes included in the IT capital. Recent studies often examine the productivity of information systems staff, or of workers who use computers.

"Labor productivity" is calculated as the level of output divided by a given level of labor input. "Multifactor productivity" (sometimes more ambitiously called "total factor productivity") is calculated as the level of output for a given level of several inputs, typically labor, capital and materials. In principle, multifactor productivity is a better measure of a firm or industry's efficiency because it adjusts for shifts among inputs, such as substituting capital equipment for labor. However, the data needed to calculate multifactor productivity are more complex.

In productivity calculations, "output" is defined as the number of units produced times their unit value, proxied by their "real" price. Determining the real price of a good or service requires the calculation of individual price "deflators" to eliminate the effects of inflation.

Trends:

The price of computing has dropped by half every 2-3 years (figure 3a and figure 3b). If progress in the rest of the economy had matched progress in the computer sector, a Cadillac would cost $4.98, while ten minutes' labor would buy a year's worth of groceries.

 There have been increasing levels of business investment in information technology equipment. These investments now account for over 10% of new investment in capital equipment by American firms (figure 4, table 2).

Information processing continues to be the principal task undertaken by America's work force. Over half the labor force is employed in information-handling activities (figure 5).

Overall productivity growth appears to have slowed significantly since the early 1970s and measured productivity growth has fallen especially sharply in the service sectors, which account for 80% of IT investment (figure 2, table 4).

White collar productivity statistics have been essentially stagnant for 20 years (figure 6).

These trends suggest the two central questions which comprise the productivity paradox: 1) Why would companies invest so heavily in information technology if it didn't add to productivity? 2) If information technology does contribute to productivity, why is its contribution so difficult to measure?

This paper builds on a number of earlier literature surveys. This review considers over 150 articles, but is not comprehensive. Rather, we aim to clarify the principal issues surrounding IT and productivity. We assimilate the results of a computerized literature search of 30 leading journals in information systems and economics. In addition, many of the leading researchers in this area identified recent research that has not yet been published.

The productivity of IT can be measured using data on the whole economy, on specified industries or on individual firms. In the 1980s and early 1990s, disappointment in information technology was chronicled in articles disclosing broad negative correlations with economy-wide productivity. Several econometric estimates also indicated low IT capital productivity in a variety of manufacturing and service industries. More recently, researchers began to find positive relationships between IT investment and various measures of economic performance at the level of individual firms. The principal empirical research studies of IT and productivity are listed in table 1.

Table 1: Principal Empirical Studies of IT and Productivity*

	Cross-sector	Manufacturing	Services
Aggregate Level Studies	Jonscher [1983], since the early 1970s and measured productivity Jonscher [1994]	Morrison & Berndt [1991]	Brand & Duke[1982]
(Economy- wide and Industry- level)	Baily [1986b], Baily & Chakrabarti [1988], Baily & Gordon [1988]	Berndt et al. [1992] Berndt & Morrison [1995]	Baily [1986a]
	Roach [1987], Roach [1988], Roach [1989b]	Siegel & Griliches [1992]	Roach [1987], Roach [1989a], Roach [1991]
	Brooke [1992]	Siegel [1994]
	Lau & Tokutsu [1992]
	Oliner & Sichel [1994]
	Jorgenson & Stiroh [1995]
Micro-Level	Osterman [1986]	Loveman [1994]	Cron & Sobol [1983]
Studies	Dos Santos [1993]	Weill [1988, 1992]	Pulley & Braunstein [1984]
	Krueger [1993]	Dudley & Lasserre [1989]	Bender [1986]
(Firm and Workers)	Brynjolfsson & Hitt [1994]	Barua, Kriebel & Mukhopadhyay [1991]	Bresnahan [1986]
	Hitt & Brynjolfsson [1994]	Brynjolfsson & Hitt [1993, 1996] Brynjolfsson & Hitt [1995]	Franke [1987]
	Lichtenberg [1995]		Strassmann [1985] Strassmann [1990]
	Brynjolfsson & Hitt [1996]		Harris & Katz [1987, 1991]
			Parsons et al. [1990]
			Diewert & Smith [1994]

II. Research on Economy-wide Productivity and Information Worker Productivity

Economists have been unable to explain the slowdown in measured productivity growth that began in the early 1970s. Labor productivity grew about 2.5% per year from 1953 to 1968, but dropped to about 0.7% per year from 1973 to 1979. Multifactor productivity growth declined from 1.75% a year to 0.32% [Baily, 1986b]. Even after accounting for factors such as the oil price shocks, changes in the quality of the labor force and potential measurement errors, most researchers still find an unexplained residual drop in productivity that roughly coincides with the rapid increase in the use of information technology.

Jorgenson and Stiroh's [1995] more recent growth accounting confirms this correlation. They calculate that average multifactor productivity growth dropped from 1.7% per year for the 1947-73 period to about 0.5% for the 1973-1992 period. At the same time, OCAM capital as a percentage of all producers' durable equipment (PDE) investment rose from about 0.5% in the 1960s to 12% in 1993. A broader category of IT capital, information processing equipment (IPE), now constitutes 34.2% of all PDE investment (table 2). Although productivity growth, especially in manufacturing, has rebounded somewhat recently, the overall negative correlation between productivity and the advent of computers underlies many of the arguments that information technology has not helped the United States' productivity and that information technology investments have been counterproductive (See, for example Baily [1986b]).

Table 2. Selected Investment Components in 1970 and 1993*

(current dollars)
	Investment	Percentage of		Investment	Percentage of
Item \ Year	1970 $billion	Fixed Investment	PDE	1993 $billion	Fixed Investment	PDE
Fixed Investment	148.1	100.0%		866.7	100.0%
Nonresidential Investment	106.7	72.05%		616.1	71.1%
PDE (nonresidential)	66.4	44.83%	100.00%	442.7	51.1%	100.0%
Information Processing	14.3	9.66%	21.54%	151.5	17.5%	34.2%
OCAM	4.1	2.77%	6.17%	53.7	6.2%	12.1%
Computer Equipment	2.7	1.82%	4.07%	47.0	5.4%	10.6%
Industrial Equipment	20.2	13.64%	30.42%	96.7	11.2%	21.8%
Transportation	16.1	10.87%	24.25%	104.2	12.0%	23.5%

Sources: Survey of Current Business, July 1994; U.S. Bureau of Economic Analysis (1992, vol. 2. Tables 5.4 and 5.8); adapted from Oliner and Sichel [1994].

Note: Information Processing Equipment: OCAM (office, computing and accounting machinery), communication equipment, and scientific and engineering equipment.

This argument was made more explicitly by Stephen Roach [1987, 1988] who focused on the productivity of information workers. In the past, office work was not very capital intensive, but recently the level of IT capital per "white collar" information worker has approached that of production capital per "blue collar" production worker. Concurrently, the ranks of information workers have ballooned and the ranks of production workers have shrunk. Roach shows that output per production worker grew by 16.9% between the 1970s and 1986, while output per information worker decreased by 6.6%. He concludes: "America's productivity shortfall [is] concentrated in that portion of the economy that is the largest employer of white-collar workers and the most heavily endowed with high-tech capital." Roach's analysis provided quantitative support for widespread reports of low office productivity.

But the economy's productivity record in the 1970s and 1980s cannot be blamed on the investment in information technology; many other factors also affect productivity and, until recently, computers were not a major share of the economy. Consider an order of magnitude estimate. In 1992, IT capital stock (OCAM) was equal to about 10% of GNP (with a base year of 1987). If, hypothetically, IT were being used efficiently and its marginal product were 50% (exceeding the return to most other capital investments), then the level of GNP would be directly increased about 5% (10% x 50%) because of the current stock of IT. However, information technology capital stock did not jump to its current level in one year; rather, the increase must be spread over about 30 years, suggesting an average annual contribution to aggregate GNP growth of 0.15%. This contribution would be very difficult to isolate because so many other factors affected GNP, especially in the relatively turbulent 1970s and 1980s. Indeed, if the marginal product of IT capital were anywhere from 0% to +65%, it would still not have affected aggregate GNP growth by more than about 0.2% per year. More comprehensive growth accounting exercises confirm this estimate. (See section V.)

Thus, very large changes in capital stock are needed to measurably change total output, although computers may have had significant effects in specific activities, such as transaction processing, and on other characteristics of the economy, such as employment shares, organizational structure, and product variety. However, as the information technology stock continues to grow and the share of the total economy accounted for by computers becomes substantial, we should begin to find changes in the level of aggregate GNP. Indeed, some recent studies do report a high contribution of computers to GDP growth. (See, for example, Oliner and Sichel [1994] and Jorgenson and Stiroh [1995].)

Just as it is hard to isolate information technology's effect on the economy, white collar productivity cannot be directly inferred from the number of information workers per unit of output. For instance, if a new delivery schedule optimizer allows a firm to substitute one clerk for two truckers, the increase in the number of white collar workers is evidence of an increase in their relative productivity as well as the firm's productivity. Osterman [1986] suggests that such efficiency improvements can explain why firms often hire more clerical workers after they introduce computers, and Berndt et al. [1992] confirm that information technology capital is, on average, a complement for white collar labor and is correlated with fewer blue collar workers. Berman, Bound and Griliches [1994] also find that the "increased use of non production workers is strongly correlated with investment in computers and in R&D." Unfortunately, it is exceedingly difficult to directly measure the productivity of office workers.

Independent of its implications for productivity, growth in the white collar work force cannot be attributed solely to information technology. Although almost half of workers now use computers in their jobs [Katz and Krueger, 1994], the ranks of information workers began to surge even before the advent of computers [Porat, 1977 ]. In fact, Jonscher [1994] argues that the increased demand for information technology created economies of scale and learning in the computer industry, thereby reducing the cost of computers.

In line with this argument, the unbalanced growth hypothesis may provide a sensible economic explanation. Economic growth may slow down because of intrinsically slow technical progress in the white collar sector, since it is less subject to automation. Then why is the white collar sector's share in the economy growing? One possible answer is the higher income elasticity (and lower price elasticity) of demand for services of this sector. As income increases, people demand more services of white collar sectors. Thus, even if information technology does not add to productivity, companies in developed countries may be forced to invest in it. Since it is difficult to measure white collar sectors' output, the story becomes complicated. Companies invest in computers to produce "unmeasurables", as argued in Griliches [1994]. In short, the increased IT use may not be a source of the productivity slowdown, but simply a response to the overall transformation of the economy. Furthermore, the main benefits from using computers appear to be in areas like improved quality, variety, timeliness, and customization, which are not well-measured in official productivity statistics [Brynjolfsson, 1994].

III. Industry-Level Studies of Information Technology Productivity

The last section has shown that contrasting the economy-wide productivity slowdown with increasing IT investment is an obtuse approach, because so many other factors may intervene. Going down to the firm-level helps to control many problems that arise from aggregation, but it is often difficult to find data representative for the whole economy. Industry-level studies may provide a middle-of-the-road alternative. Table 3 summarizes some of the important studies. While earlier studies failed to identify positive effects of IT, recent studies found more encouraging results. We start with studies on service sectors.

It has been widely reported that most of the productivity slowdown is concentrated in the service sector [Schneider, 1987; Roach, 1987, 1991]. Before about 1970, service and manufacturing productivity growth rates were comparable, but since then the trends have diverged significantly. Meanwhile services have dramatically increased as a share of total employment and to a lesser extent, as a share of total output. Because services use up to 80% of computer capital (table 4), the slow growth of productivity in the service sector has been taken as indirect evidence of poor information technology productivity.

Roach's research on white collar productivity, discussed above, focused principally on IT's performance in the service sector [1987a, 1989a, 1989b, 1991]. He argued that IT is an effective substitute for labor in most manufacturing industries, but has been associated with bloating white-collar employment in services, especially finance. He attributed this to relatively keener competitive pressures in manufacturing and foresees a period of belt-tightening and restructuring in services as they begin to face international competition.

However, studies of manufacturing also found evidence that computers may not increase productivity. Berndt and Morrison analyzed a broader data set from the U.S. Bureau of Economic Analysis (BEA) that encompasses the whole U.S. manufacturing sector. In the first paper [Morrison and Berndt, 1991], they examined a series of parameterized models of production, and found evidence that every dollar spent on IT delivered, on average, only about $0.80 of value on the margin, indicating a general overinvestment in IT. Their later paper [Berndt and Morrison, 1995] examined broad correlations of IT investment with labor productivity and multifactor productivity. This approach did not find a significant difference between the productivity of IT capital and other types of capital for a majority of the 20 industry categories examined. They did find that investment in IT was correlated with increased demand for skilled labor.

Siegel and Griliches [1992] used industry and establishment data from a variety of sources to examine several possible biases in conventional productivity estimates. They found a positive simple correlation between an industry's level of investment in computers and its multifactor productivity growth in the 1980s. They did not examine more structural approaches, in part because of troubling concerns about the reliability of the data and government measurement techniques. Their findings contrast with those of Berndt and Morrison [1995]. However, Berndt and Morrison [1995] also document positive correlations between IT capital and some measures of economic performance in the specifications where cross-sectional effects were emphasized. In addition, Berndt and Morrison's level of aggregation (two-digit SIC code) is broader than that of Siegel and Griliches' (four-digit SIC code).

Many researchers working on industry-level data express concerns about data problems, which are often caused by aggregation. For example, the BEA data is mainly used for industry-level analysis, but it is subject to subtle biases due to the techniques used to aggregate and classify establishments. One of Siegel and Griliches'[1992] principal conclusions was that "after auditing the industry numbers, we found that a non-negligible number of sectors were not consistently defined over time."

Siegel [1994] attempts to tackle the data problems that arise from two possible sources of measurement error. The first kind of error occurs when computer price and quantity are measured with error. The second source of error is more subtle: firms invest in computers not only for cost reduction but also for quality improvement. Since the quality improvement is not fully taken into account in the traditional statistics, the errors in output measurement are correlated with computer investment. These two kinds of errors cause bias and inefficiency in estimation. After attempting to control for these errors using a "multiple-indicators and multiple-causes" model, Siegel found a significant positive relationship between multifactor productivity growth and computer investment. He also finds that computer investment is positively correlated with both product quality and labor quality, a result that is consistent with Brynjolfsson [1994], Berndt and Morrison [1995], and Berman, Bound and Griliches [1994].

Table 3: Industry-Level Studies*

Study	Sector	Data source	Findings
Brand [1982]	Services	BLS*	Productivity growth of 1.3%/yr in banking
Roach [1987], Roach [1989a], Roach [1991]	Services	Principally BLS, BEA*	Vast increase in IT capital per information worker and a decrease in measured output per worker
Morrison & Berndt [1991]	Manufacturing	BEA	IT marginal benefit is 80 cents per dollar invested
Berndt et al [1992], Berndt & Morrison [1995 ]	Manufacturing	BEA, BLS	IT not correlated with higher productivity in most of industries, but correlated with more labor
Siegel & Griliches [1992]	Manufacturing	Multiple gov't sources	IT-using industries tend to be more productive; government data is unreliable
Siegel [1994]	Manufacturing	Multiple gov't sources	A multiple-indicators and multiple-causes model captures significant MFP effects of computers

BLS: U.S. Bureau of Labor Statistics

BEA: U.S. Bureau of Economic Analysis

Table 4. Investment in Computers (OCAM) in the U.S. economy.

(percentage of total in current dollars)*

Industry	1979	1989	1992
Agriculture	0.1%	0.1%	0.1%
Mining	2.4%	1.1%	0.9%
Manufacturing	29.4%	20.3%	20.2%
Construction*	0.1%	0.3%	0.2%
Non-service Total	32.0%	21.8%	21.4%
Transportation	1.3%	2.0%	1.0%
Communication	1.5%	1.4%	1.5%
Utilities	1.2%	2.8%	2.8%
Trade*	19.9%	16.3%	20.0%
Finance*	32.5%	38.7%	37.8%
Other Services*	11.6%	17.0%	13.9%
Services Total	68.0%	78.2%	78.6%
Unmeasurable Sectors*	64.1%	72.3%	71.9%
Plus consumer and
government purchases	67.7%	77.6%	77.0%
Unmeasurable sector output	63%	69%	70%

Source: BEA, adapted from Griliches [1995]

* Unmeasurable sectors: construction, trade, finance and other services; in these sectors outputs are

difficult to measure, relative to measurable sectors.

IV. Firm-Level Studies of Information Technology Productivity

Over the past ten years, many studies examined the relationship between firms' IT investment and their performance. Interestingly, studies that have used larger and more recent datasets have found evidence that IT positively affects firm performance. Research results in manufacturing often show stronger effects than studies of services, probably because of better measurement.

A. Service Sector Studies

Strassmann [1985] reports disappointing evidence in several studies. In particular, he found that there was no correlation between IT and return on investment in a sample of 38 service sector firms: some top performers invest heavily in IT, while others do not. In his later book [1990], he concludes that "there is no relation between spending for computers, profits and productivity".

Several studies have examined IT's impact on the performance of financial services firms. Parsons, Gottlieb and Denny [1990] estimated a production function for banking services in Canada. They found that the impact of IT on multifactor productivity was quite low between 1974 and 1987. They speculated that IT has positioned the industry for greater growth in the future. Similarly, Franke [1987] found that IT was associated with a sharp drop in capital productivity and stagnation in labor productivity, but remained optimistic about the future potential of IT, citing the long time lags associated with previous "technological transformations" such as the conversion to steam power. In contrast, Brand and Duke [1982] used BLS data and techniques, and found that moderate productivity growth had already occurred in banking.

Harris and Katz [1991] and Bender [1986] examined data on the insurance industry from the Life Office Management Association Information Processing Database. They found positive but sometimes weak relationships between IT expense ratios and various performance ratios. Alpar and Kim's [1991] studied 759 banks and found that a 10% increase in IT capital is associated with a 1.9% decrease in total costs. Several case studies of IT's impact on performance have also been done. Weitzendorf and Wigand [1991] developed a model of information use in two service firms; and Pulley and Braunstein [1984] studied an information services firm, and found an association between IT investment and increased economies of scope.

Estimating a production function, Brynjolfsson and Hitt [1993] found that for the service firms in their sample, gross marginal product averaged over 60 percent per year. Their 1995 study reports that IT contributes as much output in the service sector as in the manufacturing sector [Brynjolfsson and Hitt, 1995]. Because they used firm-level data, this result suggests that the productivity "slowdown" in the service sector may be an artifact of the mismeasurement of output in aggregate datasets. Indeed, even when firms were classified into "measurable" and "unmeasurable" sectors as defined by Griliches [1994], no noticeable difference in IT productivity between the sectors was found using this firm-level data.

Diewert and Smith [1994] provide an interesting case study of a large Canadian retail distribution firm. They found that the firm experienced an astounding 9.4% quarterly multifactor productivity growth, for six consecutive quarters starting at the second quarter of 1988. They argue that "these large productivity gains are made possible by the computer revolution which allows a firm to track accurately its purchase and sales of inventory items and to use the latest computer software to minimize inventory holding costs."

Measurement problems are more acute in services than in manufacturing, partly because many service transactions are idiosyncratic, and therefore not amenable to statistical aggregation. Even when data are abundant, classifications sometimes seem arbitrary. For instance, in accordance with one standard approach, Parsons, Gottlieb and Denny [1990] treat time deposits as inputs into the banking production function and demand deposits as outputs. The logic for such decisions is sometimes tenuous, and subtle changes in deposit patterns or classification standards can have disproportionate impacts.

The importance of variables other than IT is also particularly apparent in some of the service sector studies. In particular, researchers and consultants have increasingly emphasized the need to reengineer work when introducing major IT investments. As Wilson [1995] suggests, it would be interesting to know whether reengineering efforts are the main explanation for Brynjolfsson and Hitt's [1993, 1995] findings that IT is correlated with increased output. A recent survey found that, in fact, firms that had reengineered were significantly more productive than their competitors [Brynjolfsson, 1994].

Table 5: Studies of Firms in the Service Sector*

Study	Data source	Findings
Pulley & Braunstein [1984]	An info-service firm	Significant economies of scope
Clarke [1985]	Case study	Major business process redesign needed to reap benefits in investment firm
Strassmann [1985] Strassmann [1990]	Computerworld survey of 38 companies	No correlation between various IT ratios and performance measures
Bender [1986]	LOMA insurance data on 132 firms	Weak relationship between IT and various performance ratios
Franke [1987]	Finance industry data	IT was associated with a sharp drop in capital productivity and stagnant labor productivity
Harris & Katz [19 91]	LOMA insurance data for 40	Weak positive relationship between IT and various performance ratios
Noyelle [1990]	US and French industry	Severe measurement problems in services
Parsons et al. [1990]	Internal operating data from 2 large banks	IT coefficient in translog production function small and often negative
Alpar and Kim [1991]	Large number of banks	IT is cost saving, labor saving, and capital using
Weitzendorf & Wigand [1991]	Interviews at 2 companies	Interactive model of information use
Diewert & Smith [1994]	A large Canadian retail firm	Multi-factor productivity grows 9.4% per quarter over 6 quarters
Brynjolfsson & Hitt [1995]	IDG, Compustat, BEA	Marginal products of IT do not differ much in services and in the manufacturing; Firm effects account for 50% of the marginal product differential

B. Studies of Manufacturing Sector and Cross-Sector Studies

There have been several firm-level studies of IT productivity in the manufacturing sector. Some of the important results are summarized in table 6. A study by Loveman [1994] provided some of the first econometric evidence of an IT productivity shortfall, when he examined data from 60 business units using the Management Productivity and Information Technology (MPIT) subset of the Profit Impact of Market Strategy (PIMS) database. As is common in productivity literature, he used an ordinary least squares regression and assumed that production functions could be approximated by a Cobb-Douglas function. Loveman estimated that the contribution of information technology capital to final output was approximately zero over the five-year period he studied in almost every subsample. His findings were fairly robust to a number of variations on his basic formulation.

Barua, Kriebel and Mukhopadhyay [1991] traced Loveman's results back a step by looking at IT's effect on intermediate variables such as capacity utilization, inventory turnover, quality, relative price, and new product introduction. Using the same data set, they found that IT was positively related to three of these five intermediate measures, but that the effect was generally too small to measurably affect final output. Dudley and Lasserre [1989] also found econometric support for the hypothesis that better communication and information reduce the need for inventories, without explicitly relating this to bottom-line performance measures. Using a different data set, Weill [1992] disaggregated IT by use, and found that significant productivity could be attributed to transactional types of information technology (e.g., data processing), but was unable to identify gains associated with strategic systems (e.g., sales support) or informational investments (e.g., email infrastructure).

In a series of studies utilizing large firm-level surveys by International Data Group (IDG), Brynjolfsson and Hitt report that IT improves productivity. Their 1993 study found that while gross marginal product of non-computer capital ranges from 4.14% to 6.86%, that of computer capital averages 56% - 68%. The results of this and their later study [1994] imply that the following three null hypotheses can be rejected:

H1: IT capital has a zero gross marginal product.

H2: IT capital has zero net marginal benefit, after all costs have been subtracted.

H3: IT capital's marginal product is not different from that of other capital.

Their point estimates of gross marginal products indicate that at the margin computer capital generates 10 times more output than other capital of equal value. Brynjolfsson and Hitt [1995] show up to half of the excess returns imputed to IT could be attributed to firm-specific effects.

If gross marginal product of information technology capital is really so large, what friction or market failure prevents firms from investing in more computers, until the marginal products of all capital goods become equal? One reason is that computer capital has a higher user cost. According to Oliner and Sichel [1994], from 1970 to 1992 the user cost of computer capital averaged 36.6 % per year, while that of other types of capital was 15.4%. The remaining portion of the answer may come from adjustment or hidden costs of information technology investment, such as the complementary organizational investments required to realize the benefits of IT.

Lichtenberg [1995] confirms the results of Brynjolfsson and Hitt, using similar data and methods. He also analyzes Informationweek survey data and uncovers essentially the same results. His formal tests reject the above null hypotheses. Importantly, Lichtenberg extends his study to report the marginal rate of substitution between IT and non-IT workers. At the sample mean, one IT worker can apparently be substituted for six non-IT workers.

Table 6: Studies of Manufacturing Firms and Cross-Sector Firms*

Study	Data source	Findings
Loveman [1994]	PIMS/MPIT	IT investments added nothing to output
Dudley & Lasserre [1989]	US and Canadian Aggregate Data	IT and communication reduces inventories
Weill [1992]	Valve manufacturers	Contextual variables affect IT performance Transaction processing IT produce positive results
Barua, Kriebel & Mukhopadhyay [1991]	PIMS/MPIT	IT improved intermediate outputs, if not necessarily final output
Brynjolfsson & Hitt [1993]	IDG; Compustat: BEA	The gross marginal product of IT capital is over 50% per year in manufacturing
Brynjolfsson & Hitt [1995]	IDG; Compustat: BEA	Firm effects account for half of the productivity benefits of earlier study
Lichtenberg [1995]	IDG; Informationweek (cross sector)	IT has excess return; IT staff's substitution effect is large
Kwon & Stoneman [1995]	UK survey	New technology adoption especially computer use has a positive impact on output and productivity

Research in manufacturing generally finds higher returns to IT investment than in the services, probably because of better measurement. Yet the MPIT data, which both Loveman [1994] and Barua et al. [1991] use, must be scrutinized. While the point estimates for IT's contribution were quite low, the standard errors were very high so that 95% confidence interval often exceeded 200% for many of Loveman's estimates. These studies may also be unrepresentative, since the period covered by the MPIT data, 1978- 83, was unusually turbulent.

The IDG data set, which is among the largest data sets used in this research area, substantially mitigates data problems, although it contains data on large firms only, and so may not be a representative random sample. Indeed, Brynjolfsson and Hitt [1993] attribute the statistical significance of their findings partly to the large size of the IDG data set, which enables them to more precisely estimate returns for all factors. Utilizing comprehensive surveys of the UK engineering industry undertaken in 1981, 1986, and 1993, Kwon and Stoneman [1995] also find that the use of computers and numerical control machines has increased output and productivity.

V. Contribution to Consumer Surplus and Economic Growth

Some researchers have identified sizable contributions of IT to consumer surplus and to economic growth. Some important studies are summarized in table 8. Growth accounting and consumer surplus analysis are techniques to identify and measure "pecuniary externalities", which Griliches [1992, 1994] distinguishes from "non-pecuniary externalities or spill-overs."

Pecuniary externalities arise when the price of some input declines. For example, when computer prices are declining exogenously, profit-maximizing firms substitute computer systems for other input factors, such as labor or warehouse space. Lowered prices of computers and other inputs shift marginal cost curves downward. These marginal cost curves result in higher output and lower prices. The output increase is a measure of the pecuniary externality; the benefits created by the computer sector are reflected in greater output of computer-using industries. A second measure of the pecuniary externality is consumer surplus. As computer prices fall, many firms and customers that could not afford computers become able to purchase them, while the customers who were willing to pay higher prices enjoy a windfall price reduction.

Pecuniary externalities directly increase labor productivity, yet they do not necessarily increase multifactor productivity. Pecuniary externalities do not change the production function; rather they change the input mix. In contrast, non-pecuniary externalities, or spill-overs, arise from technical change; people may have found smarter ways of making goods and services using information technology. The production possibility frontier shifts out; both labor productivity and multifactor productivity should go up.

Bresnahan [1986] estimated the benefits to consumers of declining computer prices. Using the hedonic price index method, he calculates that the consumer surplus was five or more times of computer expenditures in the late 1960s financial sector. Adopting similar assumptions, Brynjolfsson [1995] estimates that, in 1987, between $69 billion and $79 billion consumer surplus was generated by $25 billion in expenditures on information technology capital.

Now we turn to several growth accounting results. Jorgenson and Stiroh's [1995] comprehensive growth accounting found that from 1979 to 1985 computers and peripherals contributed to output growth by 0.52% per year, and that from 1985 to 1992, the contribution is 0.38% per year (see Table 7). One of their study's main contributions is the careful calculation of capital's service flow. Because they assume that computers maintain their full ability until retirement, their estimation of computer capital's contribution becomes larger than that of Oliner and Sichel [1994].

Oliner and Sichel [1994] carefully examine how the various excess return hypotheses of computer capital affect growth. As a baseline they estimate that the contribution of computer capital to output growth is 0.16% per year for the 1970-1992 period. Using Romer's [1986, 1987] assumption that physical capital provides a positive externality, the contribution goes up to 0.32%. Brynjolfsson and Hitt's [1993] higher estimate for the return on computer capital raises the contribution to 0.35%. They also try to incorporate Alan Krueger's [1993] result of return on workers' computer use. If the return is equal to the difference in the marginal product between computer-using workers and non-using workers, the contribution is 0.38%. Oliver and Sichel claim that an annual contribution of up to 0.38% is not large enough to offset the approximately 1% drop in output growth since the 1970s.

Table 7. Growth Rates of Aggregate Output and Contribution of Factors (1947-92)*

Variable		Value Added			Growth Contribution
	Annual	NonComp	Computer	Capital	NonComp	Computer	Labor	Multifactor
Period	growth	Share	Share		Share	Share		productivity
47-92	3.42%	3.33%	0.09%	1.47%	1.26%	0.21%	0.92%	1.03%
47-53	5.46%	5.46%	0.00%	1.92%	1.92%	0.00%	1.26%	2.27%
53-57	2.14%	2.14%	0.00%	1.42%	1.42%	0.00%	0.19%	0.53%
57-60	2.39%	2.37%	0.02%	0.83%	0.83%	0.00%	-0.01%	1.57%
60-66	5.38%	5.30%	0.08%	1.46%	1.36%	0.10%	1.44%	2.48%
66-69	2.61%	2.54%	0.07%	1.93%	1.74%	0.20%	1.16%	-0.49%
69-73	3.67%	3.60%	0.08%	1.64%	1.40%	0.24%	0.74%	1.29%
73-79	2.63%	2.50%	0.12%	1.45%	1.19%	0.26%	1.28%	-0.10%
79-85	2.89%	2.65%	0.24%	1.28%	0.76%	0.52%	0.83%	0.78%
85-92	2.49%	2.38%	0.12%	1.26%	0.88%	0.38%	0.76%	0.47%

Source: Adapted from Jorgenson and Stiroh (1995)

The following rough calculation may provide some intuition about the size of the computer's contribution to national output. From Jorgenson and Stiroh [1995], we take the simple average contribution for the 1979-1992 period or 0.45%. We compare it with the 0.72% contribution of other capital. The share of computers in total capital stock was 1.6% in 1993, implying that one unit of computer capital contributes as much to the growth of output as 98 units of other forms of capital. In 1993, GDP grew by $173 billion. Computers contributed $29 billion; other capital contributed $46 billion. The unexplained residual (MFP) contribution is $40 billion. A rough estimate shows that the implicit marginal product of computer capital in Jorgenson and Stiroh's study is also over 60%.

Using data from 367 large firms that together generated $1.8 trillion in output per year from 1988 to 1992, Brynjolfsson and Hitt [1994] provide an interesting growth accounting result. For their sample of firms, IT capital contributes about 1% per annum to output growth - a larger growth contribution than that of ordinary capital in absolute value. Lau and Tokutsu [1992] calculate an even bigger contribution to growth, attributing approximately half of the real output growth (1.5% growth per annum) during the past three decades to computer capital. They also argue that the annual rate of inflation dropped by 1.2% per year because of the rapid decline in computer prices. In line with these studies, Roy Radner suggests that "productivity growth has slowed down for other reasons, unrelated to the IT story. Without IT, things would have been worse, and output growth would have been lower."

In summary, the weight of evidence from various studies suggests that information technology capital generates billions of dollars annually for the U.S. economy, both in terms of output growth and consumer surplus. Meanwhile, the recent firm-level analyses of Brynjolfsson and Hitt[1993, 95] and Lichtenberg [1995] have begun to remedy the shortfall of evidence regarding the productivity contribution of IT.

Table 8: Studies on Contribution to Consumer Surplus and Economic Growth*

Study	Data Source	Findings
Bresnahan [1986]	Financial service firms	Large gains in imputed consumer welfare
Lau & Tokutsu [1992]	Multiple Gov't sources	Computer capital contributes half of output growth
Brynjolfsson & Hitt [1994]	IDG*, Compustat	Growth contribution of computers is 1% per year among 367 US large firms
Oliner & Sichel [1994]	principally BEA*	Growth contribution of computers is 0.16% - 0.38% per year varying by different assumptions
Jorgenson & Stiroh [1995]	principally BEA	Growth contribution of computers for the 1979-92 period is 0.38 - 0.52% per year
Brynjolfsson [1995]	BEA	$70 billion consumer surplus is generated annually in the late 1980s.

IDG: International Data Group

BEA: U.S. Bureau of Economic Analysis

VI. Conclusion: Where Do We Go from Here?

Sections II, III, IV, and V presented a review of the principal empirical literature on the productivity of information technology. Looking at the simple relationship between the productivity slowdown of the whole US economy and the rapid growth of computer capital is too general an approach. Poor data quality for IT outputs and inputs has exacerbated this problem. Due to the application of improved methodologies and the identification of more reliable and larger datasets, researchers have made some progress with industry-level and firm-level studies. Recently, some researchers have found positive effects of IT. Careful growth accounting exercises and estimation of production and cost functions for specific sectors or industries can provide sharper insights. Consumer surplus analyses are useful exercises for identifying alternative ways to triangulate IT value. These studies suggest that without IT, the US economy would probably be in a worse situation. This section proposes further research questions and methodologies.

The first priority is to improve the data and the measurement techniques. Government statistics, especially in services and for information workers, have not kept up with the growing importance and complexity of these sectors. Therefore, researchers may have to perform their own corrections on the data, turn to private sources of secondary data, or gather data themselves. Researchers should make their data available to other researchers so that a cumulative tradition can be maintained. The studies of Weill [1992], Dos Santos et al. [1993], Berndt & Morrison [1995] and Brynjolfsson and Hitt [1993, 1995] are examples of new data identification and development.

One effective way to identify possible gaps in the data is to compare them with benefits that managers and customers expect from IT, such as quality, timeliness, customer-service, flexibility, innovation, customization and variety. In principal, many of these benefits are quantifiable. In fact, some firms already attempt such an analysis in their capital budgeting and justification processes. In addition, many companies have developed elaborate measurement programs, for example, as part of total quality management, these programs augment or even supersede financial accounting measures and can serve as a foundation for more refined metrics [Kaplan and Norton, 1992].

Many economists also have tried various methods to overcome the shortfall of government statistics, and to incorporate quality changes when estimating price indices. The long history of hedonic price index method is a good example, but some economists argue even the hedonic method does not capture all the benefits associated with product innovation and differentiation. Trajtenberg [1990] devises a new method of quality adjusted price index calculation, adopting the discrete choice model. Fisher and Griliches [1995] argue that if new inexpensive (quality-adjusted) goods are introduced and gain market share at the expense of existing goods, the official statistics by the Bureau of Labor Statistics will seriously overestimate inflation. Hausman [1994] also reports a 20% to 25% overestimation of the consumer price index for ready-to-eat cereals, based on his analysis of Apple Cinnamon Cheerios.

Unfortunately, for many services, even basic output measures must be created, because government and accounting data records only inputs. Baily and Gordon [1988] and Noyelle [1990] among others, have done much to improve measurement in areas such as banking and retailing, while relatively good statistics can be compiled from private sources in areas such as package delivery. Unfortunately, the individualized nature of many services defies aggregation. The output of a lawyer, manager or doctor cannot be extrapolated from the number of meetings attended, memoranda written or medications provided. The complexity of the "Diagnostic Related Group" approach to valuing medical care is both a step in the right direction and a testament to these difficulties. A researcher who seeks to rigorously measure the productivity of service industries generally must undertake this detailed work before jumping to conclusions based on input-based statistics. Similarly, disaggregating heterogeneous types of IT by use, as Weill [1992] did in a manufacturing study, can increase the resolution of standard statistical techniques.

Because so many factors affect firm performance, it is generally impossible to distinguish the impact of IT using simple bivariate correlations. It is essential to control for other factors such as other inputs and their prices, the macro-economic environment, demand schedules for output, and the nature of competition. Because many unobservable factors affect either the whole industry or one firm persistently, examining a panel consisting of both time series and cross-sectional data is the best approach, where feasible.

Importantly, we must remember that our tools are still blunt. Managers do not always recognize this and tend to rely too much on any one study of IT and productivity. While the studies usually state the limitations of the data and methods, sometimes only the surprising conclusions are reported by the media. Because significant investment decisions are based on these conclusions, researchers must be doubly careful to communicate the limitations of their work.

Researchers might also look to business for profitable research questions. A recurrent theme in the business press is the idea that information technology should not so much help us produce more of the same things as allow us to do entirely new things in new ways. For instance, Watts [1986] finds that information technology investments cannot be justified by cost reductions alone, but that instead managers should look to increased flexibility and responsiveness, while Brooke [1992] writes that information technology leads to greater variety but lower productivity as traditionally measured. Diewert and Smith's [1994] study makes another interesting point in respect to variety. They show that to rigorously measure the productivitwhile IT facilitates great efficiency in inventory management, aggregate inventory level of the U.S. economy did not shrink over the past 40 years, as reported in Blinder and Maccini's [1991]. Diewert and Smith argue that "a wide spread proliferation of new products into the world economy" results in no macro-level inventory change even when great micro-level improvements have been made.

This literature highlights how difficult and perhaps inappropriate it would be to translate the benefits of information technology into quantifiable productivity measures of output. Intangibles such as better responsiveness to customers and increased coordination with suppliers do not always increase the amount or even intrinsic quality of output, but they do help make sure it arrives at the right time, at the right place, with the right attributes for each customer. Berndt and Malone [1995] suggest that: "we need to spend more effort measuring new forms of value--such as capabilities for knowledge creation--rather than refining measures of productivity that are rooted in an Industrial Age mindset."

Just as managers look beyond "productivity" for some of the benefits of IT, so must researchers be prepared to look beyond conventional productivity measurement techniques. For instance, because consumers are generally assumed to be in the best position to assess the utility they gain from their purchases, so researchers might look to IT buyers for an estimate of IT value, as Bresnahan [1986] and Brynjolfsson [1995] did.

As another example, if rational investors value both the tangible and intangible aspects of firms' revenue-generating capacity, then changes in stock market value should approximate the true contribution of IT to the firm, not only in cost reductions, but also in increased variety, timeliness, and quality, and in principle, even the effectiveness of the firm in foreseeing and rapidly adapting to its changing environment. While relying on consumer or stockholder valuations begs the question of actual IT productivity to some extent, at a minimum these measures provide two additional benchmarks that can help triangulate IT value [Hitt and Brynjolfsson, 1994].

While the value of IT remains controversial, it is clear that the measurement problem is becoming more severe. Developed nations are devoting increasing shares of their economies to service- and information-intensive activities for which output measures are poor. The emerging "information age" has prompted a new approach to management accounting [Beniger, 1986; Kaplan, 1989]. Similarly, researchers should take the opportunity to rethink how we measure productivity and output.

Figures

Figure 1 -- Investment in information technology is growing at a rapid pace.

Source: Based on data from [BEA, National Income and Wealth Division], adapted from Jorgenson and Stiroh [1995].

Note: Constant dollars (base year 1987) calculated by hedonic price method, see Dulberger [1989].

Figure 2 -- Productivity in the service sector has not kept pace with that in manufacturing.

Source: Based on data from [Bureau of Labor Statistics, Productivity & Testing]

Figure 3a-The cost of computing has declined substantially relative to other capital purchases.

Source: Based on data from [U.S. Dept. of Commerce, Survey of Current Business]

Note: PDE, Producer's Durable Equipment

Figure 3b-Microchip performance has shown uninterrupted exponential growth.

Source: Grove [1990], and company data. Trend lines are by authors' estimation.

Note: P6, P7 microprocessors and 256M, 1G, 4G DRAMs are estimated by Intel and Semiconductor Industry Association.

Figure 4 -- Computers comprise about 10% of current-dollar investment in Producers' Durable Equipment

Source: Based on data from [BEA, National Income and Wealth Division]

Figure 5 -- Information processing is the largest category of employment.

Source: Porat [ 1977]. The defining criterion for information workers is whether the primary activity is knowledge creation, warehousing, or dissemination. The classification scheme includes people engaged in the following activities:

TYPOLOGY OF PRIMARY INFORMATION SECTOR INDUSTRIES USED IN FIGURE 5

KNOWLEDGE PRODUCTION AND COMMUNICATION INDUSTRIES

R&D and Inventive Industries

Private Information Services

INFORMATION DISTRIBUTION AND COMMUNICATION INDUSTRIES

Education

Public Information Services

Regulated Communication Media

Unregulated Communicated Media

RISK MANAGEMENT

Insurance Industries

Finance Industries

Speculative Brokers

SEARCH AND COORDINATION INDUSTRIES

Search and Non-Speculative Brokerage Industries

Advertising Industries

Non-Market Coordination Institutions

INFORMATION PROCESSING AND TRANSMISSION SERVICES

Non-Electronic Based Processing

Electronic Based Processing

Telecommunications Infrastructure

INFORMATION GOODS INDUSTRIES

Non-Electronic Consumption or Intermediate Goods

Non-Electronic Investment Goods

Electronic Consumption or Intermediate Goods

Electronic Investment Goods

SELECTED GOVERNMENT ACTIVITIES

Primary Information Services in the Federal Government

Postal Service

State and Local Education

SUPPORT FACILITIES

Information Structure Construction and Rental

Office Furnishings

Figure 6 -- White collar productivity appears to have stagnated.

Source: Roach [1991]

Bibliography

Ahituv, N. and Giladi, N. [1993], "Business Success and Information Technology: Are They Really Related," Proceedings of the 7th Annual Conference of Management IS, Tel Aviv University.

Allen, Thomas J. and Scott Morton, Michael S. [1994], Information Technology and the Corporation of the 1990s, Oxford University Press.

Alpar, P. and Kim, M. [1991], "A Microeconomic Approach to the Measurement of Information Technology Value," Journal of Management Information Systems, Fall, 7(2): 55-69.

Alpar, P. and Kim, M. [1990], "A Comparison of Approaches to the Measurement of Information Technology Value," Proceedings of the Twenty-Second Hawaii International Conference on System Science, Honolulu, HI.

Applegate, L., Cash, J. and Mills, D. Q. [1988], "Information Technology and Tomorrow's Manager," Harvard Business Review, November-December, pp. 128-136.

Attewell, P. and Rule, J. [1984], "Computing and Organizations: What We Know and What We Don't Know," Communications of the ACM, Vol. 27:1184-1192, (December).

Ayres, R. U. [1989], "Information, Computers, CIM and Productivity," Organization for Economic Co-operation and Development Paper, (June).

Baily, Martin Neil [1986a], "Taming the Information Monster," Bell Atlantic Quarterly, Summer, pp. 33-38.

Baily, Martin Neil [1986b], "What Has Happened to Productivity Growth?" Science, Vol. 234: 443-451.

Baily, Martin Neil and Chakrabarti, A. [1988], "Electronics and White-Collar Productivity," in Innovation and Productivity Crisis, Brookings, Washington.

Baily, Martin Neil and Gordon, R. J. [1988], "The Productivity Slowdown, Measurement Issues and the Explosion of Computer Power", Brookings Papers in Economic Activity, 1988(2): 347-431.

Bakos, J. Yannis [1987], Inter-organizational Information Systems: Strategic Implications for Competition and Cooperation, Ph.D. Dissertation, MIT School of Management.

Bakos, J. Yannis and Kemerer, Chris F. [1992], "Recent Application of Economic Theory in Information Technology Research," Decision Support System Vol. 8: 365-386.

Banker, R. D. and Kauffman, R. J. [1988], "Strategic Contributions of Information Technology: An Empirical study of ATM Networks," Proceedings of the Ninth International Conference on Information Systems, Minneapolis, Minnesota.

Barua, A., Kriebel, C. and Mukhopadhyay, T. [1991], "Information Technology and Business Value: An Analytic and Empirical Investigation," University of Texas at Austin Working Paper, (May).

Baumol, W. J. [1967], "Macroeconomics of Unbalanced Growth: The Anatomy of Urban Crisis," American Economic Review 57(3): 415-26

Baumol, W.J., S. A. Blackman, and E. N. Wolff. [1985], "Unbalanced Growth Revisited: Asymptotic Stagnancy and New Evidence," American Economic Review 74(4): 806-17

Bender, D. H. [1986], Financial Impact of Information Processing. Vol. 3(2): 22-32.

Beniger, J. [1986], The Control Revolution. Harvard University Press, Cambridge, MA.

Benjamin, R. I., Rockart, J. F., Scott Morton, M. S. et al. [1984], "Information Technology: A Strategic Opportunity," Sloan Management Review, Spring, pp. 3-10.

Berman, E., Bound, J., and Griliches, Z. [1994], "Changes in the Demand for Skilled Labor within U. S. Manufacturing: Evidence form the Annual Survey of Manufactures," Quarterly Journal of Economics, Vol. 109(2): 367-397.

Berndt, Ernst R. [1991], The Practice of Econometrics: Classic and Contemporary, Addison-Wesley, Reading, MA.

Berndt, Ernst R. and Malone, Thomas W. [1995], "Information Technology and the Productivity Paradox: Getting the Questions Right; Guest Editor's Introduction to Special Issue," Economics of Innovation and New Technology, Vol. 3: 177-182.

Berndt, Ernst R. and Morrison, Catherine J. [1995], "High-tech Capital Formation and Economic Performance in U.S. Manufacturing Industries: An Exploratory Analysis", Journal of Econometrics 65: 9-43.

Berndt, Ernst R. and Morrison, Catherine J. [1991], "Computers Aren't Pulling Their Weight," Computerworld, December 9, pp. 23-25.

Berndt, Ernst R., Morrison, Catherine J. and Rosenblum, Larry S., [1992], "High-tech Capital Formation and Labor Composition in U.S. Manufacturing Industries: an Exploratory Analysis," National Bureau of Economic Research Working Paper No. 4010, (March).

Blinder, A. S. and L.J. Maccini [1991], "Taking Stock: A Critical Assessment of Recent Research on Inventories," Journal of Economic Perspectives 5: 73-96.

Brand, H. and Duke, J. [1982], "Productivity in Commercial Banking: Computers Spur the Advance," Monthly Labor Review, Vol. 105: 19-27, (December).

Bresnahan, Timothy F. [1986], "Measuring Spillovers from Technical Advance: Mainframe Computers in Financial Services," American Economic Review 76(4), (September).

Bresnahan, Timothy F. and Trajtenberg, M. [1995], "General Purpose Technologies and Aggregate Growth," Journal of Econometrics 65: 83-108.

Brooke, G. M. [1992], "The Economics of Information Technology: Explaining the Productivity Paradox," MIT Sloan School of Management Center for Information Systems Research Working Paper No. 238, (April).

Brynjolfsson, Erik [1995], "Some Estimates of the Contribution of Information Technology to Consumer Welfare," MIT Sloan School of Management Working Paper, (August).

Brynjolfsson, Erik [1994], "Technology's True Payoff," Informationweek, October 10, pp. 34-36.

Brynjolfsson, Erik [1993], "The Productivity Paradox of Information Technology: Review and Assessment," Communications of ACM, December, 36(12), p. 67-77.

Brynjolfsson. Erik and Hitt, Lorin. [1996], "Paradox Lost? Firm-Level Evidence on the Returns to Information Systems Spending", Management Science, (April)

Brynjolfsson, Erik and Hitt, Lorin. [1995], "Information Technology as a Factor of Production: the Role of Differences among Firms," Economics of Innovation and New Technology, Vol. 3: 183-199.

Brynjolfsson, Erik and Hitt, Lorin. [1994], "Computers and Economic Growth: Firm-Level Evidence," MIT Sloan School of Management Working Paper No. 3714, (August).

Brynjolfsson. Erik and Hitt, Lorin. [1993], "Is Information Systems Spending Productive? New Evidence and New Results", The Proceedings of the 14th International Conference on Information Systems, Orlando, FL.

Brynjolfsson, Erik, Malone, T. Gurbaxani, V., and Kambil, A. [1991], "Does Information Technology Lead to Smaller Firms?" Management Science, 40:12, (December): 1628-1644.

Champy, James [1995], Reengineering Management, HaperBusiness, New York, NY.

Cecil, J. L. and Hall, E. A. [1988], "When IT Really Matters to Business Strategy," McKinsey Quarterly, pp. 2, (Autumn).

Chismar, W. G. and Kriebel, C. H. [1985], "A Method for Assessing the Economic Impact of Information Systems Technology on Organizations," Proceedings of the Sixth International Conference on Information Systems, Indianapolis, Indiana.

Clarke, R. F. [1985], The Application of Information Technology in an Investment Management Firm. Masters Thesis, Massachusetts Institute of Technology, Cambridge, MA.

Clemens, E. K. [1991], "Evaluation of Strategic Investment in Information Technology," Communications of the ACM, 34(1): 22-36.

Cooper, R. B. and Mukhopadhyay, T. [1990], "Research in MIS Effectiveness: A Microeconomic Production View," Carnegie Mellon University Working Paper.

Cron, W. L. and Sobol, M. G. [1983], The Relationship Between Computerization and Performance: A Strategy for Maximizing the Economic Benefits of Computerization. Vol. 6: 171-181.

Crowston, Kevin and Malone, Thomas W. [1988], "Information Technology and Work Organization," Chapter 49 in: M. Helander, ed., Handbook of Human-Computer Interactions. Elsevier Science, Amsterdam, pp. 1051-1070.

Crowston, Kevin and Treacy, M. E. [1986], "Assessing the Impact of Information Technology on Enterprise level Performance," MIT Center for Information Systems Research Working Paper, No. 143, (October).

Curley, K. F. and Pyburn, P. J. [1982], "Intellectual Technologies: The Key to Improving White-collar Productivity," Sloan Management Review, Fall, pp. 31-39.

Davenport, Thomas H. [1993], Process Innovation: Reengineering Work through Information Technology, Harvard Business School Press, Boston, Massachusetts.

Davenport, Thomas H. and Short, J. [1990], "The new Industrial Engineering: Information Technology and Business Process Redesign," Sloan Management Review, Vol. 31(4): 11-27.

David, Paul A. [1990], "The Dynamo and the Computer and Dynamo: A Historical Perspective on the Modern Productivity Paradox," American Economic Review Papers and Proceedings, Vol. 80(2): 355-361, (May).

Denison, Edward E. [1989], "Estimates of Productivity Change by Industry, an Evaluation and an Alternative," Brookings Institution, Washington, DC.

Denison, Edward E. [1985], "Trends in American Economic Growth, 1929-1982," Brookings, Washington D. C.

Diewert, W. Erwin and Smith, Ann Marie, [1994], "Productivity Measurement for a Distribution Firm," National Bureau of Economic Research Working Paper No. 4812, (July).

Dos Santos, B. L. Peffers. K. G. and Mauer, D. C. [1993], "The Impact of Information Technology Investment Announcements on the Market Value of the Firm," Information Systems Research, 4(1): 1-23.

Dos Santos, B. L. Peffers. K. G. and Mauer, D. C. [1991], "The Value of Investments in Information Technology: An Event Study," Kannert Graduate School of Management, Perdue University.

Dudley, L. and Lasserre, P. [1989], "Information as a Substitute for Inventories," European Economic Review, Vol. 31: 1-21.

Dulberger, Ellen R. [1989], "The Application of Hedonic Model to a Quality Adjusted Price Index for Computer Processors," in Jorgenson and Landau (Ed.), Technology and Capital Formation, MIT Press, Cambridge, MA.

Fisher, Franklin M. and Griliches, Zvi. [1995], "Aggregate Price Indices, New Goods, and Generics," Quarterly Economics, Vol. CX(1): 229-244, (February).

Franke, Richard H. [1987], "Technological Revolution and Productivity Decline: Computer Introduction in the Financial Industry," Technological Forecasting and Social Change, Vol. 31: 143-154.

Gordon, Robert J., [1990], "The Measurement of Durable Goods Prices, Chicago: University of Chicago Press," (see especially Chapter 6, Electronic Computers.)

Gordon, Robert J., [1987a], "Productivity, Wages, and Prices Inside and Outside of Manufacturing in the U.S., Japan, and Europe," European Economic Review, April 1987, 31(3), pp.685-739.

Gordon, Robert J., [1987b], "The Postwar Evolution of Computer Prices," National Bureau of Economic Research Working Paper No. 2227, Cambridge, MA.

Gordon, Robert J. and Baily M. N. [1989], "Measurement Issues and the productivity Slowdown in Five Major Industrial Countries," International Seminar on Science, Technology and Economic Growth, Paris, France.

Gremillion, L. L. and Pyburn, P. J. [1985], "Justifying Decision Support and Office Automation Systems," Journal of Management Information Systems, Vol. 2(1).

Griliches, Zvi [1995], "Comments on Measurement Issues in Relating IT Expenditures to Productivity Growth," Economics of Innovation and New Technology, Vol. 3: 317-321.

Griliches, Zvi, [1994], "Productivity, R&D, and Data Constraints," American Economic Review, 84(1), (March).

Griliches, Zvi [1992], "The Search for R&D Spillovers," Scandinavian Economics, Vol. 94, Supplement, pp. 29-47.

Griliches, Zvi and Cockburn, [1994], "Generics and New Goods in Pharmaceutical Price Indexes," American Economic Review, LXXXIV.

Griliches, Zvi. (Ed.) with the assistance of Ernst R. Berndt, Timothy F. Bresnahan, and Marilyn E. Manser. [1992], Output Measurement in the Service Sectors. NBER Studies in Income and Wealth Vol. 56, University of Chicago Press.

Grove, A. S. [1990], "The Future of the Computer Industry," California Management Review, Vol. 33(1): 148-160.

Gurbaxani, V. and Mendelson, H. [1989], "The use of Secondary Data in MIS Research," University of California, Irvine, (March).

Hammer, Michael [1990], "Reengineering Work: Don't Automate, Obliterate," Harvard Business Review, July-August, pp. 104-112.

Hammer, Michael and Champy, James [1993], Reengineering the Corporation, HarperBusiness, New York, NY.

Harris, S. E. and Katz, J. L. [1991], "Organizational Performance and Information Technology Investment Intensity in the Insurance Industry," Organizational Science, Vol. 2(3): 263-296.

Hausman, Jerry A. [1994], "Valuation of New Goods under Perfect and Imperfect Competition," National Bureau of Economic Research Working Paper No. 4970, (December).

Hitt, Lorin and Brynjolfsson, Erik. [1994], "Three Faces of IT Value: The Theory and Evidence," The Proceedings of the Fifteenth International Conference on Information Systems, (December).

Jonscher, C. [1994], "An Economic Study of the Information Technology Revolution," in Allen, Thomas J. and Scott Morton, Michael S. (Ed.), Information Technology and the Corporation of the 1990s: Research Studies, Oxford University Press, pp. 5-42.

Jonscher, C. [1983], "Information Resources and Economic Productivity," Information Economics and Policy, Vol. 1: 13-35.

Jorgenson, Dale W. and Landau, Ralph (Ed.), [1989], Technology and Capital Formation, MIT Press, Cambridge, MA.

Jorgenson, Dale W. and Stiroh, Kevin. [1995], "Computers and Growth," Economics of Innovation and New Technology, Vol. 3: 295-316.

Kaplan, Robert [1989], "Management Accounting for Advanced Technological Environments," Science, Vol. 245: 819-823, (September).

Kaplan, Robert and Norton, D. P. [1992], "The Balanced Scorecard - Measures that Drive Performance," Harvard Business Review, January-February, p. 71-79.

Katz, Lawrence F. and Krueger, Alan B. [1994], "How Computers Have Changed the Workplace, 1984-1993," Unpublished Paper, Harvard University.

Kemerer, Chris F. and Sosa, G. L. [1991], "Systems Development Risks in Strategic Information Systems," Information and Software Technology, Vol. 33(3): 212-223, (April).

Kriebel, Charles H. [1989], "Understanding the Strategic Investment in IT," in Lauden, K. C. and Turner, J. A. (Ed.) Information Technology and Management Strategy, Englewood Cliffs, NJ, Prentice Hall.

Krueger, Alan B. [1993], "How Computers Have Changed the Wage Structure: Evidence from Micro-data, 1984-1989," Quarterly Journal of Economics 108(1): 33-60.

Kwon, Myung Joong and Stoneman, Paul [1995], "The Impact of Technology Adoption on Firm Productivity." Economics of Innovation and New Technology, Vol. 3: 219-233.

Landauer, Thomas K. [1995], The Trouble with Computers, The MIT Press, Cambridge, MA.

Lasserre, P. [1988], "Project on the Impact of Information on Productivity," unpublished paper, (September).

Lau, Lawrence J. and Tokutsu, Ichiro [1992], "The Impact of Computer Technology on the Aggregate Productivity of the United States: An Indirect Approach," unpublished paper, Stanford University, (August).

Lichtenberg, Frank R. [1995], "The Output Contributions of Computer Equipment and Personal: A Firm-Level Analysis," Economics of Innovation and New Technology, Vol. 3: 201-217.

Loveman, Gary W. [1994], "An Assessment of the Productivity Impact of Information Technologies," in Allen, Thomas J. and Scott Morton, Michael S. (Ed.), Information Technology and the Corporation of the 1990s: Research Studies, Oxford University Press, pp. 84-110.

Malone, T. and Rockart, J. [1991], "Computers, Networks and the Corporation," Scientific American, Vol. 265(3): 128-136.

Mark, J. A. [1982], "Measuring Productivity in the Service Sector," Monthly Labor Review, (June).

Have McKersie, Robert. B. and Walton, Richard E. [1991], "Organizational Change," in Scott Morton (Ed.): The Corporation of the 1990s, Oxford University Press, pp. 244-277.

Morrison, Catherine J. and Berndt, Ernst. R. [1991], "Assessing the Productivity of Information Technology Equipment in U.S. Manufacturing Industries," National Bureau of Economic Research Working Paper No. 3582, (January).

Nelson, R. R. [1981], "Research on Productivity Growth and Productivity Differences: Dead Ends and new Departures," Journal of Economic Literature, Vol. 29: 1029-1064.

Noyelle, T. [1990], Skills, Wages, and Productivity in the Service Sector, Boulder, Colorado, Westview Press.

Oliner, Stephen D. and Sichel, Daniel E. [1994], "Computers and Output Growth Revisited: How Big is the Puzzle?" Brookings Papers on Economic Activity, 1994(2): 273-334.

Osterman, P. [1986], "The Impact of Computers on the Employment of Clerks and Managers," Industrial and Labor Relations Review, Vol. 39: 175-186

Parsons, D. J., Gotlieb, C. C. and Denny, M. [1990], "Productivity and Computers in Canadian Banking," University of Toronto Dept. of Economics Working Paper No. 9012, (June).

Porat, M. [1977], The Information Economy: Definition and Measurement, US Government Printing Office, Washington, DC.

Porter, M. E. and Miller, V. E. [1985], "How Information Gives You Competitive Advantage," Harvard Business Review, July-August, pp. 149-160.

Pulley, L. B. and Braunstein, Y. M. [1984], "Scope and Scale Augmenting Technological Change: An Application in the Information Sector," Juswalla and Ebedfield.

Roach, Stephen S. [1991], "Services under Siege: the Restructuring Imperative," Harvard Business Review 39(2): 82-92, (September-October).

Roach, Stephen S. [1989a], "Pitfalls of the New Assembly Line: Can Service Learn From Manufacturing?" Morgan Stanley Special Economic Study, New York, (June 22).

Roach, Stephen S. [1989b], "America's White-Collar Productivity Dilemma," Manufacturing Engineering , August , pp. 104.

Roach, Stephen S. [1987], "America's Technology Dilemma: A Profile of the Information Economy," Morgan Stanley Special Economic Study, (April).

Romer, Paul M. [1986], "Increasing Returns and Long-Run Growth," Journal of Political Economy, Vol. 94(5):1002-37

Romer, Paul M. [1987], "Crazy Explanations for the Productivity Slowdown," in Stanley Fisher (Ed.), NBER Macroeconomics Annual: 1987, MIT Press, Cambridge, MA.

Scherer, F. [1980], Industrial Market Structure and Economic Performance, Rand-McNally, Chicago, IL.

Schneider, K. [1987], "Services Hurt by Technology: Productivity is Declining," The New York Times, June 29: D1, D6.

Scott Morton, Michael S. (Ed.) [1991], The Corporation of the 1990s: Information Technology and Organizational Transformation, Oxford University Press.

Siegel, Donald [1994], "The Impact of Computers on Manufacturing Productivity Growth: A Multiple-Indicators, Multiple-Causes Approach," SUNY at Stony Brook Working Paper, (May).

Siegel, Donald and Griliches, Zvi [1992], "Purchased Services, Outsourcing, Computers, and Productivity in Manufacturing," in Griliches et al. (Ed.), Output Measurement in the Service Sectors, University of Chicago Press.

Snow, C. P. [1966], "Government Science and Public Policy," Science, Vol. 151: 650-653.

Stabell, C. B. [1982], "Office Productivity: A Microeconomic Framework for Empirical Research," in Office: Technology and People, Vol. 1: 91-106.

Strassmann, P. A. [1990], The Business Value of Computers: An Executive's Guide. New Canaan, CT, Information Economics Press.

Strassmann, P. A. [1985], Information Payoff: The Transformation of Work in the Electronic Age, Free Press, New York, NY.

Thurow, Lester [1987], "Economic Paradigms and Slow American Productivity Growth," Eastern Economic Journal, Vol. 13: 333-343.

Trajtenberg, Manuel [1990], "Product Innovations, Price Indices and the (Mis)measurement of Economic Performance," National Bureau of Economic Research Working Paper, No. 3261. (February).

Watts, L. [1986], "What Price Automation?" Northeastern University Alumni Magazine, pp. 21-24.

Weill, Peter [1992], "The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector," Information Systems Research, 3(4): 307-333.

Weitzendorf, T. and Wigand, R. [1991], "Tasks and Decisions: A Suggested Model to Demonstrate Benefits of Information Technology," Institute for Information Science Working Paper, Graz, Austria.

Wilson, Diane. D. [1995], "IT Investment and Its Productivity Effects: An Organizational Sociologist's Perspective on Directions for Future Research." Economics of Innovation and New Technology, Vol. 3: 235-251.

Zachary, G. P. [1991], "Computer Data Overload Limits Productivity Gains," Wall Street

Journal, November 11, pp. B1.