Lorin Hitt
MIT Sloan School, E53-334
Cambridge, MA 02139
617-253-6614
617-258-7579 (fax)
lhitt@mit.edu
Erik Brynjolfsson
MIT Sloan School, E53-313
Cambridge, MA 02139
617-253-4319
617-258-7579 (fax)
ebrynjo@mit.edu
May, 1995
Revised October, 1995
Copyright (c) 1995-96 Lorin Hitt and Erik Brynjolfsson
Published in MIS Quarterly, June, 1996, where it won the award for "Best Paper" for 1996. An earlier version of this paper appeared in the Proceedings of the Fifteenth International Conference on Information Systems under the title: "The Three Faces of IT Value: Theory and Evidence", where it won the awards for "Best Paper Overall" and "Best Paper on the Conference Theme".
Abstract
The business value of information technology (IT) has been debated for a number of years. While some authors have attributed large productivity improvements and substantial consumer benefits to IT, others report that IT has not had any bottom line impact on business profitability. In this paper, we focus on the fact that while productivity, consumer value and business profitability are related, they are ultimately separate questions. Accordingly, the empirical results on IT value depend heavily on which question is being addressed and what data are being used. Applying methods based on economic theory, we are able to define and examine the relevant hypotheses for each of these three questions, using recent firm-level data on IT spending by 370 large firms. Our findings indicate that IT has increased productivity and created substantial value for consumers. However, these benefits have not resulted in supranormal business profitability. We conclude that while modeling techniques need to be improved, these results are consistent with economic theory. Thus, there is no inherent contradiction between increased productivity, increased consumer value and unchanged business profitability.
Keywords: IT Productivity, Business profitability, IS Investment, Economic Theory, Consumer Surplus, Computers
Categories: AM, EF07, EI0205.04, GA01
Acknowledgments
This research has been generously supported by the MIT Center for Coordination Science, the MIT Industrial Performance Center, and the MIT International Financial Services Research Center. We would like to thank Chris Kemerer, Mary Pinder, Albert Wenger, and the referees for ICIS 94 and MISQ for helpful comments on earlier drafts of this paper. We are also grateful to International Data Group for providing essential data.
Biographical Information
Lorin M. Hitt is a doctoral student in information technology at the MIT Sloan School of Management. He received his Sc.B. and Sc.M. degrees in electrical engineering from Brown University. His research interests include assessing the productivity effects of information technology investments and using information economics and econometrics to understand the effects of IT on organizations and markets.
Erik Brynjolfsson is the Douglas Drane Associate Professor of Information Technology at the MIT Sloan School of Management. His research analyzes how information technology can transform the structures of markets and firms and assesses the productivity of information technology investments. He has written numerous articles in academic journals and served as the co-editor of special issues of Management Science and Journal of Organizational Computing. Professor Brynjolfsson holds degrees in Applied Mathematics, Decision Science, and Managerial Economics from Harvard and MIT. Before joining the MIT faculty, he directed a software development and consulting firm.
Questions about the business value of Information Technology (IT) have perplexed managers and researchers for a number of years. Businesses continue to invest enormous sums of money in computers and related technologies, presumably expecting a substantial payoff. Yet a variety of studies present contradictory evidence as to whether these expected benefits have materialized (Brynjolfsson 1993a; Wilson 1993). The debate over IT value is muddled by confusion over what question is being asked and what the appropriate null hypothesis should be. In some cases, seemingly contradictory results are not contradictory at all because different questions are being addressed. Research has been further hampered by the lack of current and comprehensive firm-level data on IT spending.
In this paper, we attempt to pinpoint the right questions regarding IT value and explicitly define the appropriate theoretically-grounded hypotheses. Now that detailed survey data on IT spending by several hundred large firms have been made available by the International Data Group (IDG), we can empirically examine each of these hypotheses using the same data set. Thus, our goals are: 1) to explain the theoretical relationships among the principal measures of IT's economic contribution, and 2) to apply the diverse models previously used to address these different measures to the same data set. In the process, we highlight some of the previous research findings in this area and go on to derive some implications for managers and researchers.
In interpreting past findings, it is useful to understand that the issue of IT value is not a single question, but is composed of several related but quite distinct issues:
1) Have investments in IT increased productivity?
2) Have investments in IT improved business profitability?
3) Have investments in IT created value for consumers?
The first question asks whether IT has enabled the production of more "output" for a given quantity of "inputs". The second considers whether firms are able to use IT to gain competitive advantage and earn higher profits than they would have earned otherwise. The final issue is concerned with the magnitude of the benefits that have been passed on to consumers, or perhaps reclaimed from them.
We argue that these three questions are logically distinct, each having different implications for how managers, researchers and policy makers should view IT investment. Because different researchers have used not only different methods, but also different data, it has been difficult to determine the cause of seemingly contradictory results.[1] We demonstrate that for the same data, IT appears to have increased productivity and provided substantial benefits to consumers, but there is no clear empirical connection between these benefits and higher business profits or stock prices. This indicates that at least some of the apparent discrepancies among earlier conclusions about IT value were not due to differences in data. Nonetheless, we show that there is no inherent contradiction in these results; they are all simultaneously consistent with economic theory. However, our findings do highlight the fact that the answers one gets will depend on the questions one asks and how one addresses them, even when the same data are used. Models matter.
The remainder of this paper is organized as follows: in Section II we review the existing literature and relevant theory, Section III presents an empirical analysis of the three approaches, Section IV discusses the results, and Section V concludes with a summary and implications.
Microeconomic theory and business strategy can provide useful foundations for assessing the benefits of IT. This section examines the relevant theory applied in many of the previous studies of IT value, and provides a guide to interpreting the various findings. In particular, three frameworks map consistently to three questions we raised in the introduction:
Issue Framework Productivity Theory of Production Business Profitability Theories of Competitive Strategy Consumer Value Theory of the Consumer
For over 60 years, the theory of production approach has been used extensively to evaluate the productivity of various firm inputs such as capital, labor and R&D expenditures (Berndt 1991). More recently, it has been used to assess IT investments. The theory posits that firms possess a method for transforming various inputs into output, represented by a production function. Different combinations of inputs can be used to produce any specific level of output, but the production function is assumed to adhere to certain mathematical assumptions.[2]
By assuming a particular form of the production function, it is possible to econometrically estimate the contribution of each input to total output in terms of the gross marginal product. This represents the additional output provided by the last dollar invested, and is distinct from the overall contribution, which is the average for all dollars invested.[3] Since firms will seek to invest in the highest value uses of an input first, theory predicts that rationally-managed firms will keep investing in an input until the last unit of that input creates no more value than it costs. Thus, in equilibrium, the net marginal product (analogous to marginal return on investment) for any input will be zero. However, because costs are positive, the gross marginal product must also be positive.
Thus, in equilibrium, the theory of production implies the following hypotheses:
H1a: IT spending has a positive gross marginal product (i.e. it contributes a positive amount to output, at the margin),
and
H1b: IT spending has zero net marginal product, after all costs have been subtracted.
These hypotheses are empirically-testable and deviations from them will require elaboration or modification of the basic theory and/or the underlying assumptions.
Several studies have employed methods based on the theory of production to evaluate IT productivity for firm- and industry-level data. Loveman (1994) found that gross marginal benefits did not deviate significantly from zero for a sample of 60 manufacturing divisions (1978-84 time period). Using more recent firm-level data for Fortune 500 manufacturing and service firms (1988-1992 period), Brynjolfsson & Hitt (1993; 1996) and Lichtenberg (1994) found gross marginal benefits of over 60%. As a practical matter, the marginal costs of IT will depend on factors such as the depreciation rate, which can be difficult to determine. Brynjolfsson and Hitt (1993; 1996) and Lichtenberg (1994) calculated net benefits using various assumptions about depreciation rates and found that net returns to IT were likely to be positive. In contrast, Morrison & Berndt (1990) explicitly estimated a cost function for 20 manufacturing industries over 1968-1986 and found that net marginal benefits were -20%. Because these studies examined different time periods and different data as well as different specifications, it is not obvious how to reconcile the results.
While the theory of production predicts that lower prices for IT will create benefits in the form of lower production costs for a given level of output, it is silent on the question of whether firms will gain competitive advantage and therefore higher profits or stock values. For that, we must turn to the business strategy field and the literature on barriers to entry.
As Porter (1980) has emphasized, in a competitive market with free entry, firms cannot earn sustainable supranormal profits because that would encourage other firms to enter and drive down prices. Although there is the possibility of exploiting an unusually profitable opportunity in the short run, long run accounting profits will be just enough to pay for the cost of capital and compensate the owners for any unique inputs to production (e.g. management expertise) that they provide.[4] Accordingly, if a firm has unique access to IT, then that firm may be in a position to earn higher profits from that access. On the other hand, IT will not confer supranormal profits to any firm in an industry if it is freely available to all participants. In this case, there is no reason to expect, a priori, that a firm spending more (or less) on IT than its competitors will have higher profits. Instead, all firms will use the amount of IT they consider optimal in equilibrium, but none will gain a competitive advantage from it. This is consistent with the argument of Clemons (1991) that IT has become a "strategic necessity", but not a source of competitive advantage.
The only way IT (or any input) can lead to sustained supranormal profits is if the industry has barriers to entry. Bain (1956) broadly defined a "barrier to entry" as anything that allows firms to earn supranormal profits, such as patents, economies of scale, search costs, product differentiation or preferential access to scarce resources. There are two possible ways in which IT value is related to barriers to entry. The first is that in industries with existing barriers to entry, it may be possible for firms in a particular industry to increase profits through the innovative use of IT, provided the barriers to entry remain intact. Second, the use of IT may raise or lower existing barriers or create new ones, thus changing the profitability of individual firms and industries.
The impact of IT on barriers to entry is ambiguous. On one hand, it may reduce economies of scale and search costs (Bakos 1993), thereby leading to lower industry profits. On the other hand, it may also enable increased product differentiation (Brooke 1992), supporting higher profits. Furthermore, if particular IT investments cannot be replicated by other firms, then firms can increase their own profits while industry profits may be either increased or decreased. However, there are relatively few IT investments which provide sustainable advantage of this sort (Clemons 1991; Kemerer and Sosa 1991). On balance, any or all of the above conditions may hold for a given industry, so competitive strategy theory does not clearly predict either a positive or negative relationship between IT and profits or market value (which, after all, represent the expected discounted value of future profits). This implies the following null hypothesis:
H2: IT spending is uncorrelated with supranormal firm profits or stock market value.
Much of the previous research in this area has examined correlations between measures of IT spending and measures of profitability (Ahituv and Giladi 1993; Dos Santos, Peffers and Mauer 1993; Markus and Soh 1993; Strassmann 1985; 1990). Some studies have attempted to examine direct correlations between IT spending and profitability ratios (Ahituv and Giladi 1993) while others examine how IT influences intermediate variables which in turn drive profits (Barua, Kriebel and Mukhopadhyay 1995; Ragowsky, Neumann and Ahituv 1994). In general, these studies find little direct correlation between IT spending and business profitability, although some models are plagued by relatively low predictive power overall and, with the exception of the paper by Barua et al., have generally not controlled for many industry-specific or firm-specific factors other than IT spending.
A third approach, also grounded in microeconomic theory, can be used to estimate the total benefit that a given purchase confers to consumers. The demand curve for a product represents how much consumers would be willing to pay (i.e., the benefit they gain) for each successive unit of that product. However, in practice they need only pay the market price, so consumers with valuations higher than the market price retain the surplus. By adding up the successive benefits of each additional unit of the good, the total benefit can be calculated as the area between the demand curve and the supply (or marginal cost) curve. Schmalensee (1976) further showed that in a competitive industry, the surplus from an input to production will be passed along to consumers, so the area under the demand curve for an input such as IT will also be an accurate estimate of consumer surplus. When an industry is not perfectly competitive, the area under the demand curve will generally understate surplus, so this computation can be viewed as a lower bound. The computation of consumer surplus from the demand curve is illustrated in figure 1.
The major difficulty with this approach is determining the locus of the demand curve.[5] Fortunately, in the case of IT, a natural experiment has occurred because the cost of computer power has dropped by several orders of magnitude. By examining how the actual quantity of IT purchased has changed over time, we can trace out an estimate of the demand curve and calculate the total consumer surplus.
As the price of IT declines, benefits are created in two ways: 1) a lower price for investments that would have been made even at the old price, and 2) new investments in IT that create additional surplus (see figure 2). In competitive equilibrium, a decline in the price of an input will lead to an increase in spending on that input and an increase in consumer surplus. If firms are making optimal investments, the additional consumer surplus should be no less than the cost of these investments, suggesting the following simple hypothesis:
H3: The consumer surplus created by IT is positive and growing over time.
The literature on the consumer surplus from IT is somewhat more sparse than the others. In addition to Bresnahan (1986), who studied the effects of IT spending on the financial services industry and found substantial benefits between 1958 and 1972, this method has been applied to data on the entire U.S. economy by Brynjolfsson (1993b), who estimated that computers generated approximately $50 billion in consumer surplus in 1987.
Productivity is most closely associated with the process of value creation. If IT investments are productive, then more output is realized for a given quantity of input, leading to increased value that can be distributed among producers, suppliers, customers or other economic agents. Business profitability and consumer surplus are also affected by value redistribution. If a firm is able to use IT to create and retain value, then IT investment can lead to increased business profitability. Alternatively, a firm can increase profitability with IT by redistributing value from customers or suppliers (i.e. using information to improve price discrimination between different types of consumers, foreclosing competition, or driving down prices paid to suppliers) without increasing the size of the total value "pie".[6] In this sense, business profitability is distinct from productivity -- productive IT can facilitate higher business profitability but is neither necessary nor sufficient. Consumer surplus represents the other side of business value. To the extent that value is being created by IT without being captured by firms, consumers will be receiving the benefits. By the same token, if firms use IT for value redistribution, consumer surplus may decrease as business profitability is increased.
The net effect of IT on these three factors thus represents a complex interplay between the types of IT investment, the ease with which these investments are copied by competitors, the nature of competition within an industry, and other industry-specific factors such as consumer demand. Under normal competitive conditions where managers are making good or optimal investments in productive technologies, consumer surplus and productivity will generally increase together. The same is not true for business profitability, where short-term profit increases from new technology will typically be eliminated by the increased competition that the new technology facilitates.
In order to investigate the effects of IT investment, we apply each of the approaches described in Section II to the same data set. It is then possible to examine how the three approaches are interrelated without the potential confusion created by comparing different studies with different data. By the same token, for each approach, we attempt to apply the same model used in the previous literature for that approach. Our results can thus be more easily compared with prior work.[7] This strategy should help highlight which differences are due to data, and which are due to models.
The data used for this analysis comprise an unbalanced panel of 370 firms over the period 1988-1992 with 1109 data points overall,[8] out of a possible 1850 data points (5x370) if the panel were complete. We obtained computer spending from IDG's annual surveys of IT spending by large firms (top half of the Fortune 500 manufacturing and service listings) from 1988-1992. These data were matched to Standard & Poor's Compustat II database to obtain values for the output, capital, labor, industry classification, and other financial data. We augmented these data with price indices from a variety of sources to remove the effects of inflation and allow inter-year comparisons on the same basis. The precise variable definitions and sources are shown in Table 1 and sample statistics for the key variables are given in Table 2.
Ideally, we would like to incorporate all components that are considered IT into our measure. A broad definition could include hardware expenses (computers, telecommunications, peripherals), software expenses (in-house or purchased), support costs and also complementary organizational investments such as training or the costs of designing and implementing IT-enabled business processes. Unfortunately, detailed data on the totality of IT expenses is generally not available.
Our measure of information technology, called "IT Stock", is comprised of two components. The first component is Computer Capital, which represents the total dollar value of central processors (mainframes, minicomputers and supercomputers) as well as the value of all PCs currently owned by the firm. The second component is IS Labor which is the labor portion of the central IS budget (total budgets cannot be used because capital expenditure on IT would be counted both in the budget and in Computer Capital). To create a single measure of "IT" we make two assumptions: 1) IS Labor represents a type of capital expenditure that produces an asset that lasts, on average, 3 years (e.g. software, templates, training); and 2) current IS spending is a good approximation of IS spending in the last 3 years. This allows us to convert the annual flow of IS labor to a "stock" of IS labor, comparable to Computer Capital which is an accumulation of spending over time. The actual computation is:
IT Stock = Computer Capital + 3 * IS Labor
where the factor of three represents the assumed service life of the asset created by IS Labor. This measure has been used in earlier work to study the relative contributions of IT in various subsectors of the economy (Brynjolfsson and Hitt 1995).
To the extent that IT Stock fails to capture the full range of IT expense, our results can only be interpreted as applying to the components we are able to measure. However, if the unmeasured components are correlated with the parts of IT we incorporate in our analysis then our estimates will partially reflect this broader definition. This is likely to be true for other types of hardware or software expenditures, although it may be less true for investments in less tangible components such as "organizational technology". An extensive discussion of the impact of omitted IT spending can be found in Brynjolfsson and Hitt (Brynjolfsson and Hitt 1996) in the context of productivity analysis.
There are a number of other limitations of this data set. First, the IDG data are self-reported, which could lead to errors in reporting and sample selection bias. However, the large size of our sample should help mitigate the impact of data errors. The high response rate (68%) suggests that the sample is likely to be reasonably representative of the target population and we find that the included firms do not appear to differ substantially from the target population in terms of size or profitability measures (return on equity, return on assets, total shareholder return). In addition, the total annual values are generally consistent with a survey done by CSC/index (Quinn, Craumer, Weaver, Buday and Waite 1993) and aggregate computer investment data by the Bureau of Economic Analysis.
Second, we use estimation procedures for some items; particularly the value of PCs and terminals and labor expenses (see Table 1). However, we tested a range of alternative estimates for these values and found that the overall results were essentially unchanged. Finally, the three year average life assumption for the IT capital created by IS labor is only an approximation. It is chosen to be between the life of Computer Capital assumed by the Bureau of Economic Analysis (7 years), and the life of IS labor if it were only an annual expense (1 year) (Bureau of Economic Analysis 1993). This assumption appears reasonable because the components of IS labor span a range of activities such as software development, software maintenance and enhancement, user support, and hardware installation that range in useful life from less than a year to the life of a system.[9] In general, the coefficient estimates in the productivity analysis do not vary much as this assumption is changed over the range of 1 to 7 years, although the implied rates of return to IT investment do change somewhat. In other analyses, the coefficients do change somewhat because the mean of IT stock changes, but overall, the basic conclusions remain the same.
We apply the production function approach to this data set using the same methods employed by previous researchers (Brynjolfsson and Hitt 1993, Lichtenberg 1994; Loveman 1994). Using the Cobb-Douglas production function, we relate three inputs, measured in constant 1990 dollars: Total IT Stock (C), Non-computer Capital (K) and Labor (L) to firm Value Added (V).[10] We also use dummy variables to control for the year the observation was made (Dt), and the industry (2-digit SIC level) or sector of the economy in which a firm operates (Dj):
After taking logarithms and adding an error term, we have the following estimating equation:
In this specification, [[beta]]1 represents the output elasticity of IT Stock, which indicates the percentage increase in output provided by a one percent increase in IT Stock. Dividing the elasticity by the ratio of IT Stock in Value Added provides an estimate of the gross marginal product of IT (which can be interpreted as a rate of return before costs of investment are subtracted).
Unbiased estimates of the parameters can be obtained by Ordinary Least Squares (OLS) provided the error term is uncorrelated with the regressors. However, following Brynjolfsson & Hitt (1993) we also employ Iterated Seemingly Unrelated Regression (ISUR) to potentially enhance estimation efficiency by incorporating the fact that the productivity of particular firms is likely to be correlated across time. Furthermore, we test the assumption that the error term is uncorrelated with the regressors by computing Two Stage Least Squares estimates (2SLS) with lagged values of the independent variables as instruments.[11]
The results of this analysis are presented in Table 3. When all industries and years are estimated simultaneously, we find that the output elasticity of IT Stock is .0883, implying a gross marginal product of approximately 94.9%.[12] The elasticity is similar for the ISUR analysis (using sector dummy variables) although the standard errors decrease somewhat, indicating a gain in estimation efficiency. The gross marginal product for other capital and labor are 7.8% and 1.22 respectively, which is approximately what would be expected for inflation-adjusted estimates of these figures,[13] and is not inconsistent with estimates of production functions performed by other researchers on a comparable set of firms (e.g. (Hall 1993a). Considering the standard error for our estimate of the gross marginal product of IT Stock, we find strong support for the hypothesis that IT has contributed positively to total output (p<.001). This is consistent with hypothesis H1a. To calculate the net returns, it is necessary to subtract an estimate of the annual cost of capital. Strikingly, even if we assume that capital costs are as high as 69% per year,[14] we can reject the hypothesis that the net return to IT Stock is zero (p<.05), contradicting hypothesis H1b.
It is possible that the high rates of return are a result of a misspecification of the productivity equation. Rather than IT causing increased output, unexpectedly large output could motivate increased investment in IT. If this were the case, our estimates could show positive productivity benefits of IT even if the actual contribution were zero. This type of bias can be addressed by estimating the equation by Two Stage Least Squares, although this results in a loss of about one third of the sample because observations are needed for the same firm in adjacent years. While our 2SLS estimates appear somewhat lower than our base OLS and ISUR estimates, about half the difference is accounted for by sample differences (see Table 3). After accounting for these changes we are unable to reject the null hypothesis that our estimates of the elasticity of IT Stock are unbiased using the Hausman specification test (Hausman 1978), although the coefficient on Non-Computer Capital does change significantly. Overall, this set of estimates suggests that our basic results regarding the high marginal product of IT are not due to simultaneity issues.
These results are consistent with similar analyses by Brynjolfsson and Hitt (Brynjolfsson and Hitt 1996) and by Lichtenberg (1994), which also provide further methodological discussion and robustness checks. However, this analysis provides an exact baseline that can be used as a comparison against the business profitability results presented below to highlight the differences caused by modeling changes as opposed to data differences.
Our business profitability model follows in the tradition of the existing IT literature on business value (Ahituv and Giladi 1993; Alpar and Kim 1990; Harris and Katz 1989; Strassmann 1990; Weill 1992). While there is not a single standard form for the estimating relationship, we begin by estimating a simple correlation, essentially replicating Strassmann's widely cited model (1990) on our data and then extending this model to include additional control variables. Following the established convention in this literature of using ratios of IT to various size measures, we assume firm profitability is a function of the ratio of IT Stock to firm employees.[15] Thus we can write:
The three measures of profitability (see Table 1 for precise definitions) that are considered here have been employed in past research: 1) Return on Assets (ROA) (Barua et al. 1995; Cron and Sobol 1983; Strassmann 1990; Weill 1992) which measures how effectively a firm has utilized its existing physical capital to earn income; 2) Return on Equity (Alpar and Kim 1990) which provides an alternative measure of how effectively a firm has utilized its financial capital, and is algebraically related to "Economic Value Added", a measure attracting increasing interest in the managerial community (Tully 1993); and 3) Total shareholder return (Dos Santos et al. 1993; Strassmann 1990) which theoretically furnishes the discounted value of future profits.
For our base case analysis we replicate the results of Strassmann (1990) by computing the correlation of IT and profitability for each year of our sample. Overall, this simple analysis is generally inconclusive, although, if anything, it suggests the possibility of a negative relationship. In the year by year regressions, only four of the fifteen coefficients are statistically significant but the majority are negative. Because we have multiple years available, we also calculate the analysis pooling all the years and including control variables for the year (Table 4b). The pooled analysis shows significant negative effects for two measures (ROA, Total Return). However, the low R2 in all three regressions indicate that these analyses explain a relatively small portion of the profitability measure variance. Furthermore, the relatively high R2 in the total return regression is primarily a result of the year dummy variables which explain 21% of the variance of this measure. The magnitudes of the coefficients on IT suggest that even large changes in IT have small effects on profitability. For instance, a 10% change in the IT Stock to employees measure implies only a 0.09% change in ROA. One interpretation of these results is that firms are using approximately the correct amount of IT: by the envelope theorem, no first order improvements can be made by raising or lowering spending on an input which is at its optimal level.
Because these estimates do not control for barriers to entry or other factors that may affect returns independent of the use of IT, they present the possibility of a spurious correlation. Two different approaches have been employed for extending these models. Weill (1992) and Barua, Kriebel and Muhkadopadhyay (1995) extend this basic model by incorporating firm specific variables that are unique to a particular theory or group of firms under study. Because these variables are not publicly available, we are unable to replicate this approach. An alternative, found in the broader literature on business profitability, is to identify generic variables that are likely to affect profitability and incorporate these variables in the model. Capon, Farley and Hoenig (1990) survey this literature and summarize the key predictors of business profitability.
We thus incorporate all the variables identified by Capon, Farley and Hoenig (1990) as having a significant impact on firm profitability that are available for the firms in our data set. A number of these relate to characteristics of particular industries such as concentration, barriers to entry or size. Assuming these factors change relatively slowly, the effects of these industry-level variables can be incorporated by including dummy variables for each industry. Other variables identify characteristics of particular firms. From these, we are able to include: growth, market share, capital investment, and debt for almost all the firms in our sample, and research and development expenditures (R&D) for about half of the firms (further description of these variables is included in Table 1 and summary statistics appear in Table 2). Several other potentially informative variables are either completely unavailable from public sources (e.g. relative price, quality) or available for very few firms in our sample (e.g. advertising).
The results of this extended analysis are presented in Table 5 (without R&D). As in the earlier analysis with industry controls, none of the IT coefficients are significant, although we do find significant effects of many of the control variables. Sales growth has a positive contribution, while debt (as measured by Debt to Equity Ratio) has a negative effect, both of which are consistent with prior research. The signs are mixed for market share and capital investment and are not always significant. Compared with the previous analysis, while the overall fit is improved, the t-statistics on the IT coefficient appear to drop (for example, in the total return regression the t-statistics drop from 2 to .4). This suggests that many of the significant effects found in the earlier models may be partially caused by a failure to adequately control for firm differences. When R&D is included in the regression (reducing the usable sample to 465), we find that R&D has a consistently positive effect (significant in the ROA and ROE regressions), but that there is no change in the signs or significance of other coefficients. We also find similar results for a related specification in which a dummy variable is included for each firm (rather than industry) which controls for all slow-changing firm characteristics.
Taken in totality, the results show little evidence of an impact of IT on supranormal profitability, which is consistent with our hypothesis H2, and much of the prior literature. However, it is interesting that the majority of our analyses show negative but insignificant effects suggesting the possibility of an overall negative effect of IT on profitability. It should be stressed that these models (both here and in the broader literature) are based on less rigorous theory than are the production function models, and, therefore, the failure to find a strong result may simply reflect inadequate modeling. Two alternative perspectives for interpreting these results will be presented in the Discussion section below.
In order to estimate consumer surplus for our sample, we use the index number method proposed by (Caves, Christensen and Diewert 1982) and applied by Bresnahan (1986). For a general utility function (the translog), the increase in consumer surplus between two periods (t, t+1) is a function of the ratio of IT Stock to Value Added (s), the Price of IT Stock (p) and Value Added (V) in the reference year, as follows:
The intuition behind this equation is that it represents the area under the demand curve between two price points. To apply this equation, we further assume that the quantity of IT can be adjusted between years, by purchasing more or less depending on prices. The primary difficulty with this analysis is that it requires two important assumptions: 1) that the increased purchases of IT are caused only by a decrease in prices and not by an exogenous shift in the entire demand schedule, and 2) that benefits received by upstream producers from their IT purchases are "competed away" and ultimately received by end consumers in the final product markets. The implications of these assumptions is discussed further in the conclusion, but overall, minor changes in these assumptions are unlikely to alter the general conclusions of this section.
Since IT Stock is a composite measure of two types of spending with different price changes over our sample period, it is important to understand the relative impact of each. The price of computers has been dropping dramatically over this period, on the order of 20% per year. However, the price of IS labor (as well as the quantity purchased) has been flat or increasing in real terms over this same period. The net impact of these two changes is that despite the rapid decrease in the cost of computers, the overall cost of IT has been declining at a much slower rate, averaging only 4% per year.
We compute annual surplus for the firms in our sample as shown in Table 6. Overall, we find that IT Stock has created significant value for consumers. Over our sample period, the price change in IT created $14.5 billion ($3.6 billion per year) in value above the cost of IT investment for the firms in our sample. This is consistent with hypothesis H3 and is proportional to the consumer surplus calculation for the economy as a whole performed by Brynjolfsson (1993b). It should be noted that the above surplus calculation follows the convention of assuming the net marginal benefit of the input (IT) is zero. Our findings of excess return to IT in the productivity analysis would suggest that consumer surplus is substantially larger.
To summarize the empirical results, we find evidence that IT investment has had a significant impact on firm output. Our production function estimates of the productivity of IT Stock suggest a gross marginal product of nearly 95%, implying positive net returns for most estimates of the cost of capital. These results are consistent with recent studies on IT and productivity by Brynjolfsson and Hitt (1996) and Lichtenberg (1994). When examining profitability as the dependent variable, we find no evidence that IT use leads to supranormal profits, and even some evidence of a small negative impact on profitability. This is similar to previous research which typically found no relationship between IT and business profitability (Strassmann 1990; Barua, Kriebel and Mukhopadhyay, 1991; Ahituv and Giladi 1993). Finally, using the consumer surplus approach, we estimate the total benefit to consumers to be substantial. The increase in surplus (above costs) is between $2 billion and $7 billion per year. This is consistent with previous approaches to this issue that used industry- or economy-level data (Bresnahan 1986; Brynjolfsson 1993b).
It is important to recognize that these results apply to the "average" firm. While IT appears to have been productive for the average firm, many firms undoubtedly made unproductive investments in IT. Similarly, while we did not discern any contribution to supranormal profits for the average firm, the high standard errors of the estimates suggest that some firms are obtaining significant competitive advantage while others are not. While the data are not sufficiently detailed to distinguish reliably characteristics of "winners" and "losers", this variation is of significant interest for future research.
The most striking aspect of the empirical results is that IT Stock appears to be correlated with substantial increases in net output and consumer surplus, but unrelated to supranormal business profitability. These findings are based on data from the same firms, over the same time period, using the same measures of IT, so the conventional explanation of incomparable data sets does not apply. Below, we put forth two possible explanations for this finding, one which takes the empirical results at face value and is based on an elaboration of the theory, and one which stresses the need for new econometric models and data.
The theoretical discussion in section II asserts that profits, productivity, and consumer value are not equivalent. Information technology is commonly characterized as reducing the coordination costs involved in locating appropriate, low cost products and services and switching production to new suppliers (Malone 1987). Such an increase in efficiency (and therefore productivity) can be shown to intensify competition by lowering barriers to entry and eliminating the market inefficiencies which enable firms to maintain a degree of monopoly over their customers (Bakos 1991). One effect of this increased competition is to reduce prices paid for firm output. A second effect is that firms will work to squeeze out "fat" by reducing their consumption of other inputs such as ordinary capital and labor (Caves and Krepps 1993). Because productivity calculations are performed after removing the effects of price changes, the reduction in input utilization and any direct contributions of IT to output will appear as productivity increases regardless of changes in output prices. However, the lower price paid for output will directly reduce profitability, possibly more than any cost savings achieved through rationalization.[16] Thus, under this theoretical scenario, the result can be higher productivity and consumer value, but lower profits.
There is some evidence that this theoretical story is consistent with business practice. In an in-depth study of the banking industry, Steiner and Teixeira (1991) found that while IT seemed to be creating enormous value, it was simultaneously intensifying competition and destroying profitable businesses by enabling entry and radically lowering prices. This reduction in prices coincided with massive layoffs in the financial services sector. Clemons and Weber (1990) discovered a similar outcome in their analysis of the "big bang", which introduced a computerized system for matching buyers and sellers in London's stock market. It is important to note that the fundamental technologies involved (e.g. ATMs and automated stock trading) were ultimately available to all competitors in an industry, so investing firms were unable to appropriate the full value they were creating.[17] Jensen (1993) makes a related argument about how technology-based productivity improvements in the tire industry created massive overcapacity, consolidation and exit from the industry for a number of firms. However, in each of these cases, large benefits have been created for consumers. Thus, there is some theoretical and anecdotal support for our econometric finding that IT can create value (in terms of productivity and consumer surplus gains) without improving profits, although a definitive finding is dependent on specific competitive conditions which cannot be fully examined in this study.
The issues of measurement and modeling shortcomings are probably the most frequently cited problems with empirical research. By considering over 1100 observations and triangulating on IT value using three modeling approaches, we may be able to mitigate the measurement problem somewhat. However, we still believe modeling weaknesses cannot be ruled out as explanations for the results of each of our models.
First, a key assumption of the production function approach is that inputs "cause" output. Yet, it may also be true that output "causes" increased investment in inputs, since capital budgets are often based on expectations of what output can be sold. If this is the case, we may overstate the contribution of IT, although without a detailed model of the reverse causality, we cannot estimate the magnitude of this bias. While we did not find direct evidence of such simultaneity in our Hausman tests, this may simply reflect the inadequacy of our instrument list.
Second, while the gross returns to IT appear to be very high, the net returns are much more difficult to calculate, especially in light of the fact that significant maintenance "liabilities" may be created whenever computer projects are undertaken (Kemerer and Sosa 1991). While we are able to reject the hypothesis that the net return to IT Stock is zero, additional unmeasured expenses (such as IS spending outside the central department or costs of IT related organizational change) could reduce the net return to close to zero. This would be consistent with economic equilibrium as well as the lack of correlation between IT and supranormal business profits. This suggests that future research should seek to identify "hidden" costs (which may be organizational) not currently considered part of the costs of IT. At the same time, other factors such as options value might lower the economic cost of IT capital, and unmeasured benefits from IT, such as greater product quality may not have been fully reflected in our estimates of IT's gross marginal product. In addition, the cost of IT may be partially driven by our assumption of the average life of IS Labor, although earlier studies have found similar results for only the Computer Capital portion of IT Stock. The bottom line is that the precise net return cannot be determined with certainty.
Third, the consumer surplus approach assumes that the demand curve is stable over time, so that increases in the quantity purchased can be directly attributed to declines in price. In reality, it is likely that diffusion of the computer "innovation" would have led to some increase in quantity even if prices had not declined, although Gurbaxani & Mendelson (1990) found that by the 1980s, the vast majority of the increase in the quantity of computers purchased could be attributed to price declines, not diffusion. In any event, as shown by Brynjolfsson (1993b), our consumer surplus estimates are likely to be underestimates to the extent they do not account for diffusion, and, therefore, our finding of significant value would only be strengthened if diffusion were explicitly modeled.
We are most concerned by the fourth modeling weakness: the possibility that the insignificant results in the profitability regressions may simply be due to the fact that these models are comparatively blunt instruments. Past models on smaller data sets have usually been unable to explain more than about 10%-20% in the variance of profitability measures, as measured by R2. This also holds true for our base analysis, although we are able to obtain some improvement by adding additional control variables.[18] As noted by Ahituv and Giladi (1993), IT is just one item in a multitude of factors that affect firm returns, and many of these other factors are not controlled for in the model. This means that the profitability regressions will tend to have low statistical power, and will not necessarily be able to detect effects even when they are present.
Furthermore, as we improved the fit of the models of profitability by adding commonly-used control variables, the IT effects only became more insignificant. This suggests that there may be omitted variables in this analysis which are correlated with the use of IT. Failure to identify these variables can lead to biased estimates. In addition, many of the controls commonly used on the literature such as sales growth and market share, are likely to be partially endogenous in the sense that they are not chosen by firms independently but are a product of other decisions. As with omitted variables, treating endogenous variables as independent variables in a regression leads to biased estimates. In sum, without a theoretically-based and complete model of the relationship between firm inputs and profitability it is more difficult to draw conclusions about the "true" relationship between IT and business profitability than it is to make inferences about the relationship between IT and output or IT and consumer surplus.
The question of IT value is far from settled. Indeed, one advantage to the comparative approach we have taken is that the existing gaps in knowledge become more apparent. For instance, our analysis underscored the relatively low power of the commonly used models of IT's effect on business profitability, and we presented some possible steps that can be taken to improve this situation.
Of equal importance, we separated the issue of IT value into three dimensions: the effect of IT on productivity, the effect of IT on business profitability, and the effect of IT on consumer surplus. Our empirical examination confirmed that, like any multidimensional object, IT's value can look different depending on the vantage point chosen. While we found evidence that IT may be increasing productivity and consumer surplus but not necessarily leading to supranormal business profits, we also showed that there is no inherent contradiction in the idea that IT can create value but destroy profits.
From a managerial perspective, it is important to understand how investment in IT affects the bottom line. Our theoretical discussion suggests that it is possible for firms to realize productivity benefits from effective management of IT, without seeing these benefits translate into higher profitability. This theoretical prediction is also borne out by our empirical analysis. Taking the theory literally, our profitability results suggest that, on average, firms are making the IT investments necessary to maintain competitive parity but are not able to gain competitive advantage.
This analysis suggests two potential insights for managers. First, when cost is the central strategic issue in an industry, our productivity results suggest that IT investment may be one way to pursue a cost leadership strategy, provided that the cost reductions cannot be emulated by other firms. However, for industries where cost is not the central strategic issue or where there are few barriers to adoption of IT, firms are unlikely to create lasting competitive advantage simply by spending more on IT. This raises the second issue: managers seeking higher profits should look beyond productivity to focus on how IT can address other strategic levers such as product position, quality, or customer service. While IT can potentially lower the cost of providing these services, attaining competitive advantage may involve using IT to radically change the way products or services are produced and delivered in a way that cannot be duplicated by competitors. This may be possible by leveraging existing advantages with IT or using technology to target other segments of the industry where competition is less intense. The key to improving business profitability may lie less in achieving productivity gains, and more in pairing the benefits of IT with an available market opportunity. Again, our results on business profitability suggest that, on average, IT spending alone is not determinative of success.
From a research perspective, by clarifying the issues and results in the existing literature on IT value, we hope to provide a mechanism for extending this literature substantially in the future. Because the value of IT was unknown, most of the previous literature focused on estimating the overall contribution of IT. Little is known about the distribution of benefits across individual firms, what characteristics of firms and industries determine which types of IT investment are productive, and which firms are effective or ineffective users of IT. Future research should go beyond estimating the "average" effects of IT and focus on differentiating successful and unsuccessful strategies. By identifying "best practices" either in terms of specific characteristics or as overall strategies of specific firms, we can provide managers with the information they need to fully exploit the value of IT.
Figure 1: Illustration of consumer surplus as the area between price and demand.
Figure 2: Components of Added Surplus - Increased value on existing units and added value from increased purchases.
As the price drops from P0 to P1, the following things occur: there is an increase in the quantity demanded from Q0 to Q1, there is added surplus due to the lower price on existing purchases (shaded area 1), and there is added surplus from the increase in purchases (shaded area 2).
Variable Computation Source
Output Gross Sales deflated by Output Price (see below). Compustat
Value Added Output minus non-labor expense. Non labor expense is Compustat
calculated as total firm expenses (excluding interest,
taxes and depreciation) divided by Output Price less
Labor (see below).
IT Stock Calculated as Computer Capital plus three times IS Calculation
Labor (see below).
Computer Capital Market value of central processors plus value of PCs IDG Survey
and terminals. Deflated by Computer price (see
below). Average value of PC determined as weighted
average of PC price from Berndt and Griliches (1990)
and value of PC from IBM. Resulting estimate is $2.84
K in 1990 dollars.
Non-Computer Deflated Book Value of Capital less Computer Capital Compustat
Capital as calculated above (for deflator see below).
IS Labor Labor portion of IS Budget. Deflated by Labor Price IDG Survey
(see below).
Labor Labor expense (when available) or estimate based on Compustat
sector average labor costs times number of employees
minus IS Labor. Deflated by Labor Price (see below).
Industry Primary industry at the 2-digit SIC level. Compustat
Total Price change plus accumulated dividends divided by Compustat
Shareholder initial price.
Return
Return on Equity Pretax income divided by total shareholders equity. Compustat
Return on Assets Pretax income divided by total assets. Compustat
Computer Price Gordon's deflator for computer systems - extrapolated (Gordon 1993)
to current period at same rate of price decline
(-19.7%/yr.).
Output Price Output deflator based on 2-digit industry from BEA Bureau of
estimates of industry price deflators. If not Economic
available, sector level deflator for intermediate Analysis 1993)
materials, supplies and components.
Labor Price Price index for total compensation. (Council of
Economic
Advisors 1992)
Capital Price GDP deflator for fixed investment. Applied at a (Council of
calculated average age based on total depreciation Economic
divided by current depreciation. Advisors 1992)
Sales Growth One year change in sales. Compustat
Capital Capital investment as percentage of total assets. Compustat
Investment
Market Share Total sales divided by industry total sales at the Compustat
2-digit SIC level. Industry total sales were computed
by adding up all firms in Compustat that report a
particular 2-digit primary SIC.
Debt to Equity Book value of total debt divided by book value of Compustat
total equity.
Research and An accumulation of annual R&D expense calculated by a Compustat
Development procedure used by Hall (1990). Represents capitalized
Stock value of R&D conducted over 20 years.
Variables Mean Std. Deviation Value Added $3.27 Bn $4.98 Bn IT Stock $304 mm $ 626 mm Non-Computer $8.84 Bn $ 13.9 Bn Capital Labor Expense $1.78 Bn $ 2.85 Bn Return on Assets 7.55% 7.37% Return on Equity 19.5% 19.5% Shareholder Rtn. 12.1% 27.6% ITRATE (IT $6.84 $6.52 K/employee Stock/Employees) K/employee Capital 7.38% 7.02% Investment (Cap. Inv./Assets) Sales Growth 5.29% 10.8% Market Share 6.3% 9.3% Debt (Debt/Equity 96.7% 93.1% Ratio) R&D Stock $1.94 Bn $3.81 Bn
Table 3: Production Function Analysis
OLS ISUR OLS on 2SLS
Estimates Estimates 2SLS Sample
IT Stock .0883* .0897* .0696* .0479*
(.0118) (.00920) (.00940) (.0219)
Non-Computer Capital .212* .225* .181* .128*
(.0125) (.00864) (.0159) (.0269)
Labor .663* .630* .725* .812*
(.0231) (.0112) (.0216) (.0415)
Dummy Variables Industry & Sector & Industry & Industry &
Year Year Year Year
N 1109 1109 765 765
R2 97.2% 94.2-95.1% 97.0% 97.0%
* - p<.05
Heteroskedasticity-consistent standard errors used for OLS and 2SLS
Multiple R2 measures are presented for ISUR because a separate R2 is computed for each year.
Return on Return on Total
Assets Equity (1 Return (1
(1 Year) Year) Year)
1988 -* - +
1989 -* - -
1990 -* - -*
1991 - - -
1992 - + -
*
- p<.05
Return on Return on Total
Assets Equity (1 Return (1
(1 Year) Year) Year)
IT Stock per -.00130* -.000668 -.00256*
Employee (.000306) (.000700) (.00122)
Dummy Variables Industry Industry & Industry &
& Year Year Year
N 1109 1109 1109
R2 3.3% 2.5% 21.8%
Profit Measure 7.6% 7.4% 19.5% 19.5% 12.1% 27.6%
Mean Std.
Deviation
* - p<.05, Heteroskedasticity-consistent standard errors in parenthesis
Table 5. Profitability Regressions with Extended Firm-Specific Control Variables
Return on Return on Total
Assets Equity (1 Return (1
(1 Year) Year) Year)
IT Stock per -.000402 .000158 -.000536
Employee (.000360) (.000845) (.00119)
Capital Intensity .204* .469* (.166) -.256
(.0536) (.0279)
Debt/Equity Ratio -.0235* -.0194* -.0358*
(.00230) (.000112) (.0106)
Market Share -.000257 .00313 -.0318
(.0336) (.103) (.158)
Sales Growth .138* .359* -.467*
(.0180) (..0614) (.0861)
Dummy Variables Industry Industry & Industry &
& Year Year Year
N 1045 1045 1045
R2 46.2% 35.2% 30.3%
*
- p<.05, Heteroskedasticity-consistent standard errors in parenthesisTable 6: Consumer Surplus Analysis
(Constant 1990 dollars)
Year IT Stock Value Added IT as a Price of Surplus
share of IT* (1990=1) using 1992
Value Added Output
1988 $48.3 Bn $677.0 Bn 7.14% 1.064 na
1989 $52.9 Bn $639.0 Bn 8.27% 1.031 $2.09 Bn
1990 $74.5 Bn $861.9 Bn 8.64% 1.000 $2.18 Bn
1991 $88.6 Bn $844.2 Bn 10.5% .959 $3.37 Bn
1992 $98.1 Bn $848.5 Bn 10.6% .892 $6.85 Bn
*
- IT price is a current period dollar weighted average of the price of Computer
Capital and the price of IS Labor
Ahituv, N. and R. Giladi, "Business Success and Information Technology: Are They Really Related", Proceedings of the 7th Annual Conference of Management IS, Tel Aviv University, 1993.
Alpar, P. and M. Kim, "A Comparison of Approaches to the Measurement of IT Value", Proceedings of the Twenty-Second Hawaii International Conference on System Science, Honolulu, HI, 1990.
Bain, J., Barriers to New Competition, Harvard University Press, Cambridge, MA, (1956).
Bakos, J. Y., "A Strategic Analysis of Electronic Marketplaces", MIS Quarterly, 15 (3): (1991).
Bakos, Y., "Reducing Market Search Costs: Implications for Electronic Marketplaces", University of California, Irvine Working Papers in Information Systems (Revised, January 1993).
Banker, R. D. and A. Maindratta, "Nonparametric Analysis and Allocative Efficiency in Production", Econometrica, 56 (6): 1315-1332, (1988).
Barua, A., C. Kriebel and T. Mukhopadhyay, "Information Technology and Business Value: An Analytic and Empirical Investigation", Information Systems Research, 6 (1): 3-23, (1995).
Berndt, E., The Practice of Econometrics: Classic and Contemporary, Addison-Wesley, Reading, MA, (1991).
Berndt, E. and Z. Griliches, "Price Indexes for Microcomputers: An Exploratory Study", NBER Working Paper 3378, (1990).
Bresnahan, T. F., "Measuring the Spillovers from Technical Advance: Mainframe Computers in Financial Services", The American Economic Review, 76 ( 4): 742-755, (1986).
Brooke, G. M., "The Economics of Information Technology: Explaining the Productivity Paradox", MIT Sloan School of Management CISR Working Paper 238, (April 1992).
Brynjolfsson, E., "The Productivity Paradox of Information Technology", Communications of the ACM, 35 (December): 66-77, (1993a).
Brynjolfsson, E., "Some Estimates of the Contribution of Information Technology to Consumer Welfare", MIT Sloan School of Management Working paper (September 1993b).
Brynjolfsson, E. and L. Hitt, "Is Information Systems Spending Productive? New Evidence and New Results", Proceedings of the 14th International Conference on Information Systems, Orlando, FL, 1993.
Brynjolfsson, E. and L. Hitt, "Information Technology as a Factor of Production: The Role of Differences Among Firms", Economics of Innovation and New Technology, 3 (4 (Special Issue on Information Technology and Productivity Paradox)): 183-200, (1995).
Brynjolfsson, E. and L. Hitt, "Paradox Lost? Firm-level Evidence on the Returns to Information Systems Spending", Management Science, (April, Forthcoming): (1996).
Bureau of Economic Analysis, U. S. D. o. C., Fixed Reproducible Tangible Wealth in the United States, 1925-85, U.S. Government Printing Office, Washington, D.C., (1993).
Capon, N., J. U. Farley and S. Hoenig, "Determinants of Financial Performance, A Meta-Analysis", Management Science, 36 (10): 1143-1159, (1990).
Caves, D. W., L. R. Christensen and W. E. Diewert, "The Economic Theory of Index Numbers and the Measurement of Input, Output and Productivity", Econometrica, 50 (6): 1393-14114, (1982).
Caves, R. and M. Krepps, "Fat: The Displacement of Nonproduction Workers from U.S. Manufacturing Industries", Brookings Papers on Economic Activity: Microeconomics, (2): 227-288, (1993).
Caves, R. E. and D. Barton, Efficiency in U.S. Manufacturing, MIT Press, Cambridge, MA, (1990).
Christensen, L. R. and D. W. Jorgenson, "The Measurement of U.S. Real Capital Input, 1929-1967", Review of Income and Wealth, 15 (4): 293-320, (1969).
Clemons, E. and B. Weber, "London's Big Bang: A Case Study of Information Technology , Competitive Impact and Organizational Change", Journal of Management Information Systems, 6 (4): 41-60, (1990).
Clemons, E. K., "Evaluation of Strategic Investments in Information Technology", Communications of the ACM, 34 (1): 22-36, (1991).
Council of Economic Advisors (ed.), Economic Report of the President, United States Government Printing Office, Washington, (1992).
Cron, W. L. and M. G. Sobol, "The Relationship Between Computerization and Performance: A Strategy for Maximizing the Economic Benefits of Computerization", Journal of Information and Management, (6): 171-181, (1983).
Dos Santos, B. L., K. G. Peffers and D. C. Mauer, "The Impact of Information Technology Investment Announcements on the Market Value of the Firm", Information Systems Research, 4 (1&127;): 1-23, (1993).
Gordon, R. J., "The Postwar Evolution of Computer Prices", National Bureau of Economic Research, Cambridge, MA Working Paper 2227, (1987).
Gordon, R. J., The Measurement of Durable Goods Prices, University of Chicago Press (for NBER), Chicago, (1993).
Gurbaxani, V. and H. Mendelson, "An Integrative Model of Information Systems Spending Growth", Information Systems Research, 1 (1&127;): 23-46, (1990).
Gurbaxani, V. and H. Mendelson, "The Use of Secondary Analysis in MIS Research", in The Information Systems Research Challenge: Survey Research Methods, Harvard Business School Press, Boston, (1991).
Hall, B. H., "The Manufacturing Sector Master File: 1959-1987, Documentation", NBER Working Paper 3366, (April 1990).
Hall, B. H., "New Evidence on the Impacts of Research and Development", Micro-BPEA Conference, Washington, DC, 1993a.
Hall, B. H., "The Stock Market's Valuation of R&D Investment During the 1980's", American Economic Review, 83 (2 (May)): 259-264, (1993b).
Harris, S. E. and J. L. Katz, "Predicting Organizational Performance Using Information Technology Managerial Control Ratios", In Proceedings of the Twenty-Second Hawaii International Conference on System Science, Honolulu, HI, 1989.
Hausman, J. A., "Specification Tests in Econometrics", Econometrica, 46 (November): 1251-1271, (1978).
Jensen, M. C., "The Modern Industrial Revolution, Exit, and the Failure of Internal Control Systems", Journal of Finance, XLVIII (3&127;): 831-881, (1993).
Kemerer, C. F. and G. L. Sosa, "Systems Development Risks in Strategic Information Systems", Information and Software Technology, 33 (3&127;): 212-223, (1991).
Lee, B. and A. Barua, "Assessing the Productivity Impact of Information Technology in the Manufacturing Sector", University of Texas at Austin Working Paper, Department of Management Science and Information Systems (1994).
Lichtenberg, F., "The Output Contributions of Computer Equipment and Personnel: A Firm Level Analysis", NBER Working Paper (October 1994).
Loveman, G. W., "An Assessment of the Productivity Impact on Information Technologies", pp. 84-110 in Information Technology and the Corporation of the 1990s: Research Studies, Allen, T. J. and M. S. Scott Morton (ed.), MIT Press, Cambridge, MA, (1994).
Malone, T. W., "Modelling Coordination in Organizations and Markets", Management Science, 33 1317-1332, (1987).
Markus, L. and C. Soh, "Banking on Information Technology: Converting IT Spending into Firm Performance", pp. 375-404 in Strategic Information Technology Management: Perspectives on Organizational Growth and Competitive Advantage, Banker, R., R. Kaufmann and M. Mahmood (ed.), Idea Group Publishing, (1993).
Morrison, C. J. and E. R. Berndt, "Assessing the Productivity of Information Technology Equipment in the U.S. Manufacturing Industries", National Bureau of Economic Research Working Paper 3582, (January 1990).
Porter, M., Competitive Strategy, Free Press, New York, (1980).
Quinn, M. A., M. A. Craumer, A. Weaver, R. S. Buday and T. J. Waite, "Critical Issues of Information Systems Management for 1993", CSC Index, Cambridge, MA The Sixth Annual Survey of Information Systems Management Issues (1993).
Ragowsky, A., S. Neumann and N. Ahituv, "Documenting the Benefit an Organization May Gain by Using Information Systems", Tel Aviv University mimeo (February 1994).
Salop, S. C. and D. T. Scheffman, "Raising Rivals' Costs", American Economic Review, 73 (2): 267-271, (1983).
Schmalensee, R., "Another Look at Social Valuation of Input Price Changes", American Economic Review, 66 (1): 239-243, (1976).
Steiner, T. D. and D. B. Teixeira, Technology in Banking, Dow Jones Irwin, Homewood, IL, (1991).
Strassmann, P. A., Information Payoff, The Transformation of Work in the Electronic Age, The Free Press, New York, (1985).
Strassmann, P. A., The Business Value of Computers, Information Economics Press, New Canaan, Connecticut, (1990).
Swanson, B. and C. Beath, Maintaining Information Systems in Organizations, Wiley, New York, (1989).
Tully, S., "The Real Key to Creating Wealth", Fortune, 30-50, (September 20, 1993).
Varian, H., Microeconomic Analysis, Third edition ed., W. W. Norton & Company, Inc., New York, (1992).
Weill, P., "The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector", Information Systems Research, 3 (4 (December)): (1992).
Wilson, D., "Assessing the Impact of Information Technology on Organizational Performance", in Strategic Information Technology Management, Banker, R., R. Kauffman and M. A. Mahmood (ed.), Idea Group, Harrisburg, PA, (1993).