Open Contracting and the Withholding of Commercially Sensitive Information: the U.S. Experience

U.S. courts and federal agencies have grappled for more than 40 years with the question of what information in a government contract should be made public and what should be withheld as “commercially sensitive.” The anticorruption community now seeks an answer to that same question.  The Open Contracting Partnership, the leading advocate for the full disclosure of every contract let by every government, acknowledged in July there should be an exemption from disclosure for such information, and a Center for Global Development working group followed in October with draft principles for determining what is commercially sensitive.

Getting the correct answer is critical, particularly for developing nations, precisely the countries where advocates believe open contracting will make the greatest difference and where the push for open contracting laws is felt the most.  Too narrow an exemption, one that would result in the release of genuinely sensitive information, will discourage companies from bidding on public tenders.  On the other hand, if the exemption is too broad, contractors can use commercial sensitivity assertions to hide information showing whether a contract was awarded fairly and honestly and whether the public is getting value for its money.

Though far different than conditions in these countries, the American experience nonetheless offers lessons to those urging developing nations to embrace open contracting. The most important being that it counsels more caution than many open contracting advocates might at first think is warranted.

Companies losing a public tender in the U.S. often file a freedom of information request to obtain a copy of the contract issued to the winner, and with high value IT and defense contracts, they almost always do. Losing bidders believe that learning what the successful bidder offered will help them win future tenders. Contract awardees share this belief and thus do everything in their power to keep competitors from seeing the contract terms. The most important information sought is that about the awardee’s pricing strategy: its profit margin; how much it will pay for raw materials, labor, and other components; what subcontractors are charging; and how much it is marking up the materials, charges, and other components.  This information will help a competitor develop a bid price for the next competition a shade under what it expects the awardee will offer.

The legal fight over disclosure of pricing information is waged over whether the information can be withheld under the Freedom of Information Act exemption for “confidential . . . commercial or financial information.” It starts with the losing bidder filing a request for the contract with the awarding agency. The agency can itself decide that parts of the contract should be withheld, and the awardee has a right to assert commercial confidentiality.  Whenever a request is contested, the law requires the contracting agency to make a detailed, thorough evaluation of a confidentiality claim, which frequently produces a record stretching to hundreds if not thousands of pages. Whether the agency upholds or denies the awardee’s claim of confidentiality or denies the requestor access to parts of the contract, an appeal of the agency’s decision to a federal district court commonly follows.  Whoever loses in the district court can, and often does, appeal to the appropriate U.S. Court of Appeals. Review by the Supreme Court of an appeals court decision is theoretically possible, but the Court has so far not accepted a case.

In reviewing whether a government agency correctly applied the exemption for “confidential . . . commercial or financial information,” courts have issued a plethora of decisions that have fleshed out the statute’s bare terms. In an early and still controlling opinion, the U.S. Court of Appeals for the District of Columbia held that “commercial or financial matter is ‘confidential’ . . . if [its] disclosure . . . is likely to . . . cause substantial harm to the competitive position of the person from whom the information was obtained.”  That court and others have gone on to rule that information causes “substantial harm” when a competitor can use it to underbid the awardee in future tenders.

One rough rule of thumb to emerge from the court decisions, although not always followed, is that the awardees’ profit margin, option prices for contract renewal, overhead costs, and its sources and costs of acquiring materials, and other data from which a competitor could divine information about pricing strategy should be withheld. A straightforward example comes from a September 2018 district court decision involving the outsourcing of the collection of unpaid taxes.  The court affirmed the Internal Revenue Service’s decision to redact the percentages private debt collectors took of the unpaid tax bill for collecting it.

The courts conduct an intensive factual review of the record and market place economics to decide when information would “cause substantial harm.” Hanging over the inquiry is the degree of certainty that harm will result that is required before information can be withheld.  The Court of Appeals for the District of Columbia favors the term “likely.” Others say “high probability” or “substantial chance.” In answering a dissent in a case involving McDonnell Douglas’ confidentiality claim, the D.C. Appeals Court  disclaimed the need for “pinpoint precision.” The vague, shifting language masks the actual standard courts apply: “I will know it when I see it.”

A fierce and still unresolved battle continues over the disclosure of line item and unit prices.  A line item contract breaks the goods, services, or works being provided down by components — labor hours of services, funding for raw materials, and so forth. An example would be the bill of quantities in a construction contract.  In a unit price contract, the total price depends upon the number of units delivered.  A contract to code documents might contain a per page price with the total payment equal to the price per page times the number of pages coded.

Arguments about whether unit- and line-item prices should be disclosed turn on such factors as the contract type, how competitive the supplier market is, and how difficult it would be, as D.C. Court of Appeals Judge Merrick Garland put it in dissent in McDonnell Douglas, to reverse engineer “sensitive strategic information.”  The majority in McDonnell, in an opinion by then Chief Judge Douglas Ginsburg, held that release of line-item information showing the price McDonnell Douglas charged for work performed by its subcontractors would allow competitors to derive McDonnell Douglas’ markup.  All they would have to do is get a quote from a subcontractor and deduct the price McDonnell charged the government. Judge Garland argued that subcontractors would quote the same prices it charged McDonnell Douglas only if the market were perfectly competitive.  Since it was not, it was quite likely any quote a competitor obtained would differ from what McDonnell Douglas got and thus be of no use in trying to discern McDonnell Douglas’ markup.

The two judges disagreed too on how important a factor price was in contracts of the kind at issue.  Judge Ginsburg argued it was critical information since “price is the only objective, or at least readily quantified, criterion among the criterion for awarding” such contracts. Hence any information that would help a competitor divine McDonnell Douglas’ price would cause it substantial harm. The contract here was for overhauling landing gear and maintaining auxiliary power units and engine thrust reverses on military aircraft which refuel other military aircraft in flight. Judge Garland said that quality, dependability, and a host of other factors were together far more important than price when awarding such contracts.

Several things are clear from the morass of cases on commercial confidentiality.  Hard and fast rules are difficult to formulate and so broad standards that require consideration of multiple factors govern decisions.  The result has been a host of questionable, unpredictable decisions that sometimes err on the side of too much disclosure and sometimes on the side of too little. A second lesson the U.S. experience teaches is that personnel at contracting agencies and companies doing business with the U.S. government must devote enormous time and effort to deciding what is and is not commercially sensitive.  As one recent commentary put it, the cases are “naturally complex” and those doing business with the government should expect a long and expensive battle if they try to protect information they believe is commercially sensitive from falling into the hands of their competitors. Many courts blanche at the work involved in reviewing agency decisions favoring disclosure, observing that the disclosure of confidential commercial information is the price companies seeking a contract from the U.S. government must pay.

The question for those mulling commercial confidentiality rules for developing states is what is workable.  How much time and effort can civil servants in these countries devote to assessing confidentiality claims? How much capacity do they have to assess them?  What about judicial review? Those familiar with the American judiciary will recognize the names Garland and Ginsburg, the judges who reached opposing conclusions in McDonnell Douglas. Both are highly regarded jurists who have been nominated by American presidents to serve on the nation’s highest court; both brought substantial intellectual capital to bear in McDonnell Douglas.

In considering a workable set of rules for developing states, open contracting advocates should pay close attention to the likely effect of the rules on potential suppliers.  Companies may be willing to risk the disclosure of sensitive commercial information to compete for large contracts let by U.S. government agencies.  Will they be willing to bear the same risk to compete in far smaller markets?  Some surely will, particularly if the market is less competitive thanks to rules discouraging some from competing.

What companies are likely to be discouraged?  I fear it will be those who have succeeded in highly competitive markets in Europe, North America, and Asia.  They are the most likely to conclude it’s not worth competing in a small market if information that could give a competitor a leg up in a larger one might be revealed.  Companies which have found success in larger markets have likely succeeded because they delivered quality products and services at the best prices.  It is of course exactly these companies that developed countries most want to see competing for their public contracts.

One solution might be to adopt easily administrable bright line rules that eliminate any uncertainty about what will and will not be disclosed.  An example would be a rule that unit and line item prices will never be disclosed. Such a rule would have its drawbacks.  Disclosure of line item prices in the United States sparked reforms in defense procurements.  But it was not simply the disclosure that produced the result.  It was too the existence of a vigorous free press staffed by highly qualified journalists who could sort through thousands of pages of contract documents and write a 30- inch story explaining in layperson’s terms what they found. It could well be that the loss from keeping line item and unit prices confidential would be minimal while the gain in terms of encouraging more firms to bid significant.

There are surely no easy answers to contract disclosure question as the never-ending litigation in the United States shows. But I hope the American experience helps those crafting rules for other countries realize that transparency in public contracting is not an unmitigated good.  There are costs as well benefits and a wise rule takes account of both.

3 thoughts on “Open Contracting and the Withholding of Commercially Sensitive Information: the U.S. Experience

  1. Rick, thanks for this great post. I found myself wondering, as I read your description of the D.C. Circuit’s substantial harm standard and its application in the McDonnell Douglas case, whether or not some bright-line rules might not be a better solution to this issue than the D.C. Circuit’s standard-based approach. This is an idea you raise yourself at the end of the post, and is one that I think potentially has some merit. The standards-based approach seems to ask judges to make decisions based on complicated economic evaluations that seem, at least potentially, more suitable for an expert administrative agency decision-maker than a generalist court. (No disrespect to Justices Garland and Ginsburg). A rules-based approach would avoid this issue, and, as you suggest, be more administrable and predictable. The risk of course is that the rule will be drawn too far on one side or the other, insisting on “too much” disclosure or, conversely, not enough.

    After reading your piece though, I do wonder if these risks—inherent in all rule-making—might be diminished in this particular case. That’s because the “too much” disclosure risk seems unlikely to materialize given that government actors generally have more incentives to favor less transparency than the other way around. And “too little” disclosure, at least in the case of government contracting, might more effectively incentivize the most competitive companies (the ones you discuss above) to invest in an emerging market. I’m not talking about a situation where the rule is set far out on pole of the disclosure spectrum. But imagining a scale where a rule compelling complete disclosure scores 100, a rule affirming complete secrecy scores 0, and the ideal rule would be set at 50, it seems to be, based on your post, that we could live with a rule set somewhere around 40. There might be some space for some shenanigans that way, but the tradeoff in terms of greater willingness on the part of market-leading companies to invest might be worth it. Does that make any sense? I guess I’m wondering if, in the emerging markets you discuss in your piece, you think there might be at least a little purchase to the argument that a slightly greater risk of corruption could be economically beneficial.

    Finally, I wonder if, in the course of thinking about this piece, you looked into alternative rules or standards used by other U.S. Courts of Appeals or other jurisdictions around the world. Are there other countries that have a radically different (or similar) approach to the U.S.? And have any jurisdictions you’re aware of experimented with a rules-based approach to government contracting disclosure laws? Thanks!

  2. When I started working for Transparency International Lithuania in 2010, we started drafting proposals to amend the Public Procurement Law and start publishing all procurement contracts, lest for the clauses where bidders had grounds for confidentiality as defined in law. I think in 2011, we submitted our first written proposals to the Parliament on this and… didn’t even receive a response. It was unthinkable. The bidders argued that most of the contracts should be confidential, the contracting agencies had little idea how to manage confidentiality, because the standard was to keep everything secret. After a couple of years, though, the national anti-corruption agency decided to prioritize these amendments in the National Anti-Corruption Program and it probably took less than another year for the amendments to be adopted in the Parliament. Making all contracts public turned around the way the bidders now approach confidentiality clauses in general, even during the bidding stage. So I really believe that setting the high standard for transparency helps to change the culture.

    Currently, the Law on Public Procurement in Lithuania obliges the contracting agencies to challenge the confidentiality clauses whenever they feel that they are not grounded. However, after having spoken to many bidders and contracting agencies representatives at home, I have come to think that what matters most is the pro-activeness of the contracting agencies. If they have capacities and competences internally, they challenge confidentiality clauses in the bids they receive by requesting the bidders to provide grounds for that. If they lack capacities and competences, they just let it be. Of course, there is still a risk that the bidder will provide a hundred pages response to why something needs to be confidential in the bid and try to drown the contracting agency in legal and technical terms, but the contracting agency can always ask for a follow up clarification. Furthermore, once such practice takes off, all the bidders kind of know that they CAN be asked to provide supporting arguments for the confidentiality clauses and maybe pay more attention at requesting something to be confidential in the first place.

    (That being said, I completely agree with the idea of this post that having some boundaries regarding confidentiality in laws is crucial too)

  3. Thank you for your interesting post, Rick. I am always amazed at the difficult economic analyses we expect judges to make with very little background in the subject. Since you have so eloquently described the costs associated with increasing transparency requirements in emerging markets, (namely, discouraging future investment by highly competitive companies and increasing the administrative burden on fledgling government agencies and courts), I wanted to offer a potential long-term cost associated with lowering transparency requirements to encourage foreign investment. While lowering transparency requirements may entice investors to come to a country in the short run, it could set a difficult precedent to overturn. After some time, these corporations will become powerful interests, and the practices governments set now will likely become further entrenched as corporations gain political influence. This concern may be less relevant if, as Jason suggests, we believe that the ultimate rule will fall somewhere in a comfortable range that does not allow for widespread corruption. However, this scenario is far from certain.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.