Should Industry Representatives Be Excluded from Federal Advisory Committees?
19 Jul 2006 in Regulatory Science, Peer Review
The Center for Science in the Public Interest is sponsoring a "Forum
on Corporate Influence on Federal Advisory Committees," July 24
at the National Press Club in Washington. At the event, CSPI promises to release a report on whether National Academy of Sciences panels are "fair and balanced."
Although agency-directed peer review panels are federal advisory committees, most advisory committees are not peer review panels.
They are comprised of representatives from a broad range of stakeholder groups, whose function is to provide policy advice based on the interests they represent. In short, advisory committee
members are expected to have financial and other interests coincident with the interests they represent.
Why would it make sense to exclude industry stakeholders from federal advisory committees charged with giving policy advice?
Neutral Source has not seen the report CSPI plans to release, but it's reasonable to expect that it will be the predominant subject of the Forum despite the participation of two industry-affiliated speakers on the program.
The Forum is not limited to the role of industry scientists on peer review panels, a subject on which we have previously posted. The subject is federal advisory committees; we have posted here, here, and here when members of EPA's Children's Health Protection Advisory Committee recently decided to give the Agency their policy opinions about EPA's reference dose for perchlorate.
Rather, the CSPI Forum is about federal advisory committees in general, to which CSPI provides helpful links. So it's important to understand what federal advisory committees are and what they do.
WHAT ARE FEDERAL ADVISORY COMMITTEES?
These groups are formally established under the Federal Advisory Committee Act (FACA). Congress explicitly acknowledged the value of advisory committees, saying "they are frequently a useful and beneficial means of furnishing expert advice, ideas, and diverse opinions to the Federal Government"and that their functions "should be advisory only." That is, "all matters under their consideration should be determined, in accordance with law, by the official, agency, or officer involved." Agencies may not legally delegate decision-making to advisory committees.
FACA section 10 establishes significant procedural burdens on federal advisory committees, including openness, record keeping and public access to records, federal agency chairmanship, leader, and control of the agenda by the federal agency and not by the non-federal members of the committee. Section 9 requires all federal advisory committees to have charters approved by the head of the agency authorizing their creation. Section 8 directs agency heads to "establish uniform administrative guidelines and management controls for advisory committees established by that agency."
By law, members of advisory committees may be compensated for their services up to a rate equivalent to GS-18 (now called SL), which is $109,808 to $143,000 per year ($54.90 to $71.50 per hour, assuming a 2,000 hour work-year). Thus, for some it can be financially rewarding as well as prestigious to serve on a federal advisory committee.
DOES THE NATIONAL ACADEMY OF SCIENCES HAVE ADVISORY COMMITTEES?
Yes. NAS committees are within the definition of an advisory committee when the work they do is funded by a federal agency. (Self-funded activities are not.) The NAS is an independent nongovernmental organization chartered by Congress and it enjoys special privileges that exempt it from virtually all of the requirements summarized above when it convenes a federally-funded advisory committee. NAS is required to comply only with FACA Section 15, which essentially delegates to NAS unfettered discretion to decide what procedures it will follow. Consequently, NAS procedures and practices are not transparent and are immune to legal challenge.
According to the limited information in the event announcement, the report CSPI will release on July 24 concerns whether National Academy of Sciences panels are "fair and balanced." Based on its extensive prior advocacy, it's reasonable to expect that CSPI's conclusion will be that they are not. Because NAS procedures and practices are opaque, and permitted by law to be that way, it may not be possible to test this conclusion. What will be interesting is what evidence CSPI brings to light, how they diagnose the problem, and what they propose be done about it.
Hints are available elsewhere on the CSPI website. For example, On July 11, the NAS released a long-awaited report on its review of EPA's dioxin reassessment. According to the report brief:
The committee concludes that EPA’s decision to rely solely on a default linear model lacked adequate scientific support. The report recommends that EPA provide risk estimates using both nonlinear and linear methods...A full list of "key findings" can be found in a text box on page 5, but the issue of nonlinear methods for extrapolation from high laboratory doses in animals to low environmental doses in humans is crucial. Nonlinear methods yield significantly lower estimates of human cancer risk.
Under NAS policy, financial conflict of interest is justifies a candidate's exclusion and cannot be "balanced," say, by the addition of another member whose financial conflicts are "opposite." NAS treats "bias" differently from conflict of interest, noting that bias is either endemic or impossible to refute. NAS policy is to ensure that its panels are "balanced" with respect to bias.
In any case, if NAS had permitted individuals with financial conflicts to serve on the dioxin reassessment committee -- a committee with a controversial assignment, if ever there was one -- it would have violated its own policies and procedures.
CSPI claims that "at least four scientists" on the NAS committee have "extensive ties to industry or direct financial conflicts of interest," of whom it identifies three. The person on the dioxin committee cited by CSPI as conflicted is Michael Denison because he "co-founded a company which holds a patent for dioxin detection."
Denison is a professor of environmental toxicology at the University of California at Davis, and the company he co-founded is Xenobiotic Detection Systems. According to the XDS website:
In 1998 XDS was awarded a patent for its proprietary CALUX® (Chemically-Activated LUciferase gene eXpression cell bioassay system) assay for dioxin-like chemicals.The company's patent concerns a laboratory test that measures the concentration of dioxins in various media. According to an XBS press release, their patented method has been shown to be more sensitive, less costly, and yield results more quickly. CSPI does not say how this equates to a financial conflict of interest, and apparently the NAS disagreed.
Concerning "extensive ties to industry," CSPI says Dennis Bier "has consulted for most major food industry groups," and Joshua Cohen "has conducted three industry-funded studies in the last five years, including the fishing industry."
Bier is Director of the Children's Nutrition Research Center and Professor of Pediatrics at the Baylor College of Medicine, a trustee of the International Life Sciences Institute, and a member of the NAS' Institute of Medicine. A current list of studies being conducted by Bier's center is here.
Cohen is lecturer at the Tufts--New England Medical Center's Institute for Clinical Research & Health Policy Studies and Center for the Evaluation of Value and Risk in Health:
CEVR is a newly-established Center focused on issues pertaining to value, cost-effectiveness, and risk in health care decisions. The Center's mission is to assess benefit-risk tradeoffs in health care choices and public health interventions, and to help decision makers target resources to improve health more efficiently and effectively...
CEVR is the new home of the Tufts-NEMC Cost-Effectiveness Registry, an online database of almost 1,000 cost-effectiveness analyses in health care, currently funded by the National Library of Medicine.What is this Registry? According to its website, the Cost-Effectiveness Registry
provides public electronic access to a comprehensive database of cost-effectiveness ratios in the published literature. Its goals are to find opportunities for targeting resources to save lives and improve health and to move towards standardization of cost-effectiveness methodology in the field.The Registry collects published reports comparing alternative medical interventions and reports their cost per quality-adjusted life year.
Previously, the Cost-Effectiveness Registry was housed at the Harvard School of Public Health's Center for Risk Analysis, which was founded by John Graham in 1989. In 2001, Graham left Harvard to become Administrator of OMB's Office of Information and Regulatory Affairs. CSPI opposed Graham's nomination.
WHAT IS THE "INTEGRITY IN SCIENCE" PROJECT?
Since 2003, CSPI's Merrill Goozner has managed its "Integrity in Science" project. The Project has two interrelated objectives: promoting disclosure and challenging conflicts of interest. With regard to disclosure, CSPI says:
The Integrity in Science project promotes full disclosure of conflicts of interest when scientists publish in journals, are quoted in the press or appear before legislative or regulatory bodies at all levels of government. The project and its allies in the scientific community believe total disclosure of conflicts of interest is mandatory if the public is to maintain its faith in the integrity of the scientific process, and the government is to remain a fair and impartial arbiter of scientific disputes that determine the laws and regulations that affect the health and safety of the American people.
CSPI has model conflict of interest disclosure guidelines that it believes all professional and scholarly journals should adopt. These guidelines call for disclosure of
Note that CSPI's definition excludes financial interests related to contracts and grants from government agencies and foundations.Any financial or other significant relations (e.g., consulting, speaker fees, corporate advisory committee memberships, expert testimony given in legal cases) of the author and the author's immediate family in the last 5 years with companies, trade associations, unions, or groups (including civic associations and public interest groups) that may gain or lose financially from the results or conclusions in the study, review, editorial, or letter (emphasis added).
With regard to conflicts of interest, CSPI says:
To guard against corporate-funded and/or questionable science dominating the regulatory process, the Integrity in Science project selectively investigates and reports on the make-up of the federal government’s more than 1,000 scientific advisory committees. The project also selectively monitors and issues reports about key non-government advisory bodies like the National Academies of Science, which Congress and regulatory agencies often turn to for advice.
Because these advisory committees provide critical guidance and scientific information for government agencies charged with protecting the public health and safety, it is imperative that these committees meet all the fairness and balance requirements of the Federal Advisory Committee Act. Every member of these committees must disclose conflicts of interest to the general public.
The Integrity in Science project challenges committee members who have represented corporations with a stake in the outcome of a scientific study or regulatory proceeding. When such members are appointed to committees for their scientific expertise, the project also intervenes to ensure that equal representation is given to independent scientists and representatives of the public interest.
CSPI addresses only "for-profit" financial
relationships and excludes "non-profit" financial
relationships, such as those resulting from contracts and
grants from federal agencies, foundations and advocacy groups. "Equal
representation" to "representatives of the public interest" is code
language for the mandatory inclusion of certain
advocacy groups (but apparently not others; see below).
The website lists
a few examples in which CSPI or an allied organization has
alleged a conflict of
interest on NAS committees, but none include Denison,
Bier or Cohen. Nothing in CSPI's objections about Bier or Cohen suggest
that either has performed industry-funded research related to dioxin
toxicity, the subject of the review. Rather, in the cases of Bier and
Cohen, the alleged conflict appears to consist
of having performed any research funded
by industry. This is a much more expansive definition than the
one
found in CSPI's model conflict of interest disclosure
guidelines, which would apply to "financial or other
significant relations" with parties "that
may gain
or lose financially from the results or conclusions."
This broad definition of industry-related conflict of interest
occurs elsewhere. CSPI
and the Environmental Working Group objected to Pfizer
scientist Jon C.
Cook serving on a Science Advisory Board peer
review of the Agency's risk assessment for perfluorooctanoic acid (PFOA), an intermediate in the production of Teflon. The nature of Cook's alleged conflict is that he previously worked for
DuPont, a manufacturer of PFOA. This employment was more than five years old and thus outside the definition in of conflict of interest in CSPI's model guidelines.
CSPI and EWG objected to five other "short list" candidates for the PFOA peer review because of their affiliations with industry, though not with any company that produced or used PFOA. They objected to two other candidates because they served as scientific advisers to the American Council on Science and Health, a nonprofit group with very different policy views about what constitutes the public interest.
CSPI's website says that their
research has shown significant
evidence of several types of improper industry conduct. However, according to the organization's 2002
tax return (Guidestar subscription may be required),
a major theme of CSPI advocacy prior to the commencement of
the Integrity in Science project has been to "examine how the demands of industry may undermine the public-interest mission of science." A strong presumption of what research will reveal is a
quintessential element of bias.
CSPI's DATABASE OF CONFLICTED SCIENTISTS
CSPI provides a database of scientists who have performed research funded by industry. The criteria used for inclusion are not specified, but they appear to be quite broad. Our
non-random search of the database revealed everything from civil litigation support to honoraria for delivering academic papers
or performing peer review.
Civil litigation support, whether on behalf of defendants or plaintiffs, is inherently and intensely dependent on client interests. In contrast, honoraria for delivering academic papers
or performing peer review are the most independent of client interests. This makes it difficult to discern strong from weak linkages because all are lumped together.
The database conveys two troubling implications:
- Industry-funded research conducted by the named scientist is biased or otherwise substandard, without regard for its substantive quality.
- All other research conducted by the named scientist is biased or otherwise substandard solely because the scientist received industry funding on other projects, also without regard for its substantive quality.
From Neutral? on 22 July 2006, 21:00
As an economist surely you would agree with the assumption that a rational firm or industry group would not fund research that was likely to cause it financial harm. Furthermore they would probably do whatever was necessary to suppress the results of unfavorable outcomes. Therefore I'm not sure why you consider implication #1 troubling.
From Editor on 18 August 2006, 18:45
I agree with Neutral?'s statement except that rational firms also care about nonfinancial harms. Firms are composed of people, and people care about other things besides narrow financial matters.
Applying this logic, a rational Congressional committee, government agency, nonprofit, or foundation also would not fund research that was likely to cause it harm -- or, stated more generally, did not advance its interests. Bias is endemic and not limited to for-profit enterprises.
Thus, I am troubled by Implication #1 because it applies a general principle about "bias," a universal phenomenon, to only a subset of the affected population. The exclusion proposed by CSPI would apply only to a subset of scientists with biases (i.e., those with industry affiliations) and not to others (e.g., those with nonprofit, foundation, or agency affiliations). Even overtly partisan scientists would not permissible under CSPI's rule so long as they are not affiliated with industry. That industry's overt partisans would be excluded but CSPI's would not seems unlikely to be accidental.
Your second sentence about the suppression of unfavorable results also applies to rational Congressional committees, government agencies, nonprofits, and foundations that fund research. You characterize the likelihood of suppression as "probable" by rational firms, but you are silent about the probsbility of suppression by other rational actors that fund research such as Congressional committees, government agencies, nonprofits, and foundations.
Markets provide a partial solution for this problem. Scientists who desire to cultivate or maintain a reputation for integrity often insist that research funders have no influence or control over whether and how results are published. They can choose not to accept research funding from sources that refuse to accept these terms.
Likewise, research funders (including firms) can include such language in the contracts and grants they award. This signals to scientists that they will not interfere with researchers' publication decisions.
This is only a partial solution because researchers know that the probability of funture funding may depend on the acceptability of current research results. But this problem is hardly limited to research funded by for-profit firms. All research funders, including government agencies, nonprofits and foundations, dislike unfavorable results and are more inclined to provide future support to researchers who bring in favorable results.
At the same time, the existence of a temptation to skew one's work to please the funder should not be interpreted as presumptive evidence of having done so. Temptations are with us always. Integrity consists of acting honorably despite them.
From Neutral? on 22 August 2006, 13:30
I agree that agencies and pro-regulation groups are also likely to produce or support research that leads to results that favor their missions. I just thought that your initial post seemed to indicate that you believed such groups were more likely to produce biased results than industry (essentially the mirror image of the CSPI proposal as you describe it). I doubt that this is true.
I tend to think the possibility of bias in industry funded work is greater than in other work since I believe that financial incentives are greater than non-financial ones. That said, I agree that bias is as you say a "universal phenomenon."
From Editor on 22 August 2006, 14:45
Which side is more likely to fund biased research is an empirical question, and I have not done the research to test any hypotheses along these lines. Your hypothesis that industry is more likely to do so is testable, but it should be tested and validated. In my initial post I said that it's not appropriate to PRESUME greater bias from scientists receiving any industry funding regardless of magnitude, significance or relevance, and rely on that presumption to automatically exclude them from peer review service. If the grounding principle for exclusion is bias on account of funding by an interested party, then evenhanded application of the principle must exclude scientists funded by government (especially regulatory agencies), NGOs and foundations. CSPI's rhetoric is fully consistent with my inference that it does not seek an evenhanded application of the principle. If you have contrary evidence, it should be easy to find. Please provide a URL in a reply.
From Neutral? on 22 August 2006, 15:15
I think that economic theory (at least classical economics) suggests that it is appropriate to presume greater bias from scientists receiving industry funding. As such, I would reverse the burden of proof. Do you know of any studies demonstrating an equal tendency toward bias?