Balneus

Australian Lefty on Politics, Governance, Science and Info Management

Evidence of Evidence

Posted by Dave Bath on 2009-02-24


How do we measure commitment to "evidence-based" policy?

While use of evidence in decision-making is important to assess any manager, the use by politicians is of great concern to all, especially if they indicate, explicitly or implicitly, that policy decisions will be evidence-based.

It’s a bit hard to figure out what such indicators might be, so I’m hoping readers will add comments with their own ideas, and perhaps point to the good, the bad, and the ugly.

To start the ball rolling, here are a couple of thoughts:

  • When an expert committee (or a lead like Garnaut) makes a very strong recommendation, and the government makes a very different decision, should the government provide an addendum to (actually it is a rebuttal of) the report explaining the reasons?
  • When the committee or lead was appointed by the Government, then a detailed rebuttal (perhaps citing more recent evidence) should be mandatory.
  • The rebuttal should be available from the same place as the initial report.
  • How big an impact should require how detailed a rebuttal?  Could this be a percentage of GDP?  What if the values are intangible (such as a recommendation to create a Bill of Rights)?
  • Should governments (and oppositions during election campaigns) detail their performance (or their performance if elected)?
  • When reporting (or promising) indicators of evidence-based policy, this shouldn’t be purely numeric, but should be weighted by impact of the decision, and perhaps the length of the rebuttal compared to the length of the evidence presented by the experts.

I’ve got to think about this, but for the moment, over to you.


Advertisements

5 Responses to “Evidence of Evidence”

  1. Ilya said

    When the government and the Rudd government in particular cite evidence-based policy, the question that needs to be asked is in preference to what? Presumably, the alternative is ideology-based policy. Since a priori both evidence (past evidence is generally either unavailable or not very useful) and ideology may not result in good policy per se (this we can only evaluate ex post), the decision-making process is necessarily rooted in a certain degree of pragmatism. As such, evidence-based policy is simply making the best available pragmatic choice.

    We should also remember that all evidence is inherently uncertain.

    It follows then that we should not necessarily impose a higher burden on governments with regard to rebuttal of evidence since this will not of itself lead to a better outcome or invalidate their choice. Rather we should educate politician in the rules of the decision-making under conditions of uncertainty. Whether evidence or ideology forms the basis of the decision is secondary to the decision-maker following the correct process in evaluating options.

  2. zombinol said

    Why do “The Australian Government Canberra” need to call it evidence based policy?

    It is meaningless in providing a certainty that the policy is soundly based in facts or is of a genuine quality. Anything can be evidence! ie. Because we all believe in it therefore it must be true, or, the finance people gave us the evidence and it made sense so we didn’t need the evidence from the sociology people who were banging on about the collapse of our society.

    Using the word evidence means any ones evidence and not necessarily all the evidence, is the evidence from a broad multidisciplinary assessment or from a private think tank funded by the peak body that will be affected by the policy?

    The core issue for Government Policy is that the Policy should be Factual Policy, I believe that this is what the people want, but its hard to do, so I am certain that “Evidence” is spin to assuage the Fact based Policy people and Evidence sounds organised and faultless and legally acceptable.

    Evidence Based Policy is a sound bite, its a deflection. Its doesn’t fix the problem that Policy will always be skewed toward the biggest tax paying lobbyist.

    I agree with Ilya we need “correct process in evaluating options” rather than more froth and lather.

  3. Dave Bath said

    Good points both…. as I said, it’s tricky.

    But… for example, how to we deal with (i.e. manage, and to manage we must be able to measure, good ‘ol CMM speak) things like the following:

    * Public deeply concerned about policy area C
    * Politician J, the PM has a policy contrary to all scientific evidence
    * During campaign, shadow-PM policitician K promises evidence-based decisions including an expert (let’s call him “G”) to report on the impact and management of C, and thus makes no promises apart from going by the evidence
    * The Economist sees this as the first national election largely determined by policy area C
    * K becomes PM
    * G strongly recommends X,Y,Z
    * K does little more than J, against all the INCREASING evidence. Gives no evidence, apart from what boils down to “We know more than all the experts… so there!”

    • zombinol said

      The only required evidence for a Policy are only those facts that satisfy the Sovereign enough to give royal ascent to a Bill, the public perception management until ascent enables the Policy through the stage managed parliament to the Executive Council.

      * Public deeply concerned about policy area C
      – Who are these public? Where is their opinion recoded – or is this a loaded PR statement from Policy C lobbyists?

      * Politician J, the PM has a policy contrary to all scientific evidence
      – If my memory serves me correctly, all scenarios like this are where the Financial outcomes of a Bill to be approved by the Sovereign out way the facts.

      * During campaign, shadow-PM politician K promises evidence-based decisions including an expert (let’s call him “G”) to report on the impact and management of C, and thus makes no promises apart from going by the evidence
      – More PR spin from K, akin to “…therefore all the policies of the Howard government were counter factual and not based on any evidence”, K is deluding himself, ALL policies are based on lack of comprehensive, reliable and often provable evidence.

      * The Economist sees this as the first national election largely determined by policy area C
      – Ok so policy area C is the new kid on the block, has any election ever been determined by exactly the same policy as a previous election? Lobbyists-> Politicians-> Publications-> Managed Population…..

      * K becomes PM
      – Given that elections are foundation of our Democracy, why is it that the evidence of the election (the paper votes) are so readily destroyed? I think that stinks and reeks and is the prime example of why any pronouncement by any politician relating to evidence is fundamentally flawed.

      * G strongly recommends X,Y,Z
      Well yes G would and only if it was in G’s cabals interests to do so as their threats would naturally increase how strongly it is being recommended.

      * K does little more than J, against all the INCREASING evidence. Gives no evidence, apart from what boils down to “We know
      more than all the experts… so there!”

      Lo are we, for we are but mere mortals, its the Sovereigns prerogative that needs to be pampered by the people of Australia for we are but commodities in the interest of the country to service the Sovereign.

      How about FACTUAL BASED POLICY, where all the evidence are tested by a Judge in Court and if found wanting then the Policy or the evidence cannot be presented, lest we create corrupted laws.

  4. […] Top Posts A Judas, not a Bonhoeffer, in the LodgeLobbyClue at GovHackBest scientific paper title of the year?Inspector Gadget copter, Strap-on wings, Da Vinci parachute vidsThe first literature, the must-read: GilgameshCO2 emissions changes since 1990Nostalgia – I see your emoticon and raise you proper ASCII artWhat if the ACCC covered political scams?Spammers: Dumb, and handled with kid glovesOtways Carbon Capture: Dumb idea?Euro v USD as premier currencyEvidence of Evidence […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: