NHS Wales and Welsh Government (WG) produce a wide range of information for internal purposes in areas such as planning, performance management, policy development and implementation, finance and public health. In addition, a number of organisations produce regular releases of information in the shape of health indicators and reports for the wider public, notably the WG Knowledge & Analytical Services (KAS) Directorate, the WG Planning & Performance Directorate, Digital Health and Care Wales (DHCW) and Public Health Wales (PHW).
Until now, there has been no formal process by which the methods for calculating various national indicators or currencies are agreed and fully documented. This has led to a number of difficulties, notably:
a) An inability of different health organisations to replicate the same results as nationally published data for local reporting requirements (e.g. for Board reporting), leading to accusations of there being “different versions of the truth”.
b) A lack of shared understanding across NHS organisations and national bodies as to the methodology used to calculate certain indicators and currencies.
c) A lack of consistent academic informatics and statistical rigour in the development of analysis methodologies underpinning indicators and currencies.
d) A lack of opportunity for Local Health Boards / Trusts to contribute NHS knowledge and experience into the development such methodologies.
In addressing some of the concerns set out above, NHS Wales is seeking to achieve a shared understanding of the methodologies used in the calculation of national indicators, ensuring a focus on the business or operational issues rather than issues with the data itself. Such discussions can often detract from the real (and useful) intelligence that lies within the data.
In April 2013, the Analysis Methodologies Group was formally established. Assurance and publication of its outputs sit within the NHS Wales Information Standards Assurance Process. The Group’s remit is to develop and review analysis methodologies associated with all routine national analyses of NHS Wales healthcare data, such as those developed in support of the national performance management framework. It is not anticipated that the group will be required to develop methodologies associated with ad-hoc and non-routine analyses.
Analysis Methodologies that have been signed off by this group are appraised by the Welsh Information Standards Board (WISB) and their decisions are communicated to the Methodology Sponsor in a formal Outcome. The resultant Analysis Method Notices (AMNs) are then published on the Data Standards website.