colinnew
EXCLUSIVE
Like many Australians cooped up during Covid lockdowns, Xin Yin Ooi found herself “obsessed” with weekly updates from health authorities, poring over case numbers, vaccinations, hospitalisations and deaths.
As a data analyst, the Sydney woman was naturally interested in the detailed statistical breakdowns provided in NSW Health’s weekly surveillance report — so when she noticed a strange figure in one table, she decided to request the underlying numbers to check the work for herself.
The only problem? The data had been “erased”.
“It was a big shock,” Ms Ooi said.
“In the early days of Omicron, our Premier, Health Minister, chief health officer, every day at the press briefing I remember vividly they were saying, ‘The data shows two doses are not enough, you need three to deal with Omicron.’ They kept repeating, ‘the data, the data’ — it’s just unbelievable they would erase the data.”
Earlier this month, the NSW Civil and Administrative Tribunal (NCAT) ruled in favour of NSW Health in a freedom-of-information battle with Ms Ooi, who first lodged a Government Information Public Access (GIPA) request with the agency more than a year ago.
NSW chief health officer Dr Kerry Chant. Picture: Gaye Gerard/NCA NewsWire
‘Long journey’
Ms Ooi had asked NSW Health to provide “patient level de-identified data” which underpinned two tables in the surveillance report published in the first week of February 2022.
Table five of that report was a breakdown of hospitalisations, ICU admissions and deaths between November 26, 2021 and February 5, 2022, sorted by vaccination status — one, two, or three or more “effective doses”, no “effective dose” or unknown.
Table six was the “proportion of cases with a severe outcome” of either ICU or death, broken down by age and vaccination status — but only showing “three or more effective doses”, “two effective doses” or “less than two effective doses”.
Ms Ooi also requested any “other relevant info that can help me understand this data”, saying in her March 2022 application that “my first analysis objective is to ungroup the last two columns on table six into [a] more refined vaccination category”.
“This is actually a very important table — it traces the cases since Omicron arrived in NSW, it was a new virus at the time, everyone wanted to know what it does,” she said.
“What is interesting is you actually see, moving from no dose to one dose, there is actually an increase [in hospitalisation] from 1 per cent to 2.7 per cent if you look at that column. I’m a data analyst by profession, I wanted to know why, so I did a GIPA. That started my long journey pursuing this.”
NCAT hearing
NSW Health denied the application the following month, saying it did not hold the information requested. Ms Ooi sought a review from the Information Commissioner, which affirmed NSW Health’s decision in June.
The matter then went to NCAT for an administrative review in December.
Ms Ooi last week published a lengthy Twitter thread outlining her concerns following NCAT’s decision in favour of NSW Health.
“In 2021, our government told us that Covid-19 is an unprecedented pandemic that requires an unprecedented solution, which is to vaccinate the entire population with few exceptions — the old, young, middle age, everyone, with Covid-19 vaccines that have just gained provisional approval from the TGA,” she wrote.
“They told us that this is our way out of the pandemic. That the vaccines will ‘stop the spread’. Later, our government said even if vaccination doesn’t stop the spread, it will make us less likely to be hospitalised, end up in the ICU, or die. Logically, NSW Health having led our Covid response, should be very keen (and obligated) to check that the vaccination program works, that it achieves the intended objectives.”
At the hearing, where Ms Ooi represented herself, Dr Christine Selvey from NSW Health’s Covid Influenza Branch explained how the weekly reports were compiled in such a way that no “intermediate records” were kept after the tables were created.
Sydney data analyst Xin Yin Ooi. Picture: @realXinOoi/Twitter
‘Complex process’
Dr Selvey told the tribunal that the former Public Health Response Branch (PHRB), which produced the reports prior to a departmental restructure in March 2022, would extract and link data from three separate sources — the Notifiable Condition Information Management System (NCIMS), the Australian Immunisation Register (AIR), and the Patient Flow Portal, a live database of hospital patients in the NSW Health system.
“As Dr Selvey explained, the process of extracting data from each of the sources and linking the data sets was an extremely complex process and took several hours,” NCAT senior member Elizabeth Bishop SC said in her decision.
“Once the data was extracted, an analyst ran the code to create the tables with the outputs which were then reviewed by the epidemiologist. This process took a further several hours.”
NSW Health staff used R, a statistical programming language, for the analysis required to produce the tables.
“A temporary database (data-frame) is created which is held within the machine RAM and only exists for the duration of the R session,” Ms Bishop said.
“No intermediary tables are created and only the required outputs are produced, following which the session ends and the data-frame is erased.”
Dr Selvey told the tribunal intermediate records were not kept because “the purpose of the analysis was to produce a snapshot at a particular point in time reflected in the published tables” and “it was unnecessary to retain copies”.
“Further, if additional analysis was required later, more up-to-date information … could be extracted at that point,” she said.
Covid patients at St Vincent’s Hospital. Picture: Kate Geraghty
Privacy concerns
She added that “access to data from AIR is only permitted for very limited purposes and is not to be retained other than for the prescribed purpose”, and that “linked patient information is extremely sensitive and the retention of those records would give rise to privacy concerns”.
The GIPA application was denied as NSW Health argued that to provide the information requested, it would have to re-do the same complex process and “create a new record”, which is outside the scope of the legislation.
Ms Bishop said she was “satisfied on the balance of probabilities” that NSW Health “did not retain the patient-level data underlying tables five and six and so does not hold the information sought by the applicant”, and also found that “intermediary records of the data were not kept (in any form) and consequently are also not held by the respondent”.
“For completeness, I note that the respondent submitted that it is not required to create a new record to meet the applicant’s request for information although it might theoretically be able to create a new record that contains the information requested,” she said.
“To do so would require an exercise of re-extraction of the data (including data held by AIR who is not the respondent), a complex compilation and analysis of the data, interpretation and the creation of a new record. Further, there is uncertainty about whether precisely the same information could be extracted again when the linked data that was used to produce tables five and six was point-in-time.”
Dominic Perrottet and Brad Hazzard. Pictures: Gaye Gerard/NCA NewsWire
‘Poor practice’
Ms Ooi, however, argued NSW Health’s process presented a “big accountability problem”.
“We had three years of this pandemic and they are saying they never leave behind a record of the data they used to do their report, which they continued to produce every week,” she said.
“There’s no checking, we can’t verify it is right because they erased everything.”
She said in her line of work she would always retain that data in order to be able to go back and “verify that it’s accurate” if something goes wrong.
“Ask any data analyst, is this something you would do?” she said.
“Create a complicated analysis that takes hours to create the output, and never save your analysis, never save any intermediary work, just publish the final thing? You can’t trace it back, there’s no audit trail. It’s just poor practice, especially during a pandemic and a vaccination program, that makes it more serious — you’re erasing the data that you are basing the policy on.”
A spokeswoman for NSW Health declined to comment.
RPA Hospital ICU intensivist Dr Richard Totaro. Picture: Richard Dobson
Not black-and-white
Associate Professor James Trauer, head of Monash University’s Epidemiological Modelling Unit, said while he did not want to comment on the legal details of the case it appeared to be “a bit of a technical debate” over the data.
“I agree with the broader point that Australian jurisdictions should have provided more disaggregated surveillance data publicly throughout the pandemic,” he said in an email after reviewing the NCAT decision and Ms Ooi’s Twitter thread.
“We know that unvaccinated people are over-represented in deaths and hospitalisations, but we need to know more about the individual characteristics of these people to calculate the true vaccine effectiveness. Finer-grained information on this would have been really helpful.”
He noted that NSW “has actually led the country in providing these data, so it absolutely isn’t a problem that’s limited to NSW”.
“However, Australia has been far behind some other countries overseas, such as the UK,” he said. “We need to think about how to improve this for future pandemics. An Australian CDC could be the primary vehicle for making this happen.”
Prof Trauer added, “I note that most of Xin Yin Ooi’s tweets are focused on data identifiability. In reality there is no easy black-and-white division between identifying and deidentified data. Even aggregate tabular data can be associated with privacy issues, for example, if there is a number one in a cell of a table.”
