An investigation is now on to determine how flawed data used to back WHO and international Covid-19 policies made it into published studies.

Two leading academic journals have published ‘expression of concern’ notices after problematic data from a little-known US healthcare analytics firm, called Surgisphere, was used in highly-cited Covid-19 studies.

According to The Guardian, Surgisphere’s chief executive, Sapan Desai, is cited as a co-author on studies which used the company’s data it claims to have obtained from more than 1,000 hospitals across the world. Surgisphere has so far failed to explain or publicise its methodologies or findings behind its data.

This is despite the fact that the company’s data was used as key evidence for the World Health Organization (WHO) and research institutes in their demand for halting all trials investigating the use of hydroxychloroquine to treat Covid-19. Today (3 June), The Lancet journal published a statement on its site saying important scientific questions have been raised about Surgisphere’s data used in one of its published studies.

“We are issuing an expression of concern to alert readers to the fact that serious scientific questions have been brought to our attention,” its editors said.

 ‘Substantive concerns’

The New England Journal of Medicine also published a similar statement about the “substantive concerns” raised about the quality of data used in a study published in May. “We have asked the authors to provide evidence that the data are reliable,” it said.

Further investigation into Surgisphere’s background by Guardian journalists has also raised concerns, such as the discovery that several of the company’s employees have little or no data or scientific background. Furthermore, the company’s LinkedIn page was found to have just a handful of employees and, overall, has little online presence.

Founded in 2008 as a medical education company and publisher of textbooks, Surgisphere claims that it has access to a database of almost 100,000 patients from 1,200 hospitals worldwide.

When asked how his small company – of which he said has 11 employees – could conduct such a massive operation, Desai said: “We use a great deal of AI and machine learning to automate this process as much as possible, which is the only way a task like this is even possible.”

‘Almost certainly a scam’

Peter Ellis, chief data scientist for a management consultancy firm called Nous Group that specialises in large-scale data integration projects, claimed the Surgisphere data was “almost certainly a scam”.

“It is not something that any hospital could realistically do. De-identifying is not just a matter of knocking off the patients’ names, it is a big and difficult process,” he said.

“I doubt hospitals even have capability to do it appropriately. It is the sort of thing national statistics agencies have whole teams working on, for years.”

Desai has rejected claims that Surgisphere’s database is flawed saying there is a “fundamental misunderstanding” of its systems, and that claims made in The Guardian contained “a number of inaccuracies and unrelated connections”.

“We do not agree with your premise or the nature of what you have put together, and I am sad to see that what should have been a scientific discussion has been denigrated into this sort of discussion,” he said.

A need for caution

Authors of the studies that used Surgisphere’s database have now commissioned an independent audit into the company’s findings which is expected to be completed next week.

Commenting for a Siliconrepublic.com piece earlier this year, Emily Jesper-Mir of the sound science campaigning charity Sense About Science warned about the dangers of publishing studies too quickly when it relates to Covid-19.

“Even when something is peer reviewed and published, there still needs to be a critique of the research by anyone who is looking at that study,” she said.

“If there’s people who post-publication critiqued it and flagged significant flaws in the study design, then that can’t be used by Donald Trump to give it oxygen that might be dangerous.”