Can we cope with too much testing?

When all this coronavirus business kicked off, it was foot and mouth that came to mind. After all it was a case of coping with a virus. But now it’s our experience with bovine TB that seems to be most relevant.

All cattle farmers are used to regular TB testing, and I’m pretty sure we’ve all heard the terms Sensitivity and Specificity.

 

Sensitivity – The probability that if the animal has the disease then the test will be positive. The higher the sensitivity the fewer false negatives but at the cost of false positives.

 

Specificity – The probability that if the animal does not have the disease then the test will be negative. The higher the specificity the fewer the false positives but the more false negatives.

 

The terms “sensitivity” and “specificity” were introduced by American biostatistician Jacob Yerushalmy in 1947. So you can blame him for picking two such similar words which makes them easy to confuse.

 

The advantage of the current skin test is that it has a high specificity (thought to be around 99.98%). This means if the animal doesn’t have the disease, the test will show it to be clear. You don’t get many false positives. This is important on a herd test, because you’re doing a lot of cattle and if you get a lot of false positives, a herd will never be clear of TB. Test a 10,000 cows and you’ll probably get one false positive.

The problem is that the skin test has a lower sensitivity, between 81% and 85%. This means that you will miss cows with the disease. On a herd basis this isn’t too worrying because herds are big enough that if the disease is there, it’s probably in several animals and you only need to pick up one for the whole herd to be locked down and repeatedly retested. Mind you it’s a nightmare for the farmer trying to get rid of the disease in his livestock.

Then you have the gamma interferon test. It has a sensitivity of over 90% which means you miss fewer cattle who have the disease. You can clear the disease out of a herd more quickly.

Unfortunately the specificity is lower than the skin test. It’s about 96.6% so you have a lot higher chance of getting false positives.

To give an example, with a 200 cow herd with the skin test you’d be unlucky to have a false positive. With the gamma interferon test you’d expect two false positives.

So using gamma interferon you actually clear the disease out of a herd faster than you do with the skin test. But in reality you never stop testing because of the levels of false positives.

This is why the vets use them in tandem. You can use gamma interferon and be confident you’ve probably detected the diseased animals. But you stick with the skin test for determining whether the herd is finally clear because you don’t get false positives.

 

So as a rule of thumb, if there’s a lot of the disease about, you want a test with a high sensitivity, because then you don’t miss the disease carriers and false positives are a very small proportion of those you find.

But if there isn’t a lot of the disease about you want a test with a high specificity because otherwise your false positives can end up outnumbering your real positives.

 

Moving on to coronavirus (COVID-19) tests, I’ve seen an article in the BMJ

https://www.bmj.com/content/bmj/369/bmj.m1808.full.pdf

 

This stated “Further evidence and independent validation of covid-19 tests are needed. As current studies show marked variation and are likely to overestimate sensitivity, we will use the lower end of current estimates from systematic reviews, with the approximate numbers of 70% for sensitivity and 95% for specificity for illustrative purposes.”

 

Whilst these are approximate numbers, they aren’t entirely encouraging. A sensitivity of 70% leaves a lot of room for false negatives. It could miss up to 30% of the people tested who have the virus. These people walk away thinking they don’t have it and might spread it through the population

On the other hand, at 95% specificity, this means that you’ll get 5% false positives.

In a large population with a lot of the disease, this isn’t too much of a problem. But let’s assume that you have a large population with very little disease.

Let us assume we test 100,000 people, and only 0.1% have the virus. This is probably the stage we’re at now and it means there are 100 people with the virus.

With our test, we would expect to get 70% of them, so we find 70 people with the virus. 30 are missed.

But with 95% specificity we’ll have 5000 people marked down as false positives. So at the end of a day’s testing we have 5070 people test positive and are asked to isolate. But actually only 70 of them actually have it and we haven’t got a clue who they are.

The problem with fetishizing testing in populations with low levels of the disease is that whilst the disease might have died out, you’ll keep finding it for ever because of the nature of the test.

Not only that but the more you test, the more false positives you’ll get. It strikes me that they’re using the tests wrong. If you have three people in a factory who go down with the virus, then by all means go in and test everybody in the factory. But I see little use in testing entire populations, especially when the level of cases gets down to the level of expected false positives.

 

Actually we’re currently field testing the specificity of the test even as I write.

On the 18^th August there were 150,174 tests and 1,089 new cases. This means that the system probably has at least 99% specificity because otherwise you’d get a lot more cases, even if they were all false positives. But of the 1,089 cases, it would be interesting to know how many of them were false positives? If has struck me that the high number of ‘asymptomatic’ cases, plus the fact that we’re not seeing people being admitted to hospital might give us a rough indication that a fair proportion of these ‘cases’ are people who don’t actually have the virus.

 

Now I’ve just heard the Minister on the radio this morning saying they’re going to roll out mass testing. Even assuming the test has a specificity of 99.99% that means that when they test us all, they’ll find 6,000 cases even if nobody has the disease. Second wave here we come.

♥♥♥♥

There again, if you’re going to be locked down for no obvious reason you might as well have a good book to read.

A collection of anecdotes, it’s the distillation of a lifetime’s experience of peasant agriculture in the North of England. I’d like to say ‘All human life is here,’ but frankly there’s more about Border Collies, Cattle and Sheep.

As a reviewer commented, “This is a delightful collection of gentle rants and witty reminiscences about life in a quiet corner of South Cumbria. Lots of sheep, cattle and collie dogs, but also wisdom, poetic insight, and humour. It was James Herriot who told us that ‘It Shouldn’t Happen to a Vet’ but Jim Webster beautifully demonstrates that it usually happened to the farmer too, but far less money changed hands.

I, for one, am hoping that this short collection of blogs finds a wide and generous audience – not least because I’m sure there’s more where this came from. And at 99p you can’t go wrong!”

Tagged: coronavirus, gamma interferon, sensitivity, skin test, specificity