Don't be fooled - getting the same result twice with expensive equipment does not mean the answer is reliable!
Don't be Fooled -
Malic Acid Testing
I recently
attended the Unified Wine & Grape Symposium in
These
questions reminded me of the of the Wine Industry Interlaboratory Testing
Program that has be going on since 1999.1,2,3 The program, set up by the ASEV and UC Davis,
is meant to help winery labs improve the quality of their results. In this program, two wine samples are sent
out a few times a year to the 40 – 55 participating winery labs. The samples are analyzed at least in
duplicate, and the results are sent back for statistical analysis. The most common method used by far was
enzymatic. A table of the results is
given below for wine samples with malic acid levels below 500 mg/L. From 10% to 23% of the results were found to
be unreliably inaccurate, and are not included in the table.
|
(Data from the
Wine Industry Interlaboratory Testing Program) |
|||||||||
|
|
|
|
Sample
1 |
|
|
||||
|
Year |
Season |
Sample |
mean |
sd |
95%
confidence |
cv |
actual
range |
samples |
|
|
|
|
No. |
|
|
interval |
|
reported |
discarded |
|
|
1999 |
Summer |
1 |
100 |
19 |
63
- 137 |
19.0 |
|
no. |
% |
|
2000 |
Summer |
3A |
440 |
54 |
334
- 546 |
12.3 |
|
|
|
|
2000 |
Summer |
3B |
210 |
47 |
118
- 302 |
22.5 |
|
|
|
|
2000 |
Fall |
4 |
180 |
54 |
74
- 286 |
29.8 |
|
|
|
|
2000 |
Winter |
5 |
160 |
55 |
52
- 268 |
34.2 |
|
|
|
|
2003 |
Fall |
15A |
96 |
24 |
49
- 143 |
25.0 |
35
- 160 |
4/37 |
11 |
|
2003 |
Fall |
15B |
103 |
34 |
36
- 170 |
33.0 |
35
- 175 |
4/37 |
11 |
|
2004 |
Summer |
17A |
165 |
44 |
79
- 251 |
26.7 |
66
- 277 |
4/43 |
9 |
|
2004 |
Summer |
17B |
118 |
42 |
36
- 200 |
35.6 |
15
- 225 |
4/43 |
9 |
|
2005 |
Spring |
19A |
112 |
43 |
28
- 196 |
38.4 |
25
- 230 |
5/47 |
11 |
|
2005 |
Spring |
19B |
135 |
41 |
55
- 215 |
30.4 |
50
- 240 |
5/47 |
11 |
|
2005 |
Fall |
21A |
165 |
56 |
55
- 275 |
33.9 |
11
- 300 |
5/49 |
10 |
|
2005 |
Fall |
21B |
163 |
54 |
57
- 269 |
33.1 |
17
- 300 |
5/49 |
10 |
|
2006 |
Summer |
23A |
275 |
34 |
208
- 342 |
12.4 |
210
- 351 |
6/44 |
14 |
|
2006 |
Summer |
23B |
176 |
34 |
109
- 243 |
19.3 |
120
- 260 |
6/44 |
14 |
|
2007 |
Summer |
26A |
218 |
76 |
69
- 367 |
34.9 |
125
- 450 |
8/51 |
16 |
|
2007 |
Summer |
26B |
110 |
62 |
0
- 232 |
56.4 |
30
- 300 |
8/51 |
16 |
|
2008 |
Summer |
29A |
98 |
41 |
18
- 178 |
41.8 |
8
- 210 |
13/57 |
23 |
|
2008 |
Summer |
29B |
99 |
41 |
19
- 179 |
41.4 |
3
- 219 |
13/57 |
23 |
|
|
|
|
|
|
|
|
|
|
|
What do
these results show?
First, in
the nine years the quality improvement program has been going on, the results
have gotten progressively worse. The
coefficient of variation, a measure of lab to lab reproducibility, has doubled. In addition, while in the initial years 10%
of the reported results were so far off the mark that they had to be excluded,
the number of totally unusable results climbed to 23% in the latest report.
Second, the
results have deteriorated to the point where about half are flat out wrong
regarding what they tell the winemaker.
In 1999, a sample with a malic acid level of 100 mg/L was analyzed, with
the results showing a range of 63 – 137 mg/L.
That’s pretty broad. In 2007 a
wine with a malic acid level of 110 mg/L was analyzed, and the results ranged
from 30 – 200 mg/L. In 2008 a sample
with a malic acid level of 96 mg/L was analyzed, and the range for the results
had ballooned to 18 – 178 mg/L. Some
winemakers were told their wines had finished MLF (which they hadn’t), while
other winemakers thought they had a few weeks yet to go (which they didn’t). And remember, that huge range was the range
reported AFTER 23% of the results had been discarded for gross unreliability. (By the
way, if an Accuvin test strip had been used to analyze the above samples, you
could expect the results to be in the range of 75 – 125 mg/L. That’s considerably better!)
In 2002 Dr.
Christian Butzke of UC Davis wrote a critique of the first few years of the
program. Regarding malic acid he wrote: “Given the corresponding consequences of
uncertainty over incomplete or deliberately prevented malolactic fermentation,
in particular CO2 evolution in the bottle and changes in perceived acidity,
this performance requires major improvements.”1 What would he say
today?
Don’t be fooled!
Spending money on expensive equipment or excessive time running
laboratory tests does not ensure reliable results.
|
What are
typical results you can expect? |
||
|
|
Actual
Value |
Expected
Range |
|
|
|
|
|
Accuvin |
98 |
73
- 123 |
|
per
the QC Survey |
98 |
18
- 178 |
|
|
|
|
|
Accuvin |
96 |
71
- 121 |
|
per
the QC Survey |
96 |
49
- 143 |
|
|
|
|
|
Accuvin |
112 |
87
- 137 |
|
per
the QC Survey |
112 |
26
- 196 |
|
|
|
|
|
Accuvin |
118 |
93
- 143 |
|
per
the QC Survey |
118 |
36
- 200 |
|
|
|
|
|
Accuvin |
103 |
78
- 128 |
|
per
the QC Survey |
103 |
36
- 170 |
|
|
|
|
|
Accuvin |
110 |
85
- 135 |
|
per
the QC Survey |
110 |
0
- 232 |
A final
thought: just getting the same answer
twice with a method performed unreliably does not make it the correct answer.
1
C. E. Butzke, Am. J. Enol. Vitic., 53 (20: 163 – 169 )
2002.
2
C. E. Butzke, Practical Winery & Vineyard, Jan. Feb. 2002
3
Collaborative Testing Services,
Inc., www.cts-interlab.com/wine/index.html,
reports, 2003 - 2008

Comments