Year
BHS (only)
BHS + AAMI
AAMI (only)
BHS (all)
AAMI (all)
ESH-IP
Total
2000
2 (20 %)
4 (40 %)
0 (0 %)
6 (60 %)
4 (40 %)
—
10
2001
1 (8 %)
5 (42 %)
1 (8 %)
6 (50 %)
6 (50 %)
—
12
2002
4 (29 %)
3 (21 %)
1 (7 %)
7 (50 %)
4 (29 %)
3 (21 %)
14
2003
2 (13 %)
0 (0 %)
6 (38 %)
2 (13 %)
6 (38 %)
8 (50 %)
16
2004
1 (6 %)
5 (29 %)
1 (6 %)
7 (41 %)
6 (35 %)
4 (24 %)
17
2005
2 (12 %)
4 (24 %)
0 (0 %)
6 (35 %)
4 (24 %)
7 (41 %)
17
2006
4 (13 %)
5 (17 %)
1 (3 %)
9 (30 %)
6 (20 %)
15 (50 %)
30
2007
1 (3 %)
2 (6 %)
3 (9 %)
3 (9 %)
5 (16 %)
24 (75 %)
32
2008
3 (9 %)
2 (6 %)
1 (3 %)
6 (17 %)
4 (11 %)
25 (71 %)
35
2009a
4 (14 %)
3 (10 %)
0 (0 %)
8 (28 %)
3 (10 %)
18 (62 %)
29
2002–2009a
21 (11 %)
24 (13 %)
13 (7 %)
48 (25 %)
38 (20 %)
104 (55 %)
190
Fig. 5.1
Cumulative graph of validation studies performed according to the European Society of Hypertension International Protocol (ESH-IP) compared with the British Hypertension Society (BHS) and Association for the Advancement of Medical Instrumentation (AAMI) protocols from 2002 (publication of ESH-IP) until June 2009. Reproduced from Stergiou et al. [25] with permission from the authors
In its introduction, the 2002 ESH-IP protocol included the sentence “It is anticipated that the relative ease of performance of the International Protocol will encourage manufacturers to submit blood pressure measuring devices for validation in order to obtain the minimum approval necessary for a device to be used in clinical medicine, and that, in time, most devices on the market will be assessed according to the protocol for basic accuracy”. The figures demonstrate that while the goal of having “most devices on the market” validated has still some way to go, the protocol did encourage manufacturers to submit devices for validation.
The analysis demonstrates the importance of practicality in the development of protocols. Protocols that are difficult and onerous to complete are less attractive both to centres and to manufacturers. Furthermore, they are rarely carried out correctly. Ideal evaluation is therefore best achieved, and perhaps can only be achieved, by a set of simple validations rather than by a single complex one.
Differences Between Protocols
At present, the ESH-IP validation protocol is the most widely used, followed by the AAMI/ANSI/ISO standard. There are some substantial differences between these validation procedures, as described below and shown in Table 5.2. A detailed comparison has also been made by Ng [28]. Nevertheless, they both have a common objective, namely the standardisation of validation procedures to establish minimum standards of accuracy and performance, and to facilitate comparison of one device with another. With the substantial data available, it is perhaps timely that a common protocol be developed that will become a world standard.
Protocol provision | ESH-IP 2010 | AAMI/ANSI/ISO 81060-2:2013 | |||||
|---|---|---|---|---|---|---|---|
Sample demographics | |||||||
Sample size | 33 | 85+ | |||||
Age | ≥25 years | Adult and adolescent | >12 years | ||||
Adult, adolescent, and children | 35 subjects 3–12 years and | ||||||
50 + subjects >25 years | |||||||
Sex | ≥10 Male and ≥10 Female | ≥30 % Male and ≥30 % Female | |||||
Arm circumference | 1 cuff | No requirements | ≥40 % upper ½, ≥40 % lower ½, | ||||
≥20 % upper ¼, ≥20 % lower ¼ | |||||||
n cuffs, n > 1 | ≥1/2n subjects/cuff | ||||||
Methodology | |||||||
Sequential same arm | 7 alternating measurements (4 observer, 3 test device) | 7 alternating measurements (4 observer, 3 test device) | |||||
Control measurement is average of 4 values (preceding and succeeding measurements of each observer) | |||||||
Control measurement is the nearer on the preceding and succeeding mean observer measurements to test measurement | Exclude measurement if observer measurements differ by >12 mmHg SBP or >8 mmHg DBP from other 2 measurements. (Max. 10 % of subjects; exclude subject if >10 %) | ||||||
Simultaneous same arm | Not permitted | Test device: Inflates ≥SBP + 20 mmHg, deflates ≤DBP—20 mmHg, deflation rate 2 mmHg/s to 3 mmHg/s | |||||
3 Pairs of measurements | |||||||
Control measurement is mean observer measurement | |||||||
Exclude subject if observer measurements differ by >12 mmHg SBP or >8 mmHg DBP | |||||||
Simultaneous opposite arm | Not permitted | 3 Pairs of measurements with test device on left arm +3 pairs with test device on right arm | |||||
Control measurement is mean observer measurement with lateral arm difference adjustment | |||||||
Exclude subject if observer measurements on same arm differ by >12 mmHg SBP or >8 mmHg DBP of if lateral difference >15 mmHg SBP or >10 mmHg DBP | |||||||
Recruitment blood pressure | |||||||
SBP | 90–100 mmHga | 10–12 | ≥1 | Initial pressures are recorded on each subject but they are not used in any calculations. Blood pressure distribution requirements are based on the reference control measurements only | |||
101–129 mmHg | |||||||
130–160 mmHg | 10–12 | ||||||
161–169 mmHg | 10–12 | ||||||
170–180 mmHga | ≥1 | ||||||
DBP | 40–50 mmHga | 10–12 | ≥1 | ||||
51–79 mmHg | |||||||
80–100 mmHg | 10–12 | ||||||
101–119 mmHg | 10–12 | ||||||
120–130 mmHga | ≥1 | ||||||
Control BP distribution | |||||||
SBP | ≤100 mmHg | 22/99–44/99 | Max. difference ≤19/99 | ≥5 % (13/255) | |||
101–129 mmHg | ≤75 % (191/255) | ||||||
130–139 mmHg | 22/99–44/99 | ||||||
140–159 mmHg | ≥20 % (51/255) | ||||||
160 mmHg | ≥5 % (13/255) | ||||||
>160 mmHg | 22/99–44/99 | ||||||
DBP | ≤60 mmHg | 22/99–44/99 | Max. difference ≤19/99 | ≥5 % | |||
61–79 mmHg | ≤75 % | ||||||
80–84 mmHg | 22/99–44/99 | ||||||
85–99 mmHg | ≥20 % | ||||||
100 mmHg | ≥5 % | ||||||
>100 mmHg | 22/99–44/99 | ||||||
Passing requirements (SBP and DBP) | |||||||
ESH-IP | Part 1 based on individual measurement errors | No. of errors | ≤5 mmHg | ≤10 mmHg | ≤15 mmHg | ||
≥2 of | ≥73/99 | ≥87/99 | ≥96/99 | ||||
All of | ≥65/99 | ≥81/99 | ≥95/99 | ||||
Part 2 based on measurement errors per subject | No. of subjects | ≥2/3 errors | 0/3 errors | ||||
≤5 mmHg | ≤5 mmHg | ||||||
Both | ≥24/33 | ≤3/33 | |||||
81060-2:2013 | Criterion 1 based on individual measurement errors | For comparison with ESH-IP, expected error rates within bands are shown when the mean error is 5 mmHg ± 8 mmHg | |||||
≤5 mmHg | ≤10 mmHg | ≤15 mmHg | Mean error (255 measurements) | Max. SD | |||
39.3 % | 70.1 % | 88.5 % | 5 mmHg | 8 mmHg | |||
Criterion 2 based on mean subject errors such that, using the normal distribution, the probability of a mean subject error being within 10 mmHg is ≥0.85. Selected examples shown | Probability of mean subject error within bands, based on normal distribution | Sample mean error and SD combinations that satisfy criteria | |||||
≤5 mmHg | ≤10 mmHg | ≤15 mmHg | Mean error (85 subject mean BPs) | Max. SD | |||
0.528 | 0.850 | 0.969 | 0 mmHg | 6.95 mmHg | |||
0.529 | 0.850 | 0.969 | 1 mmHg | 6.87 mmHg | |||
0.528 | 0.850 | 0.969 | 2 mmHg | 6.65 mmHg | |||
0.525 | 0.850 | 0.971 | 3 mmHg | 6.25 mmHg | |||
0.515 | 0.850 | 0.974 | 4 mmHg | 5.64 mmHg | |||
0.482 | 0.851 | 0.982 | 5 mmHg | 4.79 mmHg | |||
Special circumstances | |||||||
Exercise | No requirements | Supplementary study using 35 subjects | |||||
Passing Requirements
The AAMI/ISO standard requires two criteria to be fulfilled. In Criterion 1, the mean error of at least 255 measurements must be at most 5 mmHg with a standard deviation of at most 8 mmHg. In Criterion 2, the mean error of the mean measurements from at least 85 subjects must be at most 5 mmHg and the standard deviation at most a value to ensure that the mean error on at least 85 % of subjects is within 10 mmHg. (By extension, this definition also means that the expected number of subjects with average errors within the more commonly accepted error of 5 mmHg is around 50 %.) A table, showing the acceptable standard deviations for ranges of mean errors, is provided. For ABPM devices, similar criteria must also be applied to a further raised-heartrates test.
The ESH-IP protocol provides two sets of cut-off points. The primary set is for at least 73/99 measurement errors to be within 5 mmHg, at least 87/99 to be within 10 mmHg and at least 96/99 to be within 15 mmHg. The secondary set is for at least 65/99 measurement errors to be within 5 mmHg, at least 81/99 to be within 10 mmHg and at least 95/99 to be within 15 mmHg. Devices must fulfil at least two of the primary set and all of the secondary set. In addition, in at least 24/33 subjects, the errors in at least two of the three measurements recorded must be no more than 5 mmHg and there can be no more than 3/33 subjects with errors over 5 mmHg in all three measurements.
Methodology
The AAMI/ANSI/ISO standard allows simultaneous same-arm comparisons if the test device inflates to at least 20 mmHg above SBP and deflates to at least 20 mmHg below DBP at a rate of between 2 and 3 mmHg/s. Otherwise, sequential same-arm comparisons or simultaneous opposite-arm comparisons must be used. The ESH-IP protocol stipulates sequential same-arm comparisons in all circumstances.
Recruitment Requirements
For Criterion 1, the AAMI/ISO standard requires 255 measurements to be recorded in at least 85 subjects, with at most three measurements per subject. For Criterion 2, 85 subjects with at least three measurements per subject are required. This number of measurements per subject is doubled if the simultaneous opposite-arm methodology is used, with each arm being used for half the control and test measurements. For ABPM devices, a further 35 subjects are required for a validation with raised heartrates following exercise.
The ESH-IP protocol requires three measurements to be recorded in each of 33 subjects. However, strict requirements generally necessitate several more subjects to be recruited in order for these to be satisfied.
Pressure-Range Requirements
The ESH-IP protocol requires between 10 and 12 subjects with SBP in the ranges <130, 130–160 and >160 mmHg; between 10 and 12 subjects with DBP in the ranges <80, 80–100 and >100 mmHg; a total of at most for subjects with pressures outside the “ideal” range of 90–180 mmHg for SBP and 40–130 mmHg for DBP and minimum ranges of 100–170 mmHg for SBP and 50–120 mmHg for DBP. In addition, the 99 control measurements must not be statistically different from the ideal 33 measurements in each range; that is each range must contain 22 and 44 measurements and the difference between the range with the highest count and that with the lowest count cannot exceed 19.
The AAMI/ISO standard requires at least 5 % (13 measurements, requiring at least five subjects) of reference systolic blood pressures have to be at least 160 mmHg, at least 20 % (51 measurements, requiring at least 17 subjects) at least 140 mmHg and at least 5 % at most 100 mmHg. Similarly, at least 5 % of reference diastolic blood pressures have to be at least 100 mmHg, at least 20 % at least 85 mmHg and at least 5 % at most 60 mmHg.
Age Distribution Requirements
The ESH-IP protocol requires all subjects to be at least 25 years of age. The AAMI/ISO standard defines two types of studies based on age. The first is for an adult/adolescent population defined simply as on persons over the age of 12. The alternative population is a combined adult/paediatric population where 35 subjects must be between the ages of 3 and 12; the remainder (minimum 50) must be over the age of 12.
Sex Distribution Requirements
Both the ESH-IP protocol and the AAMI/ISO standard are at ad idem requiring a minimum of 30 % (stated as 10/33 in the ESH-IP protocol) male and 30 % female subjects.
< div class='tao-gold-member'>
Only gold members can continue reading. Log In or Register to continue
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
