Skip to contents

Technical reference: adult algorithm

2026-11-08

Source: vignettes/adult-algorithm.Rmd
adult-algorithm.Rmd

Purpose. This is a technical reference with an extremely detailed description of the adult algorithm. It is primarily intended for developers, but may be of interest to people who want to know more about how growthcleanr works.

This narrative documents the adult algorithm cleaning steps (Adult Steps 1, 2W, 3, 4W, 9Wa/9H/9Wb, 10H/10W, 11H/11Wa/11Wa2/11Wb, 13, 14).

Wrapper-level material lives in vignette("wrapper"): the cleangrowth() public interface, input format, unit conversion, batching, dispatch, output reassembly, and cross-algorithm working-dataframe differences (the adult algorithm’s copies-into-shrinking-dataframes pattern is contrasted there with the child algorithm’s single-data.df pattern).

The adult algorithm works with raw measurements (and BMI computed internally for some thresholds); it does not use the recentered WHO/CDC z-scores that drive the child algorithm. The one exception is Evil Twins (Adult Step 9Wa), which uses a small fixed-NHANES conditional-SD (CSD) reference for the unit-error-range flag and the median-band |z| tiebreak (see the Support Functions cross-reference to vignette("algorithm-functions")). The Z-Score Infrastructure section in the wrapper narrative is otherwise not load-bearing for the adult algorithm but may be useful context for cross-algorithm work.

Key concepts

EWMA: Exponentially Weighted Moving Average — a weighted average of a subject’s other weight measurements, where measurements closer in time receive exponentially more weight. The exponent is e=−5, making the nearest neighbor almost entirely dominant; this is closer to a near-neighbor comparison than a traditional moving average. Three variants are computed for each observation: ewma_all (all other values), ewma_before (excluding the immediately preceding value), and ewma_after (excluding the immediately following value). The before/after variants are used as confirmation — a value is only excluded when the overall and both directional deviations agree. Edge values (first/last in subject) have only one neighbor, so before/after falls back to ewma_all. Window parameter ewma_window (default 15) limits how many observations on each side contribute.

Ordered steps: Removing more extreme problems first, getting more refined at the end

Iterations: Many exclusion steps use a dynamic loop: one value is excluded per round (the worst candidate by a defined scoring rule), then the EWMA and thresholds are recomputed from scratch on the remaining values, and the process repeats. The loop runs until no candidates remain or fewer than 3 values are left. This single-worst-out-per-round approach prevents a cluster of outliers from masking each other — removing the most deviant value first often exposes the next, which would have appeared plausible if evaluated simultaneously.

Exclusion rescue: The adult algorithm does not rescue exclusions the way the child algorithm rescues carried-forward values. The one exception is the moderate EWMA step (11Wb): when 4 or more consecutive candidates are found in a single pass, they are all excluded at once with an -Error-Load code rather than one per round (in linked mode, if any of them has RV copies, all of the subject’s remaining weights are excluded). This treats a long run of EWMA failures as evidence of overall data unreliability rather than a sequence of isolated errors.

Flexibility: Defaults to looser (moderate permissiveness). Can be made more permissive (loosest) or more restrictive (tighter, tightest).

Missing measurements: A row whose measurement is missing (NA/NaN — a parameter row present in the data but carrying no value, including a 0 converted to NaN upstream) is set aside (reported as Exclude-Missing-Info) before any cleaning step runs and does not participate in any step. This matters because the algorithm derives quantities from the measurements (BMI from a same-day height+weight pair, EWMA deviations, medians) and compares them against thresholds; a missing value left in play would propagate as NA into those comparisons. Such rows are removed from the per-subject height and weight working sets at entry, so they are never a same-day BMI partner and never enter a trajectory. They are reported as Exclude-Missing-Info in the final output.

Parameters: The adult algorithm cleans height and weight only. cleangrowth() relabels adult LENGTHCM/LENGTHIN rows to HEIGHTCM/HEIGHTIN before dispatch, so adult length is cleaned as height (the submitted label is still returned). Adult head circumference is not cleaned: cleangrowth() marks every HEADCM row at or above adult_cutpoint Exclude-HC-Out-of-Range, as for child HC past 5 years. HC rows play no part in any adult height or weight step.

Rounding Tolerance

All threshold comparisons in the algorithm apply a 0.12 unit tolerance (cm for height, kg for weight). This accommodates rounding that may occur during data extraction, transformation, and loading — specifically, rounding to the nearest 0.1 cm or 0.1 kg, plus a small margin for floating-point precision.

The tolerance makes all threshold comparisons slightly more permissive: a value must exceed a limit by more than 0.12 to be excluded. For example, if the PIV minimum height is 50 cm, a recorded value of 49.9 cm is not excluded (it is within 0.12 of the threshold). When comparing two measured values (e.g., height difference in a band check), the tolerance is applied once (0.12), not doubled.

Where tolerance is applied:

  • PIV limits (Adult Step 1): value vs fixed min/max thresholds
  • Weight cap EWMA checks (Adult Step 4W): dewma_all vs 50 kg; dewma_before/dewma_after vs 45 kg (0.9 factor applied to 50 kg base); adjacency diffs vs 50 kg
  • Evil twins caps (Adult Step 9Wa): weight difference vs dynamic cap
  • Extreme EWMA (Adult Step 9Wb): dewma vs dynamic threshold
  • Height distinct (Adult Steps 10Ha/10Hb): band checks, gain/loss direction and rescue
  • Weight 2D ordered and non-ordered (Adult Steps 11Wa/11Wa2): weight difference vs wtallow
  • Moderate EWMA (Adult Step 11Wb): dewma vs wtallow (standard and alternate pathways)
  • 1D limits (Adult Step 13): value vs fixed min/max thresholds

Where tolerance is NOT applied:

  • BMI thresholds (derived from ht+wt; 0.12 in either produces <0.1 BMI unit change)
  • Perclimit ratio checks (conversion factors cancel in ratios)
  • Error load ratio (Adult Step 14; count-based, not measurement-based)
  • Exact equality checks (RV identification, SDE identical detection — same conversion applied uniformly, so identical inputs produce identical outputs)

Permissiveness Framework

The algorithm’s aggressiveness is controlled by a permissiveness argument with four levels. Each level sets defaults for multiple settings. Individual settings can be overridden; explicit values always take precedence over the preset.

Level Goal
loosest Retain all plausible values. Allows extreme weight changes (bariatric surgery, rapid fluid status change), significant height measurement error, height gain in young adults, and height loss.
looser (default) Retain most plausible values. Slightly less extreme weights allowed. Height gain allowed; height loss beyond the 3-inch band is not.
tighter Exclude plausible values likely reflecting illness with large effect on body size. Less permissive height limits. Height gain allowed; height loss beyond the 2-inch band is not. Repeated values treated as less informative.
tightest Exclude extreme measurements even if plausible. Tightest weight and height limits. No height gain or loss allowance. Repeated values treated as less informative.

For the intended use of each level and the preset values, see Section 1D of vignette("configuration").

The allofus15 weight-allowance formula used at the tightest level was adapted from the algorithm developed by Guide et al. for the All of Us Research Program:

> Guide A, Sulieman L, Garbett S, Cronin RM, Spotnitz M, Natarajan K, Carroll RJ, Harris P, Chen Q. Identifying erroneous height and weight values from adult electronic health records in the All of Us research program. J Biomed Inform. 2024 Jul;155:104660. doi: 10.1016/j.jbi.2024.104660. Epub 2024 May 23. PMID: 38788889; PMCID: PMC11973958.

Parameters controlled by permissiveness

Setting loosest looser tighter tightest
Overall (PIV) HT limits 50–244 cm 120–230 cm 142–213 cm 147–208 cm
Overall (PIV) WT limits 20–500 kg 30–270 kg 36–159 kg 39–136 kg
Overall (PIV) BMI limits¹ 5–300 12–65 16–45 18–40
Single (1D) limits wider BMI-dependent split (see Adult Step 13) same as PIV same as PIV same as PIV
wtallow_formula PW-H ("piecewise") PW-H ("piecewise") PW-L ("piecewise-lower") allofus15 ("allofus15")
UW scaling UW-based (see the wtallow formulas section) UW-based UW-based cap limited by PW-L
ET caps wtallow cap + 20 wtallow cap + 20 wtallow cap + 20 allofus15-cap-12m
perclimit_low (wt ≤45 kg) 0.5 0.5 0.7 0.7
perclimit_mid (45<wt≤80 kg) 0.4 0.4 0.4 0.4
perclimit_high (wt >80 kg) 0 (disabled) 0 (disabled) 0.4 0.4
error_load_threshold 0.41 0.41 0.29 0.29
mod_ewma_f 0.75 0.75 0.60 0.60
ht_band 3” 3” 2” 2”
allow_ht_loss TRUE FALSE FALSE FALSE
allow_ht_gain TRUE TRUE TRUE FALSE
repval_handling independent independent linked linked

¹ Adult Step 1 applies these limits only when they are wider than the single-value BMI limits (at the presets, loosest only). At the other levels they equal the single-value limits, which are applied at Adult Step 13.

Single source of truth for these preset values: vignette("configuration") §1D (which also lists the exact parameter names and per-step roles). The table above is a reading convenience; if the two ever disagree, §1D and the code (permissiveness_presets()) are canonical.

Implementation

The permissiveness argument defaults to "looser". All settings default to NULL in the function signature. At the start of cleanadult(), resolve_permissiveness() fills NULLs from the preset table and passes through any non-NULL (user-specified) values. This means:

  • cleangrowth(df) uses all looser defaults
  • cleangrowth(df, adult_permissiveness = "tighter") uses all tighter defaults
  • cleangrowth(df, adult_permissiveness = "tighter", adult_overrides = list(allow_ht_loss = TRUE)) uses tighter defaults but overrides height loss to be allowed

The parameter names in the step tables below are the names used in adult_overrides, except max_rounds (fixed, not overridable), scale_max_lbs (set with adult_scale_max_lbs), and ewma_window (set with ewma_window).

Parameters NOT controlled by permissiveness: scale_max_lbs (default Inf), ewma_window (default 15), quietly (default FALSE).

Exclusion codes and output levels

Throughout this narrative, each step’s “Exclusion Code(s)” row and inline references use the algorithm’s internal literal codes — the strings cleanadult() actually writes to its result column (Exclude-A-PIV, Exclude-A-Scale-Max, Exclude-A-Traj-Moderate, the RV / firstRV / allRV / -All / -RV-Propagated / -Error-Load variants, etc.). These internal literals are not renamed in the code.

At the end of cleangrowth(), a mapping layer translates each internal code into the two reported output levels:

  • Summary — the default exclude column. Consolidated across steps and across child/adult.
  • Detailed — the exclude_detailed column, added when exclude_detail = TRUE (or full_detail = TRUE). Keeps the -A- prefix, renames PIV→Hard-Limit, and folds the RV/firstRV/allRV/-All/-RV-Propagated passes into their base code (the -Error-Load escalation is retained as its own Detailed code). Shared same-day codes (Exclude-Identical, Exclude-Extraneous) carry no prefix at any level.

Adult internal → Detailed → Summary mapping (canonical catalog + child side: vignette("exclusion-codes")):

Code (internal) Adult Step Detailed (exclude_detailed) Summary (exclude)
Exclude-A-PIV 1H/1W Exclude-A-Hard-Limit Exclude-Hard-Limit
Exclude-A-Scale-Max (+ -Identical, -RV-Propagated) 4W Exclude-A-Scale-Max Exclude-Scale-Max
Exclude-A-Evil-Twins 9Wa Exclude-A-Evil-Twins Exclude-Extreme
Exclude-A-Identical 9H/10W Exclude-Identical Exclude-Identical
Exclude-A-Extraneous 9H/10W Exclude-Extraneous Exclude-Extraneous
Exclude-A-Traj-Extreme (+ -firstRV, -allRV, -firstRV-RV-Propagated) 9Wb Exclude-A-Traj-Extreme Exclude-Extreme
Exclude-A-Ord-Pair (+ -All) 10Ha Exclude-A-Ord-Pair Exclude-Pattern
Exclude-A-Window (+ -All) 10Hb Exclude-A-Window Exclude-Pattern
Exclude-A-2D-Ordered 11Wa Exclude-A-2D-Ordered Exclude-Pattern
Exclude-A-2D-Non-Ordered 11Wa2 Exclude-A-2D-Non-Ordered Exclude-Pattern
Exclude-A-Traj-Moderate (+ -allRV, -RV-Propagated) 11Wb Exclude-A-Traj-Moderate Exclude-Pattern
Exclude-A-Traj-Moderate-Error-Load (+ -RV) 11Wb Exclude-A-Traj-Moderate-Error-Load Exclude-Pattern
Exclude-A-Single 13 Exclude-A-Single Exclude-Single
Exclude-A-Unevaluable-Trajectory 14 Exclude-A-Unevaluable-Trajectory Exclude-Traj-Uneval

Adult has no Exclude-Pair at the Summary level — the adult 2-distinct-value HT pair/window checks are trajectory disagreement and fold into Exclude-Pattern (the child Exclude-Pair means a sparse 2-measurement subject, a different concept).

List of steps

For the most part, HT and WT are evaluated separately, although there are some shared steps. See separate tables for HT and WT steps below.

Adult Step numbering is not sequential. Adult Steps 5–8 and 12W are not present in the algorithm; numbering is preserved for consistency with prior documentation.

Height steps (include steps shared by HT and WT):

Adult Step # Adult Step title Brief description
1H PIV Exclude population implausible heights
3H Temp SDE Temporarily flag same-day height duplicates
9H Final SDE Height Resolve same-day height duplicates: exclude identical, choose keeper by category
10H Height Distinct Evaluate 2D pairs (band, loss/gain rescue, frequency) and 3+D windows (w2, loss/gain groups)
11H Mean Height Calculate mean height per subject (group-aware for loss/gain groups)

Weight steps (include steps shared by HT and WT):

Adult Step # Adult Step title Brief description
1W PIV Exclude population implausible weights
2W Repeated Values Identify groups of identical weight values; mark first vs. subsequent
3W Temp SDE Temporarily flag same-day weight duplicates
4W Scale Max Exclude weights at physical scale maximum unless EWMA confirms
9Wa Evil Twins Exclude inner members of an over-the-limit weight pair flanked by stable anchors (anchor rule)
9Wb Extreme EWMA Exclude weight outliers with large EWMA deviations (interval-specific caps)
10W Final SDE Weight Resolve same-day weight duplicates: exclude identical, choose keeper by category
11Wa 2D Ordered Weight Exclude 2D ordered weight pairs outside wtallow or perclimit
11Wa2 2D Non-Ordered Weight Exclude 2D interleaved weight pairs using wtallow + dominance rules
11Wb Moderate EWMA Exclude weight outliers using 7-step EWMA flow with trajectory rescue and error load
13 Single Distinct (1D) Exclude 1D height/weight values outside BMI-based or no-BMI limits
14 Error Load Exclude all remaining values when error ratio exceeds threshold

Configurable Arguments

For wrapper-level arguments (id, adult_cutpoint, sd.recenter, ref.data.path, batch_size, parallel, cores, log.path, ref_tables, tri_exclude, quietly), see vignette("wrapper") → Configurable Arguments (wrapper-level).

For the full cross-algorithm parameter and threshold index (including all hardcoded adult thresholds by step), see vignette("configuration").

This table covers adult-specific arguments exposed via cleangrowth(). All permissiveness settings (PIV limits, 1D limits, wtallow_formula, mod_ewma_f, perclimit_*, ht_band, allow_ht_loss, allow_ht_gain, repval_handling, error_load_threshold) are set by adult_permissiveness and documented in full in vignette("configuration") Section 1D. Individual settings can also be overridden without switching preset via the adult_overrides named list, e.g. cleangrowth(df, adult_overrides = list(overall_ht_min = 120)).

Argument Default Used in Description
adult_permissiveness "looser" All adult steps (via preset resolver) Permissiveness level: "loosest", "looser", "tighter", "tightest". Sets all settings unless individually overridden. See vignette("configuration") Section 1D for the full preset table.
adult_scale_max_lbs Inf Adult Step 4W Physical scale upper limit in pounds. Weights within ±0.1 kg of the cap are excluded if they are more than 50 kg different from their expected value, or if all of the subject’s weights (at least two) are the same value at the cap.
ewma_window 15 Adult Steps 9Wb, 11Wb Maximum observations on each side contributing to EWMA weighting. Shared with child algorithm — a single cleangrowth() argument affects both.

Support Functions

Algorithm-internal helpers defined in R/adult_support.R. Each entry lists purpose, callers, input/return contract, key invariants, and approximate code location. For step-implementing functions already fully documented in their step sections, this section provides a brief cross-reference only. For wrapper-level helpers shared across algorithms, see vignette("wrapper") → Shared Helpers.

Adult CSD reference + unit-error-range helpers (.adult_csd_ref(), .adult_csd_lookup(), .adult_z3(), .adult_csd_z(), .unit_error_range_flag())

Cross-reference. These cross-algorithm helpers — the small fixed sex-specific NHANES CSD reference (.adult_csd_ref() and its lookup/z3/per-value-z wrappers) and the shared unit-error-range band logic (.unit_error_range_flag()) — are documented in full in vignette("algorithm-functions"). They are defined in R/adult_support.R so both the adult algorithm and the standalone adult test harness can reach them. In the adult algorithm they are consumed inside Evil Twins (Adult Step 9Wa): .adult_z3() + .unit_error_range_flag() build the per-value sp_uer flag, and .adult_csd_z() provides the UER3 plausible-group split and the median-band |z| tiebreak (see the Step 9Wa walkthrough and the “Adult CSD z” / “UER3” definitions there). The constants’ provenance (NHANES 2017–2020 + 2021–2023) is recorded in that reference and in the .adult_csd_ref() roxygen.

Permissiveness configuration: permissiveness_presets() / resolve_permissiveness()

Purpose. permissiveness_presets() returns a named list of four complete parameter sets (loosest, looser, tighter, tightest), each containing the full set of adult algorithm thresholds and flags. resolve_permissiveness() merges a preset with user-supplied parameter overrides: any non-NULL override replaces the preset value; NULL values fall through to the preset default.

Callers. permissiveness_presets() is exported and may be called directly by users. It is also called from within resolve_permissiveness(). resolve_permissiveness() is called once at the top of cleanadult() before any algorithm steps run.

Inputs and return contract.

  • permissiveness_presets() — no arguments. Returns a named list of four lists; each inner list has identical parameter names (PIV/1D limits, wtallow_formula, perclimit_*, error_load_threshold, mod_ewma_f, ht_band, allow_ht_loss, allow_ht_gain, repval_handling).
  • resolve_permissiveness(permissiveness, ...) — permissiveness must be one of the four valid level strings; stops with an informative message otherwise. ... are named parameter overrides. Returns a single flat named list of resolved parameters identical in structure to one inner list from permissiveness_presets().

Key invariants. All four preset lists share the same parameter names. The resolved list keys match the named parameters that cleanadult() unpacks and passes to each step.

Code location. R/adult_support.R.

Utility helpers: check_between() / round_pt()

Purpose. Small one-liners used throughout the algorithm. check_between(vect, num_low, num_high, incl = TRUE) returns a logical vector indicating whether elements of vect fall within the given range (inclusive by default; incl = FALSE for strict). round_pt(val, pt) rounds val to the nearest multiple of pt (e.g. round_pt(78.14, 0.2) → 78.2).

Callers. check_between() is called from cleanadult() at Adult Step 4W (weight cap range check at two sites) and Adult Step 10H (height w2-window inclusion/exclusion). round_pt() has no active callers (it is listed in var_for_par for parallel export only); compute_trajectory_fails() performs the equivalent 0.2 kg rounding inline (round(x / 0.2) * 0.2), not via round_pt().

Inputs and return contract. Both functions are fully vectorized. check_between() returns a logical vector the same length as vect. round_pt() returns a numeric vector the same length as val. No input validation — callers are responsible for non-NA numeric inputs.

Code location. R/adult_support.R.

Adult EWMA: as.matrix.delta_dn() / ewma_dn()

Purpose. as.matrix.delta_dn(agedays) builds an n × n matrix of pairwise absolute age differences — the weight matrix used by adult EWMA calculations. ewma_dn() is the original adult EWMA function using |delta|^(–5) weighting (the adult exponent; the child algorithm uses (5 + |delta|)^(–1.5) instead). It returns three variants: EWMA-all, EWMA-before (excluding predecessor), EWMA-after (excluding successor).

Callers. as.matrix.delta_dn() is called from adult_ewma_cache_init() (and was also used by ewma_dn()). ewma_dn() has no active callers in the current algorithm — it was superseded by adult_ewma_cache_init() when the cache-based approach was introduced. Both functions are listed in var_for_par (the parallel-worker export list in child_clean.R) but ewma_dn() is not invoked from any algorithm step or support function.

Inputs and return contract.

  • as.matrix.delta_dn(agedays) — numeric vector; returns an n × n numeric matrix.
  • ewma_dn(agedays, meas, ewma.exp = -5, ewma.adjacent = TRUE, ewma_window = 15) — agedays and meas are aligned numeric vectors. Returns a data.frame (unlike the child ewma() which returns a named list) with columns ewma.all, ewma.before, ewma.after. When ewma.adjacent = FALSE, returns only ewma.all. Emits an unsorted-input warning but does not stop.

Key invariants. Input should be sorted by age. The window parameter zeroes out weight-matrix entries more than ewma_window positions from the diagonal (position-based, not age-based). For n ≤ 2, all three EWMA variants equal ewma.all.

Code location. R/adult_support.R.

Adult EWMA cache: adult_ewma_cache_init() / adult_ewma_cache_update()

Purpose. adult_ewma_cache_init() builds the full weight matrix once (O(n²)) and returns pre-computed EWMA-all, EWMA-before, and EWMA-after vectors as a plain named list. adult_ewma_cache_update() was designed to remove one observation from the cache in O(n) arithmetic — a performance path that avoids rebuilding the full matrix each round.

Callers. adult_ewma_cache_init() is called from remove_ewma_wt() (Adult Step 9Wb) and remove_mod_ewma_wt() (Adult Step 11Wb) — once per iteration round in each. adult_ewma_cache_update() has no active callers: both step functions rebuild the full cache from scratch each round rather than using the incremental update.

Inputs and return contract.

  • adult_ewma_cache_init(agedays, meas, ewma_exp = -5, ewma_window = 15) — sorted, aligned numeric vectors. Returns a named list: delta (n × n weight matrix), ws (weighted sums), rs (row sums), meas, agedays, n, ewma_all, ewma_before, ewma_after. Returns NULL for n = 0.
  • adult_ewma_cache_update(cache, pos_j) — cache from adult_ewma_cache_init(); pos_j is the integer row index to remove. Returns a new named list with n – 1 observations (does not mutate the input, unlike the child version which mutates an environment in place). Returns NULL when the input cache has n ≤ 1 (removal would leave n = 0).

Key invariants. Input must be sorted by age. For n ≤ 2, EWMA-before and EWMA-after equal EWMA-all (no distinct predecessor/successor to remove). The return from adult_ewma_cache_update() has the same structure as adult_ewma_cache_init() and can be passed back to adult_ewma_cache_update() iteratively.

Code location. R/adult_support.R.

PIV detection: remove_piv() / remove_piv_low() / remove_piv_high()

Purpose. remove_piv() is the PIV dispatcher: returns a logical vector indicating which rows in subj_df are population implausible — too low OR too high. remove_piv_low() and remove_piv_high() are one-line sub-helpers that check the low and high bounds independently, each applying the 0.12 cm/kg rounding tolerance.

Callers. remove_piv() is called from cleanadult() at Adult Steps 1H and 1W, once per parameter per subject.

Inputs and return contract.

  • remove_piv(subj_df, type, piv_df) — subj_df has a meas_m column; type is a row-name key into piv_df (a matrix-like object with “low” and “high” columns); returns a logical vector the same length as nrow(subj_df).
  • remove_piv_low(subj_df, type, piv_df) — returns logical: TRUE if meas_m < piv_df[type, "low"] – 0.12.
  • remove_piv_high(subj_df, type, piv_df) — returns logical: TRUE if meas_m > piv_df[type, "high"] + 0.12.

Key invariants. The 0.12 rounding tolerance is applied at every PIV check in the algorithm. piv_df must have a row named type and columns named “low” and “high”.

Code location. R/adult_support.R.

Repeated value detection: identify_rv() / redo_identify_rv()

Purpose. identify_rv() scans a single-subject weight data frame for repeated (identical) measurement values and adds two boolean columns: is_rv (TRUE for all occurrences after the first) and is_first_rv (TRUE for the first occurrence of any value that has at least one duplicate). Uses exact numeric equality. redo_identify_rv() handles the case where temp_sde() marked an is_first_rv row as extraneous — it subsets to non-extraneous rows, re-runs identify_rv(), and maps results back to the full data frame so the RV group still has a valid is_first_rv.

Callers. identify_rv() is called from cleanadult() at Adult Step 2W (initial RV scan after PIV) and again after non-SDE exclusions (weight cap, evil twins). redo_identify_rv() is called from cleanadult() immediately after each temp_sde() call that could have displaced an is_first_rv.

Inputs and return contract.

  • identify_rv(w_subj_df) — single-subject weight data frame, assumed pre-sorted by age_days / internal_id. Returns the same data frame with is_rv and is_first_rv columns added or overwritten.
  • redo_identify_rv(w_subj_df) — same structure. Only re-derives RV flags when at least one is_first_rv is currently extraneous; returns unchanged data frame otherwise.

Key invariants. Exact numeric equality only — floating-point near-duplicates (e.g. 78.1 vs 78.101) are not treated as repeated values. Does not filter by extraneous status; the identify_rv → temp_sde → redo_identify_rv cycle is the caller’s responsibility to sequence correctly.

Code location. R/adult_support.R.

Temporary SDE flagging: temp_sde()

Purpose. Selects the keeper on each same-day group, marking all other rows extraneous = TRUE. The keeper is the row with the smallest absolute deviation from the subject median; ties keep the highest internal_id (consistent with final SDE resolution). For weight (ptype = "weight"), the subject median is computed over non-RV values only; for height (ptype = "height"), over all values.

Callers. Called from cleanadult() at Adult Steps 3H and 3W (initial temp SDE), and again within the Adult Step 9H and 10W final-SDE resolution sequences.

Inputs and return contract.

  • temp_sde(subj_df, ptype = "height") — single-subject data frame with age_days, meas_m, internal_id, and (for weight) is_rv. Returns the same data frame with extraneous column added. If nrow == 0, returns as-is. If nrow == 1 or no duplicate days, returns with extraneous = FALSE for all rows.

Key invariants. For weight, the median excludes is_rv == TRUE rows. Duplicate-day detection is character-key-based (table(subj_df$age_days) → names(...)[count > 1]), so the result is stable across integer and double age_days representations.

Code location. R/adult_support.R.

Weight allowance: compute_wtallow() / compute_et_limit() / compute_perc_limit()

Purpose. Three functions for interval- and weight-based thresholds used across multiple steps. compute_wtallow() computes the maximum plausible weight change for a given interval and, optionally, upper weight (UW). compute_et_limit() computes the Evil Twins / extreme EWMA cap (wtallow-cap + 20 kg for PW-H/PW-L; allofus15 12-month cap for allofus15; fixed 70/100 for custom CSV). compute_perc_limit() returns an observation-level percentage ratio threshold tiered by weight.

See the wtallow section below for full formula derivations and design rationale.

Callers. compute_wtallow() is called from cleanadult() (Adult Steps 11Wa and 11Wa2) and from evil_twins(), remove_ewma_wt(), and remove_mod_ewma_wt(). compute_et_limit() is called from evil_twins() and remove_ewma_wt(). compute_perc_limit() is called from remove_mod_ewma_wt().

Inputs and return contract.

  • compute_wtallow(months, formula = "piecewise", uw = NULL) — months is a numeric vector of interval lengths in months; formula is "piecewise" (PW-H), "piecewise-lower" (PW-L), "allofus15", or a file path to a custom CSV with columns months and wtallow; uw is an optional upper-weight vector (NULL = base, UW = 120). Returns a numeric vector of weight allowances.
  • compute_et_limit(interval_months, formula = "piecewise", uw = NULL) — same signature; returns ET caps. Two-tier output: ≤ 6 months → et_6m cap, > 6 months → et_12m cap (except allofus15, which is flat across all intervals).
  • compute_perc_limit(meas, perclimit_low, perclimit_mid, perclimit_high) — vectorized over meas (weight in kg). Returns perclimit_low for meas ≤ 45, perclimit_mid for 45 < meas ≤ 80, perclimit_high for meas > 80.

Key mechanics not in the wtallow formulas section.

  • All three formulas cap uw_eff at 180 kg via pmin(uw, 180) before applying scale factors.
  • Custom CSV path: the file is loaded once and cached in the package-internal environment .gc_cache under .wtallow_custom_cache; a path attribute on the cache detects when a different file is passed. This side effect persists across cleangrowth() calls within the same R session.
  • compute_et_limit() for allofus15: returns a flat value (the allofus15 12-month cap for that UW) regardless of interval — unlike PW-H/PW-L which have ≤ 6m / > 6m tiers.

Code location. R/adult_support.R (compute_et_limit, compute_perc_limit, compute_wtallow).

Height evaluation: ht_allow() / ht_change_groups() / ht_3d_growth_compare()

Purpose. Three helpers supporting Adult Step 10H (Height Distinct Values). ht_allow(velocity, ageyears1, ageyears2) computes the maximum expected height gain over an age range using a log-based growth model. ht_change_groups(h_subj_df, cutoff, type) groups consecutive height measurements within 2 inches (5.08 + 0.12 cm rounding tolerance) of each other; exits early if a measurement diverges by more than 2 inches from the current group in the specified direction. ht_3d_growth_compare(mean_ht, min_age, glist, compare) checks whether adjacent height groups have plausible height differences, applying velocity tiers based on the inter-group age gap, with a 25-year age cap.

Callers. All three are called exclusively from cleanadult() in the Adult Step 10H block. ht_allow() is called only from within ht_3d_growth_compare().

Inputs and return contract.

  • ht_allow(velocity, ageyears1, ageyears2) — three scalars; returns a scalar. Formula: velocity × (log(ageyears2 – 16.9) – log(ageyears1 – 16.9)). Undefined for ageyears ≤ 16.9 — safe in practice because the adult algorithm processes only ages at or above adult_cutpoint (18–20 years).
  • ht_change_groups(h_subj_df, cutoff, type = "loss") — h_subj_df pre-sorted by age; cutoff is the maximum number of groups to form; type = "loss" exits early if any measurement drops > 5.08 + 0.12 cm below the current group minimum, type = "gain" exits if any measurement rises > 5.08 + 0.12 cm above the current group maximum. Returns a list with entries "meas" (list of named numeric vectors, names = internal_id) and "age" (list of age vectors). Returns empty lists if the early-exit condition fires before completing any groups.
  • ht_3d_growth_compare(mean_ht, min_age, glist, compare = "before") — mean_ht and min_age are numeric vectors indexed by group; compare = "before" checks each group against the immediately preceding group, compare = "first" checks against the first group. Velocity tiers: < 1-year gap → 20 cm/yr, 1–3 year gap → 15 cm/yr, > 3-year gap → 12 cm/yr. Comparison age is capped at 25 years. Returns a single boolean: TRUE if any adjacent pair fails the velocity check.

Key invariants. ht_3d_growth_compare() skips groups indexed beyond length(glist) via next, so short group lists are safe. The 0.12 cm rounding tolerance is applied to the 5.08 cm (2-inch) threshold in ht_change_groups(). The height loss/gain check uses mh2 – mh1 relative to zero and hta, with the 0.12 tolerance applied at both bounds.

Code location. R/adult_support.R.

Trajectory helpers: detect_runs() / compute_trajectory_fails()

Purpose. detect_runs(flagged) detects consecutive runs of TRUE in a logical vector and returns run metadata for each element: run_id (which run), run_len (length of that run), and run_pos (position within the run). compute_trajectory_fails(meas, age_days, err = 5) pre-computes which observations fail all three trajectory rescue methods — returning TRUE for “not rescued” (candidate eligible for exclusion). The three rescue methods: (1) within ±err kg of the linear interpolation between the two adjacent observations; (2) within ±err kg of the extrapolation from the prior pair (p2 → p1 → current); (3) within ±err kg of the extrapolation from the next pair (n2 → n1 → current). Extrapolation is not trusted when the extrapolation distance exceeds 2× the source interval.

Callers. detect_runs() is called from remove_mod_ewma_wt() twice per round (once for the standard-pathway candidates, once for the alternate-pathway candidates). compute_trajectory_fails() is called from remove_mod_ewma_wt() once per round.

Inputs and return contract.

  • detect_runs(flagged) — logical vector; returns a named list with three integer vectors of the same length: run_id (NA for non-run elements), run_len (NA for non-run elements), run_pos (NA for non-run elements). All NAs for empty input or all-FALSE input.
  • compute_trajectory_fails(meas, age_days, err = 5) — aligned numeric vectors; returns a logical vector. For n < 3, returns all TRUE (no rescue possible). NA interpolation/extrapolation results are explicitly coerced to FALSE before the final fails computation, so NA observations are treated as “rescued” (kept).

Key invariants. Extrapolated values in compute_trajectory_fails() are rounded to the nearest 0.2 kg inline (round(x / 0.2) * 0.2, equivalent to round_pt(x, 0.2)) before the ±err window is applied — matching the 0.2 kg rounding precision used elsewhere. The distance guard compares raw absolute age differences without rounding tolerance.

Code location. R/adult_support.R.

Evil Twins: evil_twins()

See Evil Twins (Adult Step 9Wa) in this narrative for full documentation.

Brief. Applies the anchor rule — excludes an inner member of an over-the-limit (OTL) weight pair (inner B–C jump > ET cap) only when both outer pairs (A–B, C–D) anchor within 60% of their cap, ceding single spikes to the EWMA steps. One exclusion per iteration, re-anchoring on the remaining values, until no OTL pair remains or fewer than 4 eligible values are left. Selection uses a windowed-median deviation (absd_med) with a UER3 unit-error check and a median band; the |z| tiebreak and UER3 plausible group use adult CSD z-scores. RVs participate as ordinary values. Returns a character vector of internal_id values to exclude.

Code location. R/adult_support.R.

Extreme EWMA + RV propagation: remove_ewma_wt() / propagate_to_rv()

See Extreme EWMA (Adult Step 9Wb) in this narrative for the remove_ewma_wt() algorithm description.

Brief — remove_ewma_wt(). Iteratively excludes weight outliers whose EWMA deviation exceeds the interval-specific ET cap (with 90% directional rule on before/after dewma). Rebuilds the EWMA cache from scratch each round via adult_ewma_cache_init(). Returns a named character vector (internal_id → exclusion code).

propagate_to_rv(). Extends firstRV exclusions to matching RV copies and extraneous values in linked-mode processing. For each excluded internal_id, finds all still-Include rows in the subject’s weight data with the same meas_m value and marks them with the source exclusion code + "-RV-Propagated". Propagation is forward-only (firstRV → later copies); this differs from the bidirectional Scale-Max propagation in Adult Step 4W. Called from cleanadult() in both the Extreme EWMA (Adult Step 9Wb) and Moderate EWMA (Adult Step 11Wb) linked-mode firstRV passes.

Inputs and return contract (propagate_to_rv).

  • propagate_to_rv(exc_codes, w_subj_df, w_subj_keep) — exc_codes is a named character vector (internal_id → exclusion code) from the firstRV pass; w_subj_df is the full weight data frame for the subject (must have meas_m and internal_id); w_subj_keep is the named exclusion-code vector for all weight rows. Returns a list: w_subj_keep (updated) and propagated_ids (character vector of IDs that received propagated codes). Returns the unchanged inputs unchanged if exc_codes is empty.

Code location. R/adult_support.R (remove_ewma_wt, propagate_to_rv).

Moderate EWMA: remove_mod_ewma_wt()

See Moderate EWMA (Adult Step 11Wb) in this narrative for full documentation of the 7-step flow.

Brief. Iteratively excludes weight outliers using wtallow-based thresholds, trajectory rescue, alternate pathway (short-gap neighbors), percentage criterion, and 4+-consecutive error-load detection. Rebuilds the EWMA cache from scratch each round. Returns a named character vector (internal_id → exclusion code).

Code location. R/adult_support.R.

2D non-ordered pairs, 1D evaluation, error load: eval_2d_nonord() / eval_1d() / eval_error_load()

See 2D Non-Ordered Weight Pairs (Adult Step 11Wa2), Single Distinct Value: 1D Evaluation (Adult Step 13), and Error Load (Adult Step 14) in this narrative for full documentation.

Brief summaries:

  • eval_2d_nonord() — identifies and applies exclusion rules to subjects with exactly two interleaved (non-time-ordered) distinct weight values: all-within-wtallow → keep all; outside-wtallow + prior non-SDE exclusions → exclude all; outside-wtallow + dominant value (> 65%) → exclude minority; outside-wtallow + neither dominant → exclude all. Returns character vector of internal_id values to exclude.
  • eval_1d() — two-pass evaluation of subjects with a single distinct height or weight value. Pass 1: evaluate with BMI when available (BMI-based limits; a BMI is formed for every same-day pairing of an included height with an included weight, and the subject counts as in range if any of them is), or without (no-BMI limits). Pass 2: re-evaluate if Pass 1 excluded a height or weight that removed BMI availability. Returns character vector of internal_id values.
  • eval_error_load() — per-subject, per-param (HT and WT processed separately) error ratio check. Enumerates param-specific error codes (HT errors: PIV, Single, Ord-Pair, Window; WT errors: PIV, Single, 2D-Ordered, 2D-Non-Ordered, Scale-Max variants, Evil-Twins; EWMA trajectory codes count as errors, except -RV-Propagated variants). Excludes SDEs from denominator. Requires denom ≥ 3. Excludes all remaining Include rows when ratio exceeds error_threshold. Returns named character vector (internal_id → exclusion code).

Code locations. eval_2d_nonord(), eval_1d(), and eval_error_load() (in file order; eval_error_load() is at the end of the file).

PIV: Population Implausible Values (Adult Step 1)

Exclude values outside absolute biological limits.

Scope Height and Weight (individually), then same-day ht+wt pairs (BMI check)
Prior Adult Step None (first step)
Next Adult Step 2W: Repeated Values (Weight); 3H: Temp SDE (Height)
Distinct/RVs N/A — runs before RV identification
Exclusion Code Exclude-A-PIV
Configurable parameter names overall_ht_min, overall_ht_max, overall_wt_min, overall_wt_max, overall_bmi_min, overall_bmi_max

Overview:

The first step removes values that are biologically impossible for an adult human. These absolute outer limits are deliberately very wide — they are meant to catch only values that could not possibly be correct (e.g., data entry errors, unit errors that produce extreme values). More refined exclusion of implausible-but-not-impossible values happens in later steps.

Key terms and variable names:

  • piv_df — data frame holding the PIV thresholds, indexed by “height” and “weight” with columns “low” and “high”
  • remove_piv() — support function that returns a logical vector identifying values outside PIV limits

Configurable setting defaults and options:

Setting loosest Unit Notes
overall_ht_min 50 cm
overall_ht_max 244 cm
overall_wt_min 20 kg Not pounds — specified because scale_max_lbs uses pounds
overall_wt_max 500 kg Not pounds — specified because scale_max_lbs uses pounds
overall_bmi_min 5 kg/m² Only meaningfully different from single_bmi_min at loosest (5 vs 10); same at other levels
overall_bmi_max 300 kg/m² Only meaningfully different from single_bmi_max at loosest (300 vs 250); same at other levels

Values shown are for loosest; see vignette("configuration") §1D for the other levels.

These thresholds use strict inequalities with 0.12 rounding tolerance (see Rounding Tolerance section): a value must be more than 0.12 below the minimum or above the maximum to be excluded. The permissiveness parameter provides preset less-permissive defaults that tighten these (and other) limits; explicit parameter values override permissiveness presets.

Logic and implementation:

  1. For each subject, evaluate all height values against the height PIV limits
  2. Exclude any height outside [overall_ht_min, overall_ht_max] (with 0.12 tolerance)
  3. For each subject, evaluate all weight values against the weight PIV limits
  4. Exclude any weight outside [overall_wt_min, overall_wt_max] (with 0.12 tolerance)
  5. BMI PIV check (Adult Step 1B): Runs only when overall_bmi_min < single_bmi_min or overall_bmi_max > single_bmi_max — i.e., only when the PIV BMI limits are wider than the Adult Step 13 single-value limits. In practice this means loosest only (overall BMI [5,300] vs single BMI [10,250]). At other levels the two sets of limits are equal, so skipping the check avoids applying thresholds earlier than the algorithm intends.
  6. For each ageday where both a surviving height and a surviving weight exist, pair the first height and first weight (by internal_id order) and compute BMI = weight_kg / (height_cm / 100)²
  7. Exclude that ht and wt pair if BMI < overall_bmi_min or BMI > overall_bmi_max. No rounding tolerance is added to the BMI comparison: tolerance is already embedded in the measurement-level checks above, and BMI is a derived value
  8. Excluded values are removed from the working dataframes so they do not participate in subsequent steps
  9. There is no need to check for same-day duplicates before this step because all values outside these limits are excluded regardless

Rationale for selected decisions:

  • The weight maximum (500 kg) is in kilograms, not pounds. This is noted explicitly because the algorithm also handles pound-to-kg conversion, and confusion between units is a common source of data error.
  • These limits are intentionally very broad. For example, a height of 50 cm (~20 inches) would be implausible for an adult, but unusually small adults and data that might represent seated heights or other measurement contexts are handled by later, more nuanced steps.
  • PIV values count toward the error load denominator in Adult Step 14 (Too Many Errors). A subject with many PIV values plus a few other exclusions could trigger error load on their remaining included values.

Repeated Value Markers (Adult Step 2W, weight only)

Identify groups of identical weight values and distinguish first occurrence from subsequent repeats.

Scope Weight only
Prior Adult Step 1W: PIV
Next Adult Step 3W: Temp SDE
Distinct/RVs Creates RV markers (this is where RVs are defined)
Exclusion Code None — this step marks but does not exclude
Configurable parameter names repval_handling (affects later EWMA steps, not this step)

Overview:

This step identifies repeated values (RVs) — weight measurements that appear more than once for a subject. The first occurrence is distinguished from subsequent identical values. RVs are not excluded at this point; instead they are flagged so that later steps (particularly the EWMA steps) can handle them differently depending on the repval_handling mode.

Height does not use RV tracking. While height does identify same-day duplicates (Adult Step 3H), it does not distinguish first vs. subsequent occurrences of the same height value across different days.

Key terms and variable names:

  • Repeated value (RV): A weight measurement value that appears more than once for a subject (exact match; 78.1 and 78.101 are not identical)
  • First RV: The earliest occurrence (by age, with internal_id as tiebreaker) of a repeated value
  • is_rv — logical flag: TRUE for non-first occurrences of a repeated value
  • is_first_rv — logical flag: TRUE for the first occurrence of a repeated value
  • Unique values have both is_rv = FALSE and is_first_rv = FALSE
  • identify_rv() — support function that sets RV flags based on current included values
  • redo_identify_rv() — support function that re-identifies RVs specifically after temp SDE marks a first_rv as extraneous

Configurable setting defaults and options:

Setting Default Options Notes
repval_handling "independent" "independent", "linked" Does not affect this step directly; controls how RVs are handled in EWMA steps (9Wb, 11Wb)
  • independent: RVs are treated as full participants in a single EWMA pass. Best when RVs are likely real measurements (or approximations).
  • linked: Two-pass structure (firstRV/allRV) with RV propagation. Best when RVs are likely carried-forward values, not independent measurements.

Logic and implementation:

  1. For each included weight value (in meas_m), count how many times it appears for the subject
  2. Values appearing only once: is_first_rv = FALSE, is_rv = FALSE
  3. Values appearing more than once:
    • The first occurrence (earliest age, internal_id tiebreaker): is_first_rv = TRUE
    • All subsequent occurrences: is_rv = TRUE
  4. RV identification depends on the dataframe being pre-sorted by age and internal_id
  5. After later exclusion steps (weight cap, evil twins), RVs are re-identified on the remaining values:
    • identify_rv() is called fresh on the remaining rows
    • temp_sde() is re-run
    • redo_identify_rv() cleans up if SDE marked a first_rv as extraneous

Rationale for selected decisions:

  • RVs are weight-only because repeated identical heights across visits are common and expected (adults don’t grow), while repeated identical weights are more likely to represent carried-forward or default values.
  • The first occurrence is kept as the “real” measurement because it is the earliest recorded instance; subsequent identical values may be copies.
  • Re-identification after exclusions is necessary because if the first occurrence of a value is excluded, the next occurrence should become the new first_rv.

Temp Same-Day Extraneous: SDE (Adult Step 3)

Temporarily flag same-day values that deviate most from patient median.

Scope Height (3H) and Weight (3W), evaluated separately
Prior Adult Step 1H: PIV (Height); 2W: Repeated Values (Weight)
Next Adult Step 4W: Weight Cap (Weight); 9H: Final SDE Height Resolution (Height)
Distinct/RVs WT median uses non-RV values only; HT median uses all values
Exclusion Code None — values are flagged as extraneous, not excluded yet
Configurable parameter names None

Overview:

When a subject has multiple measurements on the same day, this step temporarily identifies which value to keep and which to flag as extraneous. For each same-day group, the value closest to the patient’s overall median survives; the others are flagged extraneous = TRUE. This is a temporary designation — flagged values remain in the working dataframe but are skipped by most subsequent steps. Final SDE resolution (where extraneous values are actually excluded or rescued) happens later (Adult Step 9H for height, Adult Step 10W for weight).

Key terms and variable names:

  • Same-day extraneous (SDE): A measurement flagged because another measurement of the same type on the same day is a better fit for the patient
  • Extraneous: The temporary flag; extraneous = TRUE means this value is tentatively set aside
  • temp_sde() — support function that computes the median, diffs, and sets extraneous flags
  • redo_identify_rv() — called after 3W to update RV identification if a first_rv was flagged extraneous

Configurable setting defaults and options:

None. This step has no configurable settings.

Logic and implementation:

  1. Identify days with multiple measurements for the subject (same age_days)
  2. Calculate the patient’s overall median:
    • Weight: Median of all included non-RV weight values
    • Height: Median of all included height values
  3. For each value on a same-day group, compute diff = abs(measurement - median)
  4. Within each same-day group, keep the value with the smallest diff (highest internal_id tiebreaker); mark all others as extraneous = TRUE
  5. Weight only: After flagging, call redo_identify_rv() to update RV identification in case a first_rv was flagged extraneous
  6. Store extraneous flags in the subject-level tracking vector (h_extraneous or w_extraneous)

Rationale for selected decisions:

  • The median is calculated from all included values (not just non-same-day values) because the goal is to identify which same-day value best represents the patient’s overall pattern.
  • Weight uses non-RV values for the median because repeated values may be carried-forward and could bias the median. Height uses all values because the RV concept does not apply to height.
  • This is a temporary step because the “correct” same-day value may change after later exclusion steps alter the patient’s value distribution. Final resolution happens in Adult Steps 9H and 10W.

Scale Max: Weight Cap (Adult Step 4W)

Exclude weights at a physical scale maximum unless EWMA and adjacency checks confirm the value fits the patient’s pattern.

Scope Weight only
Prior Adult Step 3W: Temp SDE
Next Adult Step 9Wa: Evil Twins
Distinct/RVs EWMA and adjacency use firstRV (non-RV) values; RVs of excluded cap values are also excluded
Exclusion Codes Exclude-A-Scale-Max, Exclude-A-Scale-Max-Identical, Exclude-A-Scale-Max-RV-Propagated
Configurable parameter names scale_max_lbs

Overview:

Many clinical scales have a physical upper limit (commonly 400 lbs). When a weight exactly equals this limit, it is likely a truncated reading rather than a true weight. This step identifies weights near the scale maximum and excludes them unless they are consistent with the patient’s other weights (as determined by EWMA and adjacency checks). This step is optional — it only runs when scale_max_lbs is set to a finite value.

There are two cases:

  1. All distinct weights at cap: If a subject’s only distinct non-RV weight is the cap value, all of the subject’s weights are excluded (code: Scale-Max-Identical).
  2. Mixed cap and non-cap values: Individual cap values are evaluated using EWMA delta and adjacency difference criteria. Values must be >50 kg away from expected by both measures to be excluded (code: Scale-Max). RVs of excluded values are also excluded (code: Scale-Max-RV-propagated in linked mode).

Key terms and variable names:

  • Scale max / weight cap: The physical upper limit of a clinical scale, specified in pounds
  • EWMA (Exponentially Weighted Moving Average): A weighted average of a subject’s other measurements, where closer-in-time measurements get exponentially more weight
  • dewma: Delta from EWMA = measurement - EWMA (positive means above expected, negative means below)
  • Before/after EWMA: EWMA recalculated excluding the immediately adjacent measurement (to avoid the value being evaluated from dominating its own reference)
  • Adjacency diff: Difference between a value and its chronologically neighboring values
  • adult_ewma_cache_init() — support function computing EWMA with before/after variants using matrix operations
  • check_between() — support function checking if values fall within a range

Configurable setting defaults and options:

Setting Default Unit Notes
scale_max_lbs Inf pounds Inf disables this step entirely. Common value: 400 (lbs).

The cap detection range is scale_max_lbs converted to kg and rounded to 0.1 kg (round(scale_max_lbs / 2.2046226, 1)), ± 0.1 kg, using inclusive bounds.

Logic and implementation:

  1. Skip if scale_max_lbs == Inf or fewer than 2 included non-extraneous values
  2. Convert scale_max_lbs to kg; define detection range as kg rounded to 0.1 kg (round(kg, 1)) ± 0.1 kg
  3. Identify non-RV values at the cap and count distinct non-RV weights
  4. If only 1 distinct non-RV weight and it’s the cap: Exclude all of the subject’s weights with Scale-Max-Identical
  5. If mixed cap and non-cap, and >1 distinct non-RV weight: a. Calculate EWMA on non-RV values (exponent = -5), including ewma_before (EWMA excluding the immediately preceding observation) and ewma_after (EWMA excluding the immediately following observation). At edges, before/after EWMA equals the full EWMA. b. Calculate adjacency diffs (difference from previous and next non-RV values) c. For positive outliers (dewma50p): dewma_all > 50 + 0.12 AND dewma_before > 45 + 0.12 AND dewma_after > 45 + 0.12 AND dewma_all is not NA d. For negative outliers (dewma50m): dewma_all < −(50 + 0.12) AND dewma_before < −(45 + 0.12) AND dewma_after < −(45 + 0.12) AND dewma_all is not NA e. For positive adjacency (d50p): d_prev > 50 + 0.12 AND d_next > 50 + 0.12. Missing neighbors (first/last in series): treated as +Inf (confirms exclusion). f. For negative adjacency (d50m): d_prev < −(50 + 0.12) AND d_next < −(50 + 0.12). Missing neighbors: treated as −Inf (confirms exclusion). g. Exclude cap values where (dewma50p AND d50p) OR (dewma50m AND d50m)
  6. Propagate exclusion to RVs of excluded cap values
  7. After exclusions: re-run identify_rv(), temp_sde(), redo_identify_rv() on remaining values

Rationale for selected decisions:

  • The 50 kg threshold with 0.9 factor (45 kg) on before/after EWMA provides a conservative exclusion criterion — values near the cap are excluded only when clearly inconsistent with the patient’s pattern. All comparisons include the standard 0.12 kg rounding tolerance.
  • Missing adjacency neighbors (first or last value in series) are treated as confirming exclusion for both positive and negative directions. A cap value at the edge of a patient’s data with no neighbor on one side should still be excludable if the other neighbor confirms.
  • The step does not exclude heights, even in the “all weights at cap” case. Height exclusion is handled by other steps.
  • The 0.9 factor on before/after EWMA allows slight variation when excluding a value changes the EWMA reference.

Evil Twins (Adult Step 9Wa)

Exclude the inner members of an over-the-limit weight pair flanked by stable values (the anchor rule).

Scope Weight only
Prior Adult Step 4W: Scale Max
Next Adult Step 9H: Final SDE Height Resolution; 9Wb: Extreme EWMA
Distinct/RVs RVs participate as ordinary values (no special handling)
Exclusion Code Exclude-A-Evil-Twins
Configurable parameter names wtallow_formula (shared with EWMA steps)

Overview:

Evil twins are runs of consecutive extreme weights that distort each other’s EWMA, so that neither looks out of line on a neighbor-weighted basis and the EWMA steps alone would miss them. This step catches them up front using the anchor rule: of four age-ordered eligible weights A, B, C, D, the inner B–C jump must exceed the ET cap and both outer pairs (A–B and C–D) must anchor within 60% of their own cap. Only the inner members (B and C) are excludable; A and D anchor but are not flagged.

A missing A or D disqualifies the pair, so single spikes — at the record ends and embedded between normal values — are ceded to the Extreme/Moderate EWMA steps, which have the better-informed directional 90% rule, per-observation thresholds, and missing-neighbor handling that the non-overrideable ET cap lacks. ET runs before Extreme EWMA (9Wb) so that mutually-anchoring outlier runs are removed before they can pull the EWMA toward themselves.

The step excludes one value per iteration and re-anchors on the remaining values until no over-the-limit (OTL) pair remains or fewer than 4 eligible values are left.

Key terms and variable names:

  • 4-group (A, B, C, D): Four consecutive eligible weights ordered by age. B and C are the inner values; A and D are the outer anchors.
  • OTL (over-the-limit): Flag marking both inner members (B, C) of an evaluable pair.
  • Anchor rule: A pair (B, C) is OTL iff A exists, D exists, |B−C| > etcap(B,C) + 0.12, |A−B| ≤ 0.60·etcap(A,B) + 0.12, and |C−D| ≤ 0.60·etcap(C,D) + 0.12.
  • Upper weight (UW): The larger of the two weights in a pair, used for UW-based ET cap scaling. See the wtallow section below.
  • Count gate: Requires ≥4 eligible values (an A,B,C,D group is the floor for an evaluable anchored pair), re-checked at the top of every iteration.
  • absd_med: Absolute deviation of a candidate from a windowed median (median of the eligible weights within ±4 age-ordered positions), in kg — the adult analogue of the child’s z-space med_diff.
  • Adult CSD z: A two-sided conditional-SD z-score from a small fixed sex-specific NHANES reference (.adult_csd_ref()), used only for the UER3 plausible-group test and the median-band |z| tiebreak. NA-safe: missing sex resolves to an M/F-averaged fallback row.
  • UER3: A once-per-SP check for a clean unit-error split (a unit_error_range group of one CSD-z sign vs a plausible |z|<3 group whose mean kg differ by ~2.2046, the lb↔︎kg factor).
  • evil_twins() — support function implementing the anchor rule and selection. Signature: evil_twins(w_subj_df, wtallow_formula)
  • compute_et_limit() — ET cap for a given interval, formula, and UW. The adjacent-pair caps serve the inner and both outer comparisons and are computed once per iteration (not recomputed per pair).

Configurable setting defaults and options:

The ET cap is derived from the wtallow formula and UW (upper weight of the pair). For PW-H and PW-L formulas, ET cap = wtallow cap at 6m or 12m + 20. For allofus15, ET cap = allofus15-cap-12m (flat). For custom CSV, ET caps are fixed at 70/100.

Formula ET cap ≤6m (UW=120) ET cap >6m (UW=120) UW scaling
PW-H (loosest/looser) 70 100 Yes — caps scale with UW; see the wtallow section below
PW-L (tighter) 53.33 73.33 Yes for UW<120 (scales down); No for UW>120
allofus15 (tightest) allofus15-cap-12m allofus15-cap-12m Cap limited by PW-L effective cap at 12m
Custom CSV 70 100 No

All thresholds include the standard 0.12 kg rounding tolerance (see Rounding Tolerance section). See the wtallow section below for full formula details including UW adjustment for high and low UW.

Logic and implementation:

  1. Pre-check: Skip if <4 non-extraneous values, or if max−min range ≤ minimum possible ET cap + 0.12 (computed via compute_et_limit(1, formula, uw = min(weights)); no pair could exceed any interval-specific cap).
  2. UER3 (once, on the initial Include set): flag each weight’s unit_error_range; if the SP splits cleanly into a unit-error group (all one CSD-z sign) and a plausible group (|CSD z| < 3) whose mean kg differ by a factor in [1.7, 2.7], set uer3 = TRUE.
  3. Repeat until no OTL pair remains or <4 eligible values: a. Remove previously excluded values; break if <4 remain. b. Compute adjacent weight differences, age intervals (months, 30.4375 days/month), and per-pair ET caps (UW = larger of the pair). c. Anchor rule: mark both inner members of every pair satisfying the inner-jump and both outer-anchor conditions above. d. If no OTL value, break. e. Select one to exclude: if a single OTL candidate, exclude it; otherwise compute absd_med (windowed median) and route — if uer3, exclude the highest |CSD z|; else form a band of candidates within 0.20·etcap(B,C) of the max absd_med — if ≥2 in band exclude the highest |CSD z|, if exactly 1 exclude the highest absd_med. Tiebreak: lowest internal_id.
  4. After all exclusions: re-run identify_rv(), temp_sde(), redo_identify_rv() on remaining values.

Rationale for selected decisions:

  • The anchor rule restricts ET to its intended target — a run of consecutive extremes that mutually anchor — and cedes single spikes to the EWMA steps, which evaluate them with directional and per-observation information ET lacks.
  • Two-tier interval thresholds (≤6m / >6m) reflect that larger weight changes are more plausible over longer time periods. UW-based scaling accommodates larger absolute changes for heavier patients and tighter limits for lighter patients (see the wtallow formulas section).
  • The count gate (≥4) is a correctness floor, not an optimization: an SP with fewer than 4 eligible values cannot form an anchored A,B,C,D pair.
  • The windowed median keeps the deviation reference local, so a long trajectory elsewhere in the SP cannot mask a candidate; the |CSD z| tiebreak decides when the windowed median cannot discriminate (band ≥2).
  • The median-band width here is 0.20·etcap(B,C) (kg), mirroring the child redesign that halved its band. The child uses an absolute z-unit band; the two widths are intentionally not unified (see the comment in evil_twins() and calc_otl_evil_twins()).
  • RVs participate as ordinary values: ET applies no special repeated-value handling.
  • One exclusion per iteration with re-anchoring ensures the remaining values are re-evaluated as problems are peeled.

Final SDE Height Resolution (Adult Step 9H)

Resolve same-day height duplicates: exclude identical values, then choose keepers using category-based median selection.

Scope Height only
Prior Adult Step 3H: Temp SDE (temporary flagging)
Next Adult Step 10H: Height Distinct Pairs
Distinct/RVs Height does not use RV tracking
Exclusion Codes Exclude-A-Identical, Exclude-A-Extraneous
Configurable parameter names None

Overview:

This step does the final resolution of same-day height duplicates that were temporarily flagged in Adult Step 3H. It has two parts:

  • Part A (Identical): When all same-day values are identical, exclude all but one (keep lowest internal_id). If removing identical values eliminates all duplicates on a day, that day is no longer treated as SDE.
  • Part B (Non-identical): For remaining days with multiple different values, categorize the subject and choose which value to keep based on category-specific median comparisons.

The keeper retains its original measurement value — no averaging or mean replacement is performed.

Key terms and variable names:

  • SDE day: A day with more than one remaining included measurement after identical removal
  • daymed: Median of all values within a single day
  • nonsdemed: Subject-level median of values on non-SDE days only
  • medmed: Median of day-medians (one median per day, balancing weighting across days with different numbers of values)
  • sderatio: Ratio of SDE days to total distinct days

Configurable setting defaults and options:

None. This step has no configurable settings.

Logic and implementation:

Part A — Identical values:

  1. All values (including those temporarily marked extraneous by Adult Step 3H) remain in h_subj_df; extraneous flags identify which days have duplicates
  2. On each day with extraneous values, identify measurements that appear more than once (exact match on meas_m)
  3. Keep the first occurrence (lowest internal_id); exclude the rest with Exclude-A-Identical
  4. Re-run temp_sde() to update extraneous flags on remaining values

Part B — Non-identical SDEs:

  1. Identify remaining days with SDEs
  2. Calculate three reference medians:
    • daymed: median within each day
    • nonsdemed: median of values on non-SDE days (NA if no non-SDE days exist)
    • medmed: median of per-day medians
  3. Categorize subject:
    • Category 1 (one day): daystot == 1
    • Category 2 (high non-SDE): daystot >= 4 AND sderatio < 0.5
    • Category 3 (low non-SDE): daystot == 2 or 3 OR sderatio >= 0.5
  4. For each SDE day, choose keeper by sorting:
    • Category 1: closest to day-median, tiebreaker highest internal_id
    • Category 2: closest to non-SDE median, then day-median, then highest internal_id
    • Category 3: closest to median-of-medians, then non-SDE median, then day-median, then highest internal_id
  5. Keep the first value after sorting; exclude the rest with Exclude-A-Extraneous

Rationale for selected decisions:

  • Tiebreaker — lowest internal_id for identical, highest internal_id for non-identical: For identical values there is no meaningful difference, so keep the first-entered. For non-identical values, the later measurement may represent a more careful re-measurement.
  • Three categories: The choice of reference median depends on how much non-SDE data exists. With many non-SDE days (Category 2), the non-SDE median is the best reference. With few non-SDE days (Category 3), the median-of-medians balances across days. With only one day (Category 1), only the day-median is available.
  • No averaging: The keeper retains its original value. Mean height calculation is a separate step that happens later (Adult Step 11H).

Extreme EWMA (Adult Step 9Wb)

Exclude weight outliers whose EWMA deviation exceeds interval-specific caps.

Scope Weight only
Prior Adult Step 9Wa: Evil Twins
Next Adult Step 10W: Final SDE Weight Resolution
Distinct/RVs Independent: all values participate. Linked: firstRV pass (non-RV only), then propagate, then allRV pass.
Exclusion Codes Exclude-A-Traj-Extreme (independent), Exclude-A-Traj-Extreme-firstRV (linked), Exclude-A-Traj-Extreme-allRV (linked)
Configurable parameter names wtallow_formula, repval_handling

Overview:

This step uses Exponentially Weighted Moving Averages to identify weight values that deviate far from the patient’s expected trajectory. The EWMA gives exponentially more weight to measurements closest in time — with the exponent e=-5, the nearest neighbor almost completely dominates, making this a near-neighbor comparison rather than a traditional moving average.

A value is excluded when its EWMA deviation exceeds an interval-specific ET cap AND both directional deviations (before/after) confirm. The step iterates, removing one value per round (the worst outlier), until no candidates remain or fewer than 3 values are left.

Key terms and variable names:

  • EWMA (Exponentially Weighted Moving Average): Weighted average of other measurements, with weights proportional to |age difference|^e. With e=-5, nearest neighbor dominates.
  • dewma_all: EWMA deviation using all other values
  • dewma_before / dewma_after: EWMA deviation excluding the immediately prior / next measurement. Used for the 90% confirmation rule.
  • min_gap_months: Minimum gap (in months) to either chronological neighbor. Determines which tier (≤6m or >6m) applies.
  • Upper weight (UW): max(wt, ewma) for each observation. Used for UW-based ET cap scaling.
  • remove_ewma_wt() — support function implementing the iterative exclusion logic (shared with Moderate EWMA, Adult Step 11Wb). Signature: remove_ewma_wt(subj_df, wtallow_formula, ...)
  • compute_et_limit() — computes the ET cap per observation: compute_et_limit(min_gap_months, formula, uw). See the wtallow section below for the full specification.
  • adult_ewma_cache_init() — EWMA computation with position-based window (default 15 on each side)

Configurable setting defaults and options:

Setting loosest/looser tighter tightest Unit Notes
wtallow_formula "piecewise" "piecewise-lower" "allofus15" — Determines ET cap structure (ET cap = wtallow cap + 20 for PW-H/PW-L; see the wtallow formulas section)
repval_handling "independent" "linked" "linked" — "independent" or "linked" (controls single vs two-pass)
ewma_window 15 15 15 observations Max observations on each side for EWMA (not permissiveness-controlled)

The threshold is computed dynamically per observation: compute_et_limit(min_gap_months, formula, uw) where uw = max(wt, ewma). ET caps scale with UW — for higher UW they increase (PW-H only, capped at UW=180), for lower UW they decrease (PW-H and PW-L, with 2/3×UW ceiling). See the wtallow section below for complete formula details.

Logic and implementation:

Guards:

  1. Skip if no weight values remain
  2. Skip if <3 non-extraneous values
  3. Range gate: Skip if max-min of included values ≤ minimum possible ET cap + 0.12 (computed via compute_et_limit(1, formula, uw = min(weights))). If the entire range is within the smallest possible cap plus rounding tolerance, no value can exceed any cap.

Independent mode (single pass):

  1. Call remove_ewma_wt() with all non-extraneous values (including RVs)
  2. Each round: a. Compute minimum neighbor gap in months for each value. Missing gaps (edge values) set to Inf. b. Compute per-observation threshold: compute_et_limit(min_gap_months, formula, uw) where uw = max(wt, ewma_all) c. Compute EWMA (exponent e=-5, anchor a=0, window = ewma_window observations on each side) d. Positive outlier: dewma_all > threshold + 0.12 AND dewma_before > 0.9×threshold + 0.12 AND dewma_after > 0.9×threshold + 0.12 e. Negative outlier: dewma_all < -(threshold + 0.12) AND dewma_before < -(0.9×threshold + 0.12) AND dewma_after < -(0.9×threshold + 0.12) f. Edge values: ewma_before/ewma_after fall back to ewma_all, so the directional checks reduce to the dewma_all check and do not block exclusion g. Among candidates, select the one with largest |dewma_all|, then earliest age, then lowest internal_id h. Exclude with code "Exclude-A-Traj-Extreme" i. Rebuild EWMA for next round
  3. Iterate until no candidates or <3 values remain (no artificial round limit)
  4. After exclusions: re-identify RVs on remaining values

Linked mode (two-pass):

  1. firstRV pass: Run remove_ewma_wt() on non-RV values only, with label "Exclude-A-Traj-Extreme-firstRV". Apply same range gate (cap_short + 0.12) to firstRV subset.
  2. Propagate firstRV exclusions to their RV copies via propagate_to_rv()
  3. Re-identify RVs on remaining values
  4. allRV pass: Run remove_ewma_wt() on all remaining non-extraneous values (RVs now participate), with label "Exclude-A-Traj-Extreme-allRV". Apply range gate (minimum ET cap + 0.12) to remaining subset.
  5. Re-identify RVs after allRV exclusions

Rationale for selected decisions:

  • No round limit: The algorithm iterates until convergence — no artificial cap. In practice, extreme EWMA rarely exceeds 3 rounds.
  • 90% rule: The directional confirmation (before/after must exceed 90% of threshold) prevents excluding values that are outliers in one direction but consistent with the trend in the other. With e=-5, the 90% rule is effectively unreachable (nearest neighbor dominates so heavily that dewma_before/after are nearly equal to dewma_all), but it is retained at zero cost for consistency with the algorithm’s theoretical framework.
  • Edge values: At a record end, the missing-side EWMA falls back to ewma_all, so an extreme edge value can still be excluded based on the neighbors that do exist.
  • Two-tier structure: Both Evil Twins and Extreme EWMA use the same ET cap structure derived from the wtallow formula (ET cap = wtallow cap + 20 for PW-H/PW-L). The distinction between ET and Extreme EWMA is the detection method (adjacent pairs vs EWMA deviation), not the cap structure.
  • Sort tiebreaker: The R implementation includes internal_id as a final tiebreaker in all passes for deterministic sort order.

Height Distinct Values (Adult Step 10H)

Evaluate subjects with 2 or 3+ distinct height values: exclude pairs outside the height band (with loss/gain and frequency rescue), or values outside the best height window.

Scope Height only
Prior Adult Step 9H: Final SDE Height Resolution
Next Adult Step 10W: Final SDE Weight Resolution
Distinct/RVs Height does not use RV tracking
Exclusion Codes Exclude-A-Ord-Pair-All, Exclude-A-Ord-Pair, Exclude-A-Window-All, Exclude-A-Window
Configurable parameter names ht_band, allow_ht_loss, allow_ht_gain

Overview:

This step evaluates subjects whose remaining included heights have 2 or more distinct values. It routes to one of two substeps:

  • 10Ha (2 distinct): Check if the two values differ by more than ht_band inches. If so, attempt loss rescue (height decreased within plausible limits), gain rescue (young adult height gain within allowance), and frequency rescue (one value appears ≥4/3 as often as the other).
  • 10Hb (3+ distinct): Find the best height window (w2) — a range of ht_band inches that captures the most values with sufficient ratio to outliers. Values outside the best window are excluded, unless rescued by loss-group or gain-group analysis.

Subjects with only 1 distinct height skip this step entirely.

Key terms and variable names:

  • ht_band: Allowed height range in inches (default 3). Used with cm conversion and 0.12 cm rounding tolerance (see Rounding Tolerance section): effective threshold = ht_band * 2.54 + 0.12 cm.
  • pairhtloss / pairhtgain: Flags for whether a 2D pair qualifies for loss or gain rescue
  • htallow: Height growth allowance based on age and velocity formula: velocity * (ln(age2 - 16.9) - ln(age1 - 16.9))
  • w2 window: A height range of ht_band * 2.54 + 0.12 cm starting from each distinct height
  • w2/o2 ratio: Number of values inside vs. outside a window; must be ≥ 3/2 for the window to be viable
  • Loss group / gain group: Sequential grouping of heights into clusters within 2 inches (5.08 + 0.12 cm) of each other
  • ht_allow() — support function computing height growth allowance
  • ht_change_groups() — support function forming sequential height groups
  • ht_3d_growth_compare() — support function validating gain patterns across groups
  • check_between() — support function for inclusive range checks

Configurable setting defaults and options:

Setting loosest looser tighter tightest Unit Notes
ht_band 3 3 2 2 inches Effective threshold in cm: ht_band * 2.54 + 0.12
allow_ht_loss TRUE FALSE FALSE FALSE — When FALSE, disables loss rescue in both 2D and 3D
allow_ht_gain TRUE TRUE TRUE FALSE — When FALSE, disables gain rescue in both 2D and 3D

Logic and implementation:

Adult Step 10Ha: 2 Distinct Heights

  1. Extract the two distinct heights (ht_1, ht_2) in age order (both in cm)
  2. Band check: abs(ht_1 - ht_2) > (ht_band * 2.54 + 0.12) — if FALSE, values are within band; skip (all remain included)
  3. If band exceeded: a. Gain rescue (pairhtgain): Only evaluated if ageyears1 < 25 (young adult). Checks:
    • Height increased: (ht_2 - ht_1) > -0.12 (in cm, with rounding tolerance)
    • Within allowance: (ht_2 - ht_1) ≤ (htallow + 2) * 2.54 + 0.12 (htallow in inches, converted to cm with tolerance; 2-inch buffer)
    • Temporal ordering: all ht_2 measurements occur after all ht_1 measurements
    • htallow coefficient by age gap: <2yr → 15.5, 2–3yr (inclusive) → 13, >3yr → 12
    • Age capped at 25 for htallow formula b. Loss rescue (pairhtloss): Only evaluated if allow_ht_loss = TRUE. Checks:
    • Height decreased: ht_1 > ht_2 - 0.12 (in cm, with rounding tolerance)
    • Within loss limit: (ht_1 - ht_2) ≤ 5 * 2.54 + 0.12 for age at ht_2 < 50, or ≤ 7 * 2.54 + 0.12 for age at ht_2 ≥ 50
    • Temporal ordering: all ht_2 measurements occur after all ht_1 measurements c. If neither rescue applies: mark all values for exclusion d. Frequency rescue: If keepht1 (ht_1 count ≥ ht_2 count × 4/3), rescue ht_1 values. Same for keepht2.
  4. Exclusion code:
    • Exclude-A-Ord-Pair if frequency rescue saved some values
    • Exclude-A-Ord-Pair-All if no frequency rescue (all excluded)

Adult Step 10Hb: 3+ Distinct Heights

Part 1 — Window (w2) evaluation:

  1. Define w2 window for each distinct height: [ht, ht + ht_band * 2.54 + 0.12]
  2. Count observations and distinct values inside each window (w2) and outside (o2)
  3. Require w2/o2 ratio ≥ 3/2 for a window to be viable
  4. If multiple viable windows: select by highest score (tot_w2 + 0.5 * numdistinct_w2); ties broken by largest mean absolute distance from w2 mean to o2 values
  5. If one viable window: use it
  6. If no viable window: exclude all values (Exclude-A-Window-All)
  7. Exclude values outside the best window (Exclude-A-Window)

Part 2 — Loss group rescue (if allow_ht_loss = TRUE):

  1. Only evaluated if any values are still marked for exclusion after w2 evaluation
  2. Form sequential height groups by iterating through values in age order:
    • Current group range within 2 inches (5.08 + 0.12 cm): extend group
    • New value > 2 inches below group minimum: start new loss group
    • New value > 2 inches above group maximum: abort (wrong direction for loss)
    • Maximum 3 groups; abort if exceeded
  3. Validate loss pattern (all comparisons in cm with 0.12 tolerance):
    • G2-G1 direction: must decrease by at least 0.12 cm
    • G2-G1 magnitude: loss ≤ 5 inches (age < 50) or ≤ 7 inches (age ≥ 50)
    • G3-G2: same direction and magnitude checks
    • G3-G1 cumulative: ≤ 6/8/9 inches depending on age pattern
  4. If all checks pass: rescue all values (clear all exclusion criteria)

Part 3 — Gain group rescue:

  1. Only evaluated if any values are still marked for exclusion AND min age < 25 AND not already rescued by loss
  2. Form sequential height groups (same as loss, but maximum 6 groups; abort if height decreases)
  3. Validate gain pattern using ht_3d_growth_compare():
    • Adjacent check (compare = “before”): each group vs. previous group
    • Cumulative check (compare = “first”): each group vs. group 1
    • Direction: gain must not drop more than 0.12 cm
    • Magnitude: gain must not exceed htallow + 0.12 cm
    • htallow coefficient by age gap: <1yr → 20, 1-3yr → 15, >3yr → 12
  4. If all checks pass: rescue all values

Rationale for selected decisions:

  • 0.12 rounding tolerance: See Rounding Tolerance section. Applied to all height and weight threshold comparisons throughout the algorithm.
  • Loss rescue guarded by allow_ht_loss: This is a configurable parameter because height loss (e.g., from kyphosis, osteoporosis) is real but uncommon. Setting allow_ht_loss = FALSE makes the algorithm more conservative — any large height difference results in exclusion.
  • Gain rescue limited to age < 25: Adults over 25 do not grow, so height gain rescue only applies to young adults who might still be growing.
  • Two exclusion codes per substep: Distinguishing “all excluded” from “some rescued by frequency” (2D) or “no viable window” from “outside window” (3D) provides more granular information about why values were excluded.
  • Frequency rescue (4/3 ratio): When one height value appears substantially more often, it is likely the correct measurement. The 4/3 ratio threshold prevents marginal frequency differences from driving rescue.
  • Loss groups use means, not individual values: Group means are compared because individual measurements may have rounding noise; averaging within a group produces a more stable reference.

Final SDE Weight Resolution (Adult Step 10W)

Resolve same-day weight duplicates: exclude identical values, then choose keepers using category-based median selection.

Scope Weight only
Prior Adult Step 9Wb: Extreme EWMA
Next Adult Step 11H: Mean Height; 11Wa: Distinct Ordered Pairs (Weight)
Distinct/RVs Non-SDE median uses firstRV (non-RV) values only; re-identifies RVs after exclusions
Exclusion Codes Exclude-A-Identical, Exclude-A-Extraneous
Configurable parameter names None

Overview:

Same logic as Adult Step 9H (Final SDE Height Resolution), applied to weight. Resolves same-day weight duplicates that were temporarily flagged in Adult Step 3W. Two parts:

  • Part A (Identical): Exclude all but one when same-day values are identical (keep lowest internal_id).
  • Part B (Non-identical): Categorize subject and choose keeper using category-specific median comparisons.

The keeper retains its original measurement value — no averaging is performed. After exclusions, RVs are re-identified since a first_rv may have been excluded.

Key terms and variable names:

Same as Adult Step 9H, with one weight-specific difference:

  • nonsdemed (weight): Uses firstRV (non-RV) values on non-SDE days only, unlike height which uses all values. This prevents carried-forward RV values from biasing the reference median.

Configurable setting defaults and options:

None.

Logic and implementation:

Identical to Adult Step 9H with these weight-specific differences:

  1. After identical removal, temp_sde() is re-run with ptype = "weight" (non-RV median) and redo_identify_rv() updates RV flags
  2. Non-SDE median (nonsdemed) is calculated from non-RV values on non-SDE days only
  3. After all SDE exclusions, RVs are re-identified via identify_rv()

See Adult Step 9H for full details on categories, sorting, and tiebreakers.

Rationale for selected decisions:

  • Using firstRV (non-RV) values for the non-SDE median prevents carried-forward values from dominating the reference. Height does not need this because repeated identical heights are expected (adults don’t grow).
  • All other rationale is the same as Adult Step 9H.

Mean Height (Adult Step 11H)

Calculate the mean height for each subject, respecting loss/gain group boundaries from Adult Step 10H.

Scope Height only
Prior Adult Step 10H: Height Distinct Values
Next Adult Step 11Wa: 2D Ordered Weight Pairs
Distinct/RVs N/A — height does not use RV tracking
Exclusion Codes None — this step calculates a value, not exclusions
Configurable parameter names None

Overview:

This step computes a representative mean height for each subject. The mean_ht column can be used by researchers who want a consistent height for evaluation of body mass index. The mean is calculated from meas_m (metric measurement values) of remaining included heights. How the mean is computed depends on the outcome of Adult Step 10H:

  • Pair loss or gain (2D): Each observation retains its individual original measurement as its “mean” (no averaging across the two distinct values, since they represent different physical states)
  • 3D with validated loss groups: Mean within each loss group (observations in the same group share one mean)
  • 3D with validated gain groups: Mean within each gain group
  • All other subjects: Simple arithmetic mean of all included measurements

Key terms and variable names:

  • mean_ht — cleanadult() output column: the calculated mean height for each height observation. Carried through by cleangrowth() and returned by default when the data has adult rows (set mean_ht = FALSE to omit). Computed from finally-included heights only and populated on all of the subject’s adult height rows (see Step 11H, item 5); NA for child rows, adult weight rows, and subjects with no included height. See vignette("output").
  • meas_m — the metric measurement value (used for mean calculation)
  • loss_groups / gain_groups — group assignments from Adult Step 10H
  • pairhtloss / pairhtgain — flags from Adult Step 10Ha indicating pair rescue type

Logic and implementation:

  1. Initialize meanht as NA for all height observations
  2. If no included heights remain, leave as NA
  3. Otherwise, compute mean height based on Adult Step 10H outcome:
    • pairhtloss or pairhtgain: Use each observation’s meas_m directly (no averaging)
    • Loss groups validated: For each group in glist_loss, compute mean(meas_m) for observations in that group; all observations in the group receive the same mean
    • Gain groups validated: Same logic using glist_gain
    • Default: Compute mean(meas_m) across all included heights; all observations receive the same value
  4. Store in the mean_ht column (returned by cleanadult(); returned in cleangrowth() output by default)
  5. Finalize after Adult Steps 13/14. Steps 13 and 14 can exclude a subject’s remaining heights (always all of them). After Step 14, mean_ht is set so it reflects only finally-included heights and is populated on every height row: if the subject has no included heights left, every height row is NA; an included height keeps its Step 11H value; every other height row (excluded at any step, or with a missing measurement) takes the mean_ht of the included height nearest in age (ties go to the earlier one).

Rationale for selected decisions:

  • Uses meas_m (metric measurement): Since SDE resolution retains a keeper’s original value without averaging (Adult Step 9H), meas_m equals the raw recorded measurement for all surviving heights. The mean is based on these raw values, not any intermediate calculation.
  • Pair loss/gain keeps individual values: When a subject has two distinct heights representing different physical states (e.g., height loss from aging), averaging them would produce a value that doesn’t represent either state. Each observation keeps its own measurement.
  • Group-aware averaging: Loss/gain groups represent different time periods with different true heights. Averaging within groups but not across them preserves the temporal signal.

2D Ordered Weight Pairs (Adult Step 11Wa)

Exclude subjects with exactly 2 distinct weight values in time order when the weight difference exceeds the allowed amount or the weight ratio is too extreme.

Scope Weight only
Prior Adult Step 10W: Final SDE Weight Resolution
Next Adult Step 11Wa2: 2D Non-Ordered Weight Pairs
Distinct/RVs Routing uses firstRV (non-RV) numdistinct and ordering; allRV override for non-ordered detection; evaluation uses firstRV values
Exclusion Codes Exclude-A-2D-Ordered
Configurable parameter names wtallow_formula

Overview:

This step handles subjects whose remaining included weights have exactly 2 distinct values that are time-ordered (all instances of value 1 occur before all instances of value 2). It applies two independent tests: (1) whether the absolute weight difference exceeds a time-dependent allowance (wtallow), and (2) whether the ratio of the smaller weight to the larger weight falls below a threshold (perclimit). If either test fails, all weight observations for the subject are excluded.

This step also performs the routing that determines whether subjects go to 2D ordered (this step), 2D non-ordered (Adult Step 11Wa2), or 3+D/EWMA (Adult Step 11Wb). Subjects with 1 distinct value go to Adult Step 13 (Distinct Single).

Key terms and variable names:

  • numdistinct — number of distinct weight values among firstRV (non-RV) included observations
  • pair_distinct — TRUE if firstRV has exactly 2 distinct values and they are time-ordered (max age of value 1 < min age of value 2)
  • is_2d_nonord — TRUE if allRV has exactly 2 distinct values but they are NOT time-ordered (overrides pair_distinct)
  • wt_first / wt_last — first and last weight values (by age) among firstRV observations
  • wt_diff — wt_last - wt_first
  • ageyears_diff — time gap between the two value groups: min age of value 2 - max age of value 1 (in years)
  • wtallow (wta) — maximum allowed weight change for the given time interval, computed by compute_wtallow(months, formula, uw). UW is the upper weight of the pair (max(wt_first, wt_last)).
  • wt_perc — ratio of smaller to larger weight: min(wt_first, wt_last) / max(wt_first, wt_last)
  • perclimit — threshold for wt_perc; subject-level, max of compute_perc_limit() across observations. Weight-dependent and permissiveness-dependent (see permissiveness table above).

Configurable setting defaults and options:

  • wtallow_formula (default "piecewise"): Selects the formula for computing weight allowance from the time interval between measurements. All formulas are UW-adjusted (scaled by the upper weight of the pair). Options:

    • "piecewise" (PW-H) — Three-segment curve. Default for loosest/looser. At UW=120 (base):
      • 0–1 month: log-curved from ~10.85 kg to 20 kg (10 + 10 × log(1 + 5×months) / log(6))
      • 1–6 months: linear from 20 kg to 50 kg
      • 6–12 months: linear from 50 kg to 80 kg

      • 12 months: flat at 80 kg

      • UW > 120: caps increase by 0.25×(UW−120), capped at UW=180; 1m value = 25
      • UW < 120: caps scale as (base_cap − 20) × (UW/120) + 20; ceiling of UW × 2/3
    • "piecewise-lower" (PW-L) — Same shape as PW-H but with lower caps. Default for tighter. At UW=120 (base):
      • 0–1 month: same log curve as PW-H
      • 1–6 months: linear from 20 kg to 33.33 kg
      • 6–12 months: linear from 33.33 kg to 53.33 kg

      • 12 months: flat at 53.33 kg

      • UW > 120: NOT scaled up (same as base formula)
      • UW < 120: caps scale as (base_cap − 20) × (UW/120) + 20; ceiling of UW × 2/3
    • "allofus15" — step function for short intervals, linear ramp for long intervals. Default for tightest.
      • 0–2 days: 5 kg; 3–7 days: 10 kg; 8 days–<6 months: 15 kg
      • 6–12 months: linear from 15 kg to allofus15-cap-12m

      • 12 months: flat at allofus15-cap-12m

      • allofus15-cap-12m = min(40, effective PW-L at 12m for that UW)
    • Custom CSV path with columns months and wtallow (linearly interpolated, no UW adjustment).

See the wtallow section below for the complete specification of all formulas, UW adjustments, ET caps, and ceilings.

Logic and implementation:

Routing (numdistinct and ordering):

  1. Filter to firstRV (non-RV) included weight observations (w_nonrv)
  2. Count distinct values among firstRV observations
  3. If firstRV has exactly 2 distinct values, check ordering: max(ages of value 1) < min(ages of value 2) (strict less-than)
  4. allRV override: If allRV (all included values, including RVs) also has exactly 2 distinct values but they are NOT time-ordered, override the firstRV result — route to 2D non-ordered (Adult Step 11Wa2) instead
  5. Routing outcome:
    • 1 distinct value → Adult Step 13 (Distinct Single)
    • 2 distinct, ordered → this step (11Wa)
    • 2 distinct, not ordered → Adult Step 11Wa2
    • 3+ distinct → Adult Step 11Wb (Moderate EWMA)

Evaluation (if 2D ordered):

  1. Compute wt_first (earliest weight) and wt_last (latest weight) from firstRV observations
  2. Compute ageyears_diff = min age of value 2 - max age of value 1 (gap between the two groups)
  3. Compute wtallow using compute_wtallow(ageyears_diff * 12, formula, uw = max(wt_first, wt_last))
  4. Test 1 (wtallow): abs(wt_diff) > wtallow + 0.12 (with rounding tolerance)
  5. Compute wt_perc = min(wt_first, wt_last) / max(wt_first, wt_last)
  6. Determine perclimit: subject-level, max of compute_perc_limit() across all observations. At loosest/looser: ≤45 kg → 0.5, 45–80 kg → 0.4, >80 kg → 0 (disabled). At tighter/tightest: ≤45 kg → 0.7, >45 kg → 0.4.
  7. Test 2 (perclimit): wt_perc < perclimit (strict less-than)
  8. If either test fails, exclude ALL weight observations for the subject (including RVs)

Rationale for selected decisions:

  • firstRV for routing, allRV for non-ordered override: firstRV gives the correct numdistinct count (RVs are duplicates of existing values). However, firstRV with 2 distinct values always appears ordered (each value has one observation). The allRV check catches cases where interleaved RVs reveal that the values aren’t truly time-separated.
  • Subject-level perclimit: Uses the maximum compute_perc_limit() across all observations. At loosest/looser, perclimit is disabled (0) for weights >80 kg, so the subject-level value is driven by any lighter observations.
  • Two independent exclusion criteria: wtallow catches absolute differences that are too large for the time interval. perclimit catches extreme ratios (e.g., 40 kg vs 100 kg) even when the absolute difference might be within wtallow for a long interval.
  • Parameterized wtallow: Different datasets may warrant different sensitivity. The piecewise formula (loosest/looser default) is more permissive at short intervals and stricter at long intervals; tighter/tightest use lower-cap variants.

2D Non-Ordered Weight Pairs (Adult Step 11Wa2)

Exclude subjects with exactly 2 distinct weight values that are interleaved in time, using a four-rule decision tree based on wtallow, prior exclusions, and value dominance.

Scope Weight only
Prior Adult Step 11Wa: 2D Ordered Weight Pairs
Next Adult Step 11Wb: Moderate EWMA
Distinct/RVs Uses allRV (all included values) for both detection and evaluation
Exclusion Codes Exclude-A-2D-Non-Ordered
Configurable parameter names wtallow_formula

Overview:

This step handles subjects whose remaining included weights have exactly 2 distinct values that are NOT time-ordered — meaning the values are interleaved (e.g., 70, 110, 70, 110). Unlike 2D ordered pairs (Adult Step 11Wa), the values cannot be separated into “before” and “after” groups, so a different evaluation strategy is used.

The step applies a four-rule decision tree: (1) if all adjacent different-value pairs are within wtallow, keep everything; (2) if any pair exceeds wtallow and the subject has prior non-SDE exclusions, exclude all; (3) if one value is dominant (>65% of observations), exclude only the minority; (4) if neither value dominates, exclude all.

Key terms and variable names:

  • any_outside — TRUE if any adjacent pair of different-value observations has a weight difference exceeding wtallow for that pair’s time gap
  • prior_nonSDE — TRUE if the subject has any prior weight exclusions that are not SDE-related (not Include, Exclude-A-Identical, Exclude-A-Extraneous, empty string, or NA)
  • dominant_pct — proportion of observations belonging to the more common value: max(count_v1, count_v2) / total
  • minority_val — the less common of the two distinct values

Configurable setting defaults and options:

Same as Adult Step 11Wa — uses compute_wtallow() with wtallow_formula. UW for each adjacent pair is the larger of the two weights in the pair (uw_pair).

Logic and implementation:

  1. Sort observations by age (tiebreaker: internal_id)
  2. For each adjacent pair where the weight values differ:
    • Compute time gap in months: abs(age_days[i+1] - age_days[i]) / 30.4375
    • Compute wtallow for that gap using compute_wtallow()
    • If abs(wt[i+1] - wt[i]) > wtallow + 0.12 → set any_outside = TRUE and stop checking
    • Skip adjacent pairs with identical weight values (diff = 0, never outside wtallow)
  3. Apply decision rules:
    • Rule 1: any_outside = FALSE → all observations stay included (no action)
    • Rule 2: any_outside = TRUE AND subject has prior non-SDE exclusions → exclude ALL observations
    • Rule 3: any_outside = TRUE, no prior non-SDE exclusions, AND one value is dominant (dominant_pct > 0.65, strict) → exclude only the minority value observations
    • Rule 4: any_outside = TRUE, no prior non-SDE exclusions, AND neither value dominates (dominant_pct <= 0.65) → exclude ALL observations

Prior non-SDE exclusion check:

Checks the full subject weight history (w_subj_keep), not just currently included observations. Uses exact code matching (not substring search) to identify SDE codes. Codes that do NOT count as “prior non-SDE”: Include, Exclude-A-Identical, Exclude-A-Extraneous, empty string, NA. Everything else — including Exclude-A-Scale-Max-Identical — counts as a prior non-SDE exclusion.

Rationale for selected decisions:

  • Interleaved values need different logic than ordered: Ordered pairs have a clear “before vs. after” that can be evaluated with a single wtallow check. Interleaved values suggest either real oscillation (keep) or data quality issues (exclude), so the decision tree weighs multiple factors.
  • Prior exclusions as a signal: If the subject already has non-SDE exclusions, there’s existing evidence of data quality issues, so the threshold for excluding remaining values is lower.
  • Dominance rule (65%): When one value clearly predominates, the minority value is more likely to be erroneous. The 65% threshold prevents excluding values that are roughly equally represented, where the “error” is ambiguous.
  • All-or-nothing for balanced splits: When neither value dominates and the difference exceeds wtallow, both values are suspect and all are excluded.

Moderate EWMA (Adult Step 11Wb)

Exclude weight outliers using a 7-step EWMA flow with trajectory rescue, alternate pathway, percentage criterion, and error load detection.

Scope Weight only
Prior Adult Step 11Wa2: 2D Non-Ordered Weight Pairs
Next Adult Step 13: Distinct Single (1D)
Distinct/RVs Independent: single pass, RVs as full participants. Linked: firstRV pass (non-RV only) with RV propagation, then allRV pass.
Exclusion Codes Exclude-A-Traj-Moderate, Exclude-A-Traj-Moderate-allRV, Exclude-A-Traj-Moderate-Error-Load, Exclude-A-Traj-Moderate-Error-Load-RV
Configurable parameter names wtallow_formula, ewma_window, repval_handling, mod_ewma_f, perclimit_low, perclimit_mid, perclimit_high

Overview:

This step handles subjects with 3+ distinct weight values. It uses EWMA deviation to identify outliers, but with a more nuanced 7-step flow compared to Extreme EWMA (Adult Step 9Wb). The key differences from Extreme EWMA are:

  • Uses wtallow (time-interval-dependent) as the threshold instead of fixed caps
  • Includes trajectory rescue — values that follow the interpolation/extrapolation of surrounding measurements are protected from exclusion
  • Has an alternate pathway for values adjacent to unreliable neighbors (≤14 days, >wtallow apart)
  • Includes a percentage criterion (wt/EWMA < perclimit) as an independent exclusion pathway
  • Detects error load (4+ consecutive candidates) to flag subjects with widespread data quality issues
  • Uses graduated scoring to prioritize which candidate to exclude when multiple exist

Each round excludes at most one non-error-load value (the highest-scoring candidate). Error load values are all excluded immediately. The step iterates until no candidates remain, fewer than 3 values are left, or the maximum round limit (100) is reached.

Key terms and variable names:

  • dewma_all / dewma_bef / dewma_aft — EWMA deviation using all, all-except-predecessor, all-except-successor
  • wtallow (wta) — time-interval-dependent weight allowance, computed from compute_wtallow(months, formula, uw) using the minimum of the before and after age gaps. UW = max(wt, ewma) for the observation.
  • wta_base — wtallow at UW=120 (no UW adjustment), used as the denominator in prioritization scoring. This ensures scoring is not distorted by UW-dependent wtallow.
  • minagediff — minimum of before and after age gaps (years), used to determine wtallow
  • perclimit — observation-level ratio threshold via compute_perc_limit(meas, perclimit_low, perclimit_mid, perclimit_high). Weight-dependent and permissiveness-dependent (see permissiveness table). Note: observation-level here, unlike Adult Step 11Wa which uses subject-level max.
  • Standard-pathway candidates (exc_stand) — observations that pass the standard EWMA criterion and fail all trajectory rescue methods (Sub-step 1)
  • Alternate-pathway candidates (exc_pair) — observations flagged by the alternate pathway for being adjacent to unreliable neighbors (Sub-step 3)
  • Accumulated exclusion candidates (exc_wt_i) — combined set from standard (Sub-step 2) and alternate (Sub-step 4) pathways, used for consecutive-run detection
  • Error-load observations (error_load) — observations in runs of 4+ consecutive accumulated candidates, excluded immediately in Sub-step 7
  • ewma_window — number of observations on each side used for EWMA computation (default 15, so up to 30 total neighbors)
  • trajectory rescue — three methods (interpolation, prior extrapolation, next extrapolation) that can protect a value from standard-pathway exclusion

Configurable setting defaults and options:

Setting Default Unit Notes
wtallow_formula "piecewise" — Formula for wtallow (see Adult Step 11Wa and the wtallow formulas section)
mod_ewma_f 0.75 — Directional factor: loosest/looser = 0.75; tighter/tightest = 0.60
perclimit_low 0.5 — % criterion for wt ≤45 kg (loosest/looser); 0.7 (tighter/tightest)
perclimit_mid 0.4 — % criterion for 45<wt≤80 kg (all levels)
perclimit_high 0.0 — % criterion for wt >80 kg; 0 = disabled (loosest/looser); 0.4 (tighter/tightest)
ewma_window 15 observations Max observations on each side for EWMA. All EWMAs use this window.
repval_handling "independent" — Controls single vs two-pass EWMA
max_rounds 100 — Maximum EWMA iteration rounds

Logic and implementation:

Guards:

  1. Only subjects with 3+ distinct weight values enter this step (others went to 11Wa, 11Wa2, or 13)
  2. Must have ≥ 3 included observations
  3. Trigger check: Skip if weight range ≤ min wtallow AND (perclimit disabled OR min percent ratio ≥ max perclimit) — impossible to trigger any exclusion

EWMA computation:

  • Exponent e = -5, anchor a = 0 (same as Extreme EWMA)
  • Position-based window: only the nearest ewma_window (default 15) observations on each side contribute
  • Rebuilt from scratch each round

7-step exclusion flow (each round):

Sub-step 1 — Standard pathway + trajectory rescue:

  • Standard criteria: |dewma_all| > wtallow + 0.12 AND both directional dewma exceed mod_ewma_f × wtallow + 0.12 in the same direction (mod_ewma_f = 0.75 at loosest/looser, 0.60 at tighter/tightest)
  • Trajectory rescue checks three methods (all use ±5 kg error margin):
    1. Interpolation: Value falls between p1 and n1 (± 5 kg). Uses non-strict inequalities (≥, ≤).
    2. Extrapolation prior: Value falls within the range defined by p2 and the linear extrapolation from p2→p1. Rounded to nearest 0.2. Distance guard: extrapolation distance must be ≤ 2× source interval.
    3. Extrapolation next: Same as prior, using n1→n2 backwards. Same distance guard.
  • A value is rescued if ANY of the three methods succeeds. Must fail ALL three to remain a candidate.
  • exc_stand = standard_criteria AND fails_all_trajectory

Sub-step 2 — Standard run detection + pair/trio prioritization:

  • Detect consecutive runs of exc_stand within each subject
  • 4+ consecutive → error load (all excluded immediately in Sub-step 7)
  • Isolated (run length 1) → directly becomes exc_wt_i
  • Pairs/trios (run length 2-3) → score each member and pick the highest:
    • First of run: |dewma_aft / wtallow_aft_unrel| (wtallow computed as if before-neighbor were removed)
    • Last of run: |dewma_bef / wtallow_bef_unrel|
    • Middle of trio: |dewma_bef/wtallow_bef_unrel + dewma_aft/wtallow_aft_unrel|
    • Highest score → exc_wt_i

Sub-step 3 — Alternate pathway:

  • Catches values adjacent to unreliable neighbors (≤14 days AND >wtallow apart)
  • If prior is unreliable: only check aft dewma (|dewma_all| > wtallow + 0.12 AND |dewma_aft| > mod_ewma_f × wtallow + 0.12, same direction)
  • If next is unreliable: only check bef dewma (same logic)
  • Must not already be exc_wt_i
  • Result: exc_pair

Sub-step 4 — Alternate run detection + pair/trio prioritization:

  • Same run detection as Sub-step 2, applied to exc_pair candidates
  • Isolated → directly exc_wt_i
  • Pairs/trios → scored with same approach, accounting for committed neighbors (exc_wt_i from prior steps that are adjacent to the pair/trio)
  • Effective position adjusts when a committed neighbor is adjacent (first becomes middle, etc.)

Sub-step 5 — Percentage criterion:

  • Independent of EWMA deviation direction
  • wt/ewma_all < perclimit AND wt/ewma_bef < perclimit AND wt/ewma_aft < perclimit
  • Must not already be exc_wt_i
  • Uses observation-level perclimit via compute_perc_limit(meas, perclimit_low, perclimit_mid, perclimit_high)

Sub-step 6 — Consecutive check (all pathways combined):

  • After Adult Steps 2, 4, 5 accumulate exc_wt_i candidates, re-check for 4+ consecutive
  • Newly identified 4+ runs → error load

Sub-step 6b — End-pair median guard:

  • Applied to the non-error-load candidate set just before final prioritization, to the two positional end pairs: first/second (positions 1-2) and penult/last (positions n-1, n).
  • For an end pair, the guard fires only when all hold: subject has ≥ 4 values (low enough to reach the small-n real-data case the guard is designed for); both members are independently eligible (flagged by the standard, alternate, or percentage pathway, ignoring error load); the interior-side neighbor is NOT eligible (position n-2 for penult/last, position 3 for first/second — a 3+ run defers to the run logic); and the within-pair age gap is < endpair_gap_days (default 90).
  • When it fires, compute the median of up to 3 interior-side values (positions max(1, n-4)..n-2 for penult/last; positions 3..min(n, 5) for first/second). At n = 4 the window holds 2 values, so the “median” is the mean of those two — less robust than the 3-value form, but adequate for the partner-contamination case the guard targets. The member closer to that median is kept (removed from this round’s candidate set); the farther member is forced into the candidate set as the round’s nominee. An exact distance tie keeps the edge value (the first or the last) and nominates the inner member (the second or the penult).
  • This selects which end-pair member is the nominee; it does not itself exclude anything — the Sub-step 7 scoring below still decides whether the nominee is excluded this round. Removal stays one-per-round; the protected member is re-evaluated next round.

Sub-step 7 — Final prioritization:

  • Error load values: All excluded immediately with code "<label>-Error-Load" (e.g. Exclude-A-Traj-Moderate-Error-Load)
  • Non-error-load candidates: Score each and exclude only the single highest-scoring:
    • Edge (first/last in subject): pmax(0, |dewma_all| - wta) / wta_base where wta is UW-adjusted wtallow and wta_base is wtallow at UW=120 (no adjustment)
    • Interior: graduated multiplier × (excess_bef + excess_aft) where each excess = pmax(0, |dewma| - wta) / wta_base
      • min_excess = min(excess_bef, excess_aft)
      • multiplier = min(1.0, 0.6 + 0.4 × min_excess)
      • Full score when both directions independently exceed wtallow; 0.6× discount when only one direction is bad
    • Scoring uses excess above UW-adjusted wtallow divided by base wtallow, so a 10 kg excess at any UW always scores the same
    • Tiebreakers: smallest wtallow → closest to median age → earliest position → lowest internal_id

Mode handling:

Independent mode: Single pass with all values (including RVs) as full participants.

Linked mode:

  1. firstRV pass: Run remove_mod_ewma_wt() on non-RV values only. Label: "Exclude-A-Traj-Moderate".
    • Error load RV escalation: If any error-load value has RV copies, escalate the entire patient — all remaining Include values get "Exclude-A-Traj-Moderate-Error-Load-RV".
    • Non-error-load propagation: Propagate exclusions to RV copies via propagate_to_rv() (appends -RV-Propagated suffix).
    • Re-identify RVs on remaining values.
  2. allRV pass: Run remove_mod_ewma_wt() on all remaining values (RVs now participate). Label: "Exclude-A-Traj-Moderate-allRV".

Rationale for selected decisions:

  • wtallow instead of fixed caps: Moderate EWMA handles subjects with 3+ distinct values over varying time spans. Fixed caps (appropriate for Extreme EWMA’s large outliers) would be too coarse; wtallow scales with the time interval to catch subtler errors.
  • Trajectory rescue: Values that follow the linear trend of surrounding measurements are likely real, even if their EWMA deviation is large (e.g., a genuine rapid weight change). Rescue prevents false positives. Must fail all three methods to ensure the value genuinely doesn’t fit any trajectory.
  • Alternate pathway: When a neighbor is ≤14 days away and >wtallow apart, that neighbor is unreliable and distorts the directional dewma on that side. The alternate pathway drops the unreliable side’s confirmation requirement.
  • End-pair median guard (Adult Step 6b): At a record end, each value’s EWMA reference is dominated by its single adjacent neighbor (the e=-5 exponent makes the nearest neighbor carry nearly all the weight). When the first/second or penult/last pair diverge, the in-line member’s EWMA is contaminated by its divergent partner, so the run/score logic can exclude the in-line value and keep the outlier. The guard substitutes a clean, partner-independent reference — the median of up to 3 interior-side values — and keeps the member nearer that baseline. Trigger floor is n >= 4; at n = 4 the window holds only 2 interior-side values, so the “median” is the mean of those two and is less robust than the 3-value form at n >= 5. It is restricted to short within-pair gaps (< endpair_gap_days, default 90), since larger real shifts are expected over longer intervals, and it defers to the run logic whenever the interior-side neighbor is also flagged.
  • Graduated multiplier for interior scoring: Interior values have both before and after context, while edge values have only one direction. The graduated multiplier bridges these: when both directions independently confirm (min_ratio ≥ 1), the score is full; when only one direction is bad, the score is discounted (0.6× floor), preventing interior values from being unfairly ranked against edge values.
  • Observation-level perclimit (not subject-level): Unlike Adult Step 11Wa where the max perclimit across observations is used, here each observation uses its own weight and the current permissiveness level to determine its threshold via compute_perc_limit(). At loosest/looser, observations >80 kg have perclimit=0 (disabled).
  • Error load escalation in linked mode: If a patient has both repeated values (RVs) and 4+ consecutive flagged values, the data quality is too poor to trust any remaining values.
  • EWMA window (default 15): Limits each EWMA to the 15 nearest observations on each side. For subjects with many observations, distant measurements (which contribute negligibly due to the e=-5 exponent) are excluded from the computation, improving both efficiency and preventing numerical noise from very distant observations.
  • Max rounds (default 100): Allows the algorithm to handle subjects with many observations without being artificially limited. In practice, convergence typically occurs within a few rounds.

Single Distinct Value: 1D Evaluation (Adult Step 13)

Exclude height or weight values when a subject has only one distinct value for that parameter, using BMI-dependent or no-BMI limits.

Scope Height and Weight
Prior Adult Step 11Wb: Moderate EWMA
Next Adult Step 14: Error Load
Distinct/RVs Weight uses firstRV for numdistinct. Remaining RVs are treated as Include.
Exclusion Codes Exclude-A-Single
Configurable parameter names single_ht_min_bmi, single_ht_max_bmi, single_wt_min_bmi, single_wt_max_bmi, single_ht_min_nobmi, single_ht_max_nobmi, single_wt_min_nobmi, single_wt_max_nobmi, single_bmi_min, single_bmi_max

Overview:

After all prior steps, some subjects may have only one distinct value remaining for height or weight. These cannot be evaluated by EWMA or pair comparisons, so this step applies simple range limits. The limits depend on whether BMI can be computed (i.e., the subject has both a height and a weight on the same day):

  • BMI available and within range: Use loose limits (wide range of acceptable values). A BMI is formed for every same-day pairing of an included height with an included weight; the subject counts as within range if any of those BMIs is in range.
  • BMI available but extreme: Exclude all 1D values for the parameter when every same-day BMI is out of range (BMI outside normal range suggests data quality issue)
  • No BMI available: Use tighter limits (can’t validate with BMI, so be more conservative)

Key terms and variable names:

  • 1D — subject has exactly 1 distinct included value for a parameter (height or weight)
  • BMI — computed as weight_kg / (height_cm / 100)² from same-day height/weight pairs
  • bmi_extreme — TRUE if no same-day BMI (over all pairings of an included height with an included weight on the same day) is within [single_bmi_min, single_bmi_max] (out of range = strict < / >)
  • has_bmi — TRUE if the subject has at least one day with both an included height and an included weight

Configurable setting defaults and options:

Setting loosest looser tighter tightest
single_ht_min_bmi (cm) 60 120 142 147
single_ht_max_bmi (cm) 245 230 213 208
single_wt_min_bmi (kg) 12 30 36 39
single_wt_max_bmi (kg) 350 270 159 136
single_ht_min_nobmi (cm) 120 120 142 147
single_ht_max_nobmi (cm) 245 230 213 208
single_wt_min_nobmi (kg) 30 30 36 39
single_wt_max_nobmi (kg) 350 270 159 136
single_bmi_min (kg/m²) 10 12 16 18
single_bmi_max (kg/m²) 250 65 45 40

Note: For looser/tighter/tightest, the BMI-available and no-BMI limits are identical — BMI availability does not change which values are excluded at these levels. The BMI/no-BMI split only matters at loosest (wider limits when BMI is available: 60 cm vs 120 cm for ht_min; 12 kg vs 30 kg for wt_min). single_bmi_min/single_bmi_max match the overall PIV BMI limits at looser, tighter, and tightest; at loosest they are narrower (10–250 vs 5–300).

Logic and implementation:

The step runs as a two-pass loop over each subject (both passes use the same logic, but pass 2 catches cases where a pass 1 exclusion removes a same-day partner, causing BMI to become unavailable):

Pass 1:

  1. Determine which parameters are 1D: exactly 1 distinct included value and at least 1 observation
  2. If neither height nor weight is 1D, skip this subject
  3. Check BMI availability: find days where both an included height and an included weight exist
  4. If BMI available:
    • Compute a BMI for every same-day pairing of an included height with an included weight
    • If every BMI is extreme (bmi < single_bmi_min or bmi > single_bmi_max, strict): exclude ALL 1D height observations AND all 1D weight observations
    • If any BMI is within range: check each 1D parameter against the BMI-available limits (single_ht_min_bmi/single_ht_max_bmi, single_wt_min_bmi/single_wt_max_bmi). Exclude if outside limits, applying the 0.12 cm/kg rounding tolerance: excluded if < limit − 0.12 or > limit + 0.12.
  5. If no BMI available: check each 1D parameter against the no-BMI limits (single_ht_min_nobmi/single_ht_max_nobmi, single_wt_min_nobmi/single_wt_max_nobmi). Exclude if outside limits, applying the 0.12 cm/kg rounding tolerance: excluded if < limit − 0.12 or > limit + 0.12.

Pass 2:

  • Re-evaluate after pass 1 exclusions. If pass 1 excluded a height (or weight), the partner weight (or height) may have lost its BMI availability. In that case, re-evaluate using the tighter no-BMI limits.
  • If BMI is still available, nothing new is checked (already handled in pass 1).

Rationale for selected decisions:

  • BMI-dependent limits: When both height and weight are available on the same day, BMI provides a cross-validation that allows looser individual limits. Without BMI, tighter limits compensate for the lack of cross-validation.
  • BMI extreme → exclude all 1D: If every same-day combination of height and weight produces an implausible BMI, at least one must be wrong. Since both are 1D (no other values to compare against), there’s no way to determine which, so both are excluded.
  • Two-pass structure: A pass 1 BMI-extreme exclusion of height could leave weight without BMI availability. Pass 2 catches this by re-evaluating with tighter no-BMI limits.
  • Rounding tolerance on the range checks: The min/max range checks (BMI-available and no-BMI) carry the 0.12 cm/kg rounding tolerance used throughout the adult algorithm — a value within 0.12 of a boundary is kept. The BMI-extreme check (bmi < single_bmi_min / > single_bmi_max) is the exception: it uses strict inequalities with no tolerance.

Error Load (Adult Step 14)

Exclude all remaining included values for a parameter when the proportion of errors is too high, indicating widespread data quality issues.

Scope Height and Weight (evaluated separately)
Prior Adult Step 13: Single Distinct (1D)
Next Adult Step None (final step)
Distinct/RVs Weight uses firstRV exclusion codes for error counting
Exclusion Codes Exclude-A-Unevaluable-Trajectory (applied once per param family — HEIGHT and WEIGHT — in the long-format split; one code)
Configurable parameter names error_load_threshold

Overview:

This is the final step. It evaluates whether a subject has too many errors relative to the total number of meaningful observations. If the ratio of errors to (errors + included values) exceeds the threshold, all remaining included values for that parameter are excluded. Height and weight are evaluated independently — a subject can have error load for weight but not height, or vice versa.

Key terms and variable names:

  • error ratio — n_errors / (n_errors + n_included) for one parameter within one subject
  • n_errors — count of observations with error exclusion codes (see below)
  • n_included — count of observations still marked “Include”
  • denominator — n_errors + n_included (must be ≥ 3 to evaluate)
  • SDE exclusions — Same-day identical and extraneous exclusions are NOT counted as errors and are excluded from the denominator entirely

Configurable setting defaults and options:

Setting Default Description
error_load_threshold 0.41 at loosest/looser, 0.29 at tighter/tightest Error ratio above this triggers exclusion (strict >)

Which exclusion codes count as errors:

Height errors:

  • Exclude-A-PIV
  • Exclude-A-Single (1D)
  • Exclude-A-Ord-Pair
  • Exclude-A-Ord-Pair-All
  • Exclude-A-Window
  • Exclude-A-Window-All

Weight errors:

  • Exclude-A-PIV
  • Exclude-A-Single (1D)
  • Exclude-A-2D-Ordered
  • Exclude-A-2D-Non-Ordered
  • Exclude-A-Scale-Max
  • Exclude-A-Scale-Max-Identical
  • Exclude-A-Scale-Max-RV-Propagated
  • Exclude-A-Evil-Twins
  • All Exclude-A-Traj-* codes (Extreme and Moderate, all rounds and variants)

NOT counted as errors:

  • Same-day identical / extraneous (SDE) — excluded from denominator entirely
  • EWMA-RV-propagated codes — these represent the same underlying error as the source exclusion; counting both would inflate the ratio

Logic and implementation:

  1. For each subject, process height and weight separately
  2. Remove SDE observations from consideration (not errors, not in denominator)
  3. Count errors (observations matching error codes above)
  4. Count remaining included observations
  5. Compute denominator = errors + included. If denominator < 3, skip (too few observations to evaluate)
  6. Compute ratio = errors / denominator
  7. If ratio > error_load_threshold (0.41 at loosest/looser, 0.29 at tighter/tightest; strict >): exclude all remaining included values for that parameter

Rationale for selected decisions:

  • Threshold 0.41 (not 0.40): The 0.01 buffer (effectively 0.4 + 0.01) prevents borderline cases from triggering. A ratio of exactly 0.40 (e.g., 2 errors out of 5) is not considered excessive.
  • Minimum denominator of 3: With fewer than 3 meaningful observations, the ratio is too unstable to be informative (e.g., 1 error out of 2 = 0.50, but this single remaining value may be perfectly valid).
  • SDEs excluded from denominator: Same-day duplicates are a data collection artifact, not measurement errors. Including them would dilute the ratio.
  • EWMA-RV-propagated not counted: In linked mode, when a firstRV value is excluded by EWMA, its RV copies are propagated. These represent the same measurement error — counting both the source and propagated copies would double-count the error.
  • Scale-Max-RV-propagated IS counted: Unlike EWMA-RV-propagated, scale-max RV copies represent independent observations at the scale ceiling. Each contributes evidence of a data quality pattern.
  • Height and weight evaluated independently: A subject with many height errors but good weights should only lose the heights, not the weights.

wtallow: base formulas and adjustment

General concepts and terms

wtallow wtallow is a derived value used in multiple steps. Conceptually, it represents the threshold above which an absolute difference in weight between two values will result in exclusion of one of the two values. It is one of the most important values determining whether a value should be excluded. Importantly, most steps where wtallow is used include other constraints on exclusion or other paths to exclusion. Therefore, not every difference higher than wtallow will result in exclusion, and some differences below wtallow will result in exclusion. wtallow is dependent on two things - the interval between the two measurements being compared and the upper weight (UW) of the two measurements.

Upper weight (UW) UW is the higher of the two adjacent weights that are being evaluated when calculating wtallow. An UW of 120 is considered the base value. UW >120 are considered high and UW <120 are considered low.

wtallow-et-cap The wtallow-et-cap (ET cap) is a value related to wtallow used in Evil Twins (Adult Step 9Wa) and Extreme EWMA (Adult Step 9Wb). In Evil Twins, an adjacent-pair difference above the ET cap leads to an exclusion only when the pair is flanked by anchoring values (the anchor rule); in Extreme EWMA it is the threshold for the EWMA deviation. The wtallow-et-cap-6m is the et-cap for intervals <=6m. The wtallow-et-cap-12m is the et-cap for intervals >6m.

wtallow-cap These are values at 6m and 12m that represent a maximum value for wtallow even if the formula for wtallow would otherwise give a higher value. For intervals <=6m, no value can be above the wtallow-cap-6m. For intervals >6m, no value can be above the wtallow-cap-12m (this includes intervals >12m). The wtallow-cap values are also used in the wtallow formulas.

Piecewise-Higher (PW-H) The formula used to determine wtallow in looser and loosest permissiveness levels. It is adjusted for high UW and low UW. Piecewise-Lower (PW-L) The formula used to determine wtallow in tighter permissiveness levels. It is adjusted for low UW (not high UW). allofus15 The formula used to determine wtallow in tightest permissiveness levels, adapted from the Guide et al. (2024) All of Us algorithm. It is not adjusted based on UW.

Suffixes For PW-H and PW-L, we will use suffixes -Base, -highUW, and -lowUW to designate formulas for the three UW conditions (=120, >120, and <120).

wtallow-1-day The value of wtallow at one day. It is equal to 10 + 10 × log(1 + 5×1/30.4375) / log(6), approximately 10.85

ceiling A maximum wtallow for PW-H-lowUW and PW-L-lowUW that limits wtallow beyond the limits provided by the wtallow-cap. A ceiling is only used for PW formulas adjusted for lowUW.

Brief rationale for features of base formulas

Based on literature, input from experts, and clinical experience, the following concepts informed wtallow.

  • Weight change can be large even in a single day or very few days from fluid shifts. After 1-2 weeks, the slope of potential weight change generally slows.
  • Many of the largest weight changes occur after bariatric surgery or GLP1 agonist use. Literature shows that maximum weight loss after bariatric surgery or GLP1 agonist treatment occurs at approximately 12m, so wtallow does not rise after 12m. Of this weight change, a higher proportion occurs in the first 6m, so the slope is higher from 1m to 6m than 6m to 12m. Time courses for large weight loss with cancer can vary, but would generally not be larger or faster than those from weight loss surgery.
  • In general, PW-H was designed to include (i.e., not exclude) almost all physiologically plausible changes. PW-L was designed to include physiologically plausible changes for many, but not all, conditions. allofus15 was adapted from the Guide et al. (2024) All of Us algorithm to include changes that would be seen in people without severe or uncommon health problems.

Rationale for adjustment

People with higher weight can have larger absolute changes in body weight. Similarly, people with lower weight will typically have smaller absolute changes in body weight. The relationship is complex, varies over time, and depends on whether you are considering weight gain or weight loss. Adjusting allowed wt change by the upper weight (UW) of a pair can account for some of these differences, although it does not account for all situations. Most important, adjusting the wtallow (max wt changed allowed) can increase the risk of inappropriately including erroneous high weights. For this reason, we cap the adjustment for upper weights at the values that would be used for an upper weight of 180.

Formulas

PW-H (Piecewise-Higher)

PW-H-Base (UW = 120, no adjustment):

  • wtallow-cap-H-6m-base = 50

  • wtallow-cap-H-12m-base = 80

  • wtallow-et-cap-H-6m-base = 70

  • wtallow-et-cap-H-12m-base = 100

  • ceiling = N/A

  • months ≤ 1: 10 + 10 × log(1 + 5×months) / log(6)

  • 1 < months ≤ 6: linear increase from 20 @1m to wtallow-cap-H-6m-base @6m; works out to 20 + 6 × (months − 1)

  • 6 < months ≤ 12: linear increase from wtallow-cap-H-6m-base @6m to wtallow-cap-H-12m-base @12m; works out to 50 + 5 × (months − 6)

  • months > 12: flat at wtallow-cap-H-12m-base (80)

PW-H-highUW (Scaling up, UW>120): if UW is >180, use 180 as UW in formula

  • wtallow-cap-H-6m-highUW = 50 + 0.25 x (UW-120)

  • wtallow-cap-H-12m-highUW = 80 + 0.25 x (UW-120)

  • wtallow-et-cap-H-6m-highUW = 70 + 0.25 x (UW-120)

  • wtallow-et-cap-H-12m-highUW = 100 + 0.25 x (UW-120)

  • ceiling = N/A

  • months ≤ 1: linear increase from wtallow-1day (≈10.85) @ month 0 to 25 at 1m

  • 1 < months ≤ 6: linear increase from 25 @1m to wtallow-cap-H-6m-highUW @6m

  • 6 < months ≤ 12: linear increase from wtallow-cap-H-6m-highUW @6m to wtallow-cap-H-12m-highUW @12m

  • months > 12: flat at wtallow-cap-H-12m-highUW

PW-H-lowUW (Scaling down, UW < 120):

  • wtallow-cap-H-6m-lowUW = (50 − 20) × (UW / 120) + 20

  • wtallow-cap-H-12m-lowUW = (80 − 20) × (UW / 120) + 20

  • wtallow-et-cap-H-6m-lowUW = (50 − 20) × (UW / 120) + 40

  • wtallow-et-cap-H-12m-lowUW = (80 − 20) × (UW / 120) + 40

  • ceiling: UW × 2/3

  • months ≤ 1: 10 + 10 × log(1 + 5×months) / log(6)

  • 1 < months ≤ 6: linear increase from 20 @1m to wtallow-cap-H-6m-lowUW @6m

  • 6 < months ≤ 12: linear increase from wtallow-cap-H-6m-lowUW @6m to wtallow-cap-H-12m-lowUW @12m

  • months > 12: flat at wtallow-cap-H-12m-lowUW

  • PW-L (Piecewise-Lower)

PW-L-Base (UW = 120, no adjustment):

  • wtallow-cap-L-6m-base = 33.33

  • wtallow-cap-L-12m-base = 53.33

  • wtallow-et-cap-L-6m-base = 53.33

  • wtallow-et-cap-L-12m-base = 73.33

  • months ≤ 1: 10 + 10 × log(1 + 5×months) / log(6)

  • 1 < months ≤ 6: linear increase from 20 @1m to wtallow-cap-L-6m-base @6m

  • 6 < months ≤ 12: linear increase from wtallow-cap-L-6m-base @6m to wtallow-cap-L-12m-base @12m

  • months > 12: flat at wtallow-cap-L-12m-base

PW-L-highUW (UW>120): SAME AS BASE FORMULA

  • wtallow-cap-L-6m-highUW = 33.33

  • wtallow-cap-L-12m-highUW = 53.33

  • wtallow-et-cap-L-6m-highUW = 53.33

  • wtallow-et-cap-L-12m-highUW = 73.33

  • months ≤ 1: 10 + 10 × log(1 + 5×months) / log(6)

  • 1 < months ≤ 6: linear increase from 20 @1m to wtallow-cap-L-6m-base @6m

  • 6 < months ≤ 12: linear increase from wtallow-cap-L-6m-base @6m to wtallow-cap-L-12m-base @12m

  • months > 12: flat at wtallow-cap-L-12m-base

PW-L-lowUW (Scaling down, UW < 120):

  • wtallow-cap-L-6m-lowUW = (33.33 − 20) × (UW / 120) + 20

  • wtallow-cap-L-12m-lowUW = (53.33 − 20) × (UW / 120) + 20

  • wtallow-et-cap-L-6m-lowUW = (33.33 − 20) × (UW / 120) + 40

  • wtallow-et-cap-L-12m-lowUW = (53.33 − 20) × (UW / 120) + 40

  • ceiling: UW × 2/3

  • months ≤ 1: 10 + 10 × log(1 + 5×months) / log(6)

  • 1 < months ≤ 6: linear increase from 20 @1m to wtallow-cap-L-6m-lowUW @6m

  • 6 < months ≤ 12: linear increase from wtallow-cap-L-6m-lowUW @6m to wtallow-cap-L-12m-lowUW @12m

  • months > 12: flat at wtallow-cap-L-12m-lowUW

allofus15

allofus15 is not adjusted based on UW except that its 12m cap is limited to never exceed the PW-L 12m cap for that UW.

allofus15-cap-12m = min(40, effective PW-L at 12m for that UW)

  • For UW ≥ 120: effective PW-L-12m = 53.33, so allofus15-cap-12m = 40
  • For UW < 120: effective PW-L-12m = min(wtallow-cap-L-12m-lowUW, UW × 2/3) where wtallow-cap-L-12m-lowUW = (53.33 − 20) × (UW / 120) + 20; allofus15-cap-12m = min(40, that effective value)

allofus15 wtallow formula:

  • 1–2 days: 5
  • 3–7 days: 10
  • 8d to 6m: 15
  • 6m to 12m: linear increase from 15 @6m to allofus15-cap-12m @12m
  • >12m: flat at allofus15-cap-12m

allofus15-et-cap: allofus15-cap-12m

Permissiveness framework specification

The permissiveness levels and every preset value — overall (PIV) HT/WT/BMI limits, the 1D single-value limits, wtallow_formula, perclimit_*, mod_ewma_f, error_load_threshold, ht_band, allow_ht_loss/allow_ht_gain, repval_handling — are listed in full, with each setting’s role, in vignette("configuration") §1D; the values are also shown under Permissiveness Framework at the top of this narrative. They are not repeated in this section. (The overall/PIV and 1D limits apply the same whether or not same-day BMI is available; the BMI limits apply only when a same-day height and weight are both present.) The subsections below cover the wtallow / ET-cap mechanics and the design rationale specific to this narrative; every preset value they reference lives in §1D.

Worked examples: wtallow and ET caps by UW

The wtallow and ET-cap formulas — the PW-H / PW-L / allofus15 curves, the base values, the UW > 120 scale-up (PW-H only, capped at UW = 180), the UW < 120 scale-down with the UW × 2/3 ceiling, and ET cap = wtallow cap + 20 — are specified above under Formulas and General concepts and terms. The preset that selects the formula per permissiveness level is in vignette("configuration") §1D. The tables below give computed values at selected UW for quick reference.

wtallow tables

PW-H (loosest/looser) at selected UW values:

Months   @60     @120(base)  @160    @180(cap)
  1      20.0    20.0        25.0    25.0
  6      35.0    50.0        60.0    65.0
 12      40.0    80.0        90.0    95.0
 24      40.0    80.0        90.0    95.0

PW-L (tighter) at selected UW values:

Months   @60     @120(base)  @160
  1      20.0    20.0        20.0
  6      26.67   33.33       33.33
 12      36.67   53.33       53.33
 24      36.67   53.33       53.33

Note: PW-L is not scaled up for UW > 120.

ET cap tables

PW-H (loosest/looser) at selected UW values:

UW       ET ≤6m   ET >6m
 60 kg   55.0     70.0
120 kg   70.0     100.0
160 kg   80.0     110.0
180 kg   85.0     115.0

PW-L (tighter) at selected UW values:

UW       ET ≤6m   ET >6m
 60 kg   46.67    56.67
120 kg   53.33    73.33
160 kg   53.33    73.33

Note: PW-L is not scaled up for UW > 120.

Gap between wtallow and ET cap

The ET cap exceeds the wtallow cap by exactly 20 at the cap points (6m and 12m) for PW-H and PW-L formulas (for UW < 120, the UW × 2/3 ceiling can lower wtallow further, widening the gap). This 20 kg gap ensures that moderate EWMA outliers (wtallow < dewma < ET cap) are caught by Step 11Wb, while extreme outliers (dewma > ET cap) are caught earlier by Step 9.

The full formulas are in the Formulas section above.

Perclimit (Percentage Criterion)

The perclimit criterion is an independent exclusion pathway in the moderate EWMA step (Step 11Wb), where it excludes values where wt/ewma < perclimit for all three EWMA calculations (main, bef, aft), regardless of whether the standard EWMA or trajectory criteria are met. In the 2D ordered step (Step 11Wa) it is applied to the ratio of the smaller to the larger of the two weights.

This criterion targets very low weight values that may not trigger the absolute wtallow threshold but are implausibly small relative to the patient’s other measurements.

The three thresholds correspond to the parameters perclimit_low (wt ≤45 kg), perclimit_mid (45<wt≤80 kg), and perclimit_high (wt >80 kg).

The per-level perclimit_low / perclimit_mid / perclimit_high values are in vignette("configuration") §1D (loosest/looser disable the >80 kg tier with 0; tighter/tightest use 0.7 at ≤45 kg and 0.4 elsewhere).

Application

In the moderate EWMA step (Step 11Wb), perclimit is applied at the observation level: each weight observation’s perclimit is determined by that observation’s weight value.

In the 2D ordered step (Step 11Wa), perclimit is applied at the subject level: the maximum perclimit across all weight observations for the subject is used.

Rationale

Loosest/looser at ≤45 kg (0.5): A perclimit of 0.5 means a weight must be at least 50% of its EWMA to survive. This accommodates very underweight patients (e.g., severe cachexia, anorexia) whose weights may legitimately be quite low relative to their historical measurements. For example, a cachectic patient at 35 kg with EWMA of 52 (percewma = 0.67) would be excluded under 0.7 but retained under 0.5.

Tighter/tightest at ≤45 kg (0.7): At these levels, the goal is to exclude values likely reflecting severe illness, so the more restrictive threshold is appropriate.

All levels at 45–80 kg (0.4): At these weights, a value less than 40% of its EWMA is almost certainly an error regardless of permissiveness level.

Loosest/looser: perclimit disabled (0) for wt > 80 kg: In the code, “disabled” is implemented as perclimit = 0, meaning the check percewma < perclimit is never TRUE (no ratio can be < 0). This is distinct from Inf, which would incorrectly trigger on every observation. For weights above 80 kg, UW-adjusted wtallow always catches any value that would also be caught by perclimit = 0.4. The mathematical reason: perclimit = 0.4 fires when dewma > 1.5 × wt. For wt > 80, this requires dewma > 120. The UW-adjusted wtallow catches these values first because wtallow scales with UW (the upper weight of the pair) while the perclimit threshold grows only with the observation’s own weight. Above 80 kg the coverage gap vanishes entirely.

Tighter/tightest: perclimit 0.4 for all wt > 45 kg: With PW-L or allofus15 wtallow (not scaled up for high UW), the base caps are lower (53.33 for PW-L, 40 for allofus15). wtallow still catches values that perclimit = 0.4 would catch for wt > 45. However, the 0.4 threshold is retained at tighter and tightest as a belt-and-suspenders safeguard with negligible cost.

Error Load Threshold

The error load threshold (Step 14) controls when a subject’s remaining included values are excluded because the ratio of errors to total non-SDE values exceeds the threshold. A lower threshold means the algorithm is more aggressive about escalating when multiple errors are detected. The tolerance of the error load threshold addresses a different aspect of permissiveness than most other permissiveness thresholds, but follows the same pattern of more values being excluded in tighter/tightest.

Level error_load_threshold if 10 obs, # errors needed to trigger
loosest >0.41 5/10
looser >0.41 5/10
tighter >0.29 3/10
tightest >0.29 3/10

Moderate EWMA Directional Factor

The f factor in the Moderate EWMA step (Step 11Wb) controls how strict the directional EWMA check is. The standard pathway requires dewma_before > f * wtallow and dewma_after > f * wtallow (in addition to dewma_all > wtallow). A lower f value makes the directional check less strict, leading to more exclusions. Per-level mod_ewma_f values are in vignette("configuration") §1D (0.75 at loosest/looser, 0.60 at tighter/tightest).

Height Band, Height Loss, and Height Gain

Per-level ht_band, allow_ht_loss, and allow_ht_gain values are in vignette("configuration") §1D (ht_band 3″ at loosest/looser, 2″ at tighter/tightest; loss allowed only at loosest; gain allowed at all but tightest).

Note: htloss is not allowed in looser because allowing height loss introduces a large number of additional inclusions that are more likely to represent measurement error than true height loss. Height gain is allowed in looser and tighter because young adults may still be growing, and measurement error in height is common. Height gain rescue applies only when the earlier measurement is at age <25 years.

Repeated Value (RV) Handling

RVs are values identical to a prior value for the same subject and parameter (on a different day). Two modes are available:

independent (default for loosest and looser)

RVs are treated as fully informative values:

  • RVs contribute to EWMA calculations normally
  • RVs are evaluated against that EWMA like any other value
  • RVs are no more likely to be excluded than non-RV values

linked (default for tighter and tightest)

RVs are treated as less informative values:

  • RVs are left out of the EWMA in the firstRV pass of each EWMA step (extreme and moderate)
  • RVs do not have their own EWMA in firstRV; RV propagation serves as their exclusion criterion instead
  • In the allRV pass, the remaining RVs participate in the EWMA like any other value
  • RVs are also more likely to cause exclusion of surrounding non-RV values, because removing RVs from the EWMA can expose patterns that would otherwise be masked

Downstream effects of linked mode

Because RVs are ghosted from the EWMA, their exclusion (or survival) can affect how surrounding values are evaluated. For example:

Patient weights (close together): 60 100 95 60 60 60 70

  • Under independent: the 60s contribute to the EWMA, pulling it toward 60; the 70 looks fine relative to that EWMA and survives.
  • Under linked: the 60s are excluded from the EWMA; the 70 is evaluated against an EWMA that does not include them and may be flagged depending on the surrounding values.

This means linked mode can result in more exclusions of non-RV values in addition to the RVs themselves. Users choosing tighter or tightest should be aware that this may affect subjects with runs of repeated values followed by a return toward a prior level.

RV handling in other steps

The repval_handling setting changes processing only in the two EWMA steps (Extreme EWMA, Adult Step 9Wb; Moderate EWMA, Adult Step 11Wb). In Scale Max (Adult Step 4W) it changes only the code given to RV copies (Exclude-A-Scale-Max-RV-Propagated under linked). Evil Twins (9Wa) and the 2D ordered/non-ordered steps (11Wa/11Wa2) treat RVs the same under both modes.

User overrides

To change one or a few settings without switching preset, pass them in the adult_overrides named list to cleangrowth() — e.g. cleangrowth(df, adult_permissiveness = "tighter", adult_overrides = list(allow_ht_loss = TRUE)) applies the tighter preset but re-allows height loss. Each named value replaces that preset value for adult rows; an unrecognized name is an error. (cleanadult() is an internal function and is not called directly.) The override mechanism and the full preset table are documented in vignette("configuration") §1C/§1D. Separately, wtallow_formula accepts a custom .csv file path if a fully custom weight-allowance table is needed.