Start here: Experienced users

This is an introduction to version 3 for previous users of growthcleanr.

New: a point-and-click app. growthcleanr now includes a Shiny app, so you can clean data without writing R code. Start it with growthcleanr_app(). The app walks you through loading a data file, mapping your columns, and running growthcleanr, and lets you save the results. It runs entirely on your own computer; nothing is uploaded. See the Shiny app how-to guide.

Additional instructions and documentation are available at growthcleanr.org.

Major categories of changes include:

Usability improvements, including a simpler command, a Shiny app that facilitates use of growthcleanr without writing R code, return of a data.table instead of a vector, handling of semi-wide and wide data, ability to use date of birth and visit date to calculate age in days, expanded options for alternate column names and coding schemes, more user-relevant exclusion codes, and addition of an optional id variable to ensure complete replicability of results across growthcleanr runs.
Many algorithm additions/improvements, including improved handling of infant data, cleaning of head circumference, more refined handling of carried forwards, and a new “evil twins” step that identifies multiple abnormal values in a row.
Many efficiency improvements
Documentation improvements including two new how-to guides: Shiny app and R interface

More information on these changes is below. For a more concise list of changes, see the changelog. For further details of current behavior, see the technical documentation with browseVignettes("growthcleanr").

Updating code written for growthcleanr 2.2.3

growthcleanr version 3 is designed to be easier to use than 2.2.3 (or prior versions), but scripts written for 2.2.3 need some changes. We apologize for any inconvenience to experienced users. Because of the new data.table output and new exclusion codes, some code based on the old growthcleanr results format could produce wrong results. Check your code for each item below. More differences are covered elsewhere; this section only covers old code that we expect was frequently used and will break or be silently wrong.

1. Results will differ from 2.2.3. No setting reproduces 2.2.3 exactly. To reproduce an analysis done with 2.2.3, keep using 2.2.3 (see Installing a prior version).

2. cleangrowth() returns a data table, not a vector.

# cleangrowth() uses columns named subjid, sex, agedays, param, and measurement, unless you specify other names.

# Recommended code for version 3 using standard growthcleanr column names
# Returns your data with results added
cleaned <- cleangrowth(data)

# Code for version 3 that returns only the exclude column, as a vector
exclude_codes <- cleangrowth(data)$exclude

# Closest to code for 2.2.3: name each argument and take $exclude. The code below stores it in a column named gcr_result.
data[, gcr_result := cleangrowth(subjid = subjid, sex = sex,
                                 agedays = agedays, param = param,
                                 measurement = measurement)$exclude]

# Old code (will no longer work)
data[, gcr_result := cleangrowth(subjid, param, agedays, sex, measurement)]

growthcleanr also now takes an optional row id column. See New id column below.

Code that treats the result as a vector, such as sum(cleaned == "Include"), df[cleaned == "Include", ], or df$gcr_result <- cleaned can run without an error and give NA, wrong counts, or the wrong rows; some forms stop with an error on a data.table. Use cleaned$exclude or cleaned$bin_exclude instead. To keep only included values: analysis <- cleaned[exclude == "Include"]. The returned columns are described in The cleangrowth() output. If your data still has a gcr_result column from an earlier run, drop it first. cleangrowth() does not change user input data, so the prior gcr_result is carried through to the output unchanged.

3. Exclusion codes have new names. Filters run after cleangrowth() for exclude == "Include" will still work, but filters for specific 2.2.3 codes, such as exclude == "Exclude-Carried-Forward" or grepl("Adult", exclude), match nothing, and R gives no warning. Version 3 Summary codes do not show which algorithm cleaned a row, so select adult rows by age. (In exclude_detailed, -A- marks most adult exclusions, but not included rows.) Update any filters for specific exclusion codes using Mapping from prior exclusion codes. If you previously had special handling for values excluded as carried-forwards, note that fewer values are excluded in this category now (see vignette("cf-rescue")), so you may wish to handle these differently.

4. Some arguments were removed or renamed. For example, num.batches → cores, include.carryforward → cf_rescue, and weight_cap (kg) → adult_scale_max_lbs (pounds). A removed argument stops with a message naming what to use instead. Recentering options also changed: there is a new default recentering table based on clinical data, sd.recenter = "nhanes" and sd.recenter = "derive" are no longer available, and sd.recenter = NA (the 2.2.3 default) now uses the default built-in table. If you wish to supply your own recentering table, see “Custom recentering” in vignette("recentering"). See Removed and renamed arguments.

5. Some functions were removed; their jobs are now handled within cleangrowth(). splitinput() is replaced by checkpointing (see vignette("large-datasets")). adjustcarryforward(), testacf(), and acf_answers() are replaced by the cf_rescue argument (see vignette("cf-rescue")). sd_median() was replaced by a new recentering method (see vignette("recentering")).

6. The command-line driver moved and its flags changed. The driver is now installed at scripts/gcdriver.R rather than exec/gcdriver.R, so shell scripts that point at the old path fail with “file not found”. Find the new path with gc_cli_path():

DRIVER=$(Rscript -e 'cat(growthcleanr::gc_cli_path())')
Rscript "$DRIVER" input.csv output.csv

Flags were changed due to new options. Only long flags are accepted now; 2.2.3 short flags such as -w, -s, and -n now stop with an error. See vignette("command-line").

7. longwide() and ext_bmiz() changed. ext_bmiz() output keeps your column names: code that read age, wt, ht, or bmi from its output should use your own column names (for example agem). longwide() now returns sex as M or F, not 1 or 2, and its output can go directly to simple_bmi() and ext_bmiz(). We made this change because, for historical reasons, cleangrowth() and longwide() read numerical sex as 0=male and 1=female, while ext_bmiz() reads it as 1=male and 2=female. We recommend using M=male and F=female, which works the same for all three functions. Code that filters longwide() output on sex == 1 or sex == 2 will match no rows, without a warning; use "M" or "F" instead. ext_bmiz() now stops on numerical sex unless sex has both 1s and 2s and no other values, so recode single-sex data to M or F. If your code uses either function, see their sections in vignette("utilities") for the current arguments and output.

Changes that may simplify your workflow

These are new options. Code that does not use them does not need to change.

New input formats are supported. growthcleanr previously only worked for long data (one row per observation, with weight and height on separate rows). It can now handle semi-wide data (one row per visit, with weight, height, and head circumference on the same row) and wide data (one row per subject). Set input_format = "semi_wide" or input_format = "wide"; results come back in the same layout unless you set output_format. For ready-to-run scripts, see “Semi-wide data (one row per visit)” and “Wide data (one row per subject)” in vignette("example-scripts"). For full detail, see “Input format: long, semi-wide, and wide” in vignette("wrapper"). The Shiny app also handles these formats.

Additional input recognized. For example, date of birth (dob) and visit date (visit_date) may now be entered instead of agedays. sex now accepts the following by default: 0=male, 1=female, M/F, Male/Female, B/G, Boy/Girl, any capitalization, as well as custom codes. For ready-to-run scripts, see “Use dates instead of age in days” and “Set your sex coding (including 1 = male, 2 = female)” in vignette("example-scripts"). For full detail on accepted date formats, see “Date parsing” in vignette("wrapper"). The Shiny app also handles dates and custom sex codes.

Quick start for experienced users

Install the updated version

Note If you already have an older growthcleanr installed, a) detach it if it is loaded detach("package:growthcleanr", unload = TRUE), b) uninstall it remove.packages("growthcleanr"), c) restart R (important!), d) and then install using the instructions below.

growthcleanr version 3 is available from CRAN:

install.packages("growthcleanr")

Basic command

New option: pass a data frame directly (instead of vectors).

If your data is in standard growthcleanr format (described below) use the code below. For ready-to-run scripts for other formats (semi-wide, wide), column names, coding, etc. see vignette("example-scripts"); for parallel processing and checkpointing on larger datasets, see vignette("large-datasets").

Check your setup first

gc_runtrial() runs growthcleanr on a small sample of your patients and reports how your columns, sex coding, and parameters were read. Pass it the same arguments you plan to use with cleangrowth(). See “Check your setup first” in vignette("example-scripts").

gc_runtrial(my_data)

Full cleaning run for smaller datasets

result <- cleangrowth(my_data)   # growthcleanr now takes an optional id column (unique per row); without one, ids are generated and a warning is shown

Full cleaning run for larger datasets

result <- cleangrowth(my_data,
                      checkpoint_dir = "gc_checkpoints",
                      parallel = TRUE)

Standard growthcleanr format:

Long (each weight, height, and head circumference on a separate row)
Columns with names and coding below:
- subjid (a unique subject identifier, may be any column type)
- sex (default accepted codes: 0=male, 1=female, M/F, Male/Female, B/G, Boy/Girl, any capitalization; for other codes (for example 1 = male, 2 = female), see “Set your sex coding (including 1 = male, 2 = female)” in vignette("example-scripts"))
- agedays (age in days; a fractional age is cleaned as its whole day and returned as submitted)
- param (WEIGHTKG, HEIGHTCM, LENGTHCM, HEADCM; or for imperial units, WEIGHTLBS, HEIGHTIN, LENGTHIN, HEADIN; to use your own labels (such as WT/HT/HC), see “Use your own measurement-type labels (WT / HT / HC)” in vignette("example-scripts"))
- measurement (measurement value in the units designated by the param column (long) or column name (semi-wide or wide))
- id (optional, and new to version 3, a unique row identifier that facilitates deterministic cleaning on successive runs)

New recommended sex coding. We now recommend coding sex as M/F to facilitate use of growthcleanr and the cdcanthro package without recoding sex.

The `cleangrowth()` output

Default columns

By default, cleangrowth() returns all of your entered columns and the additional columns below. If you did not enter an id column, a generated id is also returned.

Column	Description
`exclude`	Exclusion code (Summary level; see below). A factor.
`bin_exclude`	Binary flag (character): `"Include"` or `"Exclude"`
`unit_error_range`	Whether the value is in a range common for unit errors (on by default; set `unit_error_range = FALSE` to omit), meant to provide a dataset-level signal, should not be used as a definitive indicator of a unit error
`mean_ht`	Returned when the data has adult rows (set `mean_ht = FALSE` to omit): the adult algorithm’s mean height (cm), on adult height rows only

Extra input columns are carried through by default. With carry_through = TRUE (the default), growthcleanr returns the six input columns (renamed to your names) and the result columns alongside any other columns from your input, so the results attach directly to your full dataset. Set carry_through = FALSE to return only the six input columns plus the results; in that case, keep your id column and merge on it to reattach the results. An input column with the same name as a growthcleanr output column is returned as userinput_<name>.

New features by category

This list focuses on differences from the prior CRAN release, 2.2.3. There is extensive documentation of current behavior in the package — see browseVignettes("growthcleanr") for the full set (e.g. vignette("adult-algorithm"), vignette("exclusion-codes"), vignette("output"), vignette("cf-rescue")).

Coding is not detailed here. For ready-to-run scripts for other formats, column names, coding, etc. see vignette("example-scripts"); for larger datasets, see vignette("large-datasets").

Command and output

A Shiny app (growthcleanr_app()) for cleaning data without writing R code. See the Shiny app how-to guide.
growthcleanr now accepts data in semi-wide and wide formats. See the R interface how-to guide, Changes that may simplify your workflow, “Semi-wide data (one row per visit)” and “Wide data (one row per subject)” in vignette("example-scripts"), and “Input format: long, semi-wide, and wide” in vignette("wrapper").
Users may input date of birth and visit date rather than agedays. See “Use dates instead of age in days” in vignette("example-scripts") and “Date parsing” in vignette("wrapper").
The command-line driver, which cleans a CSV file from a terminal or batch job without writing R code, was updated for the new options and gives the same results as the equivalent R call. See vignette("command-line"). If you have shell scripts that call the 2.2.3 driver, see item 6 in Updating code written for growthcleanr 2.2.3.
cleangrowth() returns a data.table, not a vector.
More user-friendly exclusion codes, reported at three levels — a binary recommendation (bin_exclude), a short Summary set (default), and a more granular Detailed set (opt-in). Details
Addition of an optional id variable that ensures deterministic, replicable handling of borderline values across multiple runs of growthcleanr on the same dataset.
growthcleanr accepts M/F, male/female, B/G, and boy/girl in the sex column without any specification (in addition to 0=male, 1=female). It is not case-sensitive. M/F and B/G can also be used in the package cdcanthro that calculates z-scores, allowing use of growthcleanr and cdcanthro on the same dataset without conversion. Other codings can be set with sex_coding.
A trial run of a small sample of your data, gc_runtrial(my_data), reports how your columns, sex coding, and parameters were read. See Check your setup first.

Efficiency

A new feature, checkpointing, reduces memory use; with it, run-time for larger datasets increases almost linearly. Checkpointing is on by default: for datasets with more than 5,000 subjects, growthcleanr cleans in checkpoints saved to a temporary folder and deletes them when the run finishes. Set checkpoint_dir to keep the checkpoint files so an interrupted run can resume, or checkpoint = FALSE to write no intermediate files to disk. See vignette("large-datasets").
Parallel processing is simpler to use: set the parallel argument to TRUE. Parallel processing and checkpointing can be used together.
Multiple internal efficiency improvements invisible to the user.

Changes to child algorithm

Improved handling of infants: The prior pediatric algorithm was optimized for children 2 years of age and older. 2.2.3 contained a preliminary version of an algorithm with new features for infants, including infants whose growth pattern is consistent with prematurity or small-for-gestational age. There is now a single combined child version incorporating modifications for infants.
The infant handling includes “correction” for an assigned gestational age when early measurements are very low. The correction is designed to improve handling for babies who are premature or small for gestational age.
Addition of head circumference. Head circumference was only evaluated by the preliminary infants algorithm in 2.2.3, which was not recommended for research use; for practical purposes HC cleaning is new in version 3.
Improved handling of carried forwards. Many consecutive identical values that would previously have been excluded as CFs are now “rescued” if it seems reasonably likely that they were independently measured rather than carried forward. Setting thresholds for rescue is extremely challenging because the performance of different thresholds varies widely based on site-specific practices, so these methods may continue to evolve.
Improved recentering. Rather than deriving recentering medians from your data (or from NHANES data for smaller datasets), this version of growthcleanr uses built-in smoothed recentering medians from multi-center data which provide protection from areas of low visit density affecting recentering and therefore results. You can supply your own recentering medians with sd.recenter =, or turn recentering off with sd.recenter = "none". See vignette("recentering") for how the defaults were built and how to customize. In the past, growthcleanr could produce different results for an individual child based on the growth patterns of other children in the dataset. Results are now reproducible regardless of the size and makeup of the dataset.

Changes to adult algorithm

Refined “wtallow” formulas that determine allowed degree of weight deviation.
Loosening of height variation limits. (The default height window is looser.)
A redesigned adult algorithm with four permissiveness levels, set via adult_permissiveness: "loosest", "looser" (the default), "tighter", and "tightest". These set all adult thresholds at once; the tightest level is intended for evaluations of relatively healthy populations. permissiveness_presets() returns the settings for each level. Of the four, "looser" (the default) is the most similar to the prior adult algorithm, though none of them reproduce it exactly. For the intended use of each level, see Section 1D of vignette("configuration"). For a ready-to-run script, see “Data that includes adults” in vignette("example-scripts"). To change individual adult thresholds without switching level, use adult_overrides (see ?cleangrowth and “User overrides” in vignette("adult-algorithm")).
When a “weight cap” (now referred to as “scale max”) was set, 2.2.3 excluded all of a subject’s weights when one was recorded at the scale maximum; weights at the cap that fit the subject’s trajectory are now kept.
Removal of steps that identified specific types of errors (e.g., hundreds). These contained bugs, were inefficient, and did not improve accuracy.

Changes to both algorithms

New “evil twins” step that excludes consecutive erroneous measures that were missed previously.
Redesigned handling of more than one measurement on the same day. The prior adult algorithm often excluded all values when there was more than one measurement per parameter on the same day. This has been fixed. When comparing results with 2.2.3, compare by subject, parameter, and day, not by row.
More customization available, especially for adults. See vignette("configuration") for more details.
Removal of unit error/swap correction for children and similar “special cases” code in adult algorithm. They were lower-yield and error-prone. Detection of errors from these causes is now folded into the rest of the algorithm.
Addition of the unit_error_range output column to provide an indicator of whether a value is in a common range for unit errors. This column is calculated independently of exclusions and is not a definitive designation of a unit error. Patients with consistent atypical growth cannot always be distinguished from patients with unit errors. In other words, if a patient’s data is entirely in the wrong units, longitudinal cleaning can mistake them for children with consistent atypical growth. If a large proportion of values have unit_error_range = TRUE, examine your data to look for unit errors. The unit error range check is a useful indicator but can be wrong. Do not assume that these values should be corrected to other units. Some values are plausible in either unit, so they are never flagged. Other true values may be flagged, such as in data with measurements from premature infants. Importantly, many types of errors can result in a value that appears to be a unit error; “correcting” these for unit errors will result in erroneous values.
Adjusted the limits beyond which all values were excluded. These were previously called BIVs and are now called hard-limits.
The longwide() command was present in 2.2.3 but not widely known. It facilitates conversion of long data to semi-wide format (height and weight on one row) for calculation of BMI. For children, this can be used with the ext_bmiz() command to calculate extended BMI z-scores in growthcleanr. growthcleanr now also accepts and returns semi-wide and wide layouts directly, via input_format and output_format.
Other refinements that improve performance but would mostly be invisible to users

Optional output columns

Additional output columns can be requested with arguments such as exclude_detail, cf_detail, and full_detail. See vignette("configuration") for the full list and vignette("output") for what each column contains.

Inspecting or troubleshooting a result

If you want to understand why a particular value was excluded, learn more about your dataset, or customize your use of growthcleanr output, these options add diagnostic columns (all off by default):

exclude_detail = TRUE – the finer Detailed exclusion code, which names the reason more specifically than the Summary code.
cf_detail = TRUE – for carried-forward decisions, the deviation values behind a rescue-or-exclude call.
corr_detail = TRUE – gestational-age correction status.
recenter_detail = TRUE – the recentered z-scores the child algorithm used (specialized output for a related tool in development).
debug = TRUE – the full set of intermediate z-scores, mostly used for debugging and algorithm refinement.
display_gc_id = TRUE – the internal row id growthcleanr used, useful when tracing same-day tiebreaks.
full_detail = TRUE – turns on all of the above at once, plus tri_exclude and internal_detail.

cf_detail, corr_detail, recenter_detail, and the debug z-scores apply to child rows only; for adult rows, use exclude_detail = TRUE.

For a ready-to-run script, see “See more detail about why values were excluded” in vignette("example-scripts").

Exclusion codes

Why the codes changed

The prior codes indicated which step a value was excluded in but were overly detailed for the vast majority of users and used unnecessarily jargon-y descriptions. We aimed for exclusion codes with clearer names in fewer categories that are more relevant to researchers.

To support different needs, the updated version reports exclusions at three levels:

Binary - growthcleanr’s binary recommendation: Include or Exclude
Summary (the default exclude column) — a small set of clear, consolidated reasons that are more aligned with what researchers would want to report.
Detailed (opt in with exclude_detail = TRUE) — finer gradations than the Summary codes, but still organized around user-relevant reasons rather than internal step mechanics. Most Detailed codes keep a -C- (child) or -A- (adult) marker showing which algorithm assigned them (e.g. Exclude-C-Pair). Every Detailed code matches to one Summary code.

The Summary exclusion codes

These are the values that can appear in the default exclude column. Exclusion codes no longer designate whether they are for Adults or not (this is available in the detailed codes). More detailed mapping of how prior codes match to these codes is available below.

Summary code	Meaning
`Include`	Value passed all checks and was not flagged.
`Exclude-Not-GC-Param`	The `param` is not one growthcleanr cleans; the row is set aside with its measurement preserved.
`Exclude-Missing-Info`	A required field is missing or unevaluable (e.g. missing/negative/zero measurement, missing age, or a missing or unrecognized value in the sex column for a pediatric measurement).
`Exclude-HC-Out-of-Range`	Head circumference after 5 years of age, beyond the available reference.
`Exclude-Identical`	A same-day duplicate identical to the value that was kept.
`Exclude-Extraneous`	A same-day value, not identical to and not selected as the kept value.
`Exclude-Hard-Limit`	A value beyond a raw or standardized limit that prevents it from being evaluated further, regardless of a subject’s other measurements; for children, also a weight or height whose same-day BMI is impossible. Previously called BIV.
`Exclude-CF`	Deemed likely carried forward from the prior value.
`Exclude-Scale-Max`	Adult only, used only if a scale maximum (e.g., 400 lbs) is entered with `adult_scale_max_lbs`. Weights within ±0.1 kg of the cap are excluded if they are more than 50 kg different from their expected value, or if all of the subject’s weights (at least two) are the same value at the cap.
`Exclude-Extreme`	An extreme outlier — an adjacent extreme value pair/group, or an extreme trajectory deviation.
`Exclude-Pattern`	A moderate trajectory or velocity disagreement with the subject’s other values.
`Exclude-Pair`	Child only. Used when, by one of the last steps of the algorithm, a subject has only two unexcluded measurements for a parameter and one of them is excluded. (For adults, pairs are included in `Exclude-Pattern`.)
`Exclude-Single`	Used when, by one of the last steps of the algorithm, a subject has only one unexcluded measurement (children) or only one distinct unexcluded value (adults) for a parameter and it is excluded based on limits that consider the subject’s other parameter when available (child: the other parameter’s z-score; adult: a same-day BMI).
`Exclude-Traj-Uneval`	A subject/parameter combination in which so many values meet error criteria that the subject’s trajectory is unevaluable.

“Hard limits” instead of “biologically implausible”

In prior versions, values far from population norms were labeled biologically implausible (BIV). The updated version calls these hard limits. The name change is driven primarily by the new adult permissiveness levels. When using any but the loosest set of limits, the hard limits will intentionally exclude biologically plausible values.

The Exclude-Single category uses different limits and methods than the hard-limit step: its limits consider the subject’s other parameter when available (child: the other parameter’s z-score; adult: a same-day BMI).

Mapping from prior exclusion codes

If your code refers to specific exclusion-code names, use the mapping below. The Summary column is what now appears in exclude by default.

Child (pediatric) measurements:

This lists the codes from the default algorithm (the preliminary infants algorithm, an opt-in that was not recommended for research, has been removed).

Prior code	Summary (`exclude`)	Detailed (`exclude_detailed`)
`Include`	`Include`	`Include`
`Missing`	`Exclude-Missing-Info`	`Exclude-Missing-Info`
`Exclude-Carried-Forward`	`Exclude-CF`	`Exclude-C-CF`
`Exclude-SD-Cutoff`	`Exclude-Hard-Limit`	`Exclude-C-Hard-Limit`
`Exclude-EWMA-Extreme`	`Exclude-Extreme`	`Exclude-C-Traj-Extreme`
`Exclude-EWMA-Extreme-Pair`	`Exclude-Pair`	`Exclude-C-Pair` (subjects with two measurements; also possibly `Exclude-Extreme` / `Exclude-Hard-Limit` when a value is extreme enough)
`Exclude-Extraneous-Same-Day`	`Exclude-Extraneous` / `Exclude-Identical`	same
`Exclude-EWMA-8` through `Exclude-EWMA-14`	`Exclude-Pattern`	`Exclude-C-Traj`
`Exclude-Min-Height-Change`, `Exclude-Max-Height-Change`	`Exclude-Pattern`	`Exclude-C-Abs-Diff`
`Exclude-Pair-Delta-17/18/19`	`Exclude-Pair`	`Exclude-C-Pair`
`Exclude-Single-Outlier`	`Exclude-Single`	`Exclude-C-Single`
`Exclude-Too-Many-Errors`	`Exclude-Traj-Uneval`	`Exclude-C-Unevaluable-Trajectory`
`Exclude-Too-Many-Errors-Other-Parameter`	(removed – no counterpart)	(removed)

Exclude-SD-Cutoff excluded values whose recentered or unrecentered z-score was more extreme than ±25 (the sd.extreme and z.extreme defaults). It maps to Exclude-Hard-Limit; the updated hard-limit step uses different limits (see Child Step 7 in vignette("child-algorithm") and Section 1B of vignette("configuration")). Exclude-Too-Many-Errors-Other-Parameter has no successor: the updated algorithm no longer propagates a high error load from one parameter to the other.

Adult (Exclude-Adult-*) measurements:

Prior code	Summary (`exclude`)	Detailed (`exclude_detailed`)
`Exclude-Adult-BIV`	`Exclude-Hard-Limit`	`Exclude-A-Hard-Limit`
`Exclude-Adult-Identical-Same-Day`	`Exclude-Identical`	`Exclude-Identical`
`Exclude-Adult-Extraneous-Same-Day`	`Exclude-Extraneous`	`Exclude-Extraneous`
`Exclude-Adult-Weight-Cap`, `Exclude-Adult-Weight-Cap-Identical`	`Exclude-Scale-Max`	`Exclude-A-Scale-Max`
`Exclude-Adult-EWMA-Extreme` (+ `-RV`)	`Exclude-Extreme`	`Exclude-A-Traj-Extreme`
`Exclude-Adult-EWMA-Moderate`	`Exclude-Pattern`	`Exclude-A-Traj-Moderate`
`Exclude-Adult-Distinct-Pairs`, `Exclude-Adult-Distinct-Ordered-Pairs`	`Exclude-Pattern`	`Exclude-A-Ord-Pair` / `Exclude-A-2D-Ordered` / `Exclude-A-2D-Non-Ordered` (by parameter)
`Exclude-Adult-Distinct-3-Or-More`	`Exclude-Pattern`	`Exclude-A-Window`
`Exclude-Adult-Unit-Errors` (+ `-RV`)	No longer identified as unit errors; such values are now excluded as `Exclude-Hard-Limit`, `Exclude-Extreme`, or `Exclude-Pattern`, depending on how far out of range they are and the permissiveness level	`Exclude-A-Hard-Limit` / `Exclude-A-Traj-Extreme` / `Exclude-A-Traj-Moderate` / `Exclude-A-Window`
`Exclude-Adult-Distinct-Single`	`Exclude-Single`	`Exclude-A-Single`
`Exclude-Adult-Too-Many-Errors`	`Exclude-Traj-Uneval`	`Exclude-A-Unevaluable-Trajectory`

Exclude-Scale-Max (adult scale maximum) was previously Exclude-Adult-Weight-Cap and Exclude-Adult-Weight-Cap-Identical.

Adult codes with no direct successor. The updated algorithm removed the standalone digit-error, transposition, and swap detection steps, so these prior codes no longer occur: Exclude-Adult-Hundreds (+ -RV), Exclude-Adult-Transpositions (+ -RV), Exclude-Adult-Swapped-Measurements, and Exclude-Adult-Possibly-Impacted-By-Weight-Cap. A genuinely extreme value that one of these once flagged should still be excluded by the Evil Twins (Exclude-Extreme), hard-limit (Exclude-Hard-Limit), or trajectory (Exclude-Pattern) steps.

Child codes with no direct successor. Unit-error and swap detection were removed from the algorithm, so these prior codes no longer occur: Unit-Error-High, Unit-Error-Low, Unit-Error-Possible, Swapped-Measurements. (An informational unit_error_range output column flags values consistent with a unit error, but the algorithm no longer corrects or excludes on that basis.)

New codes

These have no prior-version equivalent: Exclude-Not-GC-Param, Exclude-HC-Out-of-Range (head circumference after 5 years), Exclude-Identical (same-day identical values are now split out from other extraneous values for children; adults already had Exclude-Adult-Identical-Same-Day), and the extreme-pair step that feeds Exclude-Extreme.

Removed and renamed arguments

These cleangrowth() arguments no longer exist. Passing one of these stops with a message naming what to use instead.

Removed argument	Notes
`recover.unit.error`	Unit-error recovery was removed from the algorithm; no option is needed.
`sd.extreme`	The extreme z-score cutoff was replaced by the hard-limit step (Child Step 7); its limits are set with the `piv.z.*` arguments.
`z.extreme`	The extreme z-score cutoff was replaced by the hard-limit step (Child Step 7); its limits are set with the `piv.z.*` arguments.
`lt3.exclude.mode`	Handling of subjects with few measurements is built in; no option is needed.
`height.tolerance.cm`	The height-decrease tolerance is built in; no option is needed.
`ewma.exp`	The EWMA exponent is fixed internally; no option is needed.
`sdmedian.filename`	File output was removed; use `recenter_detail = TRUE` to return the recentering values as columns.
`sdrecentered.filename`	File output was removed; use `recenter_detail = TRUE` to return the recentering values as columns.
`include.carryforward`	Use `cf_rescue` instead (`"none"` is closest to `include.carryforward = FALSE`, the 2.2.3 default; `"all"` is closest to `TRUE`; the default is now `"standard"`; see below).
`num.batches`	Use `cores` instead (with `parallel = TRUE`).
`weight_cap`	Use `adult_scale_max_lbs` instead (in pounds, not kg).
`adult_columns_filename`	File output was removed; the adult mean height is returned in the `mean_ht` column.
`prelim_infants`	Infant handling is now built in; no option is needed.

Other removals (most also listed in Updating code written for growthcleanr 2.2.3):

Removed	Notes
`sd.recenter = "nhanes"`, `sd.recenter = "derive"`	The built-in reference is now the default, so omit `sd.recenter` to use it; supply a table of your own medians, or set `sd.recenter = "none"` for no recentering. See `vignette("recentering")`.
`splitinput()`	Replaced by checkpointing; see `vignette("large-datasets")`.
`adjustcarryforward()`, `testacf()`, `acf_answers()`	Replaced by the `cf_rescue` argument; see `vignette("cf-rescue")`. Calling `adjustcarryforward()` stops with a message saying so.
`sd_median()`	Replaced by a new recentering method; see `vignette("recentering")`.
`log.path` (deprecated)	Still accepted, with a warning; use `log_file`, which writes a log whenever checkpointing engages (when `checkpoint_dir` is set, or for more than `checkpoint_size` subjects under the default).

Equivalents. include.carryforward = TRUE (skip all carry-forward detection) is approximately equivalent to the new cf_rescue = "all" (rescue all potential CFs), and the 2.2.3 default (include.carryforward = FALSE) is closest to cf_rescue = "none". cf_rescue has three modes: "standard" (default; each detected carry-forward string is rescued or excluded based on how far it deviates from the originating and next measured values – see vignette("cf-rescue")), "none" (all carry-forwards excluded), and "all". The old weight_cap (kg) is replaced by adult_scale_max_lbs (pounds).

New id column

growthcleanr sometimes has to choose between two measurements that are equally plausible, such as two different values recorded on the same day. Supplying an id column (numeric or character) that uniquely identifies each row makes that choice, and therefore the full set of results, identical every time you run growthcleanr on the same data.

Efficiency improvements

Checkpointing. For larger datasets, cleangrowth(checkpoint_dir = ...) splits subjects into groups to reduce memory use and to save resumable checkpoints – see Checkpoints above.

Other efficiency improvements:

Reference preloading. This is an option to load reference tables once and reuse them. This saves roughly a second per call, and can add up for extremely large datasets.

refs <- gc_preload_refs()
for (rep in 1:n_reps) {
  result <- cleangrowth(data = my_data, ref_tables = refs)
  # ...
}

Parallel processing. Parallel processing previously did not work in all settings. This has been addressed.

result <- cleangrowth(data = my_data, parallel = TRUE, cores = 4)

Note: parallel = TRUE requires growthcleanr to be installed (the parallel workers load the installed package). (Installation is the method typically used.) parallel = TRUE will not work if you are loading the package with devtools::load_all().

Other options and improvements:

gc_partial_run() re-cleans only the subjects whose data changed since an earlier run and reuses the earlier results for everyone else. See ?gc_partial_run.
batch_size controls how many subjects are processed per in-memory batch (default 5,000); lower it if you are memory-constrained.
Progress and a summary now print by default (2.2.3 was silent); set quietly = TRUE to turn them off.
Use of matrices when possible.
“Windowing” for EWMA - by default, exponentially weighted moving averages only include the 15 measurements on either side of the value (set with ewma_window). This reduces computation time significantly without affecting performance except in extreme edge cases.
Many others.

Debugging

Many bug fixes were made.

Warnings and error messages

Extensive efforts were made to improve warnings and error messages to facilitate troubleshooting. quietly = TRUE turns off progress lines and the cleaning summary, but warnings still appear; to suppress warnings too, use R’s suppressWarnings().

Documentation

New how-to guides for the Shiny app and R user interface.
New fact sheet and researcher overview.
Additional user support documentation, such as for the command-line driver.
Expanded technical documentation; see browseVignettes("growthcleanr").
A new vignette describes how AI assistance was used in growthcleanr development (vignette("ai-role")).
More guidance is in progress; see “Documentation in progress” in vignette("new-start-here").

Other changes

Missing agedays in a row no longer causes a hard stop; affected rows are assigned Exclude-Missing-Info.
Head circumference (HEADCM) is supported as a parameter. HC measurements after 5 years of age are assigned Exclude-HC-Out-of-Range.
Recumbent length. A new length.adjust option (default FALSE) reconciles entered length and height with what the reference at that age expects, in both directions: recumbent-labeled measurements at 2 years or older are z-scored 0.7 cm lower, and height-labeled measurements under 2 years 0.7 cm higher (the measurement value itself is not changed). By default, recumbent-labeled values are relabeled to height without any measurement adjustment; the exception is gestational-age-corrected children with chronological age 2 years or older but corrected age under 2, who are presumed to be measured recumbent. Full rules: vignette("wrapper"), section “Length vs. height handling”.
cleangrowth() sex warning. A prominent warning is given if the data include children and more than one subject and every value in sex is 1 under the default coding.
longwide() sex checks. As in cleangrowth(), longwide() stops if sex has only 1s and 2s, and gives a prominent warning if every value is 1 across more than one subject. Passing the sex_coding you gave cleangrowth() removes both.
ext_bmiz() warnings. ext_bmiz() warns about missing or unrecognized sex. Logical sex is no longer read as a sex code (FALSE was read as female). It also warns about rows at about 729-730 days, which get NA because they fall just under 24 months, and if data already has one of its output columns, the returned data has the new values in that column (with a warning).
recode_sex() fix. With targetcol equal to sourcecol, it no longer recodes twice (0/1 to 1/2 in place used to make every row 2).
Docker support was dropped. The Dockerfile, .dockerignore, and image-publishing GitHub Actions workflows were removed.

Validation and review

Clinicians and experts have validated, informed, and reviewed growthcleanr at multiple stages.

The original child algorithm was validated by two pediatricians as described in the JAMIA 2017 paper (doi:10.1093/jamia/ocx037).
The adult algorithm was validated by three physicians. Results will be published in an upcoming manuscript. Note that the algorithm was specifically validated in adults 18 to 65 years of age. Overall, the sensitivity and specificity were good, but two problems were identified. There was a bug in handling of patients where the penultimate and last values both met criteria for potential exclusion. Also, the clinicians had a larger tolerance for height variability than the algorithm did. These issues were addressed in version 3.
For the algorithm updates in version 3, we obtained clinician input into thresholds and asked clinicians to review selected difficult cases. We initially planned to do formal validation. However, version 3 changes largely address edge or unusual cases, and during the planning phases for validation, agreement among clinicians for these cases was often poor. We have addressed this in several ways. Selected cases were reviewed by pediatricians and internal medicine physicians to confirm that growthcleanr agreed with at least some clinicians. We also designed tests based on literature, for example to evaluate cleaning for bariatric surgery patients, and added extensive automated testing to ensure (to the best of our ability) that updates to address one issue didn’t cause regression in performance for other issues. Finally, we added more customization options to the adult algorithm with different tolerance for different atypical patterns, specifically four levels of permissiveness. Further details will be provided in an upcoming manuscript.

What’s next?

We are working on more customization options for the child algorithm. We will continue to expand functionality and usability as resources allow.

Contact

If you have bug reports, ideas about child customization, or other new features of growthcleanr, please contact us through GitHub or email cdaymont@pennstatehealth.psu.edu.

Hint: Requests for features may help us obtain funding for implementing them.

More details of installing a prior version

To continue using a previous release:

# From CRAN:
remotes::install_version("growthcleanr", version = "2.2.3")

# Or from GitHub at a tagged release:
remotes::install_github("growth-data-tools/growthcleanr@v2.2.3")

Only one version can be installed per R library at a time. Use renv for project-specific libraries if you need more than one version available.

# Check your installed version:
packageVersion("growthcleanr")

If packageVersion() does not show the version you expect after switching versions, follow the detach/uninstall/restart/reinstall steps in the install note near the top.

2026-11-08