> ## Documentation Index > Fetch the complete documentation index at: https://docs.summand.com/llms.txt > Use this file to discover all available pages before exploring further. # Missing-value patterns > Identifies patterns in the missing data: determines which columns go missing together and whether values are missing at random or not. This component is currently in beta and may be subject to changes or instability. The **Missing-Value Patterns** component answers the question: *are missing values random or do they follow a pattern?* ## Use cases * **Retail data quality** — Are product weight and dimensions always missing together, or are these independent supplier data problems? * **Selective disclosure** — Are customers who skip the household income question also refusing to answer questions about their spending habits? * **Sensor fault attribution** — Are multiple conveyor-belt sensors dropping out together, suggesting a shared infrastructure failure? ## What it computes * $p$-value for Little's MCAR test: values below 5% suggest that data is **not** missing at random. * Missing-value pattern frequencies: which combinations of columns go missing together most often? * Conditional missingness matrix: when one column goes missing, which other column is most often missing? * Missingness correlation matrix: are some column pairs always missing together, or never missing at the same time? This is done by turning each column into a boolean indicator via the question "is this row missing in this column?" and then computing Pearson's correlation coefficient between the resulting boolean columns. A value of `+1` indicates that the two columns always go missing together; a value of `-1` indicates that for every row, exactly one of the two columns is missing: they are never both missing at the same time, and never both observed. ## Display The missing-value patterns component displays several pieces of information in the Catalog. * Result of Little's MCAR test: the $p$-value, the sample size, and which numeric columns were used. Only numeric columns with non-zero variance are included; constant columns and entirely-missing columns are excluded. * Missing-value pattern frequencies: how often does each missing-value pattern occur, as a percentage of all rows? (For datasets with many distinct missing-value patterns, only the 100 most common patterns are provided, as well as any patterns where only a single column is missing -- even if it is not among the 100 most common.) * Conditional missingness matrix: when the *reference* column is missing, how often is the *conditionally missing* columns also missing? * Missingness correlation matrix: how often two columns go missing together and how often only one of them goes missing. ## Inputs None. The component runs on the full dataset with no configuration required. ## Output shape ```json theme={null} { "little_mcar_test": { "p_value": float, "sample_size": int, "n_columns_used": int, "columns_used": list[str], # numeric columns with non-zero variance used in the test; constant columns and entirely-missing columns are excluded }, "missing_value_pattern_frequencies": [ # Sorted by descending frequency_as_percentage; ties broken lexicographically by missing_columns. { "frequency_as_percentage": float, "missing_columns": str, # column names joined by " & " in lexicographic order; # empty string for complete rows }, # ... ], "missing_value_pattern_frequencies_top_10": [...], # same schema, first 10 entries "conditional_missingness_matrix": [ # Not sorted. Use conditional_missingness_matrix_top_10 for the highest-rate entries. { "directional_column_pair": str, # "col_a -> col_b" "reference_column": str, "conditionally_missing_column": str, "conditional_missing_rate": float, # P(col_b missing | col_a missing) "n_reference_rows_used": int, # rows where col_a is missing }, # ... ], "conditional_missingness_matrix_top_10": [...], # same schema; sorted by descending conditional_missing_rate, ties broken lexicographically by (reference_column, conditionally_missing_column) "missingness_correlation_matrix": [ # Not sorted. Each unordered pair stored twice (both orderings) so that filtering on # column_1 alone finds every correlation involving that column. # Use missingness_correlation_matrix_top_10 for the entries with the largest absolute value # of the coefficient. { "column_pair": str, # "col_1 & col_2" "column_1": str, "column_2": str, "missingness_correlation_coefficient": float, }, # ... ], "missingness_correlation_matrix_top_10": [ # Unique pairs only (stored once), sorted by descending absolute value of the coefficient; ties broken lexicographically by (column_1, column_2). { "column_pair": str, "column_1": str, "column_2": str, "missingness_correlation_coefficient": float, }, # ... ], } ``` When Little's MCAR test cannot be run, `little_mcar_test` contains a single `reason_not_computed` field instead of the fields above: ```json theme={null} { "little_mcar_test": { "reason_not_computed": str # Possible values: # "[NoValidNumericColumns] ..." — no numeric columns with non-zero variance # "[NoMissingValuesInValidNumericColumns] ..." — included columns are fully observed # "[CollinearityPresent] ..." — covariance matrix is singular # "[NegativeDegreesOfFreedom] ..." — internal error; should not occur in practice }, ... } ``` ## Filtering from chat Summand can dig into the missing-value patterns in a variety of ways: ```text theme={null} # How often a specific missing-value pattern appears (columns listed alphabetically) analyze({ component: "missing_value_patterns", params: { missing_value_pattern: "dimensions & weight" } }) # How often the second column is missing given that the first column is missing analyze({ component: "missing_value_patterns", params: { conditional_missingness_directed_pair: "household_income -> spending_habits" } }) # How often all other columns go missing given that this reference column is missing analyze({ component: "missing_value_patterns", params: { conditional_missingness_reference_column: "household_income" } }) # How often this column goes missing given that one of the other columns is missing analyze({ component: "missing_value_patterns", params: { conditionally_missing_column: "spending_habits" } }) # The correlation between the boolean missingness indicators of two specific columns analyze({ component: "missing_value_patterns", params: { missingness_correlation_column_pair: "dimensions & weight" } }) # The correlation between the boolean missingness indicators of one column and all other columns analyze({ component: "missing_value_patterns", params: { missingness_correlation_column: "weight" } }) ``` ## Compute profile | Profile | Profile | | ------- | ----------- | | Fargate | 2 CPU, 8 GB |