# neurostatx.utils.preprocessing

*module*

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L1-L1)

## neurostatx.utils.preprocessing.binary_to_yes_no

*function*

```python
def binary_to_yes_no(df, cols)
```

Recode 1/0 columns to ``Yes`` / ``No`` in place.

Values of 1 become ``Yes``, 0 become ``No``, and other values become
``Don't know or missing value``.

**Parameters**

- `df` (`pd.DataFrame`) — Input table. Matching columns are modified in place.
- `cols` (`list of str`) — Column names to recode.

**Returns**

- `df` (`pd.DataFrame`) — The same table after recoding.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import binary_to_yes_no
>>> df = pd.DataFrame({"flag": [1.0, 0.0]})
>>> binary_to_yes_no(df, ["flag"])["flag"].tolist()
['Yes', 'No']
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L84-L119)

## neurostatx.utils.preprocessing.compute_correlation_coefficient

*function*

```python
def compute_correlation_coefficient(df, out_folder, context='poster', font_scale=0.2, cmap=None, annot=False)
```

Compute a Pearson correlation matrix and write a heatmap.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.
- `out_folder` (`str`) — Directory where ``correlation_heatmap.png`` is written.
- `context` (`str`) (default: `'poster'`) — Seaborn context. Defaults to ``"poster"``.
- `font_scale` (`float`) (default: `0.2`) — Seaborn font scale. Defaults to 0.2.
- `cmap` (`str`) (default: `None`) — Heatmap colormap. Defaults to None.
- `annot` (`bool`) (default: `False`) — If True, write correlation values on the heatmap. Defaults to False.

**Returns**

- `corr_mat` (`pd.DataFrame`) — Pearson correlation matrix.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import (
...     compute_correlation_coefficient)
>>> df = pd.DataFrame({"a": [1, 2, 3], "b": [2, 4, 6]})
>>> corr = compute_correlation_coefficient(df, ".")
>>> corr.loc["a", "b"]
1.0
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L229-L283)

## neurostatx.utils.preprocessing.compute_pca

*function*

```python
def compute_pca(X, n_components)
```

Fit a PCA and return scores, diagnostics, and the model.

**Parameters**

- `X` (`pd.DataFrame`) — Input table.
- `n_components` (`int`) — Number of components to keep.

**Returns**

- `scores` (`array`) — Transformed data.
- `pca` (`PCA`) — Fitted PCA model.
- `exp_var` (`array`) — Explained variance ratio.
- `components` (`array`) — Principal axes.
- `p_value` (`float`) — Bartlett sphericity p-value.
- `kmo_model` (`float`) — Overall KMO statistic.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import compute_pca
>>> X = pd.DataFrame({"a": [1.0, 2.0, 3.0, 4.0],
...                   "b": [2.0, 1.0, 4.0, 3.0]})
>>> scores, pca, exp_var, components, p_value, kmo = compute_pca(X, 1)
>>> scores.shape
(4, 1)
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L330-L373)

## neurostatx.utils.preprocessing.compute_shapiro_wilk_test

*function*

```python
def compute_shapiro_wilk_test(df)
```

Run a Shapiro-Wilk normality test on each column.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.

**Returns**

- `wilk` (`list of float`) — Shapiro-Wilk W statistics.
- `pvalues` (`list of float`) — Corresponding p-values.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import compute_shapiro_wilk_test
>>> df = pd.DataFrame({"a": [1.0, 2.0, 3.0, 4.0]})
>>> wilk, pvalues = compute_shapiro_wilk_test(df)
>>> len(wilk)
1
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L192-L226)

## neurostatx.utils.preprocessing.get_column_indices

*function*

```python
def get_column_indices(df, column_names)
```

Return integer positions for the requested column names.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.
- `column_names` (`list of str`) — Column names to look up.

**Returns**

- `indices` (`list of int`) — Column positions. Missing names are skipped.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import get_column_indices
>>> df = pd.DataFrame({"a": [1], "b": [2]})
>>> get_column_indices(df, ["b", "a"])
[1, 0]
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L122-L152)

## neurostatx.utils.preprocessing.merge_dataframes

*function*

```python
def merge_dataframes(dict_df, index, repeated_columns=False)
```

Join several tables on a shared index column.

Index values must be unique within each table.

**Parameters**

- `dict_df` (`dict`) — Mapping of labels to DataFrames.
- `index` (`str`) — Shared column used as the join index.
- `repeated_columns` (`bool`) (default: `False`) — If True, disambiguate overlapping column names with suffixes. Defaults to False.

**Returns**

- `out` (`pd.DataFrame`) — Joined table.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import merge_dataframes
>>> left = pd.DataFrame({"id": [1, 2], "a": [10, 20]})
>>> right = pd.DataFrame({"id": [1, 2], "b": [3, 4]})
>>> merge_dataframes({"l": left, "r": right}, "id")["a"].tolist()
[10, 20]
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L286-L327)

## neurostatx.utils.preprocessing.plot_distributions

*function*

```python
def plot_distributions(df, out_folder, context='poster', font_scale=1)
```

Write a histogram and ECDF for each column in ``df``.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.
- `out_folder` (`str`) — Directory where ``{column}.png`` files are written.
- `context` (`str`) (default: `'poster'`) — Seaborn context. Defaults to ``"poster"``.
- `font_scale` (`float`) (default: `1`) — Seaborn font scale. Defaults to 1.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import plot_distributions
>>> df = pd.DataFrame({"a": [1, 2, 3]})
>>> plot_distributions(df, ".")
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L155-L189)

## neurostatx.utils.preprocessing.remove_nans

*function*

```python
def remove_nans(df)
```

Split a table into rows with missing values and complete rows.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.

**Returns**

- `rows_with_nans` (`pd.DataFrame`) — Rows that contain at least one NaN.
- `complete_rows` (`pd.DataFrame`) — Rows with no missing values.

**Examples**

```pycon
>>> import pandas as pd
>>> import numpy as np
>>> from neurostatx.utils.preprocessing import remove_nans
>>> df = pd.DataFrame({"a": [1.0, np.nan], "b": [2.0, 3.0]})
>>> nans, complete = remove_nans(df)
>>> len(complete)
1
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L10-L38)

## neurostatx.utils.preprocessing.rename_columns

*function*

```python
def rename_columns(df, old_names, new_names)
```

Rename selected columns and return a copy of the table.

**Parameters**

- `df` (`pd.DataFrame`) — Input table.
- `old_names` (`list of str`) — Current column names.
- `new_names` (`list of str`) — Replacement column names, same length as ``old_names``.

**Returns**

- `new_df` (`pd.DataFrame`) — Copy of ``df`` with renamed columns.

**Examples**

```pycon
>>> import pandas as pd
>>> from neurostatx.utils.preprocessing import rename_columns
>>> df = pd.DataFrame({"a": [1], "b": [2]})
>>> rename_columns(df, ["a"], ["x"]).columns.tolist()
['x', 'b']
```

[View source](https://github.com/gagnonanthony/NeuroStatX/blob/main/neurostatx/utils/preprocessing.py#L41-L81)