NetREm: regression that knows which predictors work together

Network Regression Embeddings (Khullar et al., Bioinformatics Advances, 2024) take continuous data for candidate predictors and a response, plus a prior network of positive weights saying which predictors are related. Two knobs, α and β, set how sparse the answer is and how much to trust the network. NetREm returns the predictors that best explain the response and a coordination network showing how those predictors work together or against each other.

1 · Pick an example36 real-world settings in the Controls panel, or load your own data in section 6.

2 · Look at the inputsThe data X and y, and the prior network W with where it comes from.

3 · Move α and βWatch which predictors are kept, their signs, and how they coordinate.

4 · Compare and checkAgainst Lasso, ElasticNet, Ridge and Linear, and over 20 fresh datasets.

Run it on my own data

Inputs

NetREm needs continuous data for the response y and N candidate predictors X, and a weighted prior network W among the predictors (weights between 0 and 1). An optional shortlist from other knowledge can narrow the candidates first.

What exactly are X and y?

Prior data: X and y

Each predictor against y

Standardised values. r is the Pearson correlation: what a "one predictor at a time" view sees.

Prior network: W

Known links with positive weights. Click two predictors to add a link; click a link to change its weight or remove it.

Click a predictor to start a new link.

known link (width = weight)unknown pair, gets η

The same network as a data frame

This edge list is what you give NetREm: one row per known link, naming two predictors exactly as they appear as columns of X, with a weight in (0, 1]. The evidence column says why each link exists in this example. It updates as you edit the graph.

Where this network comes from

General guide: how to build a prior network

Optional: candidate shortlist

How to build a prior network

The prior network W should encode real knowledge about how the predictors relate: known interactions, physical connections, or relationships derived from other data. It is a hypothesis you bring to the analysis, so it should be built before, and independently of, the y you are trying to explain.

What it looks like

An undirected, weighted graph. Nodes are the predictors (the columns of X, with identical names). An edge says two predictors are related; its weight in (0, 1] says how confident or how strong. No self-loops, and each pair appears once.

node_1   node_2   weight
SOX10    EGR2     0.90
SOX10    POU3F1   0.60
EGR2     POU3F1   0.80
JUN      SOX2     0.90

Pairs you do not list are not treated as unrelated: NetREm gives them a small weight η (default 0.01) so the network is fully connected and new links can still be discovered.

Five common sources

Curated interaction databases. Proteins (STRING, BioGRID), pathways (KEGG, Reactome), metabolic reactions (Recon3D), species interactions (GloBI).
Physical or structural connections. Brain wiring from diffusion MRI, river and road networks, power-grid topology, sensor layouts, software call graphs.
Similarity in an independent resource. Phylogenetic distance between microbes, word embeddings trained on other text, neighbouring wavelengths, spatial distance between stations.
Flows and transactions. Commuter flows, supplier–customer links, input–output tables, passing networks, audience overlap.
Derived from other data or experts. Co-expression or co-occurrence estimated in a different cohort, curricula and prerequisite maps, engineers’ schematics, clinical ontologies.

Turning evidence into weights

Scores: divide by the maximum, w = s / max(s). The paper scales STRING combined scores this way.
Distances: w = exp(−d / τ), where τ sets how fast relatedness fades (5 km for air pollution, 30 km for weather, a few tree units for phylogeny).
Counts and flows: w = log(1 + n) / max log(1 + n), so a few huge flows do not swamp the rest.
Categories: fixed levels, such as 0.9 for a direct link, 0.5 for a two-step link, 0.3 for the same broad class.
Unweighted networks: set every known link to 1 (the paper does this for unweighted interaction databases).
Thresholds: drop very weak links (the paper keeps 0.01 < w ≤ 1).

Rules of thumb

Independent of y. Never build W from how predictors correlate with y in your training data; that leaks the answer into the prior. Correlations among predictors in a different dataset are fine.
Plausible and documented. Every link should have a reason you could cite: an experiment, a map, a schematic, a reference.
Coverage. Missing links are normal; η handles them. Predictors missing from the database still take part.
Check sensitivity. Refit with β near 0 and with a different source of W. Conclusions that survive both are the ones to trust.

# distances (km) between monitoring sites -> edge list
import numpy as np, pandas as pd
w = np.exp(-dist_km / 5.0)
edges = (pd.DataFrame(w, index=sites, columns=sites)
           .where(np.triu(np.ones(w.shape, bool), k=1))
           .stack().reset_index())
edges.columns = ["node_1", "node_2", "weight"]
edges = edges[edges.weight > 0.01]

NetREm’s solution

The coefficients c* say which predictors NetREm keeps and in which direction they push y. Zero means dropped. Because X and y are standardised, sizes are comparable across predictors.

Coefficients c*

Bars: NetREm’s c*. Hollow ring: each predictor’s own correlation r with y, for contrast.

Output 1: predictor → response network

Arrows from kept predictors to y. Width shows |c*|.

Reading + and −

Every number on this page has a sign. Here is what each sign means in this example, using the current results.

How c* responds to α and β

Each coloured line follows one predictor’s coefficient as you sweep one knob and hold the other at its current value. The gold line marks where you are now. Click a name in the legend to highlight that predictor; click again to clear.

Sweeping β, the network prior

Linked predictors are pulled towards similar (degree-scaled) coefficients as β grows. Very large β over-constrains the model.

Sweeping α, the sparsity penalty

As α grows, coefficients shrink and drop to exactly zero one by one. The order they leave in is a ranking of importance.

β acts first, α second

β is applied first: it changes the matrix E = XᵀX/M + βA and so reshapes the embeddings X̃ and ỹ. α then runs an ordinary Lasso on those embeddings. That means the same α does different work at different β: a predictor that Lasso drops at α = 0.05 may survive at that α once a strong link has pulled it towards a partner, and the reverse. With "Choose α by cross-validation" switched on, you can watch the chosen α itself shift as you move β. The table shows, for each predictor, the largest α at which it is still kept (its "entry point"), with the network off and at your current β.

Legend: predictors

Things to try

Compared with other models

The same standardised training data, fitted without intercepts, as in the paper’s benchmarks. Test error is mean squared error on held-out samples (standardised, so 1.0 means no better than predicting the mean). The tuning of the comparison models is set in Controls.

Why it matters: is the answer reproducible?

How often each predictor was kept

LinearElasticNetLassoNetREmtrue driver

Output 2: the coordination network

For every pair of kept predictors, NetREm gives a score B from −100 to 100. Positive: they push y the same way (cooperative). Negative: they push it opposite ways (antagonistic). The size reflects how close the two predictors sit in the embedding space, which blends their data correlation with the prior network.

Coordination among kept predictors

Same layout as the prior network, so you can compare. Solid: the pair was a known link in W. Dashed: a novel link NetREm proposes.

cooperative (B > 0)antagonistic (B < 0)novel (not in prior)

Ranked links

Similar to the package’s organize_predictor_interaction_network() output.

Why can a strongly linked pair get a small |B|? B uses the cosine between embeddings, which tracks E_ij = r_ij − βw_ij/√(d_id_j). When a positive correlation and the prior term roughly cancel, the embeddings are nearly orthogonal. Move β and watch the scores change; step 7 of the maths shows each piece.

The maths, step by step

Every number below is computed live from the current example and settings. Notation follows the paper: M samples, N candidate predictors.

Step 1 · The objective

NetREm looks for coefficients c that balance three goals:

c* = argmin_c ¹⁄_2M‖y − Xc‖² + α‖c‖₁ + ^β⁄₂ Σ_i<j w_ij ( c_i⁄√d_i − c_j⁄√d_j )² Term 1, data fit: predictions Xc should be close to y. Term 2, sparsity: the L1 penalty pushes unhelpful coefficients to exactly zero (this is Lasso). Term 3, network prior: linked predictors pay a price, proportional to their link weight w_ij, for having different coefficients. Dividing by √d (a predictor’s total link weight, its degree) stops highly connected hubs from dominating.

Set β = 0 and NetREm is exactly Lasso. Replace the network term with β‖c‖²⁄2 and you get ElasticNet. NetREm is the network-aware version of ElasticNet: instead of shrinking all coefficients towards zero, it shrinks linked coefficients towards each other.

Preprocessing · Make the network fully connected

Every pair of candidates without a known link gets a small artificial weight η (default 0.01). This keeps the maths stable, lets predictors missing from the prior still take part, and lets NetREm propose novel links later. Each predictor’s degree is d_i = Σ_k≠i w_ik.

Both X and y are standardised (mean 0, standard deviation 1) using the training data, so X^⊤X/M is the correlation matrix of the predictors.

W: known links and η-filled pairs.

Rewrite the network term as a matrix

The network penalty can be written compactly as ^β⁄₂c^⊤Ac, where A is a normalised graph Laplacian:

A_ii = 1, A_ij = − w_ij ⁄ √(d_i d_j) for i ≠ j

Strong links give strongly negative off-diagonal entries. A is symmetric and positive semi-definite, so the whole objective stays convex: there is one best answer and standard solvers find it.

A: the penalty matrix built from W.

Step 2 · Combine data and network into one matrix E

Expanding ‖y − Xc‖² shows the data term and the network term are both quadratic in c, so they merge into one matrix:

E = X^⊤X ⁄ M + βA, so E_ij = r(x_i, x_j) − β w_ij⁄√(d_id_j) Each entry blends what the data say (correlation between predictors) with what the prior says (link strength), weighted by β.

X^⊤X/M: correlations in the data

βA: the network prior, scaled

E: their sum

Build the network regression embeddings by SVD

Factorise E = UΣU^⊤ (singular value decomposition; tiny singular values below 10⁻⁶ × the largest are set to zero for stability). Then define a new, small dataset with N rows:

X̃ = √N · Σ^½U^⊤, ỹ = (√N ⁄ M) · Σ^−½U^⊤X^⊤y

Column i of X̃ is predictor i’s embedding: a vector that carries both its correlations in the data and its position in the network. These embeddings are chosen so that X̃^⊤X̃/N = E and X̃^⊤ỹ/N = X^⊤y/M.

When β = 0 the embeddings are just the principal components of X; as β grows, the network reshapes them.

X̃ (rows: latent dimensions) and ỹ (last column).

Solve an ordinary Lasso on the embeddings

Because of those two identities, the original three-term objective equals, up to a constant,

c* = argmin_c ¹⁄_2N‖ỹ − X̃c‖² + α‖c‖₁

That is a standard Lasso problem, so NetREm uses an off-the-shelf solver (Lasso with a fixed α, or LassoCV to pick α). This is the key trick: the network prior is baked into the data, so no custom optimiser is needed.

Output 1 is then the set of predictors with c* ≠ 0: positive pushes y up, negative pushes it down. The table shows the solution for the current example: the Lasso on embeddings, the same objective solved directly as a check, and an ordinary Lasso on the raw X at the same α for contrast.

Output 2 · Coordination scores B

For each pair of kept predictors, take the sign of c_i*c_j* (same direction or opposite) and the size of the cosine similarity between their embeddings, then rescale so the strongest link is ±100:

B⁰_ij = sign(c_i* c_j*) · | cos( X̃_i, X̃_j ) |, B_ij = 100 · B⁰_ij ⁄ max |B⁰| Pairs where either predictor was dropped get 0. Note cos(X̃_i, X̃_j) = E_ij ⁄ √(E_iiE_jj), so B depends on data correlation, link strength and β together.

sign(c_i*c_j*)

|cos| of embeddings

B, scaled to ±100

Run it on your own data

Paste or upload a CSV with one row per sample and one numeric column per variable (with a header row). Choose which column is y; every other numeric column becomes a candidate predictor. Optionally add an edge list for the prior network. Everything runs in your browser; nothing is uploaded anywhere.

Data: X and y

Up to 40 predictor columns and 20,000 rows. Rows with missing or non-numeric values are dropped. 70% of rows are used for training and 30% for testing (re-shuffled by "Draw new data").

Prior network: edge list (optional)

One link per line: node_1, node_2, weight (weight in (0, 1]). Names must match your column headers exactly. Leave empty to give every pair η, which is close to plain Lasso.

Beyond biology

NetREm is not a biology-only tool. It applies wherever you have a continuous outcome, many correlated candidate predictors measured on the same samples, and independent knowledge of how those predictors relate. Every card below is a working example in the lab, with a tailored prior network and the resources you would use to build it.

Bring your own problem: four questions

What is one row? A cell, a person, a week, a match, a molecule. Every column must be measured on the same rows.
What is y? One continuous outcome you want to explain: an expression level, a score, a return, a yield.
What are the candidate predictors in X? Continuous measurements that might drive y. It is fine, even expected, that many are correlated.
Where does W come from? Knowledge built independently of y: interaction databases, physical wiring, supply chains, maps, prerequisite graphs, earlier studies. Weights in (0, 1] express confidence in each link.

You then get two outputs that ordinary regressions do not give you together: a sparse, network-aware set of drivers with directions (c*), and a signed map of which drivers cooperate or compete in explaining y (B). Run it for many outcomes (every gene, every hospital, every product line) and you can stitch the results into a system-wide picture, as the paper does across all genes in a cell type.

A good fit when

predictors are correlated, so Lasso picks among them almost arbitrarily;
you have a credible relational prior, built independently of y;
you care about which groups of predictors act together, not only prediction error;
samples are limited, so the prior helps stabilise the answer.

Be careful about

a wrong prior: it pulls unrelated predictors together. Tune β and compare with β ≈ 0;
too large a β: the paper notes excess β over-constrains the model and test error rises;
building W from the same data as y: that leaks information;
reading B as proof of mechanism: it is a linear, associative score.

Using the package

The Python package is at github.com/SaniyaKhullar/NetREm. The sketch below shows the shape of a run; check the README for the exact call signature in your version.

Sketch

# Prior network: [predictor_1, predictor_2, weight in (0, 1]]
edge_list = [["TF1", "TF2", 0.8],
             ["TF4", "TF5", 0.95]]

model = netrem(edge_list=edge_list,
               beta=1, alpha_lasso=0.1,   # or model_type="LassoCV"
               y_intercept=False)
model.fit(X_train, y_train)               # samples x predictors, response

model.model_coef_df          # Output 1: c* for every predictor
model.model_nonzero_coef_df  # only the kept predictors
model.B_interaction_df       # Output 2: coordination scores B
organize_predictor_interaction_network(model)  # ranked TF-TF links

Reference

Saniya Khullar, Xiang Huang, Raghu Ramesh, John Svaren, Daifeng Wang. NetREm: Network Regression Embeddings reveal cell-type transcription factor coordination for gene regulation. Bioinformatics Advances 5(1), vbae206 (2024). doi:10.1093/bioadv/vbae206

The maths on this page follows Methods §2.1 of the paper. Running the "Paper example" with β = 1 and α = 0.1 reproduces the paper’s Fig. 2 closely (for instance TF3 c* ≈ 0.03, TF4 ≈ −0.09, TF1–TF5 B ≈ −97). The paper’s simulated data also include single-cell-style dropouts, which this page leaves out.

All data on this page are simulated in your browser. Outside the paper example, the predictor names and effects are illustrative, not findings.