---
title: "Noise2Noise researchers test L1 vs L2 in self-supervised denoising on Kodak24 and SIDD; Retraining on SIDD pairs yields 9.4–11.0 dB gains"
sdDatePublished: "2026-09-16T04:09:00Z"
source: "https://arxiv.org/pdf/2609.16788"
topics:
  - name: "scientific research"
    identifier: "medtop:20000735"
  - name: "scientific innovation"
    identifier: "medtop:20000736"
  - name: "artificial intelligence"
    identifier: "medtop:20001298"
  - name: "medical research"
    identifier: "medtop:20000737"
locations:
  - "Richmond"
  - "Redwood City"
  - "Mountain View"
  - "Breckenridge"
  - "Queens"
---


Noise2Noise researchers test L1 vs L2 in self-supervised denoising on Kodak24 and SIDD; Retraining on SIDD pairs yields 9.4–11.0 dB gains

Noise2Noise Revisited: Training Pair Distributions Dominate Loss Choice in Self-Supervised Denoising

Noise2Noise Revisited: Training Pair Distributions
Dominate Loss Choice in Self-Supervised Denoising
Dingyan Shang*
Independent Researcher
Frisco, USA
dingyanshang@gmail.com
*Corresponding author
Zhenyu Xu
Independent Researcher
Fulshear, USA
zhenyuxu0918@gmail.com
Youting Wang
Independent Researcher
Mountain View, USA
wang.yout@northeastern.edu
Bonan Shen
Independent Researcher
Long Island City, USA
shenbonan2@gmail.com
Bowen Liu
Independent Researcher
South San Francisco, USA
bliu0962@usc.edu
Abstract—Noise2Noise (N2N) trains denoisers on pairs of inde-
pendently corrupted observations, eliminating clean references.
We stress-test two natural conjectures about why the L1 loss
outperforms L2 here. First, the hypothesis that the L1 loss confers
robustness via parameter sparsity confuses the loss with Lasso
regularization: an explicit Lasso penalty produces the predicted
sparsity yet fails to reproduce L1’s cross-noise behavior, while
L1- and L2-trained weight distributions are indistinguishable.
Second, the population optima of the two losses coincide exactly
for symmetric signal posteriors and nearly so for concentrated
ones. Measured differences are therefore dominated by optimiza-
tion dynamics (bounded-influence gradients), which we probe
with gradient statistics and contaminated-target training. On
Kodak24 with five synthetic noise families, the L1 loss holds a
statistically significant edge over L2, below 1 dB PSNR, holding
across three seeds on 13 of the 14 noise columns. On real camera
noise the loss is not the decisive variable in distribution: on
official SIDD validation blocks, synthetic-Gaussian-trained N2N
models gain only 0.8 to 3.7 dB over the noisy input regardless of
loss, while retraining on SIDD’s own noisy pairs, never reading
ground truth, gains 9.4 to 11.0 dB, far ahead of BM3D. All
metrics are on raw network outputs, and the study makes no
leaderboard claim. The training pair distribution, not the loss,
carries the inductive bias. That design rule applies wherever
clean references are unobtainable, from microscopy to industrial
inspection sensors.
Index
Terms—image
denoising,
self-supervised
learning,
Noise2Noise, robust statistics, real-noise benchmarks, industrial
inspection, nondestructive evaluation
I. INTRODUCTION
Supervised deep denoisers [1], [2] require registered
clean/noisy pairs, costly or impossible in microscopy, in low-
light photography, and in industrial nondestructive testing,
where each unit under inspection is unique and a noise-free
reference of it is physically unobtainable [3]. Noise2Noise
(N2N) [4] showed that two independent noisy observations of
the same signal suffice: under zero-mean noise, the network
trained to map one observation to the other converges in
expectation to the clean-target minimizer.
Two practical questions remain. (i) How much of N2N’s
observed cross-noise robustness is attributable to the loss
function? (ii) The empirical edge of the L1 loss over L2 in
restoration networks is documented [5] and attributed there
to optimization behavior; is that attribution right, or does the
credit belong to parameter sparsity? We answer both with a
controlled protocol and make three contributions:
1) A corrected theoretical framing. We distinguish the
L1 loss (a residual penalty; maximum likelihood under
a Laplace residual model) from L1 regularization (a
weight penalty; Lasso [6]), and we delimit what the loss
can change: under zero-mean training noise the MSE
optimum is exactly the posterior mean, and the MAE
optimum coincides with it when the signal posterior
is symmetric, and approximately when it is concen-
trated. The measurable loss-function effect reduces to
finite-sample optimization dynamics (MAE’s bounded-
influence gradients [5], [7]), supported by gradient statis-
tics and contaminated-target training.
2) A control experiment dissociating the L1 loss from
L1 regularization. We train the same U-Net with MSE,
MAE, and MSE+Lasso under identical schedules. Only
the explicit Lasso penalty produces weight sparsity, and
it does not inherit the MAE cross-noise profile, ruling
out sparsity as the carrier of L1’s edge.
3) A loss-controlled quantification and closure of the
real-noise domain gap. On the official SIDD valida-
tion blocks [8], synthetic-Gaussian-trained N2N recovers
only +0.8 to +3.7 dB over the noisy input regardless of
loss. Retraining the same five losses on SIDD-Medium
noisy/noisy pairs (scene-disjoint 128/32 split, ground
truth never read during training) lifts every variant to
+9.4 to +11.0 dB, well ahead of BM3D [9]. The gap
itself is established knowledge [10], [11]; new here is
its same-architecture, same-budget measurement across
five losses within the N2N paradigm, its closure with
arXiv:2609.16788v1 [cs.LG] 15 Sep 2026

noisy pairs only, and a content-matched control that
splits the gap into ≈5 dB of scene statistics and ≈3 dB
of noise distribution. This ≈8 dB regime gap dwarfs
every in-distribution loss effect we measure (≤0.8 dB
between MAE and MSE on synthetic noise, ≤2.3 dB
across the five real-pair losses); only out of distribution
do loss effects grow large, governing robustness rather
than accuracy (Section V-D).
All evaluations report PSNR, SSIM, and LPIPS [12] on
the raw network output; a post-hoc image-enhancement stage
present in an earlier draft was removed as an evaluation
confound (Section VI-B).
II. RELATED WORK
Denoising without clean targets. N2N [4] trains on in-
dependently corrupted pairs. Blind-spot methods remove even
the second observation: Noise2Void [13] and Noise2Self [14]
predict masked pixels under a pixel-independence assump-
tion; Laine et al. [15] add noise-model-aware posterior infer-
ence to the blind-spot architecture; Noise2Same [16] relaxes
strict J-invariance; Noise2Sim [17] mines self-similar patches;
Neighbor2Neighbor [18], Recorrupted-to-Recorrupted [19],
and Noisier2Noise [20] synthesize N2N-style pairs from single
noisy images; Self2Self [21] trains on one image alone with
dropout ensembling; Blind2Unblind [22] revisits blind-spot
information loss.
Real-noise self-supervision. On real sRGB camera noise,
spatial correlation breaks the pixel-independence assumption
of blind-spot training. That Gaussian-trained denoisers transfer
poorly to real photographs is itself established: CBDNet [10]
motivates realistic noise modeling with this failure, and Brooks
et al. [11] trace it to the sRGB processing pipeline. AP-BSN
[23] and CVF-SID [24] are current self-supervised references
on SIDD (35.97 and 34.81 dB on the validation split; super-
vised state of the art is 39–40 dB). More recent systems attack
precisely the mismatch we quantify, aligning the effective
training distribution with the test noise: Positive2Negative
[25] replaces information-lossy masking and downsampling
with renoising-consistency training, next-scale prediction [26]
builds cross-scale targets that decorrelate real noise without
destroying detail (37.1 dB), and Learning-to-Recorrupt [27]
learns the recorruption map when the noise distribution is
unknown. Our real-pair N2N results (33–35 dB, Section V-D)
sit just below this self-supervised band, which is consistent
with our deliberately compact backbone and budget. We do
not compete with these systems; the real-noise setting is a
controlled probe of whether the loss function or the training
pair distribution carries the inductive bias. Our N2N-on-real-
pairs configuration is simply [4] applied to SIDD’s two noisy
shots per scene; its value is diagnostic. N2N trained on SIDD-
Medium pairs already appears as a baseline row in the raw-
domain tables of [18]; we run the configuration in sRGB, loss-
controlled.
Classical baselines. BM3D [9] remains the reference non-
learning denoiser and is still being refined, e.g. by varying the
transform in its collaborative-filtering stage [28]; we therefore
treat its configuration as an experimental variable rather than
a constant (Section VI-B).
Loss functions for restoration. Zhao et al. [5] compare
L2, L1, and SSIM-based losses for restoration networks and
attribute L1’s edge over L2 to optimization behavior rather
than statistical modeling, consistent with the framing we derive
in Section III. Charbonnier [29] and Huber losses interpolate
between the two regimes.
III. THEORY: WHAT THE LOSS CAN AND CANNOT DO
A. Likelihood View and Its Limit
For a residual model r = x −fθ(x′), minimizing P
i ρ(ri)
is maximum likelihood under p(r) ∝e−ρ(r): MSE corresponds
to a Gaussian residual (optimal predictor: conditional mean),
MAE to a Laplace one (conditional median).
In the N2N setting the training target is x2 = s + n2 with
n2 symmetric zero-mean noise independent of x1 and of the
signal s: true by construction in our synthetic arm, violated by
signal-dependent real noise (Section V-D). The MSE optimum
is then exactly the posterior mean, E[x2 | x1] = E[s | x1], with
no further assumptions. The MAE optimum is the median of
p(x2 | x1), i.e., the signal posterior p(s | x1) convolved with
the noise density, and convolving a skewed posterior with a
symmetric density does not symmetrize it. Hence, pointwise
in the predicted value z given x1,
arg min
z
E

(z −x2)2 x1

= arg min
z
E[ |z −x2| | x1] (1)
holds exactly when p(s | x1) is symmetric about its mean,
and approximately whenever the posterior is concentrated
relative to the noise, whose symmetric density then dominates
the convolution.1 Neither condition is guaranteed for natural
images, so the population-level gap between the two losses
is the mean–median distance of p(x2 | x1): bounded above
by one standard deviation of that distribution, typically far
smaller, but not zero. Systematic differences between trained
networks therefore combine this small statistical component
with the finite-sample optimization path: the MAE gradient is
sign(r), so each residual’s influence on the update is bounded
[7], while the MSE gradient grows linearly in r, so rare large
residuals dominate updates. The sub-dB gaps we measure
(Section V-B) are consistent with near-coincident optima. We
treat cross-noise evaluation as an empirical question.
B. The L1-Loss-versus-L1-Regularization Confusion
Lasso regularization augments a base loss J0 with a weight
penalty, J = J0+α P
i |wi|, whose fixed points exhibit sparse
w [6]. The loss J0 = ∥x2 −fθ(x1)∥1 penalizes pixel-domain
residuals; its gradient flows to activations, not weights, and no
mechanism connects it to weight sparsity. Section V-C tests
the dissociation directly.
1Clamping training pairs to [0, 1] truncates the noise asymmetrically at sat-
urated pixels, so the zero-mean-symmetry premise is itself only approximate
at the range bounds.

C. Hypotheses
(H1) MAE-class losses (MAE, Charbonnier, Huber) general-
ize no worse than MSE to additive heavy-tailed test noise, with
any advantage concentrated in perceptual metrics. (H2) Weight
sparsity is neither necessary nor sufficient for the MAE-class
cross-noise profile. (H3) When the test noise violates the addi-
tive zero-mean training assumption structurally (multiplicative
speckle at high variance; real signal-dependent sRGB camera
noise), N2N fails irrespective of loss, and matching the training
pair distribution to the test noise restores the guarantee.
IV. METHOD
A. Architecture
A U-Net [30] (depth 4, base width 48, BN, ReLU) with a
global residual connection: the network predicts a correction
added to its input, DnCNN-style [1]; held fixed across all loss
variants.
B. Training Distributions
Synthetic arm. 400 color images from the BSDS500
train+test splits (the standard CBSD400 set) [31] (disjoint
from all evaluation sets). Each sample is a random 128×128
crop; x1, x2 receive independent additive Gaussian noise with
σ1, σ2 ∼U(5, 50) (8-bit scale). The training family is de-
liberately narrow so that cross-noise behavior is attributable
to the loss. Real arm. SIDD-Medium