Watermark Forensics for Generative Models: An Information-Theoretic Perspective

arXiv:2607.13003v1 Announce Type: cross Abstract: A watermark in a generative model's output is usually asked only whether a text is machine-made. The same mark can do more: attribute it to the user who produced it, extract a hidden payload, or localize the part that survives editing. These form a forensic ladder, and we ask what each rung costs in the sample length n. One object organizes the answers. Let S be the secret the mark carries (a user's identity or payload), and let the information profile nu(t)=I(S;X_tmid X_{<t}) record how much the t-th token reveals about S given the earlier ones. Its total mass pays for attribution and extraction; how that mass is spread pays for localization; and detection alone is paid for not by information but by presence, the distance from the marked to the unmarked distribution. The literature's two quality models, a mark subtle on every token and one that stamps a few tokens loudly, are two incomparable ways of capping this profile. Our main theorem settles the ladder's entropy column. For statistically distortion-free schemes, attributing a text to one of N users costs Theta(log N/h) tokens over every stationary-ergodic source of entropy rate h, sharp to a (1+o(1)) factor: to our knowledge the first tight entropy-rate law for multi-user attribution (via exact alignment). The natural collision-counting analysis overcharges without bound; only a decoder thresholding each candidate by its own realized surprisal attains the rate while almost never implicating an innocent user. A matching converse makes the law two-sided, and extraction of an ell-bit payload costs Theta(ell/h). Two gaps are real, not modeling artifacts: a Theta(log N)-token window in which a text is provably machine-made yet unattributable, and a footprint-resolution uncertainty principle. Experiments on GPT-2, Pythia-410M, and Qwen2.5 recover the predicted constants.

Source: arXiv cs.LG | 2026-07-15