April 1, 2023
Conference Paper

Evaluating generative networks using Gaussian mixtures of image features

Abstract

We develop a measure for evaluating the performance of generative networks given two sets of images. A popular performance measure currently used to do this is the Fr├ęchet Inception Distance (FID). However, FID assumes that images featurized using the penultimate layer of Inception follow a Gaussian distribution. This assumption allows FID to be easily computed, since FID uses the 2-Wasserstein distance of two Gaussian distributions fitted to the featurized images. However, we show that Inception features of the ImageNet dataset are not Gaussian; in particular, each marginal is not Gaussian. To remedy this problem, we model the featurized images using Gaussian mixture models (GMMs) and compute the 2-Wasserstein distance restricted to GMMs. We define a performance measure, which we call WaM, on two sets of images by using inception (or another classifier) to featurize the images, estimate two GMMs, and use the restricted 2-Wasserstein distance to compare the GMMs. We experimentally show the advantages of WaM over FID, including how FID is more sensitive than WaM to image perturbations. By modelling the non-Gaussian features obtained from inception as GMMs and using a GMM metric, we can more accurately evaluate generative network performance.

Published: April 1, 2023

Citation

Luzi L., C.M. Ortiz Marrero, N.N. Wynar, R.G. Baraniuk, and M.J. Henry. 2023. Evaluating generative networks using Gaussian mixtures of image features. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2023), January 2-7, 2023, Waikoloa, HI, 279-288. Piscataway, New Jersey:IEEE. PNNL-SA-167150. doi:10.1109/WACV56688.2023.00036