NVP stands for non-volume-preserving transformations. This work introduces an ingenious way to create non-linear but exactly invertible transformations to map from the latent variables z to the data x. Though it is not mentioned in the paper but the transformations seem to be inspired from Fiestel structures used in encryption, an example being the DES encryption algorithm.
Once they have an invertible and non-linear transformation from z to x, they use the change of variable formula to get an expression for the density of any datapoint x. Thereafter they train to maximize the likelihood of the data. Inference of z given x is trivial here because of the invertibility of the transformation.