Role of the parameter called tail_mass

[AlbertoPresta]

Hi,
maybe it is a stupid question, but it is not clear to me what is the role of the variable called tail_mass and what happens when we increase or decrease it.

thanks in advance for the answer,
Alberto

[jonaballe]

Hi Alberto,

range coding can only process finite alphabets. So if you encode the bottleneck values, which can in theory include all the signed integers, you would need to cut them off somewhere. To get around this, we reserve one symbol as an "overflow" signal. So the range coding tables include probabilities for a given range of the latent distributions (around the most common values), plus an escape symbol that says "the encoded value is outside of the range". In that case, we use an Elias gamma code to send the actual value.

The entropy model classes automatically determine what range of values is the most probable and assign this range to the range coding tables. tail_mass is a parameter that controls how much probability mass we want the escape symbol to capture. So if you make it larger, the range coding table will be smaller, and more values are going to be transmitted using the gamma code. If you make it smaller, the range coding tables are bigger, and the escape symbol is going to be used less often.

Hope this helps!
Johannes.

[AlbertoPresta]

thanks a lot for the answer! just one last thing, this is the reason why we have that the quantized CDF is larger by 2 with respect to the cmf?

[jonaballe]

Yes, the CDF is one larger because it always begins with 0 and ends with 2^precision - 1, so it needs one more element to capture the same information. It is also one larger because one of the symbols is reserved for the overflow (in case overflow functionality is used).

[AlbertoPresta]

ok ok I got it.
May I ask you also a more theoretical question? I read that in your paper VARIATIONAL IMAGE COMPRESSION WITH A SCALE HYPERPRIOR the latent densities are convolved with standard uniform density function, and this allows them to converge to a unit uniform density, am I right?