semantic token ids will have variable lengths because of unique consecutive,...

semantic token ids will have variable lengths because of unique consecutive, so eos token must be manually selected and then used to predict the first coarse token, in the coarse transformer