Details of training script

[dreminm]

Hello!

I'm currently trying to understand the model training process and noticed the use of the --data_path ./llava/state3_after.json parameter. Could you provide more details of this file? As far as i understand this script provides stage3 finetuning. Do you have script for pretraining you model with distortion classification and score classification (stage 1+2)?

[jzhws]

The three stages are trained with the same script with only a little adjustments. So you can use this code to finish the training of the three stages. Besides, the stage-1 data is set to be private so you may only train the stage-2 and stage-3.

[dreminm]

Hello! I appreciate your response!

The paper mentions that you used 11 distortions in model pretraining: distortions = ["compression artifact", "spatial blur", "motion blur", "noise", "overexposure", "underexposure", "low contrast", "high contrast", "oversaturation", "desaturation", "block effect"]

However, in the study you referenced (LIQE), the artifacts are labeled as follows:

koniq10k = ['other realistic', 'overexposure', 'realistic blur', 'realistic contrast change', 'realistic noise', 'underexposure']
kadid10k = ['brighten', 'color blocking', 'color quantization', 'contrast', 'darken', 'desaturation', 'diffusion', 'gaussian blur', 'impulse', 'jitter', 'jpeg compression', 'jpeg2000 compression', 'lens', 'motion', 'multiplicative', 'noneccentricity patch', 'oversaturation', 'pixelate', 'quantization', 'sharpness', 'shifting', 'shifting the mean', 'white noise', 'white noise with denoise', 'white with color']
Where did you derive the artifacts ['block effect', 'compression artifact', 'high contrast', 'low contrast', 'motion blur', 'noise', 'spatial blur'] from? Is there any mapping between the labeling in LIQE and yours?