fix: improving the output of get tokenized data function for DQ

[bayo-ibm]

Description of the change

The PR aims to optimize the tokenized data function in fms_mo. Prior to now, the train data and test data were not in the same data format. Also, a unified approach was not used in generating the tokenized train_data, leaving room for further improvement of the associated code.

The training data and test dataset are now in the same data format.
The data structure for the training dataset has been changed from a list to a dict
Also, the format of the test_dataset was changed from BatchEncoding to dict.
An edit was also made in the DQ code to be able to read the test and train datasets.

Related issues or PRs

181

How to verify the PR

Was the PR tested

• I have added >=1 unit test(s) for every new method I have added (if that coverage is difficult, please briefly explain the reason)
• I have ensured all unit tests pass

Checklist for passing CI/CD:

• All commits are signed showing "Signed-off-by: Name <email@domain.com>" with git commit -signoff or equivalent
• PR title and commit messages adhere to Conventional Commits
• [x ] Contribution is formatted with tox -e fix
• Contribution passes linting with tox -e lint
• Contribution passes spellcheck with tox -e spellcheck
• Contribution passes all unit tests with tox -e unit

Note: CI/CD performs unit tests on multiple versions of Python from a fresh install. There may be differences with your local environment and the test environment.