ObjectTranslator doesn't perfectly roundtrip w3a files

[Slayer95]

Test script:

"use strict";

const fs = require('fs');
const {ObjectsTranslator} = require('wc3maptranslator');

function parseAbilities(filePath) {
  return ObjectsTranslator.warToJson('abilities', fs.readFileSync(filePath));
}

function exportAbilities(filePath, data) {
  fs.writeFileSync(filePath, ObjectsTranslator.jsonToWar('abilities', data).buffer);
}

const abilities = parseAbilities('war3map.w3a');
if (abilities.errors.length) throw new Error(`Errors found: ${abilities.errors.join(', ')}`)
exportAbilities('war3map-rt.w3a', abilities.json);

In the attached files, you can verify that war3map-rt.w3a has added a bunch of extraneous FourCC codes where there used to be a quad of zeroes.

w3a-test.zip

[ChiefOfGxBxL]

Hi @Slayer95,

It appears the difference is between the descriptions in the "Fire Shield" ability, in the .w3a file you attached. The rest of the data appears correct... it's only that ability description (for rank 1, 2, 3 and 4) that's causing an issue:

-"Fire Shield, [|cffffcc00Rank 2|r] (|cffffcc00С|r)"
+"Fire Shield, [|cffffcc00Rank 2|r] (|cffffcc00�¡|r)"

This is presumably being caused by #83:

• W3Buffer#readString uses String.fromCharCode(ch) (src), which according to MDN, "returns a string created from the specified sequence of UTF-16 code units."
• HexBuffer#addString() always writes as UTF-8 (src)

I'll test this out on a smaller example to confirm a fix, which will be to change HexBuffer#addString() to encode as UTF-16 (utf16le).

Could you please let me know what string you're using for that ability's description? Could you also attach a screenshot of how it appears in World Editor?

[Slayer95]

The rest of the data appears correct

I am afraid that's not the only issue. Please compare offset 0x88F in the test case attached to the OP.

I'll test this out on a smaller example to confirm a fix, which will be to change HexBuffer#addString() to encode as UTF-16 (utf16le).

I believe(d) writing as UTF8 was ok, so I'd suggest changing parsing of string modifications in ObjectsTranslator to use UTF-8 instead. However, yours is an alternative I haven't tested.

Could you please let me know what string you're using for that ability's description? Could you also attach a screenshot of how it appears in World Editor?

Sorry. I don't know how that byte sequence turned up in the abilities file. The map is protected and it cannot be opened in WE anymore. Additionally, War3 Object Editor fails to parse the abilities file.

[ChiefOfGxBxL]

I've added sub-issues for a number of round-trip issues I found. The master branch now has a war -> json -> war test suite which detected these issues. Many of those problems are probably coming from "information loss", since some of the translators have unsupported features.