5 comments

[ 4.2 ms ] story [ 21.3 ms ] thread
Unfortunately the source spreadsheet was not shared, but the root cause was a misreported codepage (which you can actually fix with a hex editor!)

"Â ", where the second character is a unicode non-breaking space characters (U+00A0), is the CP1252 encoding of the byte sequence [0xC2, 0xA0] which is the UTF-8 encoding of "\u00A0". So all that really needs to happen is to get Excel to think that the file is UTF-8 encoded.

The fix? For XLS, there is a special Codepage record that indicates how strings are to be interpreted. The magic value to force UTF-8 interpretation is 65001, and it is possible to use a hex editor to find exactly where that record is located and change it.

As a speaker of a Cyrillic-based language, this occurs quite frequently. It was once so common (before most sites adopted UTF8), they came up with a tool to detect the current encoding and convert it to readable Cyrillic automatically: https://www.artlebedev.ru/decoder/ (the tool exists since 1997)
That happened because the xls file was opened with something other than excel. Accept no imitations!
From my knowledge it's because Excel expects CSV files to be ANSI-encoded, but a lot of implementers just generate CSVs with UTF-8 -- which they're arguably entitled to do, it's Excel which is the one being stupid.