Docs
Browse Docs
Field guide · Text and encoding

Broken accented characters

Accented letters appear as pairs of odd symbols or as question marks, because the file was read with the wrong encoding.

What it looks like

Before
namecity
José MarinMünchen
Ana PeñaLogroño
Ren?e DuboisOrl?ans
After
namecity
José MarinMünchen
Ana PeñaLogroño
needs the sourceneeds the source
The first row is damaged but reversible: the original bytes are still there. In the third row the letter was replaced by a question mark, so the file no longer holds what it was.

How it happens

  • A file saved as UTF-8 was opened as an older single-byte encoding, so each accented letter became two characters.
  • A file passed through a system that could not store the letter and substituted a question mark.
  • The file was opened and saved again while it was being misread, which made the damage permanent.

What it breaks

  • Names and places are wrong on everything sent to the people they belong to.
  • The damaged spelling does not match the correct one, so joins and duplicate checks miss it.
  • Sorting and searching by name put the affected rows in the wrong place.

How to spot it

  • Search for the telltale pairs, most often a capital A with a tilde or a circumflex followed by another symbol.
  • Search for question marks and replacement symbols in the middle of words.
  • Check columns likely to hold accents: names, cities, and anything typed in a language other than English.

How to fix it

  • Read the original file again with the correct encoding. This is the cleanest fix when the original still exists.
  • Where only the damaged copy remains and the pairs are intact, reverse the misreading to recover the letters.
  • A question mark cannot be reversed. Recover the value from the source, or leave it flagged; guessing the letter is still guessing.
  • Save the result as UTF-8 and state the encoding wherever the file is handed on.

TermsMojibake, Encoding, UTF-8, Unicode

Karg checks for many of these when it profiles a file, applies the fixes it is confident about, and sets the rest aside for your review. See how it works, or have our team clean your data for you.