[Solved]-Java properties UTF-8 encoding in Eclipse-eclipse

If the properties are for XML or HTML, it's safest to use XML entities. They're uglier to read, but it means that the properties file can be treated as straight ASCII, so nothing will get mangled.

Note that HTML has entities that XML doesn't, so I keep it safe by using straight XML: http://www.w3.org/TR/html4/sgml/entities.html

This seems to work only for some characters ... including special characters for German, Portuguese, French. However, I ran into trouble with Russian, Hindi and Mandarin characters. These are not converted to Properties format 'native2ascii', instead get saved with ?? ?? ??
The only way I could get my app to display these characters correctly is by putting them in the properties file translated to UTF-8 format - as \u0915 instead of क, or \u044F instead of я. Any advice?

I recommend you to use Attesoro (http://attesoro.org/). Is simple and easy to use. And is made in java.

Just another Eclipse plugin for *.properties files:

Properties Editor

You can define UTF-8 .properties files to store your translations and use ResourceBundle, to get values. To avoid problems you can change encoding:

String value = RESOURCE_BUNDLE.getString(key); 
return new String(value.getBytes("ISO-8859-1"), "UTF-8");

There are too many points in the process you describe where errors can occur, so I won't try to guess what you're doing wrong, but I think I know what's happening under the hood.

EF BF BD is the UTF-8 encoded form of U+FFFD, the standard replacement character that's inserted by decoders when they encounter malformed input. It sounds like your text is being saved as ISO-8859-1, then read as if it were UTF-8, then saved as UTF-8, then converted to the Properties format using native2ascii using the platform default encoding (e.g., windows-1252).

ü              => 0xFC                // save as ISO-8859-1
0xFC           => U+FFFD              // read as UTF-8
U+FFFD         => 0xEF 0xBF 0xBD      // save as UTF-8
0xEF 0xBF 0xBD => \u00EF\u00BF\u00BD  // native2ascii

I suggest you leave the "file.encoding" property alone. Like "file.separator" and "line.separator", it's not nearly as useful as you would expect it to be. Instead, get into the habit of always specifying an encoding when reading and writing text files.

Properties props = new Properties();
URL resource = getClass().getClassLoader().getResource("data.properties");         
props.load(new InputStreamReader(resource.openStream(), "UTF8"));

this works well in java 1.6. How can i do this in 1.5, Since Properties class does not have a method to pars InputStreamReader.

There is much easier way:

props.load(new InputStreamReader(new FileInputStream("properties_file"), "UTF8"));

Properties props = new Properties();
URL resource = getClass().getClassLoader().getResource("data.properties");         
props.load(new InputStreamReader(resource.openStream(), "UTF8"));

Works like a charm

:-)

It is not a problem with Eclipse. If you are using the Properties class to read and store the properties file, the class will escape all special characters.

From the class documentation:

When saving properties to a stream or loading them from a stream, the ISO 8859-1 character encoding is used. For characters that cannot be directly represented in this encoding, Unicode escapes are used; however, only a single 'u' character is allowed in an escape sequence. The native2ascii tool can be used to convert property files to and from other character encodings.

From the API, store() method:

Characters less than \u0020 and characters greater than \u007E are written as \uxxxx for the appropriate hexadecimal value xxxx.

Answer for "pre-Java-9" is below. As of Java 9, properties files are saved and loaded in UTF-8 by default, but falling back to ISO-8859-1 if an invalid UTF-8 byte sequence is detected. See the Java 9 release notes for details.

Properties files are ISO-8859-1 by definition - see the docs for the Properties class.

Spring has a replacement which can load with a specified encoding, using PropertiesFactoryBean.

EDIT: As Laurence noted in the comments, Java 1.6 introduced overloads for load and store which take a Reader/Writer. This means you can create a reader for the file with whatever encoding you want, and pass it to load. Unfortunately FileReader still doesn't let you specify the encoding in the constructor (aargh) so you'll be stuck with chaining FileInputStream and InputStreamReader together. However, it'll work.

For example, to read a file using UTF-8:

Properties properties = new Properties();
InputStream inputStream = new FileInputStream("path/to/file");
try {
    Reader reader = new InputStreamReader(inputStream, "UTF-8");
    try {
        properties.load(reader);
    } finally {
        reader.close();
    }
} finally {
   inputStream.close();
}