The Pub Discussion Board

Get your favorite beverage, sit back, and join in the discussion

You are not logged in.

#1 2018-02-03 23:11:44

howard
Inebriated
Registered: 2016-12-01
Posts: 18

Question mark in black diamond

I am getting quite a few of these question mark symbols in some stories. At first I assumed it was a character encoding issue (UTF-8 -v- ISO-8859-1) in my browser but upon further investigation the characters are not being substituted in the browser but are actually 'replacement character' in the html page source, see https://en.wikipedia.org/wiki/Specials_(Unicode_block)#Replacement_character
Wikipedia suggests that the problem might be caused by an editor being used incorrectly (such as loading an ISO format file into an editor that expects UTF)
As this only affects the stories of some authors (including auguy and magusfang) it might be reasonable to assume it is something they are doing something wrong in exporting from their editors ready to upload their stories. Probably they simply need to tell their editors to save in UTF-8.
A file uploaded in ISO-8859-1 will not render correctly in UTF-8 and I guess part of your processing catches the characters that are invalid in UTF-8 and replaces them.
Eric, might I suggest that when you work on your website next you consider detecting these 'replacement characters' (they are hex EF BF BD) and either: Ask the author to save in UTF-8 OR automatically convert from ISO to UTF.

Thanks,

Howard

Offline

 

#2 2018-02-04 03:26:39

Eric Storm
Pub Owner
From: New Port Richey, FL
Registered: 2006-09-12
Posts: 4004
Website

Re: Question mark in black diamond

It's being worked on.  I won't bother going into detail, as it's complicated and boring.  Rest assured, I am trying to find a solution.  If I have to, I'll simply replace all of that particular character with a non-breaking space.  I think that would be less distracting than the black diamond.  (Yes, I could just remove it, but in some cases, it appears to represent a space, and so removing it could have readability consequences.)

Eric Storm


Please Remember:  The right to Freedom of Speech does not carry the proviso, "As long as it doesn't upset anyone."  The US Constitution does not grant you the right to not be offended.  If you don't like what someone's saying... IGNORE THEM.
----
AMEN! >>> Word Crimes

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2005 Rickard Andersson