Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 7 of 7 total
Thread EDBManager and Unicode
Thu, Jan 3 2008 4:13 PMPermanent Link

Dan Rootham
Tim,

ElevateDB Manager (Unicode) 1.06 Build 1

Should the SQL input panel in EDB Manager support the pasting in
of Unicode characters?

I tried to input a string of Unicode Chinese characters, but they were converted
to question marks as shown below:

INSERT INTO TestDict (
RecNo,
PartSpeech,
DictionaryTerm,
UpdatedOn
)         
VALUES (
10001,
'n',
'???: ?????(??)????,????? ??????',
CAST('2008-01-03' AS TIMESTAMP)
)

And the follow-up question is this: does the result set grid default to a font
which supports Unicode (e.g. Arial Unicode)? Is there a way to reselect the font
at runtime for:
a) the SQL input panel
b) the result set grid?

Having just written a complete "clone" of Borland's SQL Explorer to support Unicode
SQL and Unicode parameters for a site using Delphi 7, I already faced these problems.
So of course I am aware that you have to jump through many hoops to get all this working,
including the use of non-Codegear visual controls like the Tnt/TMS Unicode components.
But on this occasion, I'm the user so I can pass the buck over to you... Smiley

Thanks,
Dan

Lexicon Software Ltd, UK
Fri, Jan 4 2008 7:56 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< Should the SQL input panel in EDB Manager support the pasting in of
Unicode characters? >>

Yes, it should, although there is always the possibility that there's an
issue with the pasting.

<< I tried to input a string of Unicode Chinese characters, but they were
converted to question marks as shown below: >>

Can you load the same SQL from a Unicode .SQL file and have it display
properly ?

<< And the follow-up question is this: does the result set grid default to a
font which supports Unicode (e.g. Arial Unicode)? >>

The EDB Manager actually uses the VCL controls for all non-SQL editing and
display, so it has to convert all widestrings into ansistrings for display
purposes.  The default font used in the EDB Manager is the Tahoma font,
which should work find with all multi-byte character sets.

--
Tim Young
Elevate Software
www.elevatesoft.com

Sun, Mar 30 2008 12:43 PMPermanent Link

Dan Rootham
Tim,

Now using:  EDB Manager Unicode ver 1.08 build 1

<< The EDB Manager actually uses the VCL controls for all non-SQL editing and
display, so it has to convert all widestrings into ansistrings for display
purposes.  The default font used in the EDB Manager is the Tahoma font,
which should work find with all multi-byte character sets. >>

My original question a while back was this:
"... does the result set grid in EDBMgr default to a font which supports Unicode
(e.g. Arial Unicode)? "

I am sorry, but we are still having trouble seeing any Unicode data in EDB Manager
unless the data was originally migrated from the Win-1252 West European charset.
I fully understand that for now we are responsible for our own conversion from Win-1251,
Win-1253 (etc) and from any BLOB data. Post conversion, we have found that Russian,
Chinese and Japanese don't display in EDBMgr. In other words, all the tables which didn't
use Win-1252 West European as their DBISAM codepage

Our own conversion application reads and writes to the migrated EDB tables (it doesn't touch  
the original DBISAM tables). The utility takes Russian codepage 1251 data and passes it
through
a Windows API call to convert to Unicode. For Chinese and Japanese, our utility reads
the BLOB data migrated from DBISAM and converts that into Unicode (widestring).

After conversion, our utility correctly displays the converted Unicode data in the EDB tables
(using a Tnt / TMS Unicode grid and font Arial Unicode MS). But EDBMgr can only render the
Unicode data as question marks (????????????). The data is definitely present, because the
number of ? characters matches the expected character count. (I tried to post a screen shot
to the Binaries newsgroup, but the error message said that "a required field has not been
filled".)

Incidentally, EDBMgr displays the unconverted data just fine: Russian unconverted data
appears as various accented vowels - exactly what we expect to see when codepage 1251
(Russian) data is wrongly viewed under codepage 1252 (West European). But EDBMgr is
just seeing that data as "West European accented characters", which allows it to be visible.

Will there be a solution to this Unicode display problem in due course? We can manage for
now, but it's obviously no good when we need EDBMgr for checking actual data changes.
Is this a limitation of the font used by EDBMgr (Tahoma)? Would using Arial Unicode MS
resolve the problem?

Many thanks,
Dan

Lexicon Software Ltd, UK
Sun, Mar 30 2008 1:03 PMPermanent Link

Dan Rootham
Tim,

A follow-up, with me feeling a bit sheepish...

<< Is this a limitation of the font used by EDBMgr (Tahoma)? Would using
Arial Unicode MS resolve the problem? >>

I have just realised that Arial Unicode MS isn't even installed on the PC in question,
because MS Office isn't installed there either. I just wonder whether this means that the
version of the installed Tahoma font is also "crippled" in some way i.e. no Asian
characters?

Looking at Tahoma in Accessories > System Tools > Character Map, it seems that the
characters are there to support Cyrillic, Arabic, Hebrew and so on. But not Chinese or
Japanese.  Is it simply that Tahoma isn't a *Unicode* font? Because I can't even view
Russian data in EDB Manager Unicode.

Thanks,
Dan
Mon, Mar 31 2008 4:52 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< I have just realised that Arial Unicode MS isn't even installed on the PC
in question, because MS Office isn't installed there either. I just wonder
whether this means that the version of the installed Tahoma font is also
"crippled" in some way i.e. no Asian characters?

Looking at Tahoma in Accessories > System Tools > Character Map, it seems
that the haracters are there to support Cyrillic, Arabic, Hebrew and so on.
But not Chinese or Japanese.  Is it simply that Tahoma isn't a *Unicode*
font? Because I can't even view Russian data in EDB Manager Unicode. >>

Thoma may have limitations with certain character sets, but not because of
the font.  Rather, the issue is that even the Unicode EDB Manager is not
actually using Unicode controls, therefore it is displaying text using the
ANSI character sets and fonts.  Therefore, whatever translations occur
during the implicit Unicode->ANSI conversion that occurs before display in
the EDB Manager will determine what character points are used with the
Tahoma font.

Basically, none of this is resolvable completely until CodeGear releases the
Unicode VCL.  Use of the TNT controls is a non-starter due to the fact that
it requires almost a complete separate version of the EDB Manager in order
to function properly.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tue, Apr 1 2008 8:30 AMPermanent Link

Dan Rootham
Tim,

<< Basically, none of this is resolvable completely until CodeGear releases
the Unicode VCL.  Use of the TNT controls is a non-starter due to the fact that
it requires almost a complete separate version of the EDB Manager in order
to function properly. >>

Fair enough. At least Codegear is committed to Unicode in the next Delphi release!

Regards,
Dan
Tue, Apr 1 2008 5:27 PMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Dan,

<< Fair enough. At least Codegear is committed to Unicode in the next Delphi
release! >>

Yep, and we're ready to go. Smiley

--
Tim Young
Elevate Software
www.elevatesoft.com

Image