Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #82858 > unrolled thread

Hello World in Russian

Back to article view | Back to comp.lang.c++

Contents

  Hello World in Russian Joseph Hesse <joeh@gmail.com> - 2022-01-29 05:51 +0200
    Re: Hello World in Russian James Kuyper <jameskuyper@alumni.caltech.edu> - 2022-01-28 23:27 -0500
      Re: Hello World in Russian Muttley@dastardlyhq.com - 2022-01-29 10:21 +0000
    Re: Hello World in Russian Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-01-28 22:42 -0800
      Re: Hello World in Russian David Brown <david.brown@hesbynett.no> - 2022-01-29 13:01 +0100
        Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-29 14:12 +0100
          Re: Hello World in Russian David Brown <david.brown@hesbynett.no> - 2022-01-29 17:34 +0100
        Re: Hello World in Russian James Kuyper <jameskuyper@alumni.caltech.edu> - 2022-01-30 14:17 -0500
          Re: Hello World in Russian Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-01-30 20:17 -0800
    Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-29 11:48 +0100
      Re: Hello World in Russian Paavo Helde <eesnimi@osa.pri.ee> - 2022-01-29 17:39 +0200
        Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-30 09:18 +0100
          Re: Hello World in Russian Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-30 12:12 +0000
            Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-30 14:02 +0100
              Re: Hello World in Russian Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-30 14:45 +0000
                Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-30 21:53 +0100
                  Re: Hello World in Russian Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-30 21:00 +0000
              Re: Hello World in Russian James Kuyper <jameskuyper@alumni.caltech.edu> - 2022-01-30 14:02 -0500
            Re: Hello World in Russian Muttley@dastardlyhq.com - 2022-01-31 10:23 +0000
              Re: Hello World in Russian scott@slp53.sl.home (Scott Lurndal) - 2022-01-31 16:11 +0000
                Re: Hello World in Russian Muttley@dastardlyhq.com - 2022-01-31 16:36 +0000
                Re: Hello World in Russian Paavo Helde <eesnimi@osa.pri.ee> - 2022-01-31 20:20 +0200
              Re: Hello World in Russian Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-31 16:12 +0000
                Re: Hello World in Russian Muttley@dastardlyhq.com - 2022-01-31 16:39 +0000
                  Re: Hello World in Russian Ben Bacarisse <ben.usenet@bsb.me.uk> - 2022-01-31 23:47 +0000
          Re: Hello World in Russian Manfred <noname@add.invalid> - 2022-01-31 16:45 +0100
          Re: Hello World in Russian Andrey Tarasevich <andreytarasevich@hotmail.com> - 2022-01-31 10:10 -0800
            Re: Hello World in Russian "Alf P. Steinbach" <alf.p.steinbach@gmail.com> - 2022-01-31 19:25 +0100
        Re: Hello World in Russian Marcel Mueller <news.5.maazl@spamgourmet.org> - 2022-01-30 12:09 +0100
          Re: Hello World in Russian Paavo Helde <eesnimi@osa.pri.ee> - 2022-01-30 14:26 +0200
            Re: Hello World in Russian Richard Damon <Richard@Damon-Family.org> - 2022-01-30 08:59 -0500
              Re: Hello World in Russian Paavo Helde <eesnimi@osa.pri.ee> - 2022-01-30 18:21 +0200
                Re: Hello World in Russian Richard Damon <Richard@Damon-Family.org> - 2022-01-30 12:56 -0500
                  Re: Hello World in Russian Vir Campestris <vir.campestris@invalid.invalid> - 2022-02-07 21:05 +0000

Page 1 of 2  [1] 2  Next page →

#82858 — Hello World in Russian

In the following program, I have written "hello world", copied from the 
iternet, in Russian.  I don't understand Russian but I understand their 
alphabet is different from the English alphabet.  The program is 
supposed to output the original message and the characters in the 
original message.  My question is why are they different?
Thank you,
Joe

$ more Hello.cpp
#include <iostream>
using namespace std;

int main()
{
   const wchar_t russian[] = L"Привет мир";  //Russian for "hello, world"

   wcout << russian << endl;

   for(const wchar_t &x : russian)
     wcout << x << L", ";
   wcout << endl;
}
$ g++ -std=c++14 Hello.cpp
$ ./a.out
Privet mir                        <== original message
P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
$

[toc] | [next] | [standalone]

#82859

On 1/28/22 22:51, Joseph Hesse wrote:
> In the following program, I have written "hello world", copied from
> the iternet, in Russian. I don't understand Russian but I understand
> their alphabet is different from the English alphabet. The program is
> supposed to output the original message and the characters in the
> original message. My question is why are they different?
> Thank you,
> Joe
>
> $ more Hello.cpp
> #include <iostream>
> using namespace std;
>
> int main()
> {
> const wchar_t russian[] = L"Привет мир"; //Russian for "hello, world"

I do understand Russian (a little), and that looks correct to me.

> wcout << russian << endl;
>
> for(const wchar_t &x : russian)
> wcout << x << L", ";
> wcout << endl;
> }
> $ g++ -std=c++14 Hello.cpp
> $ ./a.out
> Privet mir <== original message
> P, r, i, v, e, t, , m, i, r, , <== wchars in original message
> $

The problem is that the behavior of your program depends upon the
locale. The default locale is the "C" locale, which doesn't support
those Russian characters. How those characters are interpreted is
implementation-defined - I think it's amusing that they get
misinterpreted as latin letters with the same sounds.

I added two lines to your program:

#include <clocale>

and as the first line of main():

std::setlocale(LC_CTYPE, "");

"" is the name of the native locale, what that means is up to your
implementation. On my desktop machine, it's the locale identified by the
LANG environment variable. I've currently got that set to en_US.UTF-8.
Because that locale supports UTF-8, it in particular supports those
Russian letters, and I get the following output:

Привет мир
П, р, и, в, е, т, , м, и, р, ,

You'll need to determine how the native locale is set on your machine,
and what the list of supported locales is, and they you'll have to find
out which one(s) support the characters you want to print.

[toc] | [prev] | [next] | [standalone]

#82861

On Fri, 28 Jan 2022 23:27:07 -0500
James Kuyper <jameskuyper@alumni.caltech.edu> wrote:
>On 1/28/22 22:51, Joseph Hesse wrote:
>> In the following program, I have written "hello world", copied from
>> the iternet, in Russian. I don't understand Russian but I understand
>> their alphabet is different from the English alphabet. The program is
>> supposed to output the original message and the characters in the
>> original message. My question is why are they different?
>> Thank you,
>> Joe
>>
>> $ more Hello.cpp
>> #include <iostream>
>> using namespace std;
>>
>> int main()
>> {
>> const wchar_t russian[] = L"Привет мир"; //Russian for "hello,
>world"
>
>I do understand Russian (a little), and that looks correct to me.
>
>> wcout << russian << endl;
>>
>> for(const wchar_t &x : russian)
>> wcout << x << L", ";
>> wcout << endl;
>> }
>> $ g++ -std=c++14 Hello.cpp
>> $ ./a.out
>> Privet mir <== original message
>> P, r, i, v, e, t, , m, i, r, , <== wchars in original message
>> $
>
>The problem is that the behavior of your program depends upon the
>locale. The default locale is the "C" locale, which doesn't support
>those Russian characters. How those characters are interpreted is
>implementation-defined - I think it's amusing that they get
>misinterpreted as latin letters with the same sounds.

Just use UTF8 and you won't need to worry about wide char nonsense. Just
use ordinary chars and let the terminal deal with translating the encoding.
On *nix anyway. Windows is probably somewhat more primitive in its approach.

[toc] | [prev] | [next] | [standalone]

#82860

On 1/28/2022 7:51 PM, Joseph Hesse wrote:
> In the following program, I have written "hello world", copied from the 
> iternet, in Russian.  I don't understand Russian but I understand their 
> alphabet is different from the English alphabet.  The program is 
> supposed to output the original message and the characters in the 
> original message.  My question is why are they different?
> Thank you,
> Joe
> 
> $ more Hello.cpp
> #include <iostream>
> using namespace std;
> 
> int main()
> {
>    const wchar_t russian[] = L"Привет мир";  //Russian for "hello, world"
> 
>    wcout << russian << endl;
> 
>    for(const wchar_t &x : russian)
>      wcout << x << L", ";
>    wcout << endl;
> }
> $ g++ -std=c++14 Hello.cpp
> $ ./a.out
> Privet mir                        <== original message
> P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
> $

Um... How did you manage to make your program output `L"Привет мир"` as

   Privet mir

?

-- 
Best regards,
Andrey Tarasevich

[toc] | [prev] | [next] | [standalone]

#82863

On 29/01/2022 07:42, Andrey Tarasevich wrote:
> On 1/28/2022 7:51 PM, Joseph Hesse wrote:
>> In the following program, I have written "hello world", copied from
>> the iternet, in Russian.  I don't understand Russian but I understand
>> their alphabet is different from the English alphabet.  The program is
>> supposed to output the original message and the characters in the
>> original message.  My question is why are they different?
>> Thank you,
>> Joe
>>
>> $ more Hello.cpp
>> #include <iostream>
>> using namespace std;
>>
>> int main()
>> {
>>    const wchar_t russian[] = L"Привет мир";  //Russian for "hello, world"
>>
>>    wcout << russian << endl;
>>
>>    for(const wchar_t &x : russian)
>>      wcout << x << L", ";
>>    wcout << endl;
>> }
>> $ g++ -std=c++14 Hello.cpp
>> $ ./a.out
>> Privet mir                        <== original message
>> P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
>> $
> 
> Um... How did you manage to make your program output `L"Привет мир"` as
> 
>   Privet mir
> 
> ?
> 

That is the Latin alphabet transliteration of the Cyrillic letters.  As
to why that conversion was done, I expect it is something to do with
locales and internationalisation libraries.  It doesn't work for all
non-ASCII characters, which is hardly surprising.  A quick test with
Norwegian letters "å", "ø" and "æ" resulted in "?", "?" and "ae".

The simple answer, at least in the *nix world, is to drop "wchar_t"
completely and forget that disaster ever existed.


#include <iostream>
using namespace std;

int main()
{
   const char russian[] = "Привет мир";  //Russian for "hello, world"

   cout << russian << endl;

   for (const char_t &x : russian)
     cout << x << L", ";
   cout << endl;
}

As long as you have UTF8 locales and fonts (and you'd need an ancient
setup, or a very limited one, not to have them), you'll get the right
output.


What I have learned from all this, is that "мир" ("mir") means "world"
as well as "peace".  I find that an interesting linguistic titbit.

[toc] | [prev] | [next] | [standalone]

#82864

On 29 Jan 2022 13:01, David Brown wrote:
> On 29/01/2022 07:42, Andrey Tarasevich wrote:
>> On 1/28/2022 7:51 PM, Joseph Hesse wrote:
>>> In the following program, I have written "hello world", copied from
>>> the iternet, in Russian.  I don't understand Russian but I understand
>>> their alphabet is different from the English alphabet.  The program is
>>> supposed to output the original message and the characters in the
>>> original message.  My question is why are they different?
>>> Thank you,
>>> Joe
>>>
>>> $ more Hello.cpp
>>> #include <iostream>
>>> using namespace std;
>>>
>>> int main()
>>> {
>>>     const wchar_t russian[] = L"Привет мир";  //Russian for "hello, world"
>>>
>>>     wcout << russian << endl;
>>>
>>>     for(const wchar_t &x : russian)
>>>       wcout << x << L", ";
>>>     wcout << endl;
>>> }
>>> $ g++ -std=c++14 Hello.cpp
>>> $ ./a.out
>>> Privet mir                        <== original message
>>> P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
>>> $
>>
>> Um... How did you manage to make your program output `L"Привет мир"` as
>>
>>    Privet mir
>>
>> ?
>>
> 
> That is the Latin alphabet transliteration of the Cyrillic letters.  As
> to why that conversion was done, I expect it is something to do with
> locales and internationalisation libraries.

Correct.

The wide streams must translate to ASCII in that situation with no 
selected locale, i.e. with the default "C" locale.


>  It doesn't work for all
> non-ASCII characters, which is hardly surprising.  A quick test with
> Norwegian letters "å", "ø" and "æ" resulted in "?", "?" and "ae".
> 
> The simple answer, at least in the *nix world, is to drop "wchar_t"
> completely and forget that disaster ever existed.

Agreed, including the qualification.


> #include <iostream>
> using namespace std;
> 
> int main()
> {
>     const char russian[] = "Привет мир";  //Russian for "hello, world"

I would just write

     const auto& russian = "Привет мир";  //Russian for "hello, world"

... preserving the length information, and not relying on the compiler 
to optimize away the copying, where the technically redundant `const` is 
for clarity and supports fast visual scanning of code.

In particular this supports forming a `constexpr stringview`.


>     cout << russian << endl;
> 
>     for (const char_t &x : russian)
>       cout << x << L", ";
>     cout << endl;
> }
> 
> As long as you have UTF8 locales and fonts (and you'd need an ancient
> setup, or a very limited one, not to have them), you'll get the right
> output.
> 
> 
> What I have learned from all this, is that "мир" ("mir") means "world"
> as well as "peace".  I find that an interesting linguistic titbit.

And, a space station. :-o

I think in these troublesome times (North Stream 2 etc., not to mention 
Hunter Biden's non-doings in Ukraine) it's worth mentioning, also a 
little off-topic, that, according to Wikipedia, Russia supported the 
American Revolution, and not just by later selling them Alaska. If not 
for Russia the American Revolution would IMO have failed, i.e., Russia 
helped to /create/ the United States. The first quoted sentence below is 
at best opinion presented as fact (unlike my 
opinion-presented-as-opinion), using words with negative connotations 
(again unlike mine), and it says a lot that it's there uncontested in 
Wikipedia, but anyway:


<<
Catherine the Great, a Russian empress who ruled from 1762–1796, played 
a modest role in the American Revolutionary War through her politicking 
with other European heads of state. Initially, the tsarina took a keen 
interest in the American struggle because it affected "English and 
European politics" and frankly believed that Britain was to blame for 
the conflict. She held a negative opinion of King George and his 
diplomats, often treating them with contempt. Nonetheless, the British 
crown still formally requested 20,000 troops in 1775 and sought an 
alliance.[20] She refused both pleas. Upon Spain's entry into the war, 
Britain once again turned to the Russian Empire, but this time, the 
English hoped for naval support. Catherine II once again ignored the 
British requests.

Perhaps Catherine the Great's greatest diplomatic contribution came from 
the creation and proclamation of the First League of Armed Neutrality in 
1780. This declaration of armed neutrality had several stipulations, but 
three crucial ones: first, "that neutral ships may freely visit the 
ports of belligerent Powers;" second, "that the goods of belligerent 
Powers on neutral ships are permitted to pass without hindrance, with 
the exception of war contraband;" and, third, "under the definition of a 
blockaded port falls only a port into which entry is actually hampered 
by naval forces." Most European nations agreed to these terms, but 
Britain refused to recognize the arrangement because it undermined the 
blockade, its most effective military strategy. After establishing a 
league of neutral parties, Catherine the Great attempted to act as a 
mediator between the United States and Britain by submitting a ceasefire 
plan. During her attempts at mediation, though, the Battle of Yorktown 
thwarted any hope of a peaceful and diplomatic solution to the American 
Revolutionary War.

These negotiations were accompanied by political intrigue. In 1780, 
during the period of Catherine II's mediating, Britain attempted to 
bribe the Russian Empire into an alliance. London offered St. Petersburg 
the island of Menorca if the Russians would agree to join the British in 
the war. Despite the economic boost such an acquisition offered, 
Catherine the Great refused this bribe and utilized it as an opportunity 
to make George III a laughing–stock of the European powers.
 >>


- Alf

[toc] | [prev] | [next] | [standalone]

#82866

On 29/01/2022 14:12, Alf P. Steinbach wrote:
> On 29 Jan 2022 13:01, David Brown wrote:
>> On 29/01/2022 07:42, Andrey Tarasevich wrote:
>>> On 1/28/2022 7:51 PM, Joseph Hesse wrote:
>>>> In the following program, I have written "hello world", copied from
>>>> the iternet, in Russian.  I don't understand Russian but I understand
>>>> their alphabet is different from the English alphabet.  The program is
>>>> supposed to output the original message and the characters in the
>>>> original message.  My question is why are they different?
>>>> Thank you,
>>>> Joe
>>>>
>>>> $ more Hello.cpp
>>>> #include <iostream>
>>>> using namespace std;
>>>>
>>>> int main()
>>>> {
>>>>     const wchar_t russian[] = L"Привет мир";  //Russian for "hello,
>>>> world"
>>>>
>>>>     wcout << russian << endl;
>>>>
>>>>     for(const wchar_t &x : russian)
>>>>       wcout << x << L", ";
>>>>     wcout << endl;
>>>> }
>>>> $ g++ -std=c++14 Hello.cpp
>>>> $ ./a.out
>>>> Privet mir                        <== original message
>>>> P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
>>>> $
>>>
>>> Um... How did you manage to make your program output `L"Привет мир"` as
>>>
>>>    Privet mir
>>>
>>> ?
>>>
>>
>> That is the Latin alphabet transliteration of the Cyrillic letters.  As
>> to why that conversion was done, I expect it is something to do with
>> locales and internationalisation libraries.
> 
> Correct.
> 
> The wide streams must translate to ASCII in that situation with no
> selected locale, i.e. with the default "C" locale.

I read your other post after I had written this one - it was very
informative.

> 
> 
>>  It doesn't work for all
>> non-ASCII characters, which is hardly surprising.  A quick test with
>> Norwegian letters "å", "ø" and "æ" resulted in "?", "?" and "ae".
>>
>> The simple answer, at least in the *nix world, is to drop "wchar_t"
>> completely and forget that disaster ever existed.
> 
> Agreed, including the qualification.
> 
> 
>> #include <iostream>
>> using namespace std;
>>
>> int main()
>> {
>>     const char russian[] = "Привет мир";  //Russian for "hello, world"
> 
> I would just write
> 
>     const auto& russian = "Привет мир";  //Russian for "hello, world"
> 
> ... preserving the length information, and not relying on the compiler
> to optimize away the copying, where the technically redundant `const` is
> for clarity and supports fast visual scanning of code.
> 
> In particular this supports forming a `constexpr stringview`.
> 

Sure.  I was just making the minor modification to the original to
remove the wide characters.

> 
>>     cout << russian << endl;
>>
>>     for (const char_t &x : russian)
>>       cout << x << L", ";
>>     cout << endl;
>> }
>>
>> As long as you have UTF8 locales and fonts (and you'd need an ancient
>> setup, or a very limited one, not to have them), you'll get the right
>> output.
>>
>>
>> What I have learned from all this, is that "мир" ("mir") means "world"
>> as well as "peace".  I find that an interesting linguistic titbit.
> 
> And, a space station. :-o

Yes - that's why I know it means "peace".

(I'm snipping the history - while I enjoy history, discussing it can run
the risk of mixing it with politics and getting into very off-topic
discussions.)

[toc] | [prev] | [next] | [standalone]

#82878

On 1/29/22 17:50, Andrey Tarasevich wrote:
> On 1/29/2022 9:19 AM, James Kuyper wrote:
[Note: I accidentally posted that message to comp.lang.c rather than
comp.lang.c++, breaking the thread. Andrey's response was also on that
newsgroup]
>> On 1/29/22 07:01, David Brown wrote:
>> ...
>>> What I have learned from all this, is that "мир" ("mir") means "world"
>>> as well as "peace".  I find that an interesting linguistic titbit.
>>
>> Which means that if a Russian leader says "Я хочу мир.", it's ambiguous
>> whether he's saying he wants peace, or he wants the world.
> 
> Not really.
> 
> The subject for "хотеть" can be placed in Accusative or Genitive case, 
> depending on the nature of the subject. This is the case for many words 
> that can used to refer to a specific object or to a some sort of 
> non-quantifiable concept or resource ("peace", "water", "fish" etc.).
> 
> "I want peace" would normally require "мир" in Genitive case, which is 
> "мира": "Я хочу мира".
> 
> If you attempt to use "Я хочу мир" in this meaning, people would 
> normally understand you properly (given enough context), but you'll 
> still be perceived as an "untrained" speaker (e.g. child, uneducated, 
> non-native, etc.)
> 
> "I want [the] world" would take "мир" in Accusative case, which is still 
> "мир": "Я хочу мир".
> 
> And this later version still doesn't sound right, since literary norms 
> usually require an extra verb specifying/qualifying the desire. Not "Я 
> хочу мир", but "Я хочу владеть миром" ("I want to own the world") или "Я 
> хочу править миром" ("I want to rule the world") etc.

I studied Russian for two years, about four decades ago, and never made
any practical use of the language. My Russian skills are sufficiently
rusty that I actually used translate.google.com to double-check before
posting. It translated "I want peace." and "I want the world." the same
way, which is a hint why translate.google.com shouldn't be used for
serious translation efforts.
However, I vaguely remember asking my Russian teacher about this same
point when I first learned about this word, and I don't remember her
answer mentioning any of the details that you just mentioned. I think
she just said something about being able to figure out the intended
meaning from context.

> I think that "I want [the] world" has the same feel in English as well. 
> It doesn't sound quite right without extra specification.

[toc] | [prev] | [next] | [standalone]

#82881

On 1/30/2022 11:17 AM, James Kuyper wrote:
> However, I vaguely remember asking my Russian teacher about this same
> point when I first learned about this word, and I don't remember her
> answer mentioning any of the details that you just mentioned. I think
> she just said something about being able to figure out the intended
> meaning from context.

Depends on how advanced your classes were and how "old-school" your 
teacher was. This is one of those intricate topics where, firstly, the 
choice between Accusative and Genitive is not always clearly defined. 
And secondly, modern users often tend to simplify the usage and just opt 
for Accusative in all cases.

See for example

http://www.abc-russian.com/2016/09/or.html

or section "Russian Genitive Case: Abstract Or Indefinite Objects" here:
https://storylearning.com/learn/russian/russian-tips/russian-genitive-case

-- 
Best regards,
Andrey Tarasevich

[toc] | [prev] | [next] | [standalone]

#82862

On 29 Jan 2022 04:51, Joseph Hesse wrote:
> In the following program, I have written "hello world", copied from the 
> iternet, in Russian.  I don't understand Russian but I understand their 
> alphabet is different from the English alphabet.  The program is 
> supposed to output the original message and the characters in the 
> original message.  My question is why are they different?
> Thank you,
> Joe
> 
> $ more Hello.cpp
> #include <iostream>
> using namespace std;
> 
> int main()
> {
>    const wchar_t russian[] = L"Привет мир";  //Russian for "hello, world"
> 
>    wcout << russian << endl;
> 
>    for(const wchar_t &x : russian)
>      wcout << x << L", ";
>    wcout << endl;
> }
> $ g++ -std=c++14 Hello.cpp
> $ ./a.out
> Privet mir                        <== original message
> P, r, i, v, e, t,  , m, i, r, ,   <== wchars in original message
> $

I reproduced your result in Ubuntu.

The reason this happens is:

* The wide streams translate to/from the external byte-oriented encoding.
* With no locale specified the external encoding wcout knows about is 
the one in the default "C" locale, namely pure ASCII.
* It uses a translation where symbols are mapped to phonetically similar 
characters in the result encoding.

Generally it's a pretty ungood idea to use the wide streams. Since they 
don't work in modern Windows (they get confused about the external 
encoding, in all implementations) they now have the direct opposite 
effect of the intended one. Namely, they make the code non-portable.

The modern way to do Russian or whatever output is to assume UTF-8:

     #include <iostream>
     using namespace std;

     auto main() -> int
     {
         const auto& russian = "Привет мир";     // Russian for "hello, 
world".
         cout << russian << endl;
     }

This works when it's built and invoked correctly, but unfortunately 
there's no error when it's not done correctly.

---

The general UTF-8 assumption can however be easily checked for the first 
two point below, about assuming UTF-8

* as the source encoding,
* as the C++ literal constants encoding (the "execution character set"), and
* as the external environment's byte oriented encoding.

The following code statically asserts that the two first points hold:

     #include <iostream>
     using namespace std;

     constexpr auto utf8_is_the_execution_character_set()
         -> bool
     {
         constexpr auto& slashed_o = "ø";
         return (sizeof( slashed_o ) == 3 and slashed_o[0] == '\xC3' and 
slashed_o[1] == '\xB8');
     }

     static_assert(
         utf8_is_the_execution_character_set(),
         "The execution character set must be UTF-8 (e.g. MSVC option 
\"/utf-8\")."
         );

     auto main() -> int
     {
         const auto& russian = "Привет мир";     // Russian for "hello, 
world".
         cout << russian << endl;
     }


You have to ensure the third aspect, that UTF-8 is the external 
encoding, in some way. In Linux it's the default. In Windows it can be 
ensured /for output/ by setting the console window's active codepage to 
65001, e.g. via a command such as `chcp 65001`, or programmatically.

---

Unfortunately, still as of Windows 11 console windows do not support 
input of UTF-8, other than the ASCII subset. :(

You can however use various 3rd party libraries to do portable UTF-8 input.

Those libraries include Boost Nowide and my own still under construction 
library code, currently called Kickstart. At the start of the Boost 
adoption process, when Nowide had been formally adopted but not yet 
released with Boost, the library still got Windows CRLF endings wrong, 
so it's perhaps not the highest quality library in the world. And 
Kickstart has been changing, e.g. filenames and file locations, about 
every time I've looked at it, in spite of my claims that /now/ it's 
pretty stable, so probably neither of them are suitable for commercial code.

The common theme is that these libraries require you to /explicitly/ use 
their UTF-8 input, instead of just overriding the behavior of `std::cin` 
or `stdin`. I believe because there are so many parties, including 
Microsoft, doing that that it gets real messy. I did that once, in an 
intended library, but it turned out that Microsoft changed things, 
introduced new sabotage-like stuff, faster than I could keep up.

And unfortunately the people in charge of the standardization of C++ 
just plain refuse to fix things, such as e.g. an ensured UTF-8 mode of 
the console i/o, and instead insist on re-educating C++ programmers into 
using modern `char8_t` instead of `char`. Like re-educating 
heterosexuals into adopting modern trans-sexual behavior. To this end in 
C++20 they broke all code that made use of critical functionality of 
`std::filesystem::path` (critical for Windows use, that is), and they 
were very much made aware of that and refused to reconsider, so it's 
very political, after 25 years, which means it will never get fixed.


- Alf

[toc] | [prev] | [next] | [standalone]

#82865

29.01.2022 12:48 Alf P. Steinbach kirjutas:

> At the start of the Boost 
> adoption process, when Nowide had been formally adopted but not yet 
> released with Boost, the library still got Windows CRLF endings wrong, 
> so it's perhaps not the highest quality library in the world.

Nowadays even Notepad has learned to cope with LF linebreaks, so there 
is essentially no reason any more to use CRLF anywhere at all. As it has 
happened with utf-8, Microsoft will be finally enforced to give up its 
stubbornness and join the sane world.

The whole notion of "text files" is seriously out of date. What's the 
point of having the file content in memory differ from the content on 
disk? Especially considering memory mapping, HTTP packets, cloud storage 
etc.

I predict the same will happen with backslashes as well, but this might 
take another 20 years. All the backslash madness in Windows is about to 
avoid typing a single space after the command name, it's about time to 
get rid of this.

[toc] | [prev] | [next] | [standalone]

#82868

On 29 Jan 2022 16:39, Paavo Helde wrote:
> 29.01.2022 12:48 Alf P. Steinbach kirjutas:
> 
>> At the start of the Boost adoption process, when Nowide had been 
>> formally adopted but not yet released with Boost, the library still 
>> got Windows CRLF endings wrong, so it's perhaps not the highest 
>> quality library in the world.
> 
> Nowadays even Notepad has learned to cope with LF linebreaks, so there 
> is essentially no reason any more to use CRLF anywhere at all.

Oh, just all the internet protocols, like NNTP. ;-)


> As it has 
> happened with utf-8, Microsoft will be finally enforced to give up its 
> stubbornness and join the sane world.
> 
> The whole notion of "text files" is seriously out of date. What's the 
> point of having the file content in memory differ from the content on 
> disk? Especially considering memory mapping, HTTP packets, cloud storage 
> etc.

My opinion is that C (and hence also C++) text mode is an abomination 
that should never have been introduced, and that in addition, given that 
it was introduced, it's designed in a stupid way with the data 
conversion applied underneath the buffer level so that one can't get a 
clear view of the raw data.

As an example, the design means that Unix `cat` can't be faithfully 
implemented in Windows using only standard C or C++, which IMO is extreme.

That said, from a C++ programming perspective data in memory is usually 
statically typed, while data on disk is untyped or effectively 
dynamically typed. In memory one knows that that thing is UTF-8 encoded 
text. On disk on doesn't know and must assume, where in Windows such 
assumptions can be partially checked (a good thing) via UTF-8 BOM.


> I predict the same will happen with backslashes as well, but this might 
> take another 20 years. All the backslash madness in Windows is about to 
> avoid typing a single space after the command name, it's about time to 
> get rid of this.

The Windows API level supports forward slashes, as did DOS before, and 
especially for `#include` directives they should be used, not backslash.

Why the Windows shells and applications generally don't support them is 
a mystery.

In some cases the fiction about what's allowed is imperfectly 
implemented. I remember in the 1990's (when I still worked) I had some 
fun demonstrating to colleagues how to completely and utterly hide some 
data on disk, using commands like


<<
[C:\root\temp]
 > type nul >poem.txt

[C:\root\temp]
 > echo "Very important secret!" > poem.txt:secret

[C:\root\temp]
 > dir | find "poem"
30 Jan 2022  09:14                 0 poem.txt

[C:\root\temp]
 > find /v "" < poem.txt:secret
"Very important secret!"
 >>


The file doesn't need to be empty, it can e.g. contain an actual poem if 
one feels like security by obscurity is a great thing.

This is just a bug in cmd.exe where it fails to check that the file name 
is "allowed" for ordinary users, so one is able to specify an internal 
NTFS stream. :-)


- Alf

[toc] | [prev] | [next] | [standalone]

#82870

"Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:

> My opinion is that C (and hence also C++) text mode is an abomination
> that should never have been introduced, and that in addition, given
> that it was introduced, it's designed in a stupid way with the data
> conversion applied underneath the buffer level so that one can't get a
> clear view of the raw data.
>
> As an example, the design means that Unix `cat` can't be faithfully
> implemented in Windows using only standard C or C++, which IMO is
> extreme.

I assume you are talking about the cases where cat defaults to reading
stdin and/or writing stdout, If so, it could be argued that it's not the
fault of the C and C++ standards, but more the fault of the
implementations not providing a useful freopen function.

But then maybe freopen simply can't be implemented in Windows for some
mysterious reason I don't get.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#82872

On 30 Jan 2022 13:12, Ben Bacarisse wrote:
> "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
> 
>> My opinion is that C (and hence also C++) text mode is an abomination
>> that should never have been introduced, and that in addition, given
>> that it was introduced, it's designed in a stupid way with the data
>> conversion applied underneath the buffer level so that one can't get a
>> clear view of the raw data.
>>
>> As an example, the design means that Unix `cat` can't be faithfully
>> implemented in Windows using only standard C or C++, which IMO is
>> extreme.
> 
> I assume you are talking about the cases where cat defaults to reading
> stdin and/or writing stdout, If so, it could be argued that it's not the
> fault of the C and C++ standards, but more the fault of the
> implementations not providing a useful freopen function.
> 
> But then maybe freopen simply can't be implemented in Windows for some
> mysterious reason I don't get.

I guess you're talking about using standard C++ code with some system 
specific knowledge such as how to specify the standard input stream as a 
file name.

To reopen the standard input so that it connects to the original source, 
which might a pipe or a file or the console, one needs to (1) identify 
that source, and (2) open that source, possibly after closing the 
original connection. Neither is feasible in general, even (AFAIK) in 
Unix environment. But one might attempt to sidestep (1) by using a 
filename that in the relevant OS denotes standard input.

A demonstration of an approach that fails is not a proof that no 
solution exists, e.g. failure to open a door doesn't prove that the door 
is stuck. Maybe the person just failed to consider using the doorknob, 
dragged the door instead of pushing, failed to note that it opens 
sideways, didn't swipe the id card, deliberately made it look as if the 
door didn't open, or something. But anyway I cooked up some code:


#include <stdlib.h>     // EXIT_...
#include <stdio.h>

#include <stdexcept>    // std::runtime_error, std::exception
#include <string>       // std::string
using namespace std;

#ifdef PORTABLE
     constexpr bool portable = true;
#else
     constexpr bool portable = false;
#endif

auto fail( const string& s ) -> bool { throw runtime_error( s ); }

void cpp_main()
{
     if constexpr( portable ) {
         freopen( nullptr, "rb", stdin ) or fail( "freopen(stdin) failed" );
         freopen( nullptr, "wb", stdout ) or fail( "freopen(stdout) 
failed" );
     } else {
         freopen( "conin$", "rb", stdin ) or fail( "freopen(stdin) 
failed" );
         freopen( "conout$", "wb", stdout ) or fail( "freopen(stdout) 
failed" );
     }
}

auto main() -> int
{
     fprintf( stderr, "%s\n", (portable? "Portable code" : 
"Windows-specific code") );
     try {
         cpp_main();
         return EXIT_SUCCESS;
     } catch( const exception& x ) {
         fprintf( stderr, "!%s\n", x.what() );
     }
     return EXIT_FAILURE;
}


It fails both as both portable code and as Windows-specific code (here 
"cl" is the Visual C++ compiler):


[C:\root\temp]
 > cl x.cpp
x.cpp

[C:\root\temp]
 > x
Windows-specific code

[C:\root\temp]
 > echo blah | x
Windows-specific code
The process tried to write to a nonexistent pipe.

[C:\root\temp]
 > cl x.cpp /D PORTABLE
x.cpp

[C:\root\temp]
 > x
Portable code
!freopen(stdin) failed


So.

- Alf

[toc] | [prev] | [next] | [standalone]

#82874

"Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:

> On 30 Jan 2022 13:12, Ben Bacarisse wrote:
>> "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
>> 
>>> My opinion is that C (and hence also C++) text mode is an abomination
>>> that should never have been introduced, and that in addition, given
>>> that it was introduced, it's designed in a stupid way with the data
>>> conversion applied underneath the buffer level so that one can't get a
>>> clear view of the raw data.
>>>
>>> As an example, the design means that Unix `cat` can't be faithfully
>>> implemented in Windows using only standard C or C++, which IMO is
>>> extreme.
>> I assume you are talking about the cases where cat defaults to reading
>> stdin and/or writing stdout, If so, it could be argued that it's not the
>> fault of the C and C++ standards, but more the fault of the
>> implementations not providing a useful freopen function.
>> But then maybe freopen simply can't be implemented in Windows for some
>> mysterious reason I don't get.
>
> I guess you're talking about using standard C++ code with some system
> specific knowledge such as how to specify the standard input stream as
> a file name.

No.  I was just talking about using freopen.

> To reopen the standard input so that it connects to the original
> source, which might a pipe or a file or the console, one needs to (1)
> identify that source, and (2) open that source, possibly after closing
> the original connection.

I don't see why you need to do that.  freopen (were it fully supported)
would allow a program to change the mode of stdin and stdout without
knowing anything about the sources.

<snip code>
> Portable code
> !freopen(stdin) failed
>
> So.

Not sure what the "so" is about.  You appeared to be blaming the C and
C++ standard for not allowing cat to be written on Windows.  I blame the
Windows standard library for not supporting freopen.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#82879

On 30 Jan 2022 15:45, Ben Bacarisse wrote:
> 
> Not sure what the "so" is about.  You appeared to be blaming the C and
> C++ standard for not allowing cat to be written on Windows.  I blame the
> Windows standard library for not supporting freopen.

No, I used it as an example of a consequence of the unreasonable design.

With a more sane design with access to the raw bytes, the problem 
wouldn't have existed, regardless of buggy or perfect `freopen`.

Still you have a point in that I gave that example based on just old 
memories. So now I added code to actually copy input to output (getchar 
+ putchar). Still fails in Windows with both Visual C++ and MinGW g++ 
for both portable and Windows-specific `freopen` args, but works in WSL 
Ubuntu for portable:


[C:\root\temp]
 > bash
alf@Alf-Windows-PC:/mnt/c/root/temp$ rm *.exe
alf@Alf-Windows-PC:/mnt/c/root/temp$ g++ -std++17 x.cpp
g++: error: unrecognized command line option ‘-std++17’; did you mean 
‘-std=c++17’?
alf@Alf-Windows-PC:/mnt/c/root/temp$ g++ -std=c++17 x.cpp
alf@Alf-Windows-PC:/mnt/c/root/temp$ echo blah | ./a.out
Windows-specific code
!freopen(stdin) failed
alf@Alf-Windows-PC:/mnt/c/root/temp$ g++ -std=c++17 x.cpp -D PORTABLE
alf@Alf-Windows-PC:/mnt/c/root/temp$ echo blah | ./a.out
Portable code
blah


That's sort of a Pyrrhic victory for the standard library design, that 
this thing that could save the day in Windows or at least be claimed to 
be the intended way to do things, works in Ubuntu where it's not needed.

Anyway. :-)


- Alf [too lazy to fix the Ubuntu prompt]

[toc] | [prev] | [next] | [standalone]

#82880

"Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:

> On 30 Jan 2022 15:45, Ben Bacarisse wrote:
>> Not sure what the "so" is about.  You appeared to be blaming the C and
>> C++ standard for not allowing cat to be written on Windows.  I blame the
>> Windows standard library for not supporting freopen.
>
> No, I used it as an example of a consequence of the unreasonable
> design.

Yes, and my point was that the inability to write "cat" on Windows is
not just down the CR/LF translation design.  A mechanism was added to
mitigate some of the problems, but it's not been taken up.  There is
blame to share around.

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#82877

On 1/30/22 08:02, Alf P. Steinbach wrote:
> On 30 Jan 2022 13:12, Ben Bacarisse wrote:
>> "Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
>>
>>> My opinion is that C (and hence also C++) text mode is an abomination
>>> that should never have been introduced, and that in addition, given
>>> that it was introduced, it's designed in a stupid way with the data
>>> conversion applied underneath the buffer level so that one can't get a
>>> clear view of the raw data.
>>>
>>> As an example, the design means that Unix `cat` can't be faithfully
>>> implemented in Windows using only standard C or C++, which IMO is
>>> extreme.
>>
>> I assume you are talking about the cases where cat defaults to reading
>> stdin and/or writing stdout, If so, it could be argued that it's not the
>> fault of the C and C++ standards, but more the fault of the
>> implementations not providing a useful freopen function.
>>
>> But then maybe freopen simply can't be implemented in Windows for some
>> mysterious reason I don't get.
>
> I guess you're talking about using standard C++ code with some system
> specific knowledge such as how to specify the standard input stream as
> a file name.
>
> To reopen the standard input so that it connects to the original
> source, which might a pipe or a file or the console, one needs to (1)
> identify that source, and (2) open that source, possibly after closing
> the original connection. Neither is feasible in general, even (AFAIK)
> in Unix environment. But one might attempt to sidestep (1) by using a
> filename that in the relevant OS denotes standard input.

No, freopen doesn't require a file name, or any of those other things.

"If filename is a null pointer, the freopen function attempts to change
the mode of the stream to that specified by mode , as if the name of the
file currently associated with the stream had been used. It is
implementation-defined which changes of mode are permitted (if any), and
under what circumstances." (C standard, 7.21.5.4p3).

If the implementation you're using fails to support changing text mode
to binary mode, that's hardly the fault of the C standard, that's up to
the implementation.

[toc] | [prev] | [next] | [standalone]

#82882

On Sun, 30 Jan 2022 12:12:33 +0000
Ben Bacarisse <ben.usenet@bsb.me.uk> wrote:
>"Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
>
>> My opinion is that C (and hence also C++) text mode is an abomination
>> that should never have been introduced, and that in addition, given
>> that it was introduced, it's designed in a stupid way with the data
>> conversion applied underneath the buffer level so that one can't get a
>> clear view of the raw data.
>>
>> As an example, the design means that Unix `cat` can't be faithfully
>> implemented in Windows using only standard C or C++, which IMO is
>> extreme.
>
>I assume you are talking about the cases where cat defaults to reading
>stdin and/or writing stdout, If so, it could be argued that it's not the
>fault of the C and C++ standards, but more the fault of the
>implementations not providing a useful freopen function.

I see no reason why cat would need freopen because it probably uses low level
I/O anyway and doesn't care where its stdin is coming from if it has to read
from it. Other utilities (eg ls) otoh will use isatty() on stdout to see if
its connected to a terminal or a pipe/file and act accordingly (ls only formats
it output if its going to a terminal).

[toc] | [prev] | [next] | [standalone]

#82884

Muttley@dastardlyhq.com writes:
>On Sun, 30 Jan 2022 12:12:33 +0000
>Ben Bacarisse <ben.usenet@bsb.me.uk> wrote:
>>"Alf P. Steinbach" <alf.p.steinbach@gmail.com> writes:
>>
>>> My opinion is that C (and hence also C++) text mode is an abomination
>>> that should never have been introduced, and that in addition, given
>>> that it was introduced, it's designed in a stupid way with the data
>>> conversion applied underneath the buffer level so that one can't get a
>>> clear view of the raw data.
>>>
>>> As an example, the design means that Unix `cat` can't be faithfully
>>> implemented in Windows using only standard C or C++, which IMO is
>>> extreme.
>>
>>I assume you are talking about the cases where cat defaults to reading
>>stdin and/or writing stdout, If so, it could be argued that it's not the
>>fault of the C and C++ standards, but more the fault of the
>>implementations not providing a useful freopen function.
>
>I see no reason why cat would need freopen because it probably uses low level
>I/O anyway and doesn't care where its stdin is coming from if it has to read
>from it.

cat(1) uses read(2)/write(2) unless -v is specified, in which case it uses
getc(3).

int
cat(fi, fname)
FILE *fi;
char *fname;
{
        register int fi_desc;
        register int nitems;

        fi_desc = fileno(fi);

        /*
         * While not end of file, copy blocks to stdout. 
         */

        while ((nitems=read(fi_desc,buffer,BUFSIZ))  > 0) {
                if ((errnbr = write(1,buffer,(unsigned)nitems)) != nitems) {

...

vcat(fi)
FILE    *fi;
{
        register int c;

        while ((c = getc(fi)) != EOF) 
        {
                /*
                 * For non-printable and non-cntrl  chars, use the "M-x" notation.
                 */
                if (!ISPRINT(c, wp) &&
                    !iscntrl(c) && !ISSET2(c) && !ISSET3(c))
                        {
                        putchar('M');
                        putchar('-');
                        c-= 0200;
                        }
...

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.c++

csiph-web