Newsgroups: perl.ldap
Path: csiph.com!xmission!news.glorb.com!usenet.stanford.edu!nntp.perl.org
Xref: csiph.com perl.ldap:626
Mailing-List: contact perl-ldap-help@perl.org; run by ezmlm
To: perl-ldap@perl.org
Subject: Re: (Net::LDAP) Automatically convert attributes into utf8 when writting
Date: Sat, 29 Aug 2015 13:54:15 +0200
Message-ID: <1722046.SSPNH8pE0O@moth>
Organization: ADPM
User-Agent: KMail/4.14.2 (Linux/4.1.0-1-amd64; KDE/4.14.2; x86_64; ; )
In-Reply-To: <668301440502635@web4g.yandex.ru>
References: <3262561440500066@web15g.yandex.ru> <55DC4BB5.6030501@keutel.de> <668301440502635@web4g.yandex.ru>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
Approved: news@nntp.perl.org
From: peter@adpm.de (Peter Marschall)

Hi,

On Tuesday, 25. August 2015 13:37:15 pe rl wrote:
> They are not necessary when reading/searching in the ldap server, since
> Net::LDAP already has a "raw" option in the constructor to automatically
> encode/decode strings. It is working for us, and the only change required
> has been to add the "raw" option to the constructor.

I think you misinterpret the purpose of the raw option.

Its goal is to convert the byte strings coming from the LDAP server that 
represent UTF-8 encoded directory strings from byte semantics to
Perl scalars with character semantics.

On the other hand, perl-ldap expects scalars in character semantics when
it comes to writing directory strings to an LDAP server.

It is not perl-ldap's job to translate between scalars in Perl's character 
semantics and various input or output encodings of your application.


> The problem appears when writting to the ldap server. I have started to
> modify our code with utf8::encode(), by adding it to every attribute in all
> of our functions. The problem is that it is very inefficient, since I will
> have to modify every attribute that appears in our programs. We have a lot
> of functions that create/modify/delete entries in the ldap server, so I
> will have to change a lot of code to manually encode attribs to utf8, and
> then test all of the changes.

It is not perl-ldap's job to translate between scalars in Perl's character 
semantics and various input or output encodings of your application.

This is the application's task.
If you - as you write - need to convert every attribute using ut8::encode(),
then your application seems to use a mixture of byte & character semantics.

In that case please do yourself a favour and switch over to character 
semantics by correctly converting input to character semantics when it 
happens:
- for file & console input you can use the ":encoding(...)" layer to make
  sure you get character semantics instead of byte semantics
- for @ARGV a simple
     $_ = Encode::decode('UTF-8' ,$_) for @ARGV;
  should be sufficient.

You may also have a look at the 'utf8::all' package that does a lot of the 
above for you automatically.

Please read the perlunicode manual page for more detailed information.

Best
PEter

-- 
Peter Marschall
peter@adpm.de