Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'example:': 0.03; 'output': 0.04; 'root': 0.04; '-*-': 0.07; 'subject:Question': 0.07; 'python': 0.09; 'enum': 0.09; 'subject:python': 0.11; 'from:addr:mrabarnett.plus.com': 0.16; 'from:addr:python': 0.16; 'from:name:mrab': 0.16; 'message-id:@mrabarnett.plus.com': 0.16; 'received:84.93': 0.16; 'received:84.93.230': 0.16; 'wrote:': 0.17; 'unicode': 0.17; 'input': 0.18; 'import': 0.21; 'example': 0.23; 'header:In-Reply-To:1': 0.25; 'header:User-Agent:1': 0.26; 'code': 0.31; 'received:84': 0.32; 'problem': 0.33; 'to:addr :python-list': 0.33; 'code:': 0.33; 'skip:u 20': 0.36; 'but': 0.36; 'should': 0.36; 'subject:: ': 0.38; 'skip:l 20': 0.38; 'to:addr:python.org': 0.39; 'received:192': 0.39; 'skip:" 10': 0.40; 'received:192.168': 0.40; 'skip:u 10': 0.60; 'remove': 0.61; 'email addr:gmail.com': 0.63; 'header:Reply-To:1': 0.68; 'reply- to:no real name:2**0': 0.72; 'flag.': 0.84; 'reply- to:addr:python.org': 0.84
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.0 cv=HO4d4PRv c=1 sm=1 a=0nF1XD0wxitMEM03M9B4ZQ==:17 a=_vX2mLoUpDQA:10 a=ZR4WZwmrTWUA:10 a=ihvODaAuJD4A:10 a=OUOv7kDek9cA:10 a=IkcTkHD0fZMA:10 a=EBOSESyhAAAA:8 a=8AHkEIZyAAAA:8 a=BiAti3QkpXgA:10 a=pGLkceISAAAA:8 a=NEPcynR3lX8ggcKH2IAA:9 a=QEXdDO2ut3YA:10 a=MSl-tDqOz04A:10 a=0nF1XD0wxitMEM03M9B4ZQ==:117
X-AUTH: mrabarnett:2500
Date: Sun, 03 Mar 2013 19:01:30 +0000
From: MRAB <python@mrabarnett.plus.com>
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130215 Thunderbird/17.0.3
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Question about Tashaphyne package in python
References: <0d0a40de-052c-49df-b43e-dff935071b49@googlegroups.com>
In-Reply-To: <0d0a40de-052c-49df-b43e-dff935071b49@googlegroups.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: python-list@python.org
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.2825.1362337289.2939.python-list@python.org>
Lines: 35
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1362337289 news.xs4all.nl 6876 [2001:888:2000:d::a6]:44355
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:40419

On 2013-03-03 03:06, yomnasalah91@gmail.com wrote:
> I have a Python code that take an Arabic word and get the root and also remove diacritics, but i I have a problem with the output. For example  : when the input is "العربيه" the output is:"عرب" which is right answer but when the input is "كاتب" the output is:"ب", and when the input is "يخاف" the output is " خف".
>
> This is my code:
>
> # -*- coding=utf-8 -*-
>
> import re
> from arabic_const import *
> import Tashaphyne
> from Tashaphyne import *
> import enum
> from enum import Enum
> search_type=Enum('unvoc_word','voc_word','root_word')
>
> HARAKAT_pat = re.compile(ur"[" + u"".join([FATHATAN, DAMMATAN, KASRATAN, FATHA, DAMMA, KASRA, SUKUN, SHADDA]) + u"]")
> HAMZAT_pat = re.compile(ur"[" + u"".join([WAW_HAMZA, YEH_HAMZA]) + u"]");
> ALEFAT_pat = re.compile(ur"[" + u"".join([ALEF_MADDA, ALEF_HAMZA_ABOVE, ALEF_HAMZA_BELOW, HAMZA_ABOVE, HAMZA_BELOW]) + u"]");
> LAMALEFAT_pat = re.compile(ur"[" + u"".join([LAM_ALEF, LAM_ALEF_HAMZA_ABOVE, LAM_ALEF_HAMZA_BELOW, LAM_ALEF_MADDA_ABOVE]) + u"]");
>
[snip]
When you're using Unicode with re in Python 2, you should include the
re.UNICODE flag. For example:

HARAKAT_pat = re.compile(ur"[" + u"".join([FATHATAN, DAMMATAN, KASRATAN, 
FATHA, DAMMA, KASRA, SUKUN, SHADDA]) + u"]", flags=re.UNICODE)

or:

HARAKAT_pat = re.compile(ur"(?u)[" + u"".join([FATHATAN, DAMMATAN, 
KASRATAN, FATHA, DAMMA, KASRA, SUKUN, SHADDA]) + u"]")

I don't know whether that will make a difference in this case because I
don't know Tashaphyne or Arabic.