Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #79644 > unrolled thread

python student at university of jordan.

Started byheba abukaff <habukaff50@yahoo.com>
First post2014-10-24 18:47 -0700
Last post2014-10-25 15:33 +1100
Articles 2 — 2 participants

Back to article view | Back to comp.lang.python


Contents

  python student at university of jordan. heba abukaff <habukaff50@yahoo.com> - 2014-10-24 18:47 -0700
    Re: python student at university of jordan. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-10-25 15:33 +1100

#79644 — python student at university of jordan.

Fromheba abukaff <habukaff50@yahoo.com>
Date2014-10-24 18:47 -0700
Subjectpython student at university of jordan.
Message-ID<mailman.15152.1414201665.18130.python-list@python.org>
Hi,
my name is heba ibrahim abukaff from jordan ,iam a computer information system student at university of jordan . 
i have a trouble using the tokenizer to find the frequency list for URL using arabic text.and iam using python 2.7.2 on winXP,I tried this code but every time i run the code appears error with first line 
COULD YOU HELP ME.
WITH REGARDS.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
> import re, codecs
import nltk 

from urllib import request
url = "http://ar.wikipedia.org/wiki/%D9%85%D9%88%D9%82%D8%B9_%D9%88%D9%8A%D8%A8"
response = request.urlopen(url)
raw = response.read().decode('utf8')
print(raw)

import re, codecs import nltk from nltk.probability
import * def construct_frequency_list(words):
    
outfile = codecs.open(r'raw.txt' , 'w')
fd = nltk.probability.FreqDist()
for w in words: fd.inc(w)
print "Total number of words: %d Vocbulary size : %d" % (fd.N(), fd.B())
print "word with highest count: %s" % (fd.max())
tokenlist = fd.iteritems()
for (key, value) in tokenlist:
print>>outfile, "%s\t%d" % (key, value)
outfile.close() def read_textfile(raw ):
lines = codecs.open(raw ,'r','utf_8').readlines()
outfile = codecs.open(r'raw.txt' ,'w','utf_8')
counter = 0 wordlist = [] for line in lines:
tokens = line.rstrip().lstrip().split()
for t in tokens: wordlist.append(t)
print>>outfile, '%s\t%d' % (t, len(t))
counter += len(t) outfile.close()
return wordlist if __name__ == "__main__":
 words = read_textfile(r'raw .txt')
construct_frequency_list(words)
print "Done!"
 

[toc] | [next] | [standalone]


#79653

FromSteven D'Aprano <steve+comp.lang.python@pearwood.info>
Date2014-10-25 15:33 +1100
Message-ID<544b2838$0$12975$c3e8da3$5496439d@news.astraweb.com>
In reply to#79644
heba abukaff wrote:

> Hi,
> my name is heba ibrahim abukaff from jordan ,iam a computer information
> system student at university of jordan . i have a trouble using the
> tokenizer to find the frequency list for URL using arabic text.and iam
> using python 2.7.2 on winXP,I tried this code but every time i run the
> code appears error with first line COULD YOU HELP ME. 

We can only help you if you help us. We cannot guess the error that you get,
you have to show us. Copy and paste the complete error message, starting
from the word "Traceback" to the end.

Given the code you showed:

[quote]
> import re, codecs
import nltk
[end quote]

Notice the greater-than sign > at the beginning of the "import re, codecs"
line? If that greater-than sign is actually in your program, you would get
a syntax error:

py> > import re, codecs
  File "<stdin>", line 1
    > import re, codecs
    ^
SyntaxError: invalid syntax


If that is not the error message you are getting, you will need to tell us
what error you actually are getting.


Regards,



-- 
Steven

[toc] | [prev] | [standalone]


Back to top | Article view | comp.lang.python


csiph-web