Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail From: Peter Otten <__peter__@web.de> Newsgroups: comp.lang.python Subject: Re: x=something, y=somethinelse and z=crud all likely to fail - how do i wrap them up Date: Sun, 31 Jan 2016 11:40:12 +0100 Organization: None Lines: 96 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 7Bit X-Trace: news.uni-berlin.de ZfuHUBfoyXZTmO12G36Fjg1dhfFCSUgKWCpnkIpVFRNA== Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'else:': 0.03; 'skip:[ 20': 0.03; 'failing': 0.05; 'none:': 0.05; 'none)': 0.07; "'no": 0.09; 'none.': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'result)': 0.09; 'skip:[ 40': 0.09; 'jan': 0.11; 'def': 0.13; '(there': 0.16; '2016': 0.16; 'chained': 0.16; 'doing:': 0.16; 'found"': 0.16; 'function?': 0.16; 'parameter,': 0.16; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:io': 0.16; 'received:plane.gmane.org': 0.16; 'received:psf.io': 0.16; 'received:t-ipconnect.de': 0.16; 'result:': 0.16; 'time.time()': 0.16; 'wrote:': 0.16; 'string': 0.17; 'skip:g 40': 0.18; 'skip:l 30': 0.18; 'stick': 0.18; 'try:': 0.18; '>>>': 0.20; 'skip:" 40': 0.20; '31,': 0.22; 'libraries': 0.22; 'ones.': 0.22; 'parse': 0.22; 'parsing': 0.22; 'pass': 0.22; 'skip:l 40': 0.23; 'header :User-Agent:1': 0.26; "doesn't": 0.26; 'header:X-Complaints-To:1': 0.26; 'chris': 0.26; 'skip:" 20': 0.26; 'function': 0.28; 'idea': 0.28; "skip:' 10": 0.28; 'initial': 0.28; '"no': 0.29; 'typically': 0.29; "i'm": 0.30; 'skip:g 30': 0.30; 'skip:[ 10': 0.31; "i'd": 0.31; 'option': 0.31; 'statement': 0.32; 'subject:all': 0.32; 'wrap': 0.33; 'except': 0.34; 'that,': 0.34; 'fail': 0.35; 'something': 0.35; 'but': 0.36; 'instead': 0.36; 'there': 0.36; 'url:org': 0.36; 'skip:{ 10': 0.36; 'to:addr :python-list': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'received:org': 0.37; 'doing': 0.38; 'stuff': 0.38; 'several': 0.38; 'sure': 0.39; 'to:addr:python.org': 0.40; 'received:de': 0.40; 'some': 0.40; 'your': 0.60; 'default': 0.61; 'skip:n 10': 0.62; 'different': 0.63; 'here': 0.66; 'price': 0.69; 'obvious': 0.76; 'chrisa': 0.84; 'pain': 0.84; 'shipping,': 0.84; 'that),': 0.91; 'try.': 0.91 X-Injected-Via-Gmane: http://gmane.org/ X-Gmane-NNTP-Posting-Host: p57bd88cd.dip0.t-ipconnect.de User-Agent: KNode/4.13.3 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.20+ Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Xref: csiph.com comp.lang.python:102344 Veek. M wrote: > Chris Angelico wrote: > >> On Sun, Jan 31, 2016 at 3:58 PM, Veek. M wrote: >>> I'm parsing html and i'm doing: >>> >>> x = root.find_class(... >>> y = root.find_class(.. >>> z = root.find_class(.. >>> >>> all 3 are likely to fail so typically i'd have to stick it in a try. >>> This is a huge pain for obvious reasons. >>> >>> try: >>> .... >>> except something: >>> x = 'default_1' >>> (repeat 3 times) >>> >>> Is there some other nice way to wrap this stuff up? >> >> I'm not sure what you're using to parse HTML here (there are several >> libraries for doing that), but the first thing I'd look for is an >> option to have it return a default if it doesn't find something - even >> if that default has to be (say) None. >> >> But failing that, you can always write your own wrapper: >> >> def find_class(root, ...): >> try: >> return root.find_class(...) >> except something: >> return 'default_1' >> >> Or have the default as a parameter, if it's different for the different >> ones. >> >> ChrisA > > I'm using lxml.html > > def parse_page(self, root): > for li_item in root.xpath('//li[re:test(@id, "^item[a-z0-9]+$")]', > namespaces={'re': "http://exslt.org/regular-expressions"}): > description = li_item.find_class('vip')[0].text_content() > link = li_item.find_class('vip')[0].get('href') > price_dollar = li_item.find_class('lvprice prc') > [0].xpath('span')[0].text > bids = li_item.find_class('lvformat')[0].xpath('span')[0].text > > tme_time = li_item.find_class('tme')[0].xpath('span') > [0].get('timems') > if tme_time: > time_hrs = int(tme_time)/1000 - time.time() > else: > time_hrs = 'No time found' > > shipping = li_item.find_class('lvshipping') > [0].xpath('span/span/span')[0].text_content()" > > print('{} {} {} {} {}'.format(link, price_dollar, time_hrs, > shipping, bids)) > print('-----------------------------------------------------------------') When you use XPath instead of the chained function calls your initial > Pass the statement as a string to a try function? idea works out naturally: def parse_page(self, root): def get_xpath(path, default=""): result = li_item.xpath(path) if result: return " ".join(part.strip() for part in result) return default for li_item in root.xpath( '//li[re:test(@id, "^item[a-z0-9]+$")]', namespaces={'re': "http://exslt.org/regular-expressions"}): description = get_xpath("*[@class='vip']//text()") link = get_xpath("*[@class='vip']/@href") price = get_xpath("*[@class='lvprice prc']/span/text()") bids = get_xpath("*[@class='lvformat']/span/text()") tme_time = get_xpath("*[@class='tme']/span/@timems", None) if tme_time is not None: time_hrs = int(tme_time)/1000 - time.time() else: time_hrs = "No time found" shipping = get_xpath( "*[@class='lvshipping']/span/span/span//text()")