Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #32850 > unrolled thread

java.util.Scanner hasNext returns true, next throws NoSuchElementException

Started bylndresnick@gmail.com
First post2015-10-09 07:12 -0700
Last post2015-10-27 19:25 +0100
Articles 5 — 3 participants

Back to article view | Back to comp.lang.java.programmer

Contents

  java.util.Scanner hasNext returns true, next throws NoSuchElementException lndresnick@gmail.com - 2015-10-09 07:12 -0700
    Re: java.util.Scanner hasNext returns true, next throws NoSuchElementException Jeff Higgins <jeff@invalid.invalid> - 2015-10-09 11:16 -0400
      Re: java.util.Scanner hasNext returns true, next throws NoSuchElementException lndresnick@gmail.com - 2015-10-09 08:29 -0700
        Re: java.util.Scanner hasNext returns true, next throws NoSuchElementException lndresnick@gmail.com - 2015-10-27 09:57 -0700
    Re: java.util.Scanner hasNext returns true, next throws NoSuchElementException Jan Burse <janburse@fastmail.fm> - 2015-10-27 19:25 +0100

#32850 — java.util.Scanner hasNext returns true, next throws NoSuchElementException

Fromlndresnick@gmail.com
Date2015-10-09 07:12 -0700
Subjectjava.util.Scanner hasNext returns true, next throws NoSuchElementException
Message-ID<f0815dfd-0228-4a1a-af8d-40034de9aeb1@googlegroups.com>
Debugging an exception in my code, found out that under certain conditions java.util.Scanner hasNext() returns true, but next() throws NoSuchElementException. I submitted a bug against java to Oracle for this today, but throwing it out here so folks searching can find.  Very happy to hear if I'm wrong, any comments welcome.  But the documented interface is that if hasNext() returns true, there is a token available.  For next(), the documented interface is that if there is no token available, NoSuchElementException is thrown.  In the code below, NoSuchElementException is thrown for input lengths of 1019, 1022, 2046, 4094, and 8190 (plus added delimiter).  This bug appears to be triggered if a fuzzy matched delimiter crosses the internal java.util.Scanner buffer size of 1024

Easy enough to work around, mind you.  Just catch the exception locally and consider it "success".

Code follows:

import java.util.Scanner;
import java.util.NoSuchElementException;

public class Boundary {

    static private boolean parse(String data, String boundary) {
        String dataCopy = new String(data);
        try {
            Scanner scanner = new Scanner(dataCopy);
            try {
                scanner.useDelimiter("-" + boundary + "(-)?");

                while (scanner.hasNext()) {
                    String bodyPart = scanner.next();
                }
            } finally {
                if (scanner != null)
                    scanner.close();
            }
        } catch (NoSuchElementException e) {
            System.out.println("Caught NoSuchElementException " + e);
            e.printStackTrace();
            return false;
        }
		
        return true;
    }
    
    public static void main (String argv[]) {
        final String boundary = ";";

        for ( int i = 0; i < 10000; ++i ) {
            StringBuilder data = new StringBuilder();
            data.append("--" + boundary);
            for (int j = 0; j< i; ++j) {
                data.append(j%10);
            }

            data.append("-" + boundary + "-");
           
            boolean ret=parse(data.toString(), boundary);
            if (!ret) { 
                System.out.println("parse returned " + ret + " i=" +i);
            }
        }
    }
}

[toc] | [next] | [standalone]

#32852

FromJeff Higgins <jeff@invalid.invalid>
Date2015-10-09 11:16 -0400
Message-ID<mv8lmj$bqg$1@dont-email.me>
In reply to#32850
On 10/09/2015 10:12 AM, lndresnick@gmail.com wrote:
> Debugging an exception in my code, found out that under certain conditions java.util.Scanner hasNext() returns true, but next() throws NoSuchElementException. I submitted a bug against java to Oracle for this today, but throwing it out here so folks searching can find.

Just searched Oracle bugs database for Scanner. Several bugs seem to
describe the behavior you describe. Didn't find a bug report for a
Scanner dated today.

   Very happy to hear if I'm wrong, any comments welcome.  But the 
documented interface is that if hasNext() returns true, there is a token 
available.  For next(), the documented interface is that if there is no 
token available, NoSuchElementException is thrown.  In the code below, 
NoSuchElementException is thrown for input lengths of 1019, 1022, 2046, 
4094, and 8190 (plus added delimiter).  This bug appears to be triggered 
if a fuzzy matched delimiter crosses the internal java.util.Scanner 
buffer size of 1024
>
> Easy enough to work around, mind you.  Just catch the exception locally and consider it "success".
>
> Code follows:
>
> import java.util.Scanner;
> import java.util.NoSuchElementException;
>
> public class Boundary {
>
>      static private boolean parse(String data, String boundary) {
>          String dataCopy = new String(data);
>          try {
>              Scanner scanner = new Scanner(dataCopy);
>              try {
>                  scanner.useDelimiter("-" + boundary + "(-)?");
>
>                  while (scanner.hasNext()) {
>                      String bodyPart = scanner.next();
>                  }
>              } finally {
>                  if (scanner != null)
>                      scanner.close();
>              }
>          } catch (NoSuchElementException e) {
>              System.out.println("Caught NoSuchElementException " + e);
>              e.printStackTrace();
>              return false;
>          }
> 		
>          return true;
>      }
>
>      public static void main (String argv[]) {
>          final String boundary = ";";
>
>          for ( int i = 0; i < 10000; ++i ) {
>              StringBuilder data = new StringBuilder();
>              data.append("--" + boundary);
>              for (int j = 0; j< i; ++j) {
>                  data.append(j%10);
>              }
>
>              data.append("-" + boundary + "-");
>
>              boolean ret=parse(data.toString(), boundary);
>              if (!ret) {
>                  System.out.println("parse returned " + ret + " i=" +i);
>              }
>          }
>      }
> }
>

[toc] | [prev] | [next] | [standalone]

#32853

Fromlndresnick@gmail.com
Date2015-10-09 08:29 -0700
Message-ID<6ca929bc-5bd0-4c1e-84a2-aa288a3f6973@googlegroups.com>
In reply to#32852
On Friday, October 9, 2015 at 11:17:44 AM UTC-4, Jeff Higgins wrote:
> On 10/09/2015 10:12 AM, lndresnick@gmail.com wrote:
> > Debugging an exception in my code, found out that under certain conditions java.util.Scanner hasNext() returns true, but next() throws NoSuchElementException. I submitted a bug against java to Oracle for this today, but throwing it out here so folks searching can find.
> 
> Just searched Oracle bugs database for Scanner. Several bugs seem to
> describe the behavior you describe. Didn't find a bug report for a
> Scanner dated today.
> 

Thanks.  I submitted the bug and got an auto-reply.  Based on that auto-reply, I gather they triage submissions before adding to actual bug DB. 

I found in the Oracle defect database 3 java.util.Scanner bugs related to the internal buffer size, but not quite this one (i.e. none have the NoSuchElementException).

[toc] | [prev] | [next] | [standalone]

#32940

Fromlndresnick@gmail.com
Date2015-10-27 09:57 -0700
Message-ID<033473dd-d4cb-406a-a5cd-17873610885b@googlegroups.com>
In reply to#32853
On Friday, October 9, 2015 at 11:29:26 AM UTC-4, lndre...@gmail.com wrote:
> On Friday, October 9, 2015 at 11:17:44 AM UTC-4, Jeff Higgins wrote:
> > On 10/09/2015 10:12 AM, lndresnick@gmail.com wrote:
> > > Debugging an exception in my code, found out that under certain conditions java.util.Scanner hasNext() returns true, but next() throws NoSuchElementException. I submitted a bug against java to Oracle for this today, but throwing it out here so folks searching can find.
> > 
> > Just searched Oracle bugs database for Scanner. Several bugs seem to
> > describe the behavior you describe. Didn't find a bug report for a
> > Scanner dated today.
> > 
> 
> Thanks.  I submitted the bug and got an auto-reply.  Based on that auto-reply, I gather they triage submissions before adding to actual bug DB. 
> 
> I found in the Oracle defect database 3 java.util.Scanner bugs related to the internal buffer size, but not quite this one (i.e. none have the NoSuchElementException).

There is a bug now.

http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8139414

Given the other bugs going back a few years, shame this rather basic code hasn't been fixed...

[toc] | [prev] | [next] | [standalone]

#32941

FromJan Burse <janburse@fastmail.fm>
Date2015-10-27 19:25 +0100
Message-ID<n0ofj0$6p3$1@news.albasani.net>
In reply to#32850
Hi,

I see more tasks for program verification!

Bye

Stories such as:
http://envisage-project.eu/proving-android-java-and-python-sorting-algorithm-is-broken-and-how-to-fix-it

lndresnick@gmail.com schrieb:
> Debugging an exception in my code, found out that under certain conditions java.util.Scanner hasNext() returns true, but next() throws NoSuchElementException. I submitted a bug against java to Oracle for this today, but throwing it out here so folks searching can find.  Very happy to hear if I'm wrong, any comments welcome.  But the documented interface is that if hasNext() returns true, there is a token available.  For next(), the documented interface is that if there is no token available, NoSuchElementException is thrown.  In the code below, NoSuchElementException is thrown for input lengths of 1019, 1022, 2046, 4094, and 8190 (plus added delimiter).  This bug appears to be triggered if a fuzzy matched delimiter crosses the internal java.util.Scanner buffer size of 1024
>
> Easy enough to work around, mind you.  Just catch the exception locally and consider it "success".
>
> Code follows:
>
> import java.util.Scanner;
> import java.util.NoSuchElementException;
>
> public class Boundary {
>
>      static private boolean parse(String data, String boundary) {
>          String dataCopy = new String(data);
>          try {
>              Scanner scanner = new Scanner(dataCopy);
>              try {
>                  scanner.useDelimiter("-" + boundary + "(-)?");
>
>                  while (scanner.hasNext()) {
>                      String bodyPart = scanner.next();
>                  }
>              } finally {
>                  if (scanner != null)
>                      scanner.close();
>              }
>          } catch (NoSuchElementException e) {
>              System.out.println("Caught NoSuchElementException " + e);
>              e.printStackTrace();
>              return false;
>          }
> 		
>          return true;
>      }
>
>      public static void main (String argv[]) {
>          final String boundary = ";";
>
>          for ( int i = 0; i < 10000; ++i ) {
>              StringBuilder data = new StringBuilder();
>              data.append("--" + boundary);
>              for (int j = 0; j< i; ++j) {
>                  data.append(j%10);
>              }
>
>              data.append("-" + boundary + "-");
>
>              boolean ret=parse(data.toString(), boundary);
>              if (!ret) {
>                  System.out.println("parse returned " + ret + " i=" +i);
>              }
>          }
>      }
> }
>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web