Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #65502 > unrolled thread

how to reduce bugs due to incorrect indentation

Back to article view | Back to comp.lang.python

Contents

  how to reduce bugs due to incorrect indentation msustik@gmail.com - 2014-02-05 19:02 -0800
    Re: how to reduce bugs due to incorrect indentation Dan Sommers <dan@tombstonezero.net> - 2014-02-06 03:08 +0000
      Re: how to reduce bugs due to incorrect indentation Chris Angelico <rosuav@gmail.com> - 2014-02-06 15:47 +1100
    Re: how to reduce bugs due to incorrect indentation Asaf Las <roegltd@gmail.com> - 2014-02-05 21:23 -0800
    Re: how to reduce bugs due to incorrect indentation Terry Reedy <tjreedy@udel.edu> - 2014-02-06 03:09 -0500
    Re: how to reduce bugs due to incorrect indentation Grant Edwards <invalid@invalid.invalid> - 2014-02-06 14:49 +0000
      Re: how to reduce bugs due to incorrect indentation msustik@gmail.com - 2014-02-06 10:51 -0800
    Re: how to reduce bugs due to incorrect indentation Roel Schroeven <roel@roelschroeven.net> - 2014-02-06 21:29 +0100
      Re: how to reduce bugs due to incorrect indentation msustik@gmail.com - 2014-02-06 17:20 -0800
        Re: how to reduce bugs due to incorrect indentation Chris Angelico <rosuav@gmail.com> - 2014-02-07 12:57 +1100
        Re: how to reduce bugs due to incorrect indentation Roel Schroeven <roel@roelschroeven.net> - 2014-02-07 23:08 +0100
        Re: how to reduce bugs due to incorrect indentation Jurko Gospodnetić <jurko.gospodnetic@pke.hr> - 2014-02-08 12:11 +0100
    Re: how to reduce bugs due to incorrect indentation Larry Martell <larry.martell@gmail.com> - 2014-02-06 15:36 -0500
    Re: how to reduce bugs due to incorrect indentation Ethan Furman <ethan@stoneleaf.us> - 2014-02-06 13:32 -0800
    Re: how to reduce bugs due to incorrect indentation Larry Martell <larry.martell@gmail.com> - 2014-02-06 17:09 -0500
    Re: how to reduce bugs due to incorrect indentation Chris Angelico <rosuav@gmail.com> - 2014-02-07 09:30 +1100
      Re: how to reduce bugs due to incorrect indentation Asaf Las <roegltd@gmail.com> - 2014-02-06 15:01 -0800
        Re: how to reduce bugs due to incorrect indentation Chris Angelico <rosuav@gmail.com> - 2014-02-07 11:08 +1100
    Re: how to reduce bugs due to incorrect indentation Roel Schroeven <roel@roelschroeven.net> - 2014-02-07 23:06 +0100

#65502 — how to reduce bugs due to incorrect indentation

I had a bug in a Python script recently. The code in question was something along the lines of:

if a == 1:
    x = y
else:
    x = z
y = z + y
z = z + 1

While editing this file I accidentally pushed TAB on the line with 'y = z + y'.

My changes were elsewhere and I did not notice the above one line change when I looked at the diffs before commit. I should have noticed it...

It was rare that a was 1 and therefore the problem did not show up for a while. (I know I should have had tests exercising all cases...)

When the bug showed up, it was kind of difficult to remember what was the original intent. Fortunately, looking at old versions allowed me to find the problem commit and the bug.

Any suggestion on how to avoid this type of error in the future?

Thanks!

[toc] | [next] | [standalone]

#65503

On Wed, 05 Feb 2014 19:02:09 -0800, msustik wrote:

> My changes were elsewhere and I did not notice the above one line
> change when I looked at the diffs before commit. I should have noticed
> it...
> 
> It was rare that a was 1 and therefore the problem did not show up for
> a while. (I know I should have had tests exercising all cases...)
> 
> When the bug showed up, it was kind of difficult to remember what was
> the original intent. Fortunately, looking at old versions allowed me
> to find the problem commit and the bug.
> 
> Any suggestion on how to avoid this type of error in the future?

I think you just mentioned the best suggestions:  (a) have a good
collection of unit tests (where "good" means data and branch coverage),
and (b) look carefully, or have someone else look carefully, at every
commit.

Dan

[toc] | [prev] | [next] | [standalone]

#65509

On Thu, Feb 6, 2014 at 2:08 PM, Dan Sommers <dan@tombstonezero.net> wrote:
> On Wed, 05 Feb 2014 19:02:09 -0800, msustik wrote:
>
>> My changes were elsewhere and I did not notice the above one line
>> change when I looked at the diffs before commit. I should have noticed
>> it...
>>
>> It was rare that a was 1 and therefore the problem did not show up for
>> a while. (I know I should have had tests exercising all cases...)
>>
>> When the bug showed up, it was kind of difficult to remember what was
>> the original intent. Fortunately, looking at old versions allowed me
>> to find the problem commit and the bug.
>>
>> Any suggestion on how to avoid this type of error in the future?
>
> I think you just mentioned the best suggestions:  (a) have a good
> collection of unit tests (where "good" means data and branch coverage),
> and (b) look carefully, or have someone else look carefully, at every
> commit.

Yep, I'd agree with that. Additionally, a habit of small and
conceptual commits helps hugely, even if you don't look at them too
carefully as they go through; get familiar with 'git gui blame' or
whatever the equivalent is for your system (or just browsing through
'gitk filename.py' or equiv) and you can go back in time to pin down
bugs. I had to fight heavily to get my boss to understand this,
because he said he _never_ went back through the repo's history;
meanwhile, I was doing so frequently, and knowing exactly what code
was changed, when, and why. His attitude to bugs was "don't make any,
and you're a bad programmer if you do". My attitude to bugs was, and
is, "be honest with yourself, you will make them, so design your
systems to handle that".

When your commits are small and tidy, you can often find bugs really
REALLY easily just by looking at the commit that changed some
particular line. With gitk, I'll often make a dummy edit to a line,
then highlight the red "this line deleted" line in the uncommitted
changes display, right-click, "Show origin of this line".
Alternatively, 'git blame' or other annotation can work, but I
generally find that annotating a whole file is overkill and trying to
ask for annotations of one small section is tedious. But whichever way
you do it, you should be able to VERY quickly go from "Hmm, I wonder
why this is indented like this?" to "Here's the commit that made it
like that", and then you can easily decide whether it was right or
not.

Of course, your commit messages have to be useful. I can't honestly
say my own commit messages are perfect, but at least they're not like
these:

https://github.com/douglascrockford/JSLint/commits/master

Note to future readers: That link will take you to the most recent
commits on that repo. It may be (it may be!) that, by the time you
look at it, he'll have changed his practice and started writing
exemplary messages. If so, just page back through the history and find
how it looked in 2014. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#65510

On Thursday, February 6, 2014 5:02:09 AM UTC+2, msu...@gmail.com wrote:
> I had a bug in a Python script recently. The code in question was 
> something along the lines of:
> if a == 1:
>     x = y
> else:
>     x = z
> y = z + y
> z = z + 1
> 
> While editing this file I accidentally pushed TAB on the line with 
> 'y = z + y'.
> My changes were elsewhere and I did not notice the above one line 
> change when I looked at the diffs before commit. I should have noticed 
> it...
> 
> It was rare that a was 1 and therefore the problem did not show up 
> for a while. (I know I should have had tests exercising all cases...)
> When the bug showed up, it was kind of difficult to remember what 
> was the original intent. Fortunately, looking at old versions allowed 
> me to find the problem commit and the bug.
> Any suggestion on how to avoid this type of error in the future?
> Thanks!

I see. 

The only solution to mimic C style curly brackets comes to my mind is 
to use hash to mark start and stop of your block as:

if a == 1:#
    x = y
#
else:#
    x = z
#
y = z + y
z = z + 1

and PEP ... Nulla poena sine lege :-)

/Asaf

[toc] | [prev] | [next] | [standalone]

#65518

On 2/5/2014 10:02 PM, msustik@gmail.com wrote:

> if a == 1:
>      x = y
> else:
>      x = z
> y = z + y
> z = z + 1
>
> While editing this file I accidentally pushed TAB on the line with 'y = z + y'.

In this particular case, remove the indentation with
x = y if a == 1 else z
and indenting the next line is  syntax error.
-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#65542

On 2014-02-06, msustik@gmail.com <msustik@gmail.com> wrote:
> I had a bug in a Python script recently. The code in question was something along the lines of:
>
> if a == 1:
>     x = y
> else:
>     x = z
> y = z + y
> z = z + 1
>
> While editing this file I accidentally pushed TAB on the line 
> with 'y = z + y'.
>
> Any suggestion on how to avoid this type of error in the future?

The best advice is to pay closer attention to what you're doing.  Look
at the code while you're editing, not your fingers.  Before you commit
the change, spend some time looking carefully at it to verify that
changes were intentional.

-- 
Grant Edwards               grant.b.edwards        Yow! I left my WALLET in
                                  at               the BATHROOM!!
                              gmail.com            

[toc] | [prev] | [next] | [standalone]

#65552

Thanks for all the suggestions!
Best,
-Matyas

[toc] | [prev] | [next] | [standalone]

#65555

msustik@gmail.com schreef:
> I had a bug in a Python script recently. The code in question was something along the lines of:
> 
> if a == 1:
>     x = y
> else:
>     x = z
> y = z + y
> z = z + 1
> 
> While editing this file I accidentally pushed TAB on the line with 'y = z + y'.
> 
> My changes were elsewhere and I did not notice the above one line change when I looked at the diffs before commit. I should have noticed it...
> 
> It was rare that a was 1 and therefore the problem did not show up for a while. (I know I should have had tests exercising all cases...)
> 
> When the bug showed up, it was kind of difficult to remember what was the original intent. Fortunately, looking at old versions allowed me to find the problem commit and the bug.
> 
> Any suggestion on how to avoid this type of error in the future?

My suggestion: configure your editor to insert the appropriate amount of 
spaces instead of a tab when you press the tab key.


Best regards,
Roel

-- 
"Met een spitsvondig citaat bewijs je niets."
         -- Voltaire

[toc] | [prev] | [next] | [standalone]

#65569

On Thursday, February 6, 2014 12:29:36 PM UTC-8, Roel Schroeven wrote:
> 
> My suggestion: configure your editor to insert the appropriate amount of 
> 
> spaces instead of a tab when you press the tab key.

You misunderstood the problem, but managed to start a Tab war! :-)

My emacs inserts 4 spaces in python mode when I press the tab key. Python uses the indentation to decide how many lines following the else are executed in the else branch or after the else branch (always that is).

By pressing inadvertently the Tab key I changed the semantics of the code while it is still syntactically correct and PEP8 compliant. This was indeed a user error and my question was towards practices reducing the chance of this happening again.

Based on the responses I arrived to the conclusion that there is no better solution than trying to be careful and have good testing suites.

It would be possible to disable the Tab key completely and type in the spaces all the time. (It is much less likely that one would press the space bar accidentally four times or hold it down to get 4 spaces by mistake.)

Unfortunately this means giving up the indentation help of the editor and that will slow down coding. It will also lead to many indentation mistakes during development (most of which will be caught right away however. Maybe a coloring of the background based on tab position could assist in this.

I also considered adding an extra blank line after the if-else block (similarly for loops) in the hope that it would reduce the chance of missing an inadvertent indentation after the block.

However, this defeats somewhat the python paradigm that got rid of closing braces and endif-s etc. used in other languages to allow more compact code.

-Matyas


> 
> 
> 
> 
> 
> Best regards,
> 
> Roel
> 
> 
> 
> -- 
> 
> "Met een spitsvondig citaat bewijs je niets."
> 
>          -- Voltaire

[toc] | [prev] | [next] | [standalone]

#65573

On Fri, Feb 7, 2014 at 12:20 PM,  <msustik@gmail.com> wrote:
> It would be possible to disable the Tab key completely and type in the spaces all the time. (It is much less likely that one would press the space bar accidentally four times or hold it down to get 4 spaces by mistake.)
>
> Unfortunately this means giving up the indentation help of the editor and that will slow down coding. It will also lead to many indentation mistakes during development (most of which will be caught right away however. Maybe a coloring of the background based on tab position could assist in this.
>

I don't know that it'd really help much anyway. You might reduce one
chance of making errors by hitting a single key, but at the cost of
stupid syntactic salt (indentation requires hitting a key four times?
No thanks), and your fingers would just get used to
whack-whack-whack-whack. No change.

You can spend all your time trying to warp your coding style around
preventing this bug or that bug from happening, or you can just
acknowledge that bugs WILL happen and handle them after the event.
(Hence, source control.) Suppose you come up with a solution to the
accidental-indentation problem. What are you going to do about this
one?

def foo(bar):
    if not bar: bat = [0]
    for x in bar:
        print(len(bar),x)

Now, why is your empty-list handling not working? Oh, there was a
typo. How are you going to deal with that? Well, you could bring in
C-style variable declarations; then you'd get an immediate error
('bat' is undeclared), but somehow I don't think most Python
programmers would prefer this :) Now personally, I do quite like
declared variables, because they allow infinitely-nested scoping, and
I find that feature worth the effort of declaring all my locals; but
it's a tradeoff, and I wouldn't go to that level of effort *just* to
catch typos in variable names. What if there had been a 'bat' at a
higher scope? Then the typo just means the code does something else
wrong. No fundamental difference.

There was a scheme posted to this list a little while ago to have
variable names shown in different colors, which might have helped. (I
disagree with the author's idea that similar names should be in
similar colors - I think that similar names should be in DISsimilar
colors, specifically to catch this sort of error. But anyway.) That's
a theory that might help... but it still might not. And what if your
error is in a literal string that later gets parsed? No, there's no
way that you can catch everything beforehand.

Bugs happen. Find 'em, fix 'em.

ChrisA

[toc] | [prev] | [next] | [standalone]

#65616

msustik@gmail.com schreef:
> On Thursday, February 6, 2014 12:29:36 PM UTC-8, Roel Schroeven wrote:
>> My suggestion: configure your editor to insert the appropriate amount of 
>>
>> spaces instead of a tab when you press the tab key.
> 
> You misunderstood the problem, but managed to start a Tab war! :-)

Indeed I did; sorry for both :-)

-- 
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
   -- Isaac Asimov

Roel Schroeven

[toc] | [prev] | [next] | [standalone]

#65668

   Hi,

On 7.2.2014. 2:20, msustik@gmail.com wrote:
> Based on the responses I arrived to the conclusion that there
> is no better solution than trying to be careful and have good
> testing suites.
>
> It would be possible to disable the Tab key completely
> ...[snipped]...
> Maybe a coloring of the background based on tab position
> ...[snipped]...
> I also considered
> ...[snipped]...

   YMMV, but for me, just reading through this fun thread took more
time then ever debugging issues caused by bad Python code
indentation. :-D

   So, my suggestion would be to just ignore the problem and deal
with any resulting issues as they occur.

   Clean coding & development practices, some of which have been
mentioned earlier in this thread and are useful for many other
reasons as well, will additionally reduce the chance of such
errors causing any non-trivial issues.

   Best regards,
     Jurko Gospodnetić

[toc] | [prev] | [next] | [standalone]

#65556

On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven <roel@roelschroeven.net> wrote:
> msustik@gmail.com schreef:
>>
>> I had a bug in a Python script recently. The code in question was
>> something along the lines of:
>>
>> if a == 1:
>>     x = y
>> else:
>>     x = z
>> y = z + y
>> z = z + 1
>>
>> While editing this file I accidentally pushed TAB on the line with 'y = z
>> + y'.
>>
>> My changes were elsewhere and I did not notice the above one line change
>> when I looked at the diffs before commit. I should have noticed it...
>>
>> It was rare that a was 1 and therefore the problem did not show up for a
>> while. (I know I should have had tests exercising all cases...)
>>
>> When the bug showed up, it was kind of difficult to remember what was the
>> original intent. Fortunately, looking at old versions allowed me to find the
>> problem commit and the bug.
>>
>> Any suggestion on how to avoid this type of error in the future?
>
>
> My suggestion: configure your editor to insert the appropriate amount of
> spaces instead of a tab when you press the tab key.

+1 - tabs are evil.

[toc] | [prev] | [next] | [standalone]

#65560

On 02/06/2014 12:36 PM, Larry Martell wrote:
> On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven wrote:
>> msustik@gmail.com schreef:
>>>
>>> While editing this file I accidentally pushed TAB on the line with 'y = z
>>> + y'.
>>
>> My suggestion: configure your editor to insert the appropriate amount of
>> spaces instead of a tab when you press the tab key.
>
> +1 - tabs are evil.

Tabs are not evil, and an argument can be made that tabs are better (a decent editor can be configured to show x many 
spaces per tab, then users could decide how much indentation they preferred to see... but I digress).

Using spaces instead of tabs would also have not prevented the error that Msustik encountered, and for that matter we 
don't know whether he was using tabs or spaces in his source file, only that he hit the Tab key -- surely you are not 
suggesting everyone rip out their tab key and just hit the space bar four times for each level of indentation?  ;)

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#65561

On Thu, Feb 6, 2014 at 4:32 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
> On 02/06/2014 12:36 PM, Larry Martell wrote:
>>
>> On Thu, Feb 6, 2014 at 3:29 PM, Roel Schroeven wrote:
>>>
>>> msustik@gmail.com schreef:
>>>>
>>>>
>>>> While editing this file I accidentally pushed TAB on the line with 'y =
>>>> z
>>>> + y'.
>>>
>>>
>>> My suggestion: configure your editor to insert the appropriate amount of
>>> spaces instead of a tab when you press the tab key.
>>
>>
>> +1 - tabs are evil.
>
>
> Tabs are not evil, and an argument can be made that tabs are better (a
> decent editor can be configured to show x many spaces per tab, then users
> could decide how much indentation they preferred to see... but I digress).
>
> Using spaces instead of tabs would also have not prevented the error that
> Msustik encountered, and for that matter we don't know whether he was using
> tabs or spaces in his source file, only that he hit the Tab key -- surely
> you are not suggesting everyone rip out their tab key and just hit the space
> bar four times for each level of indentation?  ;)

The Tab key is not evil, it's the tab character (Ctrl-I). I have been
bitten by this many time when I had to work on a program written by
another. They had their tab stops set at 5 or 6, mine is set at 4, or
they did not have expandtab set, but I did. So you get either a script
that looks misaligned, but works, or one that does not look misaligned
but doesn't work. When I have to pick up someone else's script the
first thing I do is replace the tabs with spaces.

[toc] | [prev] | [next] | [standalone]

#65562

On Fri, Feb 7, 2014 at 9:09 AM, Larry Martell <larry.martell@gmail.com> wrote:
> The Tab key is not evil, it's the tab character (Ctrl-I). I have been
> bitten by this many time when I had to work on a program written by
> another. They had their tab stops set at 5 or 6, mine is set at 4, or
> they did not have expandtab set, but I did. So you get either a script
> that looks misaligned, but works, or one that does not look misaligned
> but doesn't work. When I have to pick up someone else's script the
> first thing I do is replace the tabs with spaces.

All you've proven is that *mixing* spaces and tabs is evil. It's like
arguing that oil is evil because, when you mix it with water, weird
stuff happens. But that doesn't mean I want to fry my bacon in water.

Mmm, bacon.

Sorry. I'm back now. Ahem. Arguably, a better fix is to replace spaces
with tabs, because they're more obvious. But mainly, just be
consistent. Whatever one file uses, it uses exclusively. It'd be
pretty easy to create a git commit hook that checks files for leading
indentation and rejects the commit if it's mismatched; I would guess
the same is true in Mercurial.

But none of this would solve the OP's original issue. Whether it's a
tab or spaces, unexpectedly indenting a line of code is a problem.
It's no different from accidentally hitting Ctrl-T in SciTE and
reordering two lines, when one line depends on the other. It's a bug.
So you look at your commits before you make them (to give yourself a
chance to catch it quickly), and you make sure you can always look
back over your commits (in case you didn't catch it quickly). Much
better than blaming the characters involved. Poor innocent U+0009.

ChrisA

[toc] | [prev] | [next] | [standalone]

#65563

On Friday, February 7, 2014 12:30:17 AM UTC+2, Chris Angelico wrote:
> On Fri, Feb 7, 2014 at 9:09 AM, Larry Martell <lar...@gmail.com> wrote:
> 
> > The Tab key is not evil, it's the tab character (Ctrl-I). I have been
> > bitten by this many time when I had to work on a program written by
> > another. They had their tab stops set at 5 or 6, mine is set at 4, or
> > they did not have expandtab set, but I did. So you get either a script
> > that looks misaligned, but works, or one that does not look misaligned
> > but doesn't work. When I have to pick up someone else's script the
> > first thing I do is replace the tabs with spaces.
> 
> All you've proven is that *mixing* spaces and tabs is evil. It's like
> arguing that oil is evil because, when you mix it with water, weird
> stuff happens. But that doesn't mean I want to fry my bacon in water.
> Mmm, bacon.
> Sorry. I'm back now. Ahem. Arguably, a better fix is to replace spaces
> with tabs, because they're more obvious. But mainly, just be
> consistent. Whatever one file uses, it uses exclusively. It'd be
> pretty easy to create a git commit hook that checks files for leading
> indentation and rejects the commit if it's mismatched; I would guess
> the same is true in Mercurial.
> 
> But none of this would solve the OP's original issue. Whether it's a
> tab or spaces, unexpectedly indenting a line of code is a problem.
> It's no different from accidentally hitting Ctrl-T in SciTE and
> reordering two lines, when one line depends on the other. It's a bug.
> So you look at your commits before you make them (to give yourself a
> chance to catch it quickly), and you make sure you can always look
> back over your commits (in case you didn't catch it quickly). Much
> better than blaming the characters involved. Poor innocent U+0009.
> ChrisA

pep8 pushed \t to dark side in Python. 

though it is better that spaces sometimes. let say someone's indented code
with 2 spaces and user is comfortable with 4. if \t then it is done by 
editor's conf without touching code.

/Asaf

[toc] | [prev] | [next] | [standalone]

#65566

On Fri, Feb 7, 2014 at 10:01 AM, Asaf Las <roegltd@gmail.com> wrote:
> pep8 pushed \t to dark side in Python.

Only for the Python stdlib, and only because a decision has to be made
one way or the other. I believe Guido stated at one point that there
was only a very weak push toward "spaces only" rather than "tabs
only", just that it had to be one of those.

ChrisA

[toc] | [prev] | [next] | [standalone]

#65614

Chris Angelico schreef:
> But none of this would solve the OP's original issue. Whether it's a
> tab or spaces, unexpectedly indenting a line of code is a problem.

I had misread. I thought the problem was that the OP did want to indent, 
but accidentally used the tab key instead of the space bar to do so, 
introducing the dreaded mixing of tabs and spaces. Remapping the tab key 
would solve that problem.

But the real problem was different, so my advice was indeed not a 
solution to the problem. I should've read better.

-- 
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
   -- Isaac Asimov

Roel Schroeven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web