Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c++ > #19113 > unrolled thread

Tips to analyze a project with thousands of files

Back to article view | Back to comp.lang.c++

Contents

  Tips to analyze a project with thousands of files "Like Learn" <like2learn@live.ca> - 2012-10-19 23:07 -0600
    Re: Tips to analyze a project with thousands of files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-10-20 07:49 +0000
      Re: Tips to analyze a project with thousands of files "Like Learn" <like2learn@live.ca> - 2012-10-21 00:09 -0600
      Re: Tips to analyze a project with thousands of files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-10-21 11:42 +0000
    Re: Tips to analyze a project with thousands of files Öö Tiib <ootiib@hot.ee> - 2012-10-20 09:14 -0700
      Re: Tips to analyze a project with thousands of files "Like Learn" <like2learn@live.ca> - 2012-10-21 00:12 -0600
    Re: Tips to analyze a project with thousands of files ctgqumgf@sharklasers.com - 2012-10-21 01:34 -0700
      Re: Tips to analyze a project with thousands of files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-10-21 11:55 +0000
      Re: Tips to analyze a project with thousands of files "Like Learn" <like2learn@live.ca> - 2012-10-21 22:39 -0600
    Re: Tips to analyze a project with thousands of files Stuart <DerTopper@web.de> - 2012-10-23 13:00 +0200
    Re: Tips to analyze a project with thousands of files drew@furrfu.invalid (Drew Lawson) - 2012-10-24 15:33 +0000
    Re: Tips to analyze a project with thousands of files Werner <werasm@gmail.com> - 2012-10-24 09:29 -0700
      Re: Tips to analyze a project with thousands of files Jorgen Grahn <grahn+nntp@snipabacken.se> - 2012-10-25 20:44 +0000
    Re: Tips to analyze a project with thousands of files Luca Risolia <luca.risolia@studio.unibo.it> - 2012-10-24 22:42 +0200

#19113 — Tips to analyze a project with thousands of files

My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
baseband chip with thousands of files (literally) to fix a bug. There is no 
design documents about the project. The project is qriting in C++, halve of 
them are commented in Doxygen format, while the other halve not. I don't 
have the hardware to run the firmware or debugging. The only thing I have is 
a list of debugging outputs of the printfs in firmware collected by the 
client.

I spent one day today and even couldn't find out where the firmware starts. 
Obviously it was not the main function. The comments are not in pure 
English, frustrated. Any tip to find out the starting line of an ARM 
project?

BTW, any suggestion for debugging tools toward Qualcomm MDM9x00? I believe 
it belongs to ARM family and has multiple-processors.




--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

[toc] | [next] | [standalone]

#19118

On Sat, 2012-10-20, Like Learn wrote:
> My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
> baseband chip with thousands of files (literally) to fix a bug. There is no 
> design documents about the project. The project is qriting in C++, halve of 
> them are commented in Doxygen format, while the other halve not. I don't 
> have the hardware to run the firmware or debugging. The only thing I have is 
> a list of debugging outputs of the printfs in firmware collected by the 
> client.

The task is almost never to "analyze" something.  IME, it's only by
doing actual useful work (changing a feature, doing internal changes
or testing) that you can learn more about a large piece of code.

I have often been in that situation. It's frustrating.  But you have
to accept it:

"I cannot understand all of this at once.  There are big parts I will
probably never understand.  Most of this code is fit for
thedailywtf.com. But I can fix this particular bug, and learn
something while doing it, and the code will be better afterwards."

> I spent one day today and even couldn't find out where the firmware starts. 
> Obviously it was not the main function. The comments are not in pure 
> English, frustrated. Any tip to find out the starting line of an ARM 
> project?

Build it and look at the symbol table of the final image.  That is
often a good supplement to looking at the source code.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#19144

Thanks. So I am not the only one afraid of school bus.

I will learn to play with the cards I have.


"Jorgen Grahn" <grahn+nntp@snipabacken.se> wrote in message 
news:slrnk84lsu.1d3.grahn+nntp@frailea.sa.invalid...
> On Sat, 2012-10-20, Like Learn wrote:
>> My task is to analyze a legacy firmware project using Qualcomm MDM9X00
>> baseband chip with thousands of files (literally) to fix a bug. There is 
>> no
>> design documents about the project. The project is qriting in C++, halve 
>> of
>> them are commented in Doxygen format, while the other halve not. I don't
>> have the hardware to run the firmware or debugging. The only thing I have 
>> is
>> a list of debugging outputs of the printfs in firmware collected by the
>> client.
>
> The task is almost never to "analyze" something.  IME, it's only by
> doing actual useful work (changing a feature, doing internal changes
> or testing) that you can learn more about a large piece of code.
>
> I have often been in that situation. It's frustrating.  But you have
> to accept it:
>
> "I cannot understand all of this at once.  There are big parts I will
> probably never understand.  Most of this code is fit for
> thedailywtf.com. But I can fix this particular bug, and learn
> something while doing it, and the code will be better afterwards."
>
>> I spent one day today and even couldn't find out where the firmware 
>> starts.
>> Obviously it was not the main function. The comments are not in pure
>> English, frustrated. Any tip to find out the starting line of an ARM
>> project?
>
> Build it and look at the symbol table of the final image.  That is
> often a good supplement to looking at the source code.
>
> /Jorgen
>
> -- 
>  // Jorgen Grahn <grahn@  Oo  o.   .     .
> \X/     snipabacken.se>   O  o   . 



--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

[toc] | [prev] | [next] | [standalone]

#19165

On Sat, 2012-10-20, Jorgen Grahn wrote:
> On Sat, 2012-10-20, Like Learn wrote:
>> My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
>> baseband chip with thousands of files (literally) to fix a bug.
...
> The task is almost never to "analyze" something.  IME, it's only by
> doing actual useful work (changing a feature, doing internal changes
> or testing) that you can learn more about a large piece of code.
...

Only now did I see that "Like Learn" didn't say the task was just "to
analyze", but "to analyze ... to fix a bug".  So my advice must have
looked a bit strange. Sorry!

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#19129

On Saturday, 20 October 2012 08:07:35 UTC+3, Like Learn  wrote:
> My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
> baseband chip with thousands of files (literally) to fix a bug. There is no 
> design documents about the project.

Nice task. You have something interesting to do. No documentation and likely no
unit-tests means that you should carefully consider twice before you change
one character in such code.

> The project is qriting in C++, halve of them are commented in Doxygen 
> format, while the other halve not.

That is half better than no design documentation whatsoever.

> I don't have the hardware to run the firmware or debugging. The only thing
> I have is a list of debugging outputs of the printfs in firmware collected
> by the client.

Then your client is eager to pay for at least 10 times less efficient
development process than normal? That instead of giving you some little
device for debugging? It is likely someting like mobile phone so transport can't be too expensive. Excellent news! Both rich and stupid clients! That
is most golden treasure imaginable.

> I spent one day today and even couldn't find out where the firmware starts. 
> Obviously it was not the main function. The comments are not in pure 
> English, frustrated. Any tip to find out the starting line of an ARM 
> project?

Why they did choose you as developer? There is nothing worse 
than stupid managers. Run perhaps? If not then compile it into binary and
examine that. It has to start somehow and there is nothing magic about it.
Other option is to just to do whatever you like and week later to tell that 
you tried hard but it seems that you are not qualified enough for the work.

> BTW, any suggestion for debugging tools toward Qualcomm MDM9x00? I believe 
> it belongs to ARM family and has multiple-processors.

It is mobile wireless modem chip. I suspect that the 
software is not for that chip but for some other embedded processor that uses it as a modem. 

[toc] | [prev] | [next] | [standalone]

#19145

No idea why they chose me. Since they chose me, I hate to disappoint people.

"Öö Tiib" <ootiib@hot.ee> wrote in message 
news:454eaa0d-e057-48fa-8285-d6d54e4967c6@googlegroups.com...
> On Saturday, 20 October 2012 08:07:35 UTC+3, Like Learn  wrote:
>> My task is to analyze a legacy firmware project using Qualcomm MDM9X00
>> baseband chip with thousands of files (literally) to fix a bug. There is 
>> no
>> design documents about the project.
>
> Nice task. You have something interesting to do. No documentation and 
> likely no
> unit-tests means that you should carefully consider twice before you 
> change
> one character in such code.
>
>> The project is qriting in C++, halve of them are commented in Doxygen
>> format, while the other halve not.
>
> That is half better than no design documentation whatsoever.
>
>> I don't have the hardware to run the firmware or debugging. The only 
>> thing
>> I have is a list of debugging outputs of the printfs in firmware 
>> collected
>> by the client.
>
> Then your client is eager to pay for at least 10 times less efficient
> development process than normal? That instead of giving you some little
> device for debugging? It is likely someting like mobile phone so transport 
> can't be too expensive. Excellent news! Both rich and stupid clients! That
> is most golden treasure imaginable.
>
>> I spent one day today and even couldn't find out where the firmware 
>> starts.
>> Obviously it was not the main function. The comments are not in pure
>> English, frustrated. Any tip to find out the starting line of an ARM
>> project?
>
> Why they did choose you as developer? There is nothing worse
> than stupid managers. Run perhaps? If not then compile it into binary and
> examine that. It has to start somehow and there is nothing magic about it.
> Other option is to just to do whatever you like and week later to tell 
> that
> you tried hard but it seems that you are not qualified enough for the 
> work.
>
>> BTW, any suggestion for debugging tools toward Qualcomm MDM9x00? I 
>> believe
>> it belongs to ARM family and has multiple-processors.
>
> It is mobile wireless modem chip. I suspect that the
> software is not for that chip but for some other embedded processor that 
> uses it as a modem. 



--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

[toc] | [prev] | [next] | [standalone]

#19159

Given such a task it might be an option using a static code analyzer. As far as I know fixing a bug also a deadline...

My hope would be that the output of the static code checker can give me some ideas. Well, reading your description I guess there will be hundreds of warnings. I know this game too good.

The art is then to ignore minor warnings (e.g. signed/unsigned assignments/comparisons, ...) and filter out real bad ones (possible division by 0, array out of bounds, access of memory already released, ...).

Given a vague description of the bug (the crash happens when I do this or that) you might be lucky...

I have been working with QAC++ and PC-Lint so far. There might also be open-source tools, like cppcheck which I never had a look at.

[BTW: I cannot tell you how good these tools are at finding dynamic memory problems. There are other tools (like Boundschecker, Purify, ...). But I guess in your case (Qualcomm) they won't help as they normally run instrumented binaries on the host.]

[BTW: GCC with -pedantic option and highest warning level isn't that bad at finding problems either]

[toc] | [prev] | [next] | [standalone]

#19167

On Sun, 2012-10-21, ctgqumgf@sharklasers.com wrote:

[LL has a big heap of unknown code, and they want him/her to fix a bug
which she cannot reproduce herself]

> Given such a task it might be an option using a static code
> analyzer. As far as I know fixing a bug also a deadline...

> My hope would be that the output of the static code checker can give
> me some ideas.

IME, it *is* a good idea to crank up the warning level of the
compiler, and run whatever other static tools you have access to --
but it is rather unlikely that the particular bug you're interested in
shows up there.

Even more unlikely in sloppily written code, which doesn't try to turn
runtime errors into compile-time errors: little use of const, lots of
casts, lots of raw pointers, C code disguised as C++ ...

So yeah, do it, but don't spend too much time on it.

> Given a vague description of the bug (the crash happens when I do
> this or that) you might be lucky...

If it *is* a crash you may be lucky -- any half-decent system gives
you a register and stack dump in that case.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#19190

I will five them a try. Thanks!

<ctgqumgf@sharklasers.com> wrote in message 
news:65d52104-da7b-4eb6-8174-4177810002e1@googlegroups.com...
> Given such a task it might be an option using a static code analyzer. As 
> far as I know fixing a bug also a deadline...
>
> My hope would be that the output of the static code checker can give me 
> some ideas. Well, reading your description I guess there will be hundreds 
> of warnings. I know this game too good.
>
> The art is then to ignore minor warnings (e.g. signed/unsigned 
> assignments/comparisons, ...) and filter out real bad ones (possible 
> division by 0, array out of bounds, access of memory already released, 
> ...).
>
> Given a vague description of the bug (the crash happens when I do this or 
> that) you might be lucky...
>
> I have been working with QAC++ and PC-Lint so far. There might also be 
> open-source tools, like cppcheck which I never had a look at.
>
> [BTW: I cannot tell you how good these tools are at finding dynamic memory 
> problems. There are other tools (like Boundschecker, Purify, ...). But I 
> guess in your case (Qualcomm) they won't help as they normally run 
> instrumented binaries on the host.]
>
> [BTW: GCC with -pedantic option and highest warning level isn't that bad 
> at finding problems either] 



--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

[toc] | [prev] | [next] | [standalone]

#19211

On 10/20/12 Like Learn wrote:
> My task is to analyze a legacy firmware project using Qualcomm MDM9X00
> baseband chip with thousands of files (literally) to fix a bug. There is no
> design documents about the project.


Documenting the "overall" design is -- though the most important 
documentation artefact of a project -- almost always neglected because 
at the time the project was being conceived everybody knew the design by 
heart (lotsa meetings and presentations).

After only half a year most of the design decisions will have been 
forgotten by the management. Five years later the last of the 
programmers who took part in all those kick-off meeting will be 
transferred to another project/leave the company.

Everything will run smoothly for another year or two.

Then something real serious bug will suddenly appear and some poor 
bugger (e.g. you) will be hired to "simply" fix it in a day or two. 
Turns out that even the original project leader is no longer working in 
the company and the only ressource of information is the 
soon-to-be-retired secretary who could tell you all the funny episodes 
about the project but, of course, does not have any technical 
understanding about it.

So basically you are faced a situation where your boss dropped his key 
into the Hoover dam and hands you a snorkel and a diving mask. Story of 
our lives, I guess.


 > The project is qriting in C++, halve of
> them are commented in Doxygen format, while the other halve not. I don't
> have the hardware to run the firmware or debugging. The only thing I have is
> a list of debugging outputs of the printfs in firmware collected by the
> client.
>
> I spent one day today and even couldn't find out where the firmware starts.
> Obviously it was not the main function. The comments are not in pure
> English, frustrated.


Been there. I was able to convince my boss to re-write the whole damn 
project, but it was only 100KLOC, so it took me only a year to get the 
thing working again.

The original code actually used some of the code obfuscating technique 
that I read in one of these funny "how to make yourself indispensable" 
guidelines. The original author (who could of course not be ascertained 
because there were no comments and no source control system) was 
apparently worried about memory allocations, so he introduced at lot of 
members i1, i2, ..., i15 in a class.

These member variables were then used as substitute for local variables 
in the methods of the class (so that the compiler does not have to 
allocate too much stack memory). Because each method used those 
variables in a completely different way it was impossible to give them a 
proper name, so they got the generic i1 name.

Had I known that such a page like dailywtf.com existed at that time ...


 > Any tip to find out the starting line of an ARM
> project?
>
> BTW, any suggestion for debugging tools toward Qualcomm MDM9x00? I believe
> it belongs to ARM family and has multiple-processors.

Sorry, can't help you there.

All I can say is to be brave, always use a bit of spit to clean your 
scuba mask (helps defogging), and keep an eye open for job 
advertisements. I got a pretty good job-offering just recently, but I 
turned it down because of all the shit-shovelling I would have to do. I 
think a programmer can only do so much shit-shovelling, and I had enough 
of it to last me three livetimes.

Regards,
Stuart

[toc] | [prev] | [next] | [standalone]

#19226

In article <k5tbim$ua7$1@adenine.netfront.net>
	"Like Learn" <like2learn@live.ca> writes:
>
>I spent one day today and even couldn't find out where the firmware starts. 
>Obviously it was not the main function. The comments are not in pure 
>English, frustrated. Any tip to find out the starting line of an ARM 
>project?

For firmware, the starting point is probably specified by the
hardware.  And then something, somewhere in the project will use
platform specific code to sync up with that.  One possibility is
that the build places certain routines at fixed memory locations.

Last time I was in that situation, it was in the startup assembly
code, but there are probably ways to do it in a higher level language.

Keep an eye out for global function pointers (or a global instance
of some class).


You have my sympathies.  I didn't set out to decrypt legacy code,
but somehow that's what I've been doing for ~25 years.

-- 
Drew Lawson            |  If dreams were thunder,
                       |    and lightning was desire,
                       |  This old house would have burnt down
                       |    a long time ago

[toc] | [prev] | [next] | [standalone]

#19227

On Saturday, October 20, 2012 7:07:35 AM UTC+2, Like Learn wrote:
> My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
> 
> baseband chip with thousands of files (literally) to fix a bug. There is no 
> 
> design documents about the project. The project is qriting in C++, halve of 
> 
> them are commented in Doxygen format, while the other halve not.
[snip]

- Compile with various compilers with high warning levels, and allow
 no warnings (even if it takes explicitly disabling some that you 
 understand).
- Replace all sprintf's and printf's and functions taking varargs 
with something that is typesafe, alternatively check each one. I 
think the gcc compiler actually warns when this is used wrong.
- Replace all memcpy and strcpy functions with typesafe 
 alternatives.
- Verify your inputs, especially inputs that realize to (cause) 
 strings.
- Check conversion between signed and unsigned, especially those
 that calculate buffer sizes, and are used as arguments to 
 strncpcy/memcpy (which should not be used anyway)...
- Check all bald pointers. Make sure they are at least initialized.
 (I use a lightweight pointer wrapper to ensure this).
- Ensure that RAII is employed everywhere if not GC exists
 (I've no experiene with other forms of garbage collection). This
 means wrapping all bald pointers anyway.

Kind regards,

Werner

Regards,

Werner

[toc] | [prev] | [next] | [standalone]

#19237

On Wed, 2012-10-24, Werner wrote:
> On Saturday, October 20, 2012 7:07:35 AM UTC+2, Like Learn wrote:
>> My task is to analyze a legacy firmware project using Qualcomm MDM9X00 
>> 
>> baseband chip with thousands of files (literally) to fix a bug. There is no 
>> design documents about the project. The project is qriting in C++, halve of 
>> them are commented in Doxygen format, while the other halve not.
> [snip]
>
> - Compile with various compilers with high warning levels, and allow
>   no warnings (even if it takes explicitly disabling some that you 
>   understand).
> - Replace all sprintf's and printf's and functions taking varargs 
>   with something that is typesafe, alternatively check each one. I 
>   think the gcc compiler actually warns when this is used wrong.

It does, and you can tell it "this function of mine takes
printf-like arguments, please check it too".

> - Replace all memcpy and strcpy functions with typesafe 
>  alternatives.

- Do the same for bzero(), memcmp(), memset(), ... most functions
  which deal with void*.

- Audit the C-style casts (g++ -Wold-style-cast).

> - Verify your inputs, especially inputs that realize to (cause) 
>  strings.
> - Check conversion between signed and unsigned, especially those
>  that calculate buffer sizes, and are used as arguments to 
>  strncpcy/memcpy (which should not be used anyway)...
> - Check all bald pointers. Make sure they are at least initialized.
>  (I use a lightweight pointer wrapper to ensure this).
> - Ensure that RAII is employed everywhere if not GC exists
>  (I've no experiene with other forms of garbage collection). This
>  means wrapping all bald pointers anyway.

- Apply 'const' to increase readability.

- Look for small bunches of related data which should be small
  classes.  People tend to write the big, long-lived classes and
  leave the small things as 'int', even if they have very specific
  behavior (e.g. statistics counters which start at 0 and only can be
  incremented).

That (what I listed, and almost all of what Werner listed) is some of
the things I try to do when I work with legacy code.

Although your situation may be different ... if for example
you're doing the maintenance on old releases and some other
team is doing new development, noone will thank you for doing
drastic changes. Unfortunately.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .     .
\X/     snipabacken.se>   O  o   .

[toc] | [prev] | [next] | [standalone]

#19230

On 20/10/2012 07:07, Like Learn wrote:
> My task is to analyze a legacy firmware project using Qualcomm MDM9X00
> baseband chip with thousands of files (literally) to fix a bug. There is no
> design documents about the project. The project is qriting in C++, halve of
> them are commented in Doxygen format, while the other halve not. I don't
> have the hardware to run the firmware or debugging. The only thing I have is
> a list of debugging outputs of the printfs in firmware collected by the
> client.
>
> I spent one day today and even couldn't find out where the firmware starts.
> Obviously it was not the main function. The comments are not in pure
> English, frustrated. Any tip to find out the starting line of an ARM
> project?
>
> BTW, any suggestion for debugging tools toward Qualcomm MDM9x00? I believe
> it belongs to ARM family and has multiple-processors.

https://en.wikipedia.org/wiki/BOUML#Code_Generation_and_Reverse_Engineering


--- news://freenews.netfront.net/ - complaints: news@netfront.net ---

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c++

csiph-web