Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1419714 > unrolled thread
| Started by | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| First post | 2016-06-10 22:50 +0200 |
| Last post | 2016-06-11 07:30 +0200 |
| Articles | 20 on this page of 44 — 4 participants |
Back to article view | Back to linux.kernel
[PATCH 0/4] scripts/coccicheck: add paramap and indexing options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 22:50 +0200
[PATCH 2/4] coccicheck: enable paramap support "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 22:50 +0200
Re: [Cocci] [PATCH 2/4] coccicheck: enable paramap support SF Markus Elfring <elfring@users.sourceforge.net> - 2016-06-11 08:00 +0200
Re: [Cocci] [PATCH 2/4] coccicheck: enable paramap support Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 08:00 +0200
[PATCH 3/4] scripts: add glimpse.sh for indexing the kernel "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 22:50 +0200
Re: [Cocci] [PATCH 3/4] scripts: add glimpse.sh for indexing the kernel SF Markus Elfring <elfring@users.sourceforge.net> - 2016-06-11 19:20 +0200
[PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 22:50 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-10 23:10 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 23:20 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-10 23:30 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-10 23:50 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-10 23:50 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-11 00:00 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-11 00:10 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-11 00:30 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-11 07:50 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 08:00 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-11 08:10 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-13 20:40 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-13 21:00 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-13 21:50 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-13 23:30 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-14 07:10 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 07:20 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Wolfram Sang <wsa@the-dreams.de> - 2016-06-11 08:00 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 08:10 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 07:30 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-13 20:50 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-11 07:20 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-13 21:40 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-13 22:00 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-13 23:30 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-14 07:30 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-14 21:30 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-14 22:50 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-14 23:20 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-14 23:20 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-15 00:10 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-15 09:50 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-15 17:40 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options Julia Lawall <julia.lawall@lip6.fr> - 2016-06-15 17:50 +0200
Re: [PATCH 4/4] coccicheck: add indexing enhancement options "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-06-15 20:00 +0200
Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options SF Markus Elfring <elfring@users.sourceforge.net> - 2016-06-11 08:00 +0200
Re: [Cocci] [PATCH 0/4] scripts/coccicheck: add paramap and indexing options SF Markus Elfring <elfring@users.sourceforge.net> - 2016-06-11 07:30 +0200
Page 2 of 3 — ← Prev page 1 [2] 3 Next page →
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-13 21:50 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJG1X-2jr-23@gated-at.bofh.it> |
| In reply to | #1421153 |
On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > Is there another scripts/coccinelle/ file I can use to test against to demo > > against glimpse/idutils/gitgrep best? > > I'd think this one may be a candidate: > > scripts/coccinelle/misc/irqf_oneshot.cocci > > Not too many, but quite some matches over the tree. Seems like a reasonable choice. julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-13 23:30 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJHAK-3pk-11@gated-at.bofh.it> |
| In reply to | #1421258 |
On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > Is there another scripts/coccinelle/ file I can use to test against to demo > > > against glimpse/idutils/gitgrep best? > > > > I'd think this one may be a candidate: > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > Not too many, but quite some matches over the tree. > > Seems like a reasonable choice. With this one on a 32-core system, I get: glimpse: real 0m6.549s user 0m49.136s sys 0m3.076s idutils: real 0m6.749s user 0m51.936s sys 0m3.876s gitgrep: real 0m6.805s user 0m51.572s sys 0m4.432s coccigrep: real 0m16.369s user 0m58.712s sys 0m5.064s I redirected stderr to stdout, and verified glimpse output has: glimpse request = request_threaded_irq Does this match expectations? Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-14 07:10 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJOLU-e2-11@gated-at.bofh.it> |
| In reply to | #1421349 |
On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > On Mon, Jun 13, 2016 at 09:48:30PM +0200, Julia Lawall wrote: > > > > > > On Mon, 13 Jun 2016, Wolfram Sang wrote: > > > > > > > > > Is there another scripts/coccinelle/ file I can use to test against to demo > > > > against glimpse/idutils/gitgrep best? > > > > > > I'd think this one may be a candidate: > > > > > > scripts/coccinelle/misc/irqf_oneshot.cocci > > > > > > Not too many, but quite some matches over the tree. > > > > Seems like a reasonable choice. > > With this one on a 32-core system, I get: > > glimpse: > real 0m6.549s > user 0m49.136s > sys 0m3.076s > > idutils: > real 0m6.749s > user 0m51.936s > sys 0m3.876s > > gitgrep: > real 0m6.805s > user 0m51.572s > sys 0m4.432s > > coccigrep: > real 0m16.369s > user 0m58.712s > sys 0m5.064s > > I redirected stderr to stdout, and verified glimpse output has: > > glimpse request = request_threaded_irq > > Does this match expectations? Yes. I'm not sure that gitgrep would work as well when there are multiple keywords. It may descend to coccigrep. julia
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-11 07:20 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rIJuW-7j9-5@gated-at.bofh.it> |
| In reply to | #1419756 |
On Sat, 11 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:51:26PM +0200, Wolfram Sang wrote: > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > > > Sweet, testing that now. It's not as efficient as glimpse because the query language is simpler. So more filtering has to be done at the ocaml level. But it's probably fine in most cases. julia
[toc] | [prev] | [next] | [standalone]
| From | Wolfram Sang <wsa@the-dreams.de> |
|---|---|
| Date | 2016-06-11 08:00 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rIK7D-7w3-1@gated-at.bofh.it> |
| In reply to | #1419909 |
[Multipart message — attachments visible in raw view] — view raw
> It's not as efficient as glimpse because the query language is simpler. Interesting, what is missing compared to glimpse? > So more filtering has to be done at the ocaml level. But it's probably > fine in most cases. For me, it has two advantages over glimpse: a) it is in the debian package repository b) the same database can be used with the code browser 'seascope' which can do nice things by feeding ctags on the fly with data from id-utils. Mileages vary, of course, just wanted to mention it to give pointers.
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-11 08:10 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rIKhj-7OF-9@gated-at.bofh.it> |
| In reply to | #1419918 |
On Sat, 11 Jun 2016, Wolfram Sang wrote: > > > It's not as efficient as glimpse because the query language is simpler. > > Interesting, what is missing compared to glimpse? Glimpse allows queries that are arbitrary formulas, up to a limited level of complexity, involving both && and ||. For idutils, Coccinelle runs lid on each of the tokens in the formula, and then does unions and intersections on the result. julia > > So more filtering has to be done at the ocaml level. But it's probably > > fine in most cases. > > For me, it has two advantages over glimpse: > > a) it is in the debian package repository > b) the same database can be used with the code browser 'seascope' > which can do nice things by feeding ctags on the fly with data > from id-utils. > > Mileages vary, of course, just wanted to mention it to give pointers. > >
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-11 07:30 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rIJEB-7mp-7@gated-at.bofh.it> |
| In reply to | #1419755 |
On Fri, 10 Jun 2016, Wolfram Sang wrote: > > AFAICT coccinelle does not have integration support for id-utils though. > > I used it just today ;) -- "--use-idutils ./ID" > > ID was generated with simple 'mkid -s'. Coccinelle includes a script scripts/idutils_index.sh This does mkid -i C --output .id-utils.index * julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-13 20:50 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJF5W-1G3-81@gated-at.bofh.it> |
| In reply to | #1419914 |
On Sat, Jun 11, 2016 at 07:24:51AM +0200, Julia Lawall wrote: > > > On Fri, 10 Jun 2016, Wolfram Sang wrote: > > > > AFAICT coccinelle does not have integration support for id-utils though. > > > > I used it just today ;) -- "--use-idutils ./ID" > > > > ID was generated with simple 'mkid -s'. > > Coccinelle includes a script scripts/idutils_index.sh > > This does mkid -i C --output .id-utils.index * I'll add support for detecting both. An issue of course is if any of these indexes grows stale. So I'll advise against these and recommend gitgrep unless the user has a hook to update index regularly. Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-11 07:20 +0200 |
| Subject | Re: [Cocci] [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rIJuW-7j9-3@gated-at.bofh.it> |
| In reply to | #1419751 |
On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:43:57PM +0200, Wolfram Sang wrote: > > > > Well, slightly better. > > > > > > No, it should be much better. You would have to look at the standard > > > > I use id-utils regularly and it is indeed at least a magnitude better. > > The indexing often pays off already with the first coccinelle run for > > me. Highly recommended. > > AFAICT coccinelle does not have integration support for id-utils though. Yes it does, for years. julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-13 21:40 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJFSi-2fB-27@gated-at.bofh.it> |
| In reply to | #1419738 |
On Fri, Jun 10, 2016 at 11:21:28PM +0200, Julia Lawall wrote: > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > On Fri, Jun 10, 2016 at 11:02:38PM +0200, Julia Lawall wrote: > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > Enable indexing optimizations heuristics. Coccinelle has > > > > support to make use of its own enhanced "grep" mechanisms > > > > instead of using regular grep for searching code 'coccigrep', > > > > in practice though this seems to not perform better than > > > > regular grep however its expected to help with some use cases > > > > so we use that if you have no other indexing options in place > > > > available. > > > > > > > > Since git has its own index, support for using 'git grep' has been > > > > added to Coccinelle, that should on average perform better than > > > > using the internal cocci grep, and regular grep. Lastly, Coccinelle > > > > has had support for glimpseindex for a long while, however the > > > > tool was previously closed source, its now open sourced, and > > > > provides the best performance, so support that if we can detect > > > > you have a glimpse index. > > > > > > > > These tests have been run on an 8 core system: > > > > > > > > Before: > > > > > > > > $ export COCCI=scripts/coccinelle/free/kfree.cocci > > > > $ time make coccicheck MODE=report > > > > > > > > Before this patch with no indexing or anything: > > > > > > > > real 16m22.435s > > > > user 128m30.060s > > > > sys 0m2.712s > > > > > > > > Using coccigrep (after this patch if you have no .git): > > > > > > > > real 16m27.650s > > > > user 128m47.904s > > > > sys 0m2.176s > > > > > > > > If you have .git and therefore use gitgrep: > > > > > > > > real 16m21.220s > > > > user 129m30.940s > > > > sys 0m2.060s > > > > > > > > And if you have a .glimpse_index: > > > > > > > > real 16m14.794s > > > > user 128m42.356s > > > > sys 0m1.880s > > > > > > I don't see any convincing differences in these times. > > > > > > I believe that Coccinelle's internal grep is always used, even with no > > > option. > > > > Ah that would explain it. This uses coccinelle 1.0.5, is the default > > there to use --use-coccigrep if no other index is specified ? > > It has been the default for a long time. > > > > I'm puzzled why glimpse gives no benefit. > > > > Well, slightly better. > > No, it should be much better. You would have to look at the standard > error to see if you are getting any benefit. There should be very few > occurrences of Skipping if you are really using glimpse. In any case, if > you asked for glimpse and it was not able to provide it, there should be > warning messages at the top of stderr. I'll redirect stderr to stdout by default when parmap support is used then. Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-13 22:00 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJGbE-2nm-13@gated-at.bofh.it> |
| In reply to | #1421248 |
On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > On Fri, Jun 10, 2016 at 11:21:28PM +0200, Julia Lawall wrote: > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > On Fri, Jun 10, 2016 at 11:02:38PM +0200, Julia Lawall wrote: > > > > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > > > Enable indexing optimizations heuristics. Coccinelle has > > > > > support to make use of its own enhanced "grep" mechanisms > > > > > instead of using regular grep for searching code 'coccigrep', > > > > > in practice though this seems to not perform better than > > > > > regular grep however its expected to help with some use cases > > > > > so we use that if you have no other indexing options in place > > > > > available. > > > > > > > > > > Since git has its own index, support for using 'git grep' has been > > > > > added to Coccinelle, that should on average perform better than > > > > > using the internal cocci grep, and regular grep. Lastly, Coccinelle > > > > > has had support for glimpseindex for a long while, however the > > > > > tool was previously closed source, its now open sourced, and > > > > > provides the best performance, so support that if we can detect > > > > > you have a glimpse index. > > > > > > > > > > These tests have been run on an 8 core system: > > > > > > > > > > Before: > > > > > > > > > > $ export COCCI=scripts/coccinelle/free/kfree.cocci > > > > > $ time make coccicheck MODE=report > > > > > > > > > > Before this patch with no indexing or anything: > > > > > > > > > > real 16m22.435s > > > > > user 128m30.060s > > > > > sys 0m2.712s > > > > > > > > > > Using coccigrep (after this patch if you have no .git): > > > > > > > > > > real 16m27.650s > > > > > user 128m47.904s > > > > > sys 0m2.176s > > > > > > > > > > If you have .git and therefore use gitgrep: > > > > > > > > > > real 16m21.220s > > > > > user 129m30.940s > > > > > sys 0m2.060s > > > > > > > > > > And if you have a .glimpse_index: > > > > > > > > > > real 16m14.794s > > > > > user 128m42.356s > > > > > sys 0m1.880s > > > > > > > > I don't see any convincing differences in these times. > > > > > > > > I believe that Coccinelle's internal grep is always used, even with no > > > > option. > > > > > > Ah that would explain it. This uses coccinelle 1.0.5, is the default > > > there to use --use-coccigrep if no other index is specified ? > > > > It has been the default for a long time. > > > > > > I'm puzzled why glimpse gives no benefit. > > > > > > Well, slightly better. > > > > No, it should be much better. You would have to look at the standard > > error to see if you are getting any benefit. There should be very few > > occurrences of Skipping if you are really using glimpse. In any case, if > > you asked for glimpse and it was not able to provide it, there should be > > warning messages at the top of stderr. > > I'll redirect stderr to stdout by default when parmap support is used then. Usually I put them in different files. julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-13 23:30 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJHAK-3pk-19@gated-at.bofh.it> |
| In reply to | #1421272 |
On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote: > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > > > On Fri, Jun 10, 2016 at 11:21:28PM +0200, Julia Lawall wrote: > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > On Fri, Jun 10, 2016 at 11:02:38PM +0200, Julia Lawall wrote: > > > > > > > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > > > > > Enable indexing optimizations heuristics. Coccinelle has > > > > > > support to make use of its own enhanced "grep" mechanisms > > > > > > instead of using regular grep for searching code 'coccigrep', > > > > > > in practice though this seems to not perform better than > > > > > > regular grep however its expected to help with some use cases > > > > > > so we use that if you have no other indexing options in place > > > > > > available. > > > > > > > > > > > > Since git has its own index, support for using 'git grep' has been > > > > > > added to Coccinelle, that should on average perform better than > > > > > > using the internal cocci grep, and regular grep. Lastly, Coccinelle > > > > > > has had support for glimpseindex for a long while, however the > > > > > > tool was previously closed source, its now open sourced, and > > > > > > provides the best performance, so support that if we can detect > > > > > > you have a glimpse index. > > > > > > > > > > > > These tests have been run on an 8 core system: > > > > > > > > > > > > Before: > > > > > > > > > > > > $ export COCCI=scripts/coccinelle/free/kfree.cocci > > > > > > $ time make coccicheck MODE=report > > > > > > > > > > > > Before this patch with no indexing or anything: > > > > > > > > > > > > real 16m22.435s > > > > > > user 128m30.060s > > > > > > sys 0m2.712s > > > > > > > > > > > > Using coccigrep (after this patch if you have no .git): > > > > > > > > > > > > real 16m27.650s > > > > > > user 128m47.904s > > > > > > sys 0m2.176s > > > > > > > > > > > > If you have .git and therefore use gitgrep: > > > > > > > > > > > > real 16m21.220s > > > > > > user 129m30.940s > > > > > > sys 0m2.060s > > > > > > > > > > > > And if you have a .glimpse_index: > > > > > > > > > > > > real 16m14.794s > > > > > > user 128m42.356s > > > > > > sys 0m1.880s > > > > > > > > > > I don't see any convincing differences in these times. > > > > > > > > > > I believe that Coccinelle's internal grep is always used, even with no > > > > > option. > > > > > > > > Ah that would explain it. This uses coccinelle 1.0.5, is the default > > > > there to use --use-coccigrep if no other index is specified ? > > > > > > It has been the default for a long time. > > > > > > > > I'm puzzled why glimpse gives no benefit. > > > > > > > > Well, slightly better. > > > > > > No, it should be much better. You would have to look at the standard > > > error to see if you are getting any benefit. There should be very few > > > occurrences of Skipping if you are really using glimpse. In any case, if > > > you asked for glimpse and it was not able to provide it, there should be > > > warning messages at the top of stderr. > > > > I'll redirect stderr to stdout by default when parmap support is used then. > > Usually I put them in different files. We can do that as well but I would only want to deal with parmap support case. Any preference? How about .coccicheck.stderr.$PID where PID would be the PID of the shell script? Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-14 07:30 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rJP5f-ls-1@gated-at.bofh.it> |
| In reply to | #1421352 |
On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote: > > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote: > > > > > On Fri, Jun 10, 2016 at 11:21:28PM +0200, Julia Lawall wrote: > > > > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > > > On Fri, Jun 10, 2016 at 11:02:38PM +0200, Julia Lawall wrote: > > > > > > > > > > > > > > > > > > On Fri, 10 Jun 2016, Luis R. Rodriguez wrote: > > > > > > > > > > > > > Enable indexing optimizations heuristics. Coccinelle has > > > > > > > support to make use of its own enhanced "grep" mechanisms > > > > > > > instead of using regular grep for searching code 'coccigrep', > > > > > > > in practice though this seems to not perform better than > > > > > > > regular grep however its expected to help with some use cases > > > > > > > so we use that if you have no other indexing options in place > > > > > > > available. > > > > > > > > > > > > > > Since git has its own index, support for using 'git grep' has been > > > > > > > added to Coccinelle, that should on average perform better than > > > > > > > using the internal cocci grep, and regular grep. Lastly, Coccinelle > > > > > > > has had support for glimpseindex for a long while, however the > > > > > > > tool was previously closed source, its now open sourced, and > > > > > > > provides the best performance, so support that if we can detect > > > > > > > you have a glimpse index. > > > > > > > > > > > > > > These tests have been run on an 8 core system: > > > > > > > > > > > > > > Before: > > > > > > > > > > > > > > $ export COCCI=scripts/coccinelle/free/kfree.cocci > > > > > > > $ time make coccicheck MODE=report > > > > > > > > > > > > > > Before this patch with no indexing or anything: > > > > > > > > > > > > > > real 16m22.435s > > > > > > > user 128m30.060s > > > > > > > sys 0m2.712s > > > > > > > > > > > > > > Using coccigrep (after this patch if you have no .git): > > > > > > > > > > > > > > real 16m27.650s > > > > > > > user 128m47.904s > > > > > > > sys 0m2.176s > > > > > > > > > > > > > > If you have .git and therefore use gitgrep: > > > > > > > > > > > > > > real 16m21.220s > > > > > > > user 129m30.940s > > > > > > > sys 0m2.060s > > > > > > > > > > > > > > And if you have a .glimpse_index: > > > > > > > > > > > > > > real 16m14.794s > > > > > > > user 128m42.356s > > > > > > > sys 0m1.880s > > > > > > > > > > > > I don't see any convincing differences in these times. > > > > > > > > > > > > I believe that Coccinelle's internal grep is always used, even with no > > > > > > option. > > > > > > > > > > Ah that would explain it. This uses coccinelle 1.0.5, is the default > > > > > there to use --use-coccigrep if no other index is specified ? > > > > > > > > It has been the default for a long time. > > > > > > > > > > I'm puzzled why glimpse gives no benefit. > > > > > > > > > > Well, slightly better. > > > > > > > > No, it should be much better. You would have to look at the standard > > > > error to see if you are getting any benefit. There should be very few > > > > occurrences of Skipping if you are really using glimpse. In any case, if > > > > you asked for glimpse and it was not able to provide it, there should be > > > > warning messages at the top of stderr. > > > > > > I'll redirect stderr to stdout by default when parmap support is used then. > > > > Usually I put them in different files. > > We can do that as well but I would only want to deal with parmap support > case. Any preference? How about .coccicheck.stderr.$PID where PID would > be the PID of the shell script? I don't understand the connection with parmap. Originally our use of parmap made output files based on pids. Maybe this is the default for parmap. I found this completely unusable. I guess one could look at the dates to see which file is the most recent one, but it seems tedious. If you are putting the standard output in x.out, then put the standard error in x.err. julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-14 21:30 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rK2c9-Fp-19@gated-at.bofh.it> |
| In reply to | #1421582 |
On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
>
>
> On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
>
> > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > >
> > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > >
> > > Usually I put them in different files.
> >
> > We can do that as well but I would only want to deal with parmap support
> > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > be the PID of the shell script?
>
> I don't understand the connection with parmap.
When parmap support is not available the cocciscript will currently
disregard stderr, output is provided as it comes to stdout from each
thread I guess.
> Originally our use of parmap made output files based on pids. Maybe this
> is the default for parmap. I found this completely unusable. I guess one
> could look at the dates to see which file is the most recent one, but it
> seems tedious. If you are putting the standard output in x.out, then put
> the standard error in x.err.
I'll use ${DIR}/coccicheck.$$.err for stderr.
Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-14 22:50 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rK3rz-1nX-13@gated-at.bofh.it> |
| In reply to | #1422247 |
On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> >
> >
> > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> >
> > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > >
> > > >
> > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > >
> > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > >
> > > > Usually I put them in different files.
> > >
> > > We can do that as well but I would only want to deal with parmap support
> > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > be the PID of the shell script?
> >
> > I don't understand the connection with parmap.
>
> When parmap support is not available the cocciscript will currently
> disregard stderr, output is provided as it comes to stdout from each
> thread I guess.
Deepa's recent patch to coccicheck made apparent that Coccicheck uses
--very-quiet, so there is standard error.
> > Originally our use of parmap made output files based on pids. Maybe this
> > is the default for parmap. I found this completely unusable. I guess one
> > could look at the dates to see which file is the most recent one, but it
> > seems tedious. If you are putting the standard output in x.out, then put
> > the standard error in x.err.
>
> I'll use ${DIR}/coccicheck.$$.err for stderr.
What is ${DIR}? and what is $$?
julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-14 23:20 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rK3UB-1OB-9@gated-at.bofh.it> |
| In reply to | #1422319 |
On Tue, Jun 14, 2016 at 10:47:32PM +0200, Julia Lawall wrote:
>
>
> On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
>
> > On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > >
> > > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > > >
> > > > >
> > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > >
> > > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > > >
> > > > > Usually I put them in different files.
> > > >
> > > > We can do that as well but I would only want to deal with parmap support
> > > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > > be the PID of the shell script?
> > >
> > > I don't understand the connection with parmap.
> >
> > When parmap support is not available the cocciscript will currently
> > disregard stderr, output is provided as it comes to stdout from each
> > thread I guess.
>
> Deepa's recent patch to coccicheck made apparent that Coccicheck uses
> --very-quiet, so there is standard error.
OK I'm disegarding the redirect for non-parmap for now but we'd have to
determine if we want to append or add one per PID... I rather leave that
stuff as-is and encourage folks to upgrade coccinelle.
> > > Originally our use of parmap made output files based on pids. Maybe this
> > > is the default for parmap. I found this completely unusable. I guess one
> > > could look at the dates to see which file is the most recent one, but it
> > > seems tedious. If you are putting the standard output in x.out, then put
> > > the standard error in x.err.
> >
> > I'll use ${DIR}/coccicheck.$$.err for stderr.
>
> What is ${DIR}? and what is $$?
When you run scripts/coccicheck we take the absolute directory
of it and then go down one level of directory, so in this case it
would be the base directory of the Linux kernel.
$$ is the PID of the bash script.
Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-14 23:20 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rK3UB-1OB-27@gated-at.bofh.it> |
| In reply to | #1422344 |
On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> On Tue, Jun 14, 2016 at 10:47:32PM +0200, Julia Lawall wrote:
> >
> >
> > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> >
> > > On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> > > >
> > > >
> > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > >
> > > > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > > > >
> > > > > >
> > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > > >
> > > > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > > > >
> > > > > > Usually I put them in different files.
> > > > >
> > > > > We can do that as well but I would only want to deal with parmap support
> > > > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > > > be the PID of the shell script?
> > > >
> > > > I don't understand the connection with parmap.
> > >
> > > When parmap support is not available the cocciscript will currently
> > > disregard stderr, output is provided as it comes to stdout from each
> > > thread I guess.
> >
> > Deepa's recent patch to coccicheck made apparent that Coccicheck uses
> > --very-quiet, so there is standard error.
>
> OK I'm disegarding the redirect for non-parmap for now but we'd have to
> determine if we want to append or add one per PID... I rather leave that
> stuff as-is and encourage folks to upgrade coccinelle.
If coccicheck is using --very-quiet, there will not be much stderr of
interest when using parmap either.
I'm not sure to understan the issue about appending. Appending what to
what? While Coccinelle is running, you have a directory with the same
name as your semantic patch (or a directory name that you specify with
--tmp-dir) that has separate files for each core's standard output and
standard error. At the end, when all the code has been treated,
Coccinelle writes the successive stdouts to standard output, and the
sucessive stderrs to standard error.
> > > > Originally our use of parmap made output files based on pids. Maybe this
> > > > is the default for parmap. I found this completely unusable. I guess one
> > > > could look at the dates to see which file is the most recent one, but it
> > > > seems tedious. If you are putting the standard output in x.out, then put
> > > > the standard error in x.err.
> > >
> > > I'll use ${DIR}/coccicheck.$$.err for stderr.
> >
> > What is ${DIR}? and what is $$?
>
> When you run scripts/coccicheck we take the absolute directory
> of it and then go down one level of directory, so in this case it
> would be the base directory of the Linux kernel.
>
> $$ is the PID of the bash script.
OK. I still don't find PIDs useful, but I guess if we are talking about
the entire output of coccicheck, there is not much else to do. Normally,
I don't want these files accumulating, and just write over the old ones.
julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-15 00:10 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rK4GZ-2lJ-23@gated-at.bofh.it> |
| In reply to | #1422349 |
On Tue, Jun 14, 2016 at 11:17:13PM +0200, Julia Lawall wrote:
>
>
> On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
>
> > On Tue, Jun 14, 2016 at 10:47:32PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> > >
> > > > On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> > > > >
> > > > >
> > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > >
> > > > > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > > > >
> > > > > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > > > > >
> > > > > > > Usually I put them in different files.
> > > > > >
> > > > > > We can do that as well but I would only want to deal with parmap support
> > > > > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > > > > be the PID of the shell script?
> > > > >
> > > > > I don't understand the connection with parmap.
> > > >
> > > > When parmap support is not available the cocciscript will currently
> > > > disregard stderr, output is provided as it comes to stdout from each
> > > > thread I guess.
> > >
> > > Deepa's recent patch to coccicheck made apparent that Coccicheck uses
> > > --very-quiet, so there is standard error.
> >
> > OK I'm disegarding the redirect for non-parmap for now but we'd have to
> > determine if we want to append or add one per PID... I rather leave that
> > stuff as-is and encourage folks to upgrade coccinelle.
>
> If coccicheck is using --very-quiet, there will not be much stderr of
> interest when using parmap either.
OK I don't follow. Does coccinelle only direct error to stderr when --very-quiet
is used ? Or does using --very-quiet suppress stderr ?
> I'm not sure to understan the issue about appending. Appending what to
> what?
> While Coccinelle is running, you have a directory with the same
> name as your semantic patch (or a directory name that you specify with
> --tmp-dir) that has separate files for each core's standard output and
> standard error. At the end, when all the code has been treated,
> Coccinelle writes the successive stdouts to standard output, and the
> sucessive stderrs to standard error.
Ah I see. OK yeah never mind about appending.
> > > > > Originally our use of parmap made output, specia files based on pids. Maybe this
> > > > > is the default for parmap. I found this completely unusable. I guess one
> > > > > could look at the dates to see which file is the most recent one, but it
> > > > > seems tedious. If you are putting the standard output in x.out, then put
> > > > > the standard error in x.err.
> > > >
> > > > I'll use ${DIR}/coccicheck.$$.err for stderr.
> > >
> > > What is ${DIR}? and what is $$?
> >
> > When you run scripts/coccicheck we take the absolute directory
> > of it and then go down one level of directory, so in this case it
> > would be the base directory of the Linux kernel.
> >
> > $$ is the PID of the bash script.
>
> OK. I still don't find PIDs useful, but I guess if we are talking about
> the entire output of coccicheck, there is not much else to do. Normally,
> I don't want these files accumulating, and just write over the old ones.
Which is why I would much prefer to instead just redirect in coccicheck
case stderr to stdout from coccinelle. Is that preferred?
Luis
[toc] | [prev] | [next] | [standalone]
| From | Julia Lawall <julia.lawall@lip6.fr> |
|---|---|
| Date | 2016-06-15 09:50 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rKdKi-83X-23@gated-at.bofh.it> |
| In reply to | #1422380 |
On Wed, 15 Jun 2016, Luis R. Rodriguez wrote:
> On Tue, Jun 14, 2016 at 11:17:13PM +0200, Julia Lawall wrote:
> >
> >
> > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> >
> > > On Tue, Jun 14, 2016 at 10:47:32PM +0200, Julia Lawall wrote:
> > > >
> > > >
> > > > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> > > >
> > > > > On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> > > > > >
> > > > > >
> > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > >
> > > > > > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > > > > >
> > > > > > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > > > > > >
> > > > > > > > Usually I put them in different files.
> > > > > > >
> > > > > > > We can do that as well but I would only want to deal with parmap support
> > > > > > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > > > > > be the PID of the shell script?
> > > > > >
> > > > > > I don't understand the connection with parmap.
> > > > >
> > > > > When parmap support is not available the cocciscript will currently
> > > > > disregard stderr, output is provided as it comes to stdout from each
> > > > > thread I guess.
> > > >
> > > > Deepa's recent patch to coccicheck made apparent that Coccicheck uses
> > > > --very-quiet, so there is standard error.
> > >
> > > OK I'm disegarding the redirect for non-parmap for now but we'd have to
> > > determine if we want to append or add one per PID... I rather leave that
> > > stuff as-is and encourage folks to upgrade coccinelle.
> >
> > If coccicheck is using --very-quiet, there will not be much stderr of
> > interest when using parmap either.
>
> OK I don't follow. Does coccinelle only direct error to stderr when --very-quiet
> is used ? Or does using --very-quiet suppress stderr ?
--very-quiet suppresses most administrative messages that go to stderr.
There are still actual errors. Bu you don't see eg what file is being
currently processed.
> > > > > > Originally our use of parmap made output, specia files based on pids. Maybe this
> > > > > > is the default for parmap. I found this completely unusable. I guess one
> > > > > > could look at the dates to see which file is the most recent one, but it
> > > > > > seems tedious. If you are putting the standard output in x.out, then put
> > > > > > the standard error in x.err.
> > > > >
> > > > > I'll use ${DIR}/coccicheck.$$.err for stderr.
> > > >
> > > > What is ${DIR}? and what is $$?
> > >
> > > When you run scripts/coccicheck we take the absolute directory
> > > of it and then go down one level of directory, so in this case it
> > > would be the base directory of the Linux kernel.
> > >
> > > $$ is the PID of the bash script.
> >
> > OK. I still don't find PIDs useful, but I guess if we are talking about
> > the entire output of coccicheck, there is not much else to do. Normally,
> > I don't want these files accumulating, and just write over the old ones.
>
> Which is why I would much prefer to instead just redirect in coccicheck
> case stderr to stdout from coccinelle. Is that preferred?
Then things will be merged in strange ways.
Why not just let the user decide what to do with these things?
julia
[toc] | [prev] | [next] | [standalone]
| From | "Luis R. Rodriguez" <mcgrof@kernel.org> |
|---|---|
| Date | 2016-06-15 17:40 +0200 |
| Subject | Re: [PATCH 4/4] coccicheck: add indexing enhancement options |
| Message-ID | <rKl57-4l9-15@gated-at.bofh.it> |
| In reply to | #1422734 |
On Wed, Jun 15, 2016 at 09:39:49AM +0200, Julia Lawall wrote:
>
>
> On Wed, 15 Jun 2016, Luis R. Rodriguez wrote:
>
> > On Tue, Jun 14, 2016 at 11:17:13PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> > >
> > > > On Tue, Jun 14, 2016 at 10:47:32PM +0200, Julia Lawall wrote:
> > > > >
> > > > >
> > > > > On Tue, 14 Jun 2016, Luis R. Rodriguez wrote:
> > > > >
> > > > > > On Tue, Jun 14, 2016 at 07:22:03AM +0200, Julia Lawall wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > > >
> > > > > > > > On Mon, Jun 13, 2016 at 09:50:15PM +0200, Julia Lawall wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, 13 Jun 2016, Luis R. Rodriguez wrote:
> > > > > > > > > >
> > > > > > > > > > I'll redirect stderr to stdout by default when parmap support is used then.
> > > > > > > > >
> > > > > > > > > Usually I put them in different files.
> > > > > > > >
> > > > > > > > We can do that as well but I would only want to deal with parmap support
> > > > > > > > case. Any preference? How about .coccicheck.stderr.$PID where PID would
> > > > > > > > be the PID of the shell script?
> > > > > > >
> > > > > > > I don't understand the connection with parmap.
> > > > > >
> > > > > > When parmap support is not available the cocciscript will currently
> > > > > > disregard stderr, output is provided as it comes to stdout from each
> > > > > > thread I guess.
> > > > >
> > > > > Deepa's recent patch to coccicheck made apparent that Coccicheck uses
> > > > > --very-quiet, so there is standard error.
> > > >
> > > > OK I'm disegarding the redirect for non-parmap for now but we'd have to
> > > > determine if we want to append or add one per PID... I rather leave that
> > > > stuff as-is and encourage folks to upgrade coccinelle.
> > >
> > > If coccicheck is using --very-quiet, there will not be much stderr of
> > > interest when using parmap either.
> >
> > OK I don't follow. Does coccinelle only direct error to stderr when --very-quiet
> > is used ? Or does using --very-quiet suppress stderr ?
>
> --very-quiet suppresses most administrative messages that go to stderr.
> There are still actual errors. Bu you don't see eg what file is being
> currently processed.
OK thanks. I remove --very-quiet now if --profile is used within SPFLAGS, I'll extend
this to also avoid --very-quiet if --show-trying is used. SPFLAGS is where you can
specify extra options that the script doesn't specifically support.
> > > > > > > Originally our use of parmap made output, specia files based on pids. Maybe this
> > > > > > > is the default for parmap. I found this completely unusable. I guess one
> > > > > > > could look at the dates to see which file is the most recent one, but it
> > > > > > > seems tedious. If you are putting the standard output in x.out, then put
> > > > > > > the standard error in x.err.
> > > > > >
> > > > > > I'll use ${DIR}/coccicheck.$$.err for stderr.
> > > > >
> > > > > What is ${DIR}? and what is $$?
> > > >
> > > > When you run scripts/coccicheck we take the absolute directory
> > > > of it and then go down one level of directory, so in this case it
> > > > would be the base directory of the Linux kernel.
> > > >
> > > > $$ is the PID of the bash script.
> > >
> > > OK. I still don't find PIDs useful, but I guess if we are talking about
> > > the entire output of coccicheck, there is not much else to do. Normally,
> > > I don't want these files accumulating, and just write over the old ones.
> >
> > Which is why I would much prefer to instead just redirect in coccicheck
> > case stderr to stdout from coccinelle. Is that preferred?
>
> Then things will be merged in strange ways.
>
> Why not just let the user decide what to do with these things?
Sure, what should be the default?
Luis
[toc] | [prev] | [next] | [standalone]
Page 2 of 3 — ← Prev page 1 [2] 3 Next page →
Back to top | Article view | linux.kernel
csiph-web