Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1231225 > unrolled thread

Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and findfs objects

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Matt Fleming <matt@codeblueprint.co.uk> - 2015-09-23 10:30 +0200
    Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Jiri Olsa <jolsa@redhat.com> - 2015-09-23 10:40 +0200
      Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Matt Fleming <matt@codeblueprint.co.uk> - 2015-09-23 12:10 +0200
        Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Michael Petlan <mpetlan@redhat.com> - 2015-09-24 17:10 +0200
      Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Arnaldo Carvalho de Melo <acme@redhat.com> - 2015-09-23 15:50 +0200
        Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Jiri Olsa <jolsa@redhat.com> - 2015-09-23 16:00 +0200
          Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Arnaldo Carvalho de Melo <acme@redhat.com> - 2015-09-23 16:00 +0200
        Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Matt Fleming <matt@codeblueprint.co.uk> - 2015-09-24 14:20 +0200
          Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and  findfs objects Arnaldo Carvalho de Melo <acme@redhat.com> - 2015-09-24 16:30 +0200

#1231225 — Re: [tip:perf/core] tools lib api fs: Remove debugfs, tracefs and findfs objects

On Mon, 21 Sep, at 05:20:03PM, Vinson Lee wrote:
> On Mon, Sep 14, 2015 at 11:59 PM, tip-bot for Jiri Olsa
> <tipbot@zytor.com> wrote:
> > Commit-ID:  60a1133a5b39738671eff1e4d77bedc1ee3fa528
> > Gitweb:     http://git.kernel.org/tip/60a1133a5b39738671eff1e4d77bedc1ee3fa528
> > Author:     Jiri Olsa <jolsa@kernel.org>
> > AuthorDate: Wed, 2 Sep 2015 09:56:44 +0200
> > Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
> > CommitDate: Mon, 14 Sep 2015 12:50:15 -0300
> >
> > tools lib api fs: Remove debugfs, tracefs and findfs objects
> >
> > We have all the functionality in fs.c, let's remove unneeded
> > objects.
> >
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > Cc: David Ahern <dsahern@gmail.com>
> > Cc: Matt Fleming <matt@codeblueprint.co.uk>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Cc: Raphael Beamonte <raphael.beamonte@gmail.com>
> > Cc: Steven Rostedt <rostedt@goodmis.org>
> > Link: http://lkml.kernel.org/r/1441180605-24737-15-git-send-email-jolsa@kernel.org
> > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> 
> Hi.
> 
> This commit seems to have introduced a build failure with tools/vm.
> 
> $ make -C tools vm
> [...]
> gcc -Wall -Wextra -I../lib/ -o page-types page-types.c ../lib/api/libapi.a
> page-types.c:45:28: fatal error: api/fs/debugfs.h: No such file or directory
>  #include <api/fs/debugfs.h>

Given the ferocious pace of development of tools/perf, is there not
some kind of automated build that happens when new patches are picked
up, before they're pushed out?

Things are refactored and changed so fast in this area (I dare say
faster than almost any other part of the kernel source tree) that not
having the safety net of automated builds just seems suicidal.

And that doesn't even begin to cover runtime testing, since I've
noticed things breaking in tools/perf and people not catching it
immediately.

Does automated testing exist for perf tools development?

-- 
Matt Fleming, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1231231

On Wed, Sep 23, 2015 at 09:23:02AM +0100, Matt Fleming wrote:
> On Mon, 21 Sep, at 05:20:03PM, Vinson Lee wrote:
> > On Mon, Sep 14, 2015 at 11:59 PM, tip-bot for Jiri Olsa
> > <tipbot@zytor.com> wrote:
> > > Commit-ID:  60a1133a5b39738671eff1e4d77bedc1ee3fa528
> > > Gitweb:     http://git.kernel.org/tip/60a1133a5b39738671eff1e4d77bedc1ee3fa528
> > > Author:     Jiri Olsa <jolsa@kernel.org>
> > > AuthorDate: Wed, 2 Sep 2015 09:56:44 +0200
> > > Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
> > > CommitDate: Mon, 14 Sep 2015 12:50:15 -0300
> > >
> > > tools lib api fs: Remove debugfs, tracefs and findfs objects
> > >
> > > We have all the functionality in fs.c, let's remove unneeded
> > > objects.
> > >
> > > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> > > Cc: David Ahern <dsahern@gmail.com>
> > > Cc: Matt Fleming <matt@codeblueprint.co.uk>
> > > Cc: Namhyung Kim <namhyung@kernel.org>
> > > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > > Cc: Raphael Beamonte <raphael.beamonte@gmail.com>
> > > Cc: Steven Rostedt <rostedt@goodmis.org>
> > > Link: http://lkml.kernel.org/r/1441180605-24737-15-git-send-email-jolsa@kernel.org
> > > Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > 
> > Hi.
> > 
> > This commit seems to have introduced a build failure with tools/vm.
> > 
> > $ make -C tools vm
> > [...]
> > gcc -Wall -Wextra -I../lib/ -o page-types page-types.c ../lib/api/libapi.a
> > page-types.c:45:28: fatal error: api/fs/debugfs.h: No such file or directory
> >  #include <api/fs/debugfs.h>
> 
> Given the ferocious pace of development of tools/perf, is there not
> some kind of automated build that happens when new patches are picked
> up, before they're pushed out?
> 
> Things are refactored and changed so fast in this area (I dare say
> faster than almost any other part of the kernel source tree) that not
> having the safety net of automated builds just seems suicidal.
> 
> And that doesn't even begin to cover runtime testing, since I've
> noticed things breaking in tools/perf and people not catching it
> immediately.
> 
> Does automated testing exist for perf tools development?

heh, we've been playing game "who first mention it in public will implement it" ... you won! ;-)

AFAIK we have: 
  - 'perf test' for perf specific functionality
  - 'make -f tests/make' for building
  - build framework tests

I 'try' to run those before sending anything out, but we dont have
automated thing that would run it any time Arnaldo push new perf/core.

The RedHat QE has some more perf tool tests. There was some movement
to make those public, but not sure how it ended up.. ccing Michael Petlan
for news on this ;-)

jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1231303

On Wed, 23 Sep, at 10:39:06AM, Jiri Olsa wrote:
> On Wed, Sep 23, 2015 at 09:23:02AM +0100, Matt Fleming wrote:
> > 
> > Does automated testing exist for perf tools development?
> 
> heh, we've been playing game "who first mention it in public will implement it" ... you won! ;-)
 
Hehe, whoops!

> AFAIK we have: 
>   - 'perf test' for perf specific functionality
>   - 'make -f tests/make' for building
>   - build framework tests
> 
> I 'try' to run those before sending anything out, but we dont have
> automated thing that would run it any time Arnaldo push new perf/core.
 
Right. The problem with manual steps is that they're easy to forget.
Furthermore, it actively discourages you from adding new testing
functionality that requires more manual steps (who wants to remember
to type another command?).

Yes, you can script it, but then every developer ends up with their
own version, which get out of sync, or work slightly differently etc.

Also, now that we've potentially got perf arch tests coming [1] you or
Arnaldo may not always have the hardware available to ensure that no
regressions were introduced to the runtime testing, or the OS
installations to perform build testing, for say, Ubuntu or OpenSUSE.

That is kind of a separate problem (automated testing of a matrix of
OS and hardware configs), but having a single, standard way to
automate build/runtime testing of tools/perf is the first step.

> The RedHat QE has some more perf tool tests. There was some movement
> to make those public, but not sure how it ended up.. ccing Michael Petlan
> for news on this ;-)

Cool! I'd definitely be interested in knowing the details.

[1] - https://lkml.kernel.org/r/1441479742-15402-1-git-send-email-matt@codeblueprint.co.uk

-- 
Matt Fleming, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232175

On Wed, 2015-09-23 at 11:08 +0100, Matt Fleming wrote:

[SNIP]

> > The RedHat QE has some more perf tool tests. There was some movement
> > to make those public, but not sure how it ended up.. ccing Michael Petlan
> > for news on this ;-)
> 
> Cool! I'd definitely be interested in knowing the details.
> 

Hi!

Yes, we have some tests, but they really need some refactoring and then
extending.

There are many "regression" tests that cover some extreme situations
that failed with some kernel/perf version on some hardware. They are
probably not very useful for the purpose mentioned here.

Then there are some tests that should cover basic functionality and
check for the correctness of perf's behaviour. Since it became being
pretty messy, I have got an idea to rewrite that in a more structured
and robust way and make it public.

So I started with some skeleton and tests for perf stat builtin sub
command [1]. My idea is to port there all the meaningful tests that
we have at Red Hat. Then I will be happy if someone else is interested
in contributing some more coverage, ideas or whatever...

I am on a PTO for two weeks from now, so I will respond after it, if you
have any questions, suggestions or ideas.


Regards,
Michael



[1] https://github.com/rfmvh/perftool-testsuite


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1231453

Em Wed, Sep 23, 2015 at 10:39:06AM +0200, Jiri Olsa escreveu:
> On Wed, Sep 23, 2015 at 09:23:02AM +0100, Matt Fleming wrote:
> > On Mon, 21 Sep, at 05:20:03PM, Vinson Lee wrote:
> > > This commit seems to have introduced a build failure with tools/vm.
> > > 
> > > $ make -C tools vm
> > > [...]
> > > gcc -Wall -Wextra -I../lib/ -o page-types page-types.c ../lib/api/libapi.a
> > > page-types.c:45:28: fatal error: api/fs/debugfs.h: No such file or directory
> > >  #include <api/fs/debugfs.h>
> > 
> > Given the ferocious pace of development of tools/perf, is there not
> > some kind of automated build that happens when new patches are picked
> > up, before they're pushed out?

> > Things are refactored and changed so fast in this area (I dare say
> > faster than almost any other part of the kernel source tree) that not
> > having the safety net of automated builds just seems suicidal.

Well, I don't want to die, and I work with people that would kill me if
I behaved that way, so I think its not _that_ bad, there are safeguards,
and we're always thinking about adding some more. 8-)

> > And that doesn't even begin to cover runtime testing, since I've
> > noticed things breaking in tools/perf and people not catching it
> > immediately.
> > 
> > Does automated testing exist for perf tools development?
 
> heh, we've been playing game "who first mention it in public will
> implement it" ... you won! ;-)

Nah, you did lotsa already with tools/perf/tests/make

[acme@zoo linux]$ grep ^make tools/perf/tests/make 
make_clean_all      := clean all
make_python_perf_so := python/perf.so
make_debug          := DEBUG=1
make_no_libperl     := NO_LIBPERL=1
make_no_libpython   := NO_LIBPYTHON=1
make_no_scripts     := NO_LIBPYTHON=1 NO_LIBPERL=1
make_no_newt        := NO_NEWT=1
make_no_slang       := NO_SLANG=1
make_no_gtk2        := NO_GTK2=1
make_no_ui          := NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
make_no_demangle    := NO_DEMANGLE=1
make_no_libelf      := NO_LIBELF=1
make_no_libunwind   := NO_LIBUNWIND=1
make_no_libdw_dwarf_unwind := NO_LIBDW_DWARF_UNWIND=1
make_no_backtrace   := NO_BACKTRACE=1
make_no_libnuma     := NO_LIBNUMA=1
make_no_libaudit    := NO_LIBAUDIT=1
make_no_libbionic   := NO_LIBBIONIC=1
make_no_auxtrace    := NO_AUXTRACE=1
make_tags           := tags
make_cscope         := cscope
make_help           := help
make_doc            := doc
make_perf_o           := perf.o
make_util_map_o       := util/map.o
make_util_pmu_bison_o := util/pmu-bison.o
make_install        := install
make_install_bin    := install-bin
make_install_doc    := install-doc
make_install_man    := install-man
make_install_html   := install-html
make_install_info   := install-info
make_install_pdf    := install-pdf
make_install_prefix       := install prefix=/tmp/krava
make_install_prefix_slash := install prefix=/tmp/krava/
make_static         := LDFLAGS=-static
make_minimal        := NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1
make_minimal        += NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1
NO_BACKTRACE=1
make_minimal        += NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1
make_minimal        += NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1
make_kernelsrc:
make_kernelsrc_tools:
[acme@zoo linux]$ 

This takes a lot of testing, I plan on using TypeChef to speed that up
and increase the number of tests:

https://github.com/ckaestne/TypeChef-LinuxAnalysis/blob/master/README.md

And 'perf test' has 40 tests, with some being really a multiplexor, like
the perf_event_attr ones, that will run the tools and look at how they
set up perf_event_attr for multiple command line options:

[root@zoo ~]# perf test | tail -10
31: Test output sorting of hist entries                      : Ok
32: Test cumulation of child hist entries                    : Ok
33: Test tracking with sched_switch                          : Ok
34: Filter fds with revents mask in a fdarray                : Ok
35: Add fd to a fdarray, making it autogrow                  : Ok
36: Test kmod_path__parse function                           : Ok
37: Test thread map                                          : Ok
38: Test LLVM searching and compiling                        : (skip bpf parsing) Ok
39: Test x86 instruction decoder - new instructions          : Ok
40: Test topology in session                                 : Ok
[root@zoo ~]# 

New stuff normally comes with new 'perf test' entries, Intel PT borrowed
the kernel x86 instruction decoder: added a 'perf test' entry, AFAIK
there was no similar test for it in the kernel proper, IIRC Masami plans
to do it.

The attr one you can look at:

[acme@zoo linux]$ ls -la tools/perf/tests/attr/test-* | wc -l
33
 
> AFAIK we have: 
>   - 'perf test' for perf specific functionality
>   - 'make -f tests/make' for building
>   - build framework tests
> 
> I 'try' to run those before sending anything out, but we dont have
> automated thing that would run it any time Arnaldo push new perf/core.

Well, I do run it in multiple distros, like RHEL5, RHEL6 and RHEL7
besides Fedora 21.

We're getting used to tools/{lib,include}/ so this happened, but
otherwise I don't feel like there are that many problems cropping up as
you seem to think :-\

Of course, in these days of CI, I'd love if someone would hook 'make -C
tools/perf build-test' and 'perf test' somewhere to be run for every
changeset.
 
> The RedHat QE has some more perf tool tests. There was some movement
> to make those public, but not sure how it ended up.. ccing Michael Petlan
> for news on this ;-)

Yeah, this too has helped catch and fix problems.

BTW, tools/vm/ was reported yesterday and a fix is already in
tip/perf/core/:

https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/tools/vm?id=f6489bc2d402c0db84aa64f13b864d17f7eecb07

Age       Commit message (Expand)                                           Author                   Files Lines
12 hours  tools vm: Fix build due to removal of tools/lib/api/fs/debugfs.h  Arnaldo Carvalho de Melo	1  -3/+3

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1231455

On Wed, Sep 23, 2015 at 10:44:56AM -0300, Arnaldo Carvalho de Melo wrote:

SNIP

> New stuff normally comes with new 'perf test' entries, Intel PT borrowed
> the kernel x86 instruction decoder: added a 'perf test' entry, AFAIK
> there was no similar test for it in the kernel proper, IIRC Masami plans
> to do it.
> 
> The attr one you can look at:
> 
> [acme@zoo linux]$ ls -la tools/perf/tests/attr/test-* | wc -l
> 33
>  
> > AFAIK we have: 
> >   - 'perf test' for perf specific functionality
> >   - 'make -f tests/make' for building
> >   - build framework tests
> > 
> > I 'try' to run those before sending anything out, but we dont have
> > automated thing that would run it any time Arnaldo push new perf/core.
> 
> Well, I do run it in multiple distros, like RHEL5, RHEL6 and RHEL7
> besides Fedora 21.
> 
> We're getting used to tools/{lib,include}/ so this happened, but
> otherwise I don't feel like there are that many problems cropping up as
> you seem to think :-\
> 
> Of course, in these days of CI, I'd love if someone would hook 'make -C
> tools/perf build-test' and 'perf test' somewhere to be run for every
> changeset.

yep, thats what I meant.. having this hooked up to your perf/core
would be big help

jirka
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1231456

Em Wed, Sep 23, 2015 at 03:50:12PM +0200, Jiri Olsa escreveu:
> On Wed, Sep 23, 2015 at 10:44:56AM -0300, Arnaldo Carvalho de Melo wrote:
> 
> SNIP
> 
> > New stuff normally comes with new 'perf test' entries, Intel PT borrowed
> > the kernel x86 instruction decoder: added a 'perf test' entry, AFAIK
> > there was no similar test for it in the kernel proper, IIRC Masami plans
> > to do it.
> > 
> > The attr one you can look at:
> > 
> > [acme@zoo linux]$ ls -la tools/perf/tests/attr/test-* | wc -l
> > 33
> >  
> > > AFAIK we have: 
> > >   - 'perf test' for perf specific functionality
> > >   - 'make -f tests/make' for building
> > >   - build framework tests
> > > 
> > > I 'try' to run those before sending anything out, but we dont have
> > > automated thing that would run it any time Arnaldo push new perf/core.
> > 
> > Well, I do run it in multiple distros, like RHEL5, RHEL6 and RHEL7
> > besides Fedora 21.
> > 
> > We're getting used to tools/{lib,include}/ so this happened, but
> > otherwise I don't feel like there are that many problems cropping up as
> > you seem to think :-\
> > 
> > Of course, in these days of CI, I'd love if someone would hook 'make -C
> > tools/perf build-test' and 'perf test' somewhere to be run for every
> > changeset.
> 
> yep, thats what I meant.. having this hooked up to your perf/core
> would be big help

Till then, I'll turn more machines on here at my lab to do do it
manually and add an entry for:

 make -C tools/vm/

In that 'make -C tools/perf build-test'

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232083

On Wed, 23 Sep, at 10:44:56AM, Arnaldo Carvalho de Melo wrote:
> 
> Of course, in these days of CI, I'd love if someone would hook 'make -C
> tools/perf build-test' and 'perf test' somewhere to be run for every
> changeset.
  
Yes please!

> BTW, tools/vm/ was reported yesterday and a fix is already in
> tip/perf/core/:
> 
> https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/tools/vm?id=f6489bc2d402c0db84aa64f13b864d17f7eecb07
> 
> Age       Commit message (Expand)                                           Author                   Files Lines
> 12 hours  tools vm: Fix build due to removal of tools/lib/api/fs/debugfs.h  Arnaldo Carvalho de Melo	1  -3/+3

It's not that this wasn't fixed quickly (kudos for that, btw), rather
it's that the breakage should have been avoided altogether.

But if this is an isolated incident, then fair enough, I'll stop
whining.

-- 
Matt Fleming, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1232131

Em Thu, Sep 24, 2015 at 01:15:51PM +0100, Matt Fleming escreveu:
> On Wed, 23 Sep, at 10:44:56AM, Arnaldo Carvalho de Melo wrote:
> > Of course, in these days of CI, I'd love if someone would hook 'make -C
> > tools/perf build-test' and 'perf test' somewhere to be run for every
> > changeset.
>   
> Yes please!

But then even this one would have not been caught, because the test in
place don't include trying to build tools/vm/, i.e. from time to time
something will pass and will be caught by people like Vinson, reported
and fixed :-)

> > BTW, tools/vm/ was reported yesterday and a fix is already in
> > tip/perf/core/:
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/tools/vm?id=f6489bc2d402c0db84aa64f13b864d17f7eecb07
> > 
> > Age       Commit message (Expand)                                           Author                   Files Lines
> > 12 hours  tools vm: Fix build due to removal of tools/lib/api/fs/debugfs.h  Arnaldo Carvalho de Melo	1  -3/+3
 
> It's not that this wasn't fixed quickly (kudos for that, btw), rather
> it's that the breakage should have been avoided altogether.
 
> But if this is an isolated incident, then fair enough, I'll stop
> whining.

Expressing concern is not a problem, its an opportunity for us to try
and get them addressed and improve so that others don't get afraid of
the processes in place.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web