Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1559777 > unrolled thread

[PATCH v2 0/3] xen: optimize xenbus performance

Back to article view | Back to linux.kernel

Contents

  [PATCH v2 0/3] xen: optimize xenbus performance Juergen Gross <jgross@suse.com> - 2017-01-16 15:20 +0100
    [PATCH v2 1/3] xen: clean up xenbus internal headers Juergen Gross <jgross@suse.com> - 2017-01-16 15:20 +0100
      Re: [PATCH v2 1/3] xen: clean up xenbus internal headers Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-01-18 00:30 +0100
    Re: [PATCH v2 3/3] xen: optimize xenbus driver for multiple  concurrent xenstore accesses Boris Ostrovsky <boris.ostrovsky@oracle.com> - 2017-01-19 00:00 +0100
      Re: [PATCH v2 3/3] xen: optimize xenbus driver for multiple  concurrent xenstore accesses Juergen Gross <jgross@suse.com> - 2017-01-19 08:50 +0100

#1559777 — [PATCH v2 0/3] xen: optimize xenbus performance

The xenbus driver used for communication with Xenstore (all kernel
accesses to Xenstore and in case of Xenstore living in another domain
all accesses of the local domain to Xenstore) is rather simple
especially regarding multiple concurrent accesses: they are just being
serialized in spite of Xenstore being capable to handle multiple
parallel accesses.

Clean up the external interface(s) of xenbus and optimize its
performance by allowing multiple concurrent accesses to Xenstore.

V2:
- patch 1: update commit message, re-add lost copyright
- patch 3: address comments of Boris Ostrovsky:
    - guard xs_request_id by lock
    - move state struct definitions to the place where they are being
      used
    - rate limit some warnings
    - use barrier() instead of cpu_relax()
    - add/remove some comments
    - minor changes like naming of variables

Juergen Gross (3):
  xen: clean up xenbus internal headers
  xen: modify xenstore watch event interface
  xen: optimize xenbus driver for multiple concurrent xenstore accesses

 drivers/block/xen-blkback/xenbus.c         |   6 +-
 drivers/net/xen-netback/xenbus.c           |   8 +-
 drivers/xen/cpu_hotplug.c                  |   5 +-
 drivers/xen/manage.c                       |   6 +-
 drivers/xen/xen-balloon.c                  |   2 +-
 drivers/xen/xen-pciback/xenbus.c           |   2 +-
 drivers/xen/xenbus/xenbus.h                | 135 +++++++
 drivers/xen/xenbus/xenbus_client.c         |   6 +-
 drivers/xen/xenbus/xenbus_comms.c          | 315 +++++++++++++++--
 drivers/xen/xenbus/xenbus_comms.h          |  51 ---
 drivers/xen/xenbus/xenbus_dev_backend.c    |   2 +-
 drivers/xen/xenbus/xenbus_dev_frontend.c   | 213 ++++++-----
 drivers/xen/xenbus/xenbus_probe.c          |  14 +-
 drivers/xen/xenbus/xenbus_probe.h          |  88 -----
 drivers/xen/xenbus/xenbus_probe_backend.c  |  11 +-
 drivers/xen/xenbus/xenbus_probe_frontend.c |  17 +-
 drivers/xen/xenbus/xenbus_xs.c             | 547 +++++++++++++----------------
 drivers/xen/xenfs/super.c                  |   2 +-
 drivers/xen/xenfs/xenstored.c              |   2 +-
 include/xen/xenbus.h                       |  18 +-
 20 files changed, 838 insertions(+), 612 deletions(-)
 create mode 100644 drivers/xen/xenbus/xenbus.h
 delete mode 100644 drivers/xen/xenbus/xenbus_comms.h
 delete mode 100644 drivers/xen/xenbus/xenbus_probe.h

Cc: konrad.wilk@oracle.com
Cc: roger.pau@citrix.com
Cc: wei.liu2@citrix.com
Cc: paul.durrant@citrix.com
Cc: netdev@vger.kernel.org
-- 
2.10.2

[toc] | [next] | [standalone]

#1559780 — [PATCH v2 1/3] xen: clean up xenbus internal headers

The xenbus driver has an awful mixture of internally and globally
visible headers: some of the internally used only stuff is defined in
the global header include/xen/xenbus.h while some stuff defined in
internal headers is used by other drivers, too.

Clean this up by moving the externally used symbols to
include/xen/xenbus.h and the symbols used internally only to a new
header drivers/xen/xenbus/xenbus.h replacing xenbus_comms.h and
xenbus_probe.h

Signed-off-by: Juergen Gross <jgross@suse.com>
---
v2: update commit message, re-add lost copyright
---
 drivers/xen/xenbus/{xenbus_probe.h => xenbus.h} | 63 ++++++++++++++-----------
 drivers/xen/xenbus/xenbus_client.c              |  2 +-
 drivers/xen/xenbus/xenbus_comms.c               |  2 +-
 drivers/xen/xenbus/xenbus_comms.h               | 51 --------------------
 drivers/xen/xenbus/xenbus_dev_backend.c         |  2 +-
 drivers/xen/xenbus/xenbus_dev_frontend.c        |  4 +-
 drivers/xen/xenbus/xenbus_probe.c               |  3 +-
 drivers/xen/xenbus/xenbus_probe_backend.c       |  3 +-
 drivers/xen/xenbus/xenbus_probe_frontend.c      |  3 +-
 drivers/xen/xenbus/xenbus_xs.c                  |  3 +-
 drivers/xen/xenfs/super.c                       |  2 +-
 drivers/xen/xenfs/xenstored.c                   |  2 +-
 include/xen/xenbus.h                            | 12 ++---
 13 files changed, 52 insertions(+), 100 deletions(-)
 rename drivers/xen/xenbus/{xenbus_probe.h => xenbus.h} (59%)
 delete mode 100644 drivers/xen/xenbus/xenbus_comms.h

diff --git a/drivers/xen/xenbus/xenbus_probe.h b/drivers/xen/xenbus/xenbus.h
similarity index 59%
rename from drivers/xen/xenbus/xenbus_probe.h
rename to drivers/xen/xenbus/xenbus.h
index c9ec7ca..a6b007d 100644
--- a/drivers/xen/xenbus/xenbus_probe.h
+++ b/drivers/xen/xenbus/xenbus.h
@@ -1,7 +1,5 @@
-/******************************************************************************
- * xenbus_probe.h
- *
- * Talks to Xen Store to figure out what devices we have.
+/*
+ * Private include for xenbus communications.
  *
  * Copyright (C) 2005 Rusty Russell, IBM Corporation
  * Copyright (C) 2005 XenSource Ltd.
@@ -31,8 +29,8 @@
  * IN THE SOFTWARE.
  */
 
-#ifndef _XENBUS_PROBE_H
-#define _XENBUS_PROBE_H
+#ifndef _XENBUS_XENBUS_H
+#define _XENBUS_XENBUS_H
 
 #define XEN_BUS_ID_SIZE			20
 
@@ -54,35 +52,46 @@ enum xenstore_init {
 	XS_LOCAL,
 };
 
+extern enum xenstore_init xen_store_domain_type;
 extern const struct attribute_group *xenbus_dev_groups[];
 
-extern int xenbus_match(struct device *_dev, struct device_driver *_drv);
-extern int xenbus_dev_probe(struct device *_dev);
-extern int xenbus_dev_remove(struct device *_dev);
-extern int xenbus_register_driver_common(struct xenbus_driver *drv,
-					 struct xen_bus_type *bus,
-					 struct module *owner,
-					 const char *mod_name);
-extern int xenbus_probe_node(struct xen_bus_type *bus,
-			     const char *type,
-			     const char *nodename);
-extern int xenbus_probe_devices(struct xen_bus_type *bus);
+int xs_init(void);
+int xb_init_comms(void);
+void xb_deinit_comms(void);
+int xb_write(const void *data, unsigned int len);
+int xb_read(void *data, unsigned int len);
+int xb_data_to_read(void);
+int xb_wait_for_data_to_read(void);
 
-extern void xenbus_dev_changed(const char *node, struct xen_bus_type *bus);
+int xenbus_match(struct device *_dev, struct device_driver *_drv);
+int xenbus_dev_probe(struct device *_dev);
+int xenbus_dev_remove(struct device *_dev);
+int xenbus_register_driver_common(struct xenbus_driver *drv,
+				  struct xen_bus_type *bus,
+				  struct module *owner,
+				  const char *mod_name);
+int xenbus_probe_node(struct xen_bus_type *bus,
+		      const char *type,
+		      const char *nodename);
+int xenbus_probe_devices(struct xen_bus_type *bus);
 
-extern void xenbus_dev_shutdown(struct device *_dev);
+void xenbus_dev_changed(const char *node, struct xen_bus_type *bus);
 
-extern int xenbus_dev_suspend(struct device *dev);
-extern int xenbus_dev_resume(struct device *dev);
-extern int xenbus_dev_cancel(struct device *dev);
+void xenbus_dev_shutdown(struct device *_dev);
 
-extern void xenbus_otherend_changed(struct xenbus_watch *watch,
-				    const char **vec, unsigned int len,
-				    int ignore_on_shutdown);
+int xenbus_dev_suspend(struct device *dev);
+int xenbus_dev_resume(struct device *dev);
+int xenbus_dev_cancel(struct device *dev);
 
-extern int xenbus_read_otherend_details(struct xenbus_device *xendev,
-					char *id_node, char *path_node);
+void xenbus_otherend_changed(struct xenbus_watch *watch,
+			     const char **vec, unsigned int len,
+			     int ignore_on_shutdown);
+
+int xenbus_read_otherend_details(struct xenbus_device *xendev,
+				 char *id_node, char *path_node);
 
 void xenbus_ring_ops_init(void);
 
+void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg);
+
 #endif
diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c
index 056da6e..23edf53 100644
--- a/drivers/xen/xenbus/xenbus_client.c
+++ b/drivers/xen/xenbus/xenbus_client.c
@@ -47,7 +47,7 @@
 #include <xen/xen.h>
 #include <xen/features.h>
 
-#include "xenbus_probe.h"
+#include "xenbus.h"
 
 #define XENBUS_PAGES(_grants)	(DIV_ROUND_UP(_grants, XEN_PFN_PER_PAGE))
 
diff --git a/drivers/xen/xenbus/xenbus_comms.c b/drivers/xen/xenbus/xenbus_comms.c
index ecdecce..c21ec02 100644
--- a/drivers/xen/xenbus/xenbus_comms.c
+++ b/drivers/xen/xenbus/xenbus_comms.c
@@ -40,7 +40,7 @@
 #include <asm/xen/hypervisor.h>
 #include <xen/events.h>
 #include <xen/page.h>
-#include "xenbus_comms.h"
+#include "xenbus.h"
 
 static int xenbus_irq;
 
diff --git a/drivers/xen/xenbus/xenbus_comms.h b/drivers/xen/xenbus/xenbus_comms.h
deleted file mode 100644
index 867a2e4..0000000
--- a/drivers/xen/xenbus/xenbus_comms.h
+++ /dev/null
@@ -1,51 +0,0 @@
-/*
- * Private include for xenbus communications.
- *
- * Copyright (C) 2005 Rusty Russell, IBM Corporation
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License version 2
- * as published by the Free Software Foundation; or, when distributed
- * separately from the Linux kernel or incorporated into other
- * software packages, subject to the following license:
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this source file (the "Software"), to deal in the Software without
- * restriction, including without limitation the rights to use, copy, modify,
- * merge, publish, distribute, sublicense, and/or sell copies of the Software,
- * and to permit persons to whom the Software is furnished to do so, subject to
- * the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- */
-
-#ifndef _XENBUS_COMMS_H
-#define _XENBUS_COMMS_H
-
-#include <linux/fs.h>
-
-int xs_init(void);
-int xb_init_comms(void);
-void xb_deinit_comms(void);
-
-/* Low level routines. */
-int xb_write(const void *data, unsigned len);
-int xb_read(void *data, unsigned len);
-int xb_data_to_read(void);
-int xb_wait_for_data_to_read(void);
-extern struct xenstore_domain_interface *xen_store_interface;
-extern int xen_store_evtchn;
-extern enum xenstore_init xen_store_domain_type;
-
-extern const struct file_operations xen_xenbus_fops;
-
-#endif /* _XENBUS_COMMS_H */
diff --git a/drivers/xen/xenbus/xenbus_dev_backend.c b/drivers/xen/xenbus/xenbus_dev_backend.c
index 4a41ac9..1126701 100644
--- a/drivers/xen/xenbus/xenbus_dev_backend.c
+++ b/drivers/xen/xenbus/xenbus_dev_backend.c
@@ -16,7 +16,7 @@
 #include <xen/events.h>
 #include <asm/xen/hypervisor.h>
 
-#include "xenbus_comms.h"
+#include "xenbus.h"
 
 static int xenbus_backend_open(struct inode *inode, struct file *filp)
 {
diff --git a/drivers/xen/xenbus/xenbus_dev_frontend.c b/drivers/xen/xenbus/xenbus_dev_frontend.c
index 79130b3..e2bc9b3 100644
--- a/drivers/xen/xenbus/xenbus_dev_frontend.c
+++ b/drivers/xen/xenbus/xenbus_dev_frontend.c
@@ -57,12 +57,12 @@
 #include <linux/miscdevice.h>
 #include <linux/init.h>
 
-#include "xenbus_comms.h"
-
 #include <xen/xenbus.h>
 #include <xen/xen.h>
 #include <asm/xen/hypervisor.h>
 
+#include "xenbus.h"
+
 /*
  * An element of a list of outstanding transactions, for which we're
  * still waiting a reply.
diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c
index 4bdf654..6baffbb 100644
--- a/drivers/xen/xenbus/xenbus_probe.c
+++ b/drivers/xen/xenbus/xenbus_probe.c
@@ -62,8 +62,7 @@
 
 #include <xen/hvm.h>
 
-#include "xenbus_comms.h"
-#include "xenbus_probe.h"
+#include "xenbus.h"
 
 
 int xen_store_evtchn;
diff --git a/drivers/xen/xenbus/xenbus_probe_backend.c b/drivers/xen/xenbus/xenbus_probe_backend.c
index 37929df..f46b4dc 100644
--- a/drivers/xen/xenbus/xenbus_probe_backend.c
+++ b/drivers/xen/xenbus/xenbus_probe_backend.c
@@ -53,8 +53,7 @@
 #include <xen/xenbus.h>
 #include <xen/features.h>
 
-#include "xenbus_comms.h"
-#include "xenbus_probe.h"
+#include "xenbus.h"
 
 /* backend/<type>/<fe-uuid>/<id> => <type>-<fe-domid>-<id> */
 static int backend_bus_id(char bus_id[XEN_BUS_ID_SIZE], const char *nodename)
diff --git a/drivers/xen/xenbus/xenbus_probe_frontend.c b/drivers/xen/xenbus/xenbus_probe_frontend.c
index 6d40a97..d7b77a6 100644
--- a/drivers/xen/xenbus/xenbus_probe_frontend.c
+++ b/drivers/xen/xenbus/xenbus_probe_frontend.c
@@ -27,8 +27,7 @@
 
 #include <xen/platform_pci.h>
 
-#include "xenbus_comms.h"
-#include "xenbus_probe.h"
+#include "xenbus.h"
 
 
 
diff --git a/drivers/xen/xenbus/xenbus_xs.c b/drivers/xen/xenbus/xenbus_xs.c
index 6afb993..4c49d87 100644
--- a/drivers/xen/xenbus/xenbus_xs.c
+++ b/drivers/xen/xenbus/xenbus_xs.c
@@ -48,8 +48,7 @@
 #include <asm/xen/hypervisor.h>
 #include <xen/xenbus.h>
 #include <xen/xen.h>
-#include "xenbus_comms.h"
-#include "xenbus_probe.h"
+#include "xenbus.h"
 
 struct xs_stored_msg {
 	struct list_head list;
diff --git a/drivers/xen/xenfs/super.c b/drivers/xen/xenfs/super.c
index 8559a71..328c398 100644
--- a/drivers/xen/xenfs/super.c
+++ b/drivers/xen/xenfs/super.c
@@ -16,10 +16,10 @@
 #include <linux/magic.h>
 
 #include <xen/xen.h>
+#include <xen/xenbus.h>
 
 #include "xenfs.h"
 #include "../privcmd.h"
-#include "../xenbus/xenbus_comms.h"
 
 #include <asm/xen/hypervisor.h>
 
diff --git a/drivers/xen/xenfs/xenstored.c b/drivers/xen/xenfs/xenstored.c
index fef20db..82fd2a3 100644
--- a/drivers/xen/xenfs/xenstored.c
+++ b/drivers/xen/xenfs/xenstored.c
@@ -4,9 +4,9 @@
 #include <linux/fs.h>
 
 #include <xen/page.h>
+#include <xen/xenbus.h>
 
 #include "xenfs.h"
-#include "../xenbus/xenbus_comms.h"
 
 static ssize_t xsd_read(struct file *file, char __user *buf,
 			    size_t size, loff_t *off)
diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h
index 271ba62..98f73a2 100644
--- a/include/xen/xenbus.h
+++ b/include/xen/xenbus.h
@@ -38,6 +38,7 @@
 #include <linux/notifier.h>
 #include <linux/mutex.h>
 #include <linux/export.h>
+#include <linux/fs.h>
 #include <linux/completion.h>
 #include <linux/init.h>
 #include <linux/slab.h>
@@ -175,16 +176,9 @@ void xs_suspend(void);
 void xs_resume(void);
 void xs_suspend_cancel(void);
 
-/* Used by xenbus_dev to borrow kernel's store connection. */
-void *xenbus_dev_request_and_reply(struct xsd_sockmsg *msg);
-
 struct work_struct;
 
-/* Prepare for domain suspend: then resume or cancel the suspend. */
-void xenbus_suspend(void);
-void xenbus_resume(void);
 void xenbus_probe(struct work_struct *);
-void xenbus_suspend_cancel(void);
 
 #define XENBUS_IS_ERR_READ(str) ({			\
 	if (!IS_ERR(str) && strlen(str) == 0) {		\
@@ -235,4 +229,8 @@ const char *xenbus_strstate(enum xenbus_state state);
 int xenbus_dev_is_online(struct xenbus_device *dev);
 int xenbus_frontend_closed(struct xenbus_device *dev);
 
+extern const struct file_operations xen_xenbus_fops;
+extern struct xenstore_domain_interface *xen_store_interface;
+extern int xen_store_evtchn;
+
 #endif /* _XEN_XENBUS_H */
-- 
2.10.2

[toc] | [prev] | [next] | [standalone]

#1561113 — Re: [PATCH v2 1/3] xen: clean up xenbus internal headers

On 01/16/2017 09:15 AM, Juergen Gross wrote:
> The xenbus driver has an awful mixture of internally and globally
> visible headers: some of the internally used only stuff is defined in
> the global header include/xen/xenbus.h while some stuff defined in
> internal headers is used by other drivers, too.
>
> Clean this up by moving the externally used symbols to
> include/xen/xenbus.h and the symbols used internally only to a new
> header drivers/xen/xenbus/xenbus.h replacing xenbus_comms.h and
> xenbus_probe.h
>
> Signed-off-by: Juergen Gross <jgross@suse.com>


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

[toc] | [prev] | [next] | [standalone]

#1562258 — Re: [PATCH v2 3/3] xen: optimize xenbus driver for multiple concurrent xenstore accesses

On 01/16/2017 09:15 AM, Juergen Gross wrote:
> +
> +static uint32_t xs_request_enter(struct xb_req_data *req)
> +{
> +	uint32_t rq_id;
> +
> +	req->type = req->msg.type;
> +
> +	spin_lock(&xs_state_lock);
> +	for (;;) {
> +		if (req->msg.tx_id != 0)
> +			break;
> +		if (xs_suspend_active) {
> +			spin_unlock(&xs_state_lock);
> +			wait_event(xs_state_enter_wq, xs_suspend_active == 0);
> +			spin_lock(&xs_state_lock);
> +			continue;
> +		}
> +		if (req->type == XS_TRANSACTION_START)
> +			xs_state_users++;
> +		break;
> +	}
> +	xs_state_users++;
> +	rq_id = xs_request_id++;
> +	spin_unlock(&xs_state_lock);
> +
> +	return rq_id;
> +}

I should have noticed this last time but I've been looking at this code
again and I don't think I understand why you are incrementing count for
XS_TRANSACTION_START inside the loop.

In fact, why not just 'while(xs_suspend_active) {}' loop?

-boris

[toc] | [prev] | [next] | [standalone]

#1562446 — Re: [PATCH v2 3/3] xen: optimize xenbus driver for multiple concurrent xenstore accesses

On 18/01/17 21:14, Boris Ostrovsky wrote:
> On 01/16/2017 09:15 AM, Juergen Gross wrote:
>> +
>> +static uint32_t xs_request_enter(struct xb_req_data *req)
>> +{
>> +	uint32_t rq_id;
>> +
>> +	req->type = req->msg.type;
>> +
>> +	spin_lock(&xs_state_lock);
>> +	for (;;) {
>> +		if (req->msg.tx_id != 0)
>> +			break;
>> +		if (xs_suspend_active) {
>> +			spin_unlock(&xs_state_lock);
>> +			wait_event(xs_state_enter_wq, xs_suspend_active == 0);
>> +			spin_lock(&xs_state_lock);
>> +			continue;
>> +		}
>> +		if (req->type == XS_TRANSACTION_START)
>> +			xs_state_users++;
>> +		break;
>> +	}
>> +	xs_state_users++;
>> +	rq_id = xs_request_id++;
>> +	spin_unlock(&xs_state_lock);
>> +
>> +	return rq_id;
>> +}
> 
> I should have noticed this last time but I've been looking at this code
> again and I don't think I understand why you are incrementing count for
> XS_TRANSACTION_START inside the loop.
> 
> In fact, why not just 'while(xs_suspend_active) {}' loop?

That's a valid question.

I'll change it. The reason to have the larger loop body isn't existing
any longer.


Juergen

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web