Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1548765 > unrolled thread
| Started by | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| First post | 2016-12-31 00:10 +0100 |
| Last post | 2016-12-31 07:50 +0100 |
| Articles | 10 — 1 participant |
Back to article view | Back to linux.kernel
[RFC 00/32] State of MARS Reo-Redundancy Module Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 17/32] mars: add new module xio_bio Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 19/32] mars: add new module xio_client Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 28/32] mars: add new module mars_proc Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 02/32] mars: add new module brick_say Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 03/32] mars: add new module brick_mem Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 13/32] mars: add new module xio Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 26/32] mars: add new module net Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
[RFC 10/32] mars: add new module lib_limiter Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 00:10 +0100
Re: [RFC 00/32] State of MARS Reo-Redundancy Module Thomas Schoebel-Theuer <tst@schoebel-theuer.de> - 2016-12-31 07:50 +0100
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 00/32] State of MARS Reo-Redundancy Module |
| Message-ID | <sUejv-2jF-3@gated-at.bofh.it> |
Hi together, here is my traditional annual status report on the development of MARS [1]. In the meantime, the out-of-tree MARS has replaced DRBD as the backbone of the 1&1 geo-redundancy feature as publicly advertised for 1&1 Shared Hosting Linux (ShaHoLin). MARS is also running on several other 1&1 clusters. Some other people over the world have also seemingly started to use it. At 1&1, MARS is now running on more than 2000 Servers and on more than 2 * 8 petabytes of data, and it has collected more than 20 millions of operation hours. The slides [1] are explaining why the sharding architecture supported by MARS has no problems scaling out to such numbers, while some other non-MARS and non-blocklevel 1&1 clusters (architecturally called "Big Clusters" in the slides, although they have less than 1 PB of data) have seemingly reached their _practical_ scaling limits at their practical dimensioning and practical workload, although they were originally advertized as scaling almost "unlimited" in theory, and some people had seemed to _believe_ this in the past. Some of the reasons for massive differences in scalability are explained in the slides, and some more explanations will hopefully follow in 2017. During 2016, I published several bugfix releases for the stable branch, and some portability improvements to some newer kernels for the out-of-tree (OOT) version of MARS at the github repo [2]. There is also a prototype of a prepatch-less WIP-compatibility branch which is not yet merged with master. Some minor developments have also started: there is a lab prototye for md5 checksumming of 4k blocks on the underlying disk devices. This was motivated by the observation that most operational incidents are due to hardware defects, and we want to catch them as early as possible. Conversely, this also implies that MARS is considered more stable than the hardware, but of course this is _expected_ from a HA solution ;) In January 2016, I took a few holidays for improving the upstream version of MARS a little bit, mainly improving some checkpatch issues. After that, I had to do much other work at 1&1, so unfortunately the development of this part got stuck at that point. Sorry. The attached code is more or less just for your personal information that this part is not dead, and that development will continue. This autumn, I got a new boss and some new objectives for 2017. One of the new objectives will involve MARS. The current ShaHoLin sharding architecture has been diretly migrated from the former DRBD hardware setup to MARS: it just consists of about 1000 _pairs_ of hard-iron iSCSI storage servers and some hard-iron standalone servers with local hardware RAIDs. They are hosting about 500 MARS resources (originally DRBD resources) just for the web servers; there are even more resources at (already virtualized) database servers. The former should be virtualized during 2017 for reducing the server iron, likely using LXC and/or KVM. The resulting future system should increase flexibility by MARS resource data migration among the _whole_ pool. This means that MARS will give up the traditional DRBD-like pairing in favour of a new feature: treating all of the existing storage like one big "virtual LVM-like storage pool". Notice that MARS's internal _architecture_ can already do this: it allows for k > 2 replicas, and already has dynamic join-cluster and join-resource and leave-resource operations which can be easily used for runtime data migration during operation (while resources are loaded), and even for very big resources. After adding a new operation "merge-cluster" which checks that all the resource names are disjoint, this _would_ even work with the current version of MARS, at least in theory. However, the current limitation is at the internal _metadata_ updates (not at the IO data paths): currently all cluster members are exchanging all _metadata_ with all other nodes, leading to O(n^2) _metadata_ communications. A future version of MARS will reduce this to the necessary scale (only among the nodes involved in some resources), and it will communicate the other metadata less frequently and no longer full-mesh. As a side note: hopefully I will also get the necessary time for replacing the current symlink tree by metadata files, which should also improve the metadata scaling properties. The goal will be a very low number of MARS clusters (one for US, one for EU, and both probably split for web hosting versus databases) consisting of several thousands of nodes. The realtime-critical data IO paths will remain at the sharding principle, leading to excellent scalability. Only the _metadata_ updates will follow the "big cluster" architectural approach, and only as far as necessary. They are not time-critical anyway. As always, for the opensource community part of my work: it would be nice if some other kernel hackers would start joining the MARS development in 2017, at least for helping me getting it upstream. I would be excited if I would be invited to the next kernel summit or a similar meeting. A happy new year from your devoted Thomas [1] https://github.com/schoebel/mars/blob/master/docu/MARS_GUUG2016.pdf [2] https://github.com/schoebel/mars Thomas Schoebel-Theuer (32): mars: add new module lamport mars: add new module brick_say mars: add new module brick_mem mars: add new module brick_checking mars: add new module meta mars: add new module brick mars: add new module lib_pairing_heap mars: add new module lib_queue mars: add new module lib_rank mars: add new module lib_limiter mars: add new module lib_timing mars: add new module vfs_compat mars: add new module xio mars: add new module xio_net mars: add new module lib_mapfree mars: add new module lib_log mars: add new module xio_bio mars: add new module xio_sio mars: add new module xio_client mars: add new module xio_if mars: add new module xio_copy mars: add new module xio_trans_logger mars: add new module xio_server mars: add new module strategy mars: add new module main_strategy mars: add new module net mars: add new module server_strategy mars: add new module mars_proc mars: add new module mars_main mars: add new module Makefile mars: add new module Kconfig mars: activate build drivers/staging/Kconfig | 2 + drivers/staging/Makefile | 1 + drivers/staging/mars/Kconfig | 266 + drivers/staging/mars/Makefile | 96 + drivers/staging/mars/brick.c | 723 +++ drivers/staging/mars/brick_mem.c | 1080 ++++ drivers/staging/mars/brick_say.c | 920 +++ drivers/staging/mars/lamport.c | 61 + drivers/staging/mars/lib/lib_limiter.c | 163 + drivers/staging/mars/lib/lib_rank.c | 87 + drivers/staging/mars/lib/lib_timing.c | 68 + drivers/staging/mars/mars/main_strategy.c | 2135 +++++++ drivers/staging/mars/mars/mars_main.c | 6160 ++++++++++++++++++++ drivers/staging/mars/mars/mars_proc.c | 389 ++ drivers/staging/mars/mars/mars_proc.h | 34 + drivers/staging/mars/mars/net.c | 109 + drivers/staging/mars/mars/server_strategy.c | 436 ++ drivers/staging/mars/mars/strategy.h | 239 + drivers/staging/mars/xio_bricks/lib_log.c | 506 ++ drivers/staging/mars/xio_bricks/lib_mapfree.c | 382 ++ drivers/staging/mars/xio_bricks/xio.c | 227 + drivers/staging/mars/xio_bricks/xio_bio.c | 845 +++ drivers/staging/mars/xio_bricks/xio_client.c | 1083 ++++ drivers/staging/mars/xio_bricks/xio_copy.c | 1005 ++++ drivers/staging/mars/xio_bricks/xio_if.c | 892 +++ drivers/staging/mars/xio_bricks/xio_net.c | 1849 ++++++ drivers/staging/mars/xio_bricks/xio_server.c | 493 ++ drivers/staging/mars/xio_bricks/xio_sio.c | 578 ++ drivers/staging/mars/xio_bricks/xio_trans_logger.c | 3410 +++++++++++ include/linux/brick/brick.h | 620 ++ include/linux/brick/brick_checking.h | 107 + include/linux/brick/brick_mem.h | 218 + include/linux/brick/brick_say.h | 89 + include/linux/brick/lamport.h | 26 + include/linux/brick/lib_limiter.h | 52 + include/linux/brick/lib_pairing_heap.h | 109 + include/linux/brick/lib_queue.h | 165 + include/linux/brick/lib_rank.h | 136 + include/linux/brick/lib_timing.h | 182 + include/linux/brick/meta.h | 106 + include/linux/brick/vfs_compat.h | 48 + include/linux/xio/lib_log.h | 333 ++ include/linux/xio/lib_mapfree.h | 84 + include/linux/xio/xio.h | 319 + include/linux/xio/xio_bio.h | 85 + include/linux/xio/xio_client.h | 105 + include/linux/xio/xio_copy.h | 115 + include/linux/xio/xio_if.h | 109 + include/linux/xio/xio_net.h | 177 + include/linux/xio/xio_server.h | 91 + include/linux/xio/xio_sio.h | 68 + include/linux/xio/xio_trans_logger.h | 271 + 52 files changed, 27854 insertions(+) create mode 100644 drivers/staging/mars/Kconfig create mode 100644 drivers/staging/mars/Makefile create mode 100644 drivers/staging/mars/brick.c create mode 100644 drivers/staging/mars/brick_mem.c create mode 100644 drivers/staging/mars/brick_say.c create mode 100644 drivers/staging/mars/lamport.c create mode 100644 drivers/staging/mars/lib/lib_limiter.c create mode 100644 drivers/staging/mars/lib/lib_rank.c create mode 100644 drivers/staging/mars/lib/lib_timing.c create mode 100644 drivers/staging/mars/mars/main_strategy.c create mode 100644 drivers/staging/mars/mars/mars_main.c create mode 100644 drivers/staging/mars/mars/mars_proc.c create mode 100644 drivers/staging/mars/mars/mars_proc.h create mode 100644 drivers/staging/mars/mars/net.c create mode 100644 drivers/staging/mars/mars/server_strategy.c create mode 100644 drivers/staging/mars/mars/strategy.h create mode 100644 drivers/staging/mars/xio_bricks/lib_log.c create mode 100644 drivers/staging/mars/xio_bricks/lib_mapfree.c create mode 100644 drivers/staging/mars/xio_bricks/xio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_bio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_client.c create mode 100644 drivers/staging/mars/xio_bricks/xio_copy.c create mode 100644 drivers/staging/mars/xio_bricks/xio_if.c create mode 100644 drivers/staging/mars/xio_bricks/xio_net.c create mode 100644 drivers/staging/mars/xio_bricks/xio_server.c create mode 100644 drivers/staging/mars/xio_bricks/xio_sio.c create mode 100644 drivers/staging/mars/xio_bricks/xio_trans_logger.c create mode 100644 include/linux/brick/brick.h create mode 100644 include/linux/brick/brick_checking.h create mode 100644 include/linux/brick/brick_mem.h create mode 100644 include/linux/brick/brick_say.h create mode 100644 include/linux/brick/lamport.h create mode 100644 include/linux/brick/lib_limiter.h create mode 100644 include/linux/brick/lib_pairing_heap.h create mode 100644 include/linux/brick/lib_queue.h create mode 100644 include/linux/brick/lib_rank.h create mode 100644 include/linux/brick/lib_timing.h create mode 100644 include/linux/brick/meta.h create mode 100644 include/linux/brick/vfs_compat.h create mode 100644 include/linux/xio/lib_log.h create mode 100644 include/linux/xio/lib_mapfree.h create mode 100644 include/linux/xio/xio.h create mode 100644 include/linux/xio/xio_bio.h create mode 100644 include/linux/xio/xio_client.h create mode 100644 include/linux/xio/xio_copy.h create mode 100644 include/linux/xio/xio_if.h create mode 100644 include/linux/xio/xio_net.h create mode 100644 include/linux/xio/xio_server.h create mode 100644 include/linux/xio/xio_sio.h create mode 100644 include/linux/xio/xio_trans_logger.h -- 2.11.0
[toc] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 17/32] mars: add new module xio_bio |
| Message-ID | <sUetc-2Cs-27@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/xio_bricks/xio_bio.c | 845 ++++++++++++++++++++++++++++++
include/linux/xio/xio_bio.h | 85 +++
2 files changed, 930 insertions(+)
create mode 100644 drivers/staging/mars/xio_bricks/xio_bio.c
create mode 100644 include/linux/xio/xio_bio.h
diff --git a/drivers/staging/mars/xio_bricks/xio_bio.c b/drivers/staging/mars/xio_bricks/xio_bio.c
new file mode 100644
index 000000000000..97bc4fc46f3e
--- /dev/null
+++ b/drivers/staging/mars/xio_bricks/xio_bio.c
@@ -0,0 +1,845 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+/* Bio brick (interface to blkdev IO via kernel bios) */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/bio.h>
+
+#include <linux/xio/xio.h>
+#include <linux/brick/lib_timing.h>
+#include <linux/xio/lib_mapfree.h>
+
+#include <linux/xio/xio_bio.h>
+static struct timing_stats timings[2];
+
+struct threshold bio_submit_threshold = {
+ .thr_ban = &xio_global_ban,
+ .thr_parent = &global_io_threshold,
+ .thr_limit = BIO_SUBMIT_MAX_LATENCY,
+ .thr_factor = 100,
+ .thr_plus = 0,
+};
+
+struct threshold bio_io_threshold[2] = {
+ [0] = {
+ .thr_ban = &xio_global_ban,
+ .thr_parent = &global_io_threshold,
+ .thr_limit = BIO_IO_R_MAX_LATENCY,
+ .thr_factor = 10,
+ .thr_plus = 10000,
+ },
+ [1] = {
+ .thr_ban = &xio_global_ban,
+ .thr_parent = &global_io_threshold,
+ .thr_limit = BIO_IO_W_MAX_LATENCY,
+ .thr_factor = 10,
+ .thr_plus = 10000,
+ },
+};
+
+/************************ own type definitions ***********************/
+
+/************************ own helper functions ***********************/
+
+/* This is called from the kernel bio layer.
+ */
+static
+void bio_callback(struct bio *bio)
+{
+ struct bio_aio_aspect *aio_a = bio->bi_private;
+ struct bio_brick *brick;
+ unsigned long flags;
+
+ CHECK_PTR(aio_a, err);
+ CHECK_PTR(aio_a->output, err);
+ brick = aio_a->output->brick;
+ CHECK_PTR(brick, err);
+
+ aio_a->status_code = bio->bi_error;
+
+ spin_lock_irqsave(&brick->lock, flags);
+ list_del(&aio_a->io_head);
+ list_add_tail(&aio_a->io_head, &brick->completed_list);
+ atomic_inc(&brick->completed_count);
+ spin_unlock_irqrestore(&brick->lock, flags);
+
+ wake_up_interruptible(&brick->response_event);
+ goto out_return;
+err:
+ XIO_FAT("cannot handle bio callback\n");
+out_return:;
+}
+
+/* Map from kernel address/length to struct page (if not already known),
+ * check alignment constraints, create bio from it.
+ * Return the length (may be smaller than requested).
+ */
+static
+int make_bio(
+struct bio_brick *brick, void *data, int len, loff_t pos, struct bio_aio_aspect *private, struct bio **_bio)
+{
+ unsigned long long sector;
+ int sector_offset;
+ int data_offset;
+ int page_offset;
+ int page_len;
+ int bvec_count;
+ int rest_len = len;
+ int result_len = 0;
+ int status;
+ int i;
+ struct bio *bio = NULL;
+ struct block_device *bdev;
+
+ status = -EINVAL;
+ CHECK_PTR(brick, out);
+ bdev = brick->bdev;
+ CHECK_PTR(bdev, out);
+
+ if (unlikely(rest_len <= 0)) {
+ XIO_ERR("bad bio len %d\n", rest_len);
+ goto out;
+ }
+
+ sector = pos >> 9; /* TODO: make dynamic */
+ sector_offset = pos & ((1 << 9) - 1); /* TODO: make dynamic */
+ data_offset = ((unsigned long)data) & ((1 << 9) - 1); /* TODO: make dynamic */
+
+ if (unlikely(sector_offset > 0)) {
+ XIO_ERR("odd sector offset %d\n", sector_offset);
+ goto out;
+ }
+ if (unlikely(sector_offset != data_offset)) {
+ XIO_ERR("bad alignment: sector_offset %d != data_offset %d\n", sector_offset, data_offset);
+ goto out;
+ }
+ if (unlikely(rest_len & ((1 << 9) - 1))) {
+ XIO_ERR("odd length %d\n", rest_len);
+ goto out;
+ }
+
+ page_offset = ((unsigned long)data) & (PAGE_SIZE - 1);
+ page_len = rest_len + page_offset;
+ bvec_count = (page_len - 1) / PAGE_SIZE + 1;
+ if (bvec_count > brick->bvec_max) {
+ bvec_count = brick->bvec_max;
+ } else if (unlikely(bvec_count <= 0)) {
+ XIO_WRN("bvec_count=%d\n", bvec_count);
+ bvec_count = 1;
+ }
+
+ bio = bio_alloc(GFP_BRICK, bvec_count);
+ status = -ENOMEM;
+
+ for (i = 0; i < bvec_count && rest_len > 0; i++) {
+ struct page *page;
+ int this_rest = PAGE_SIZE - page_offset;
+ int this_len = rest_len;
+
+ if (this_len > this_rest)
+ this_len = this_rest;
+
+ page = brick_iomap(data, &page_offset, &this_len);
+ if (unlikely(!page)) {
+ XIO_ERR("cannot iomap() kernel address %p\n", data);
+ status = -EINVAL;
+ goto out;
+ }
+
+ bio->bi_io_vec[i].bv_page = page;
+ bio->bi_io_vec[i].bv_len = this_len;
+ bio->bi_io_vec[i].bv_offset = page_offset;
+
+ data += this_len;
+ rest_len -= this_len;
+ result_len += this_len;
+ page_offset = 0;
+ }
+
+ if (unlikely(rest_len != 0)) {
+ XIO_ERR("computation of bvec_count %d was wrong, diff=%d\n", bvec_count, rest_len);
+ status = -EINVAL;
+ goto out;
+ }
+
+ bio->bi_vcnt = i;
+ bio->bi_iter.bi_idx = 0;
+ bio->bi_iter.bi_size = result_len;
+ bio->bi_iter.bi_sector = sector;
+ bio->bi_bdev = bdev;
+ bio->bi_private = private;
+ bio->bi_end_io = bio_callback;
+ bio->bi_rw = 0; /* must be filled in later */
+ status = result_len;
+
+out:
+ if (unlikely(status < 0)) {
+ XIO_ERR("error %d\n", status);
+ if (bio) {
+ bio_put(bio);
+ bio = NULL;
+ }
+ }
+ *_bio = bio;
+ return status;
+}
+
+/***************** own brick * input * output operations *****************/
+
+#define PRIO_INDEX(aio) ((aio)->io_prio + 1)
+
+static int bio_get_info(struct bio_output *output, struct xio_info *info)
+{
+ struct bio_brick *brick = output->brick;
+ struct inode *inode;
+ int status = -ENOENT;
+
+ if (unlikely(!brick->mf ||
+ !brick->mf->mf_filp ||
+ !brick->mf->mf_filp->f_mapping)) {
+ goto done;
+ }
+ inode = brick->mf->mf_filp->f_mapping->host;
+ if (unlikely(!inode))
+ goto done;
+
+ info->tf_align = 512;
+ info->tf_min_size = 512;
+ brick->total_size = i_size_read(inode);
+ info->current_size = brick->total_size;
+ XIO_DBG("determined device size = %lld\n", info->current_size);
+ status = 0;
+
+done:
+ return status;
+}
+
+static int bio_io_get(struct bio_output *output, struct aio_object *aio)
+{
+ struct bio_aio_aspect *aio_a;
+ int status = -EINVAL;
+
+ CHECK_PTR(output, done);
+ CHECK_PTR(output->brick, done);
+
+ if (aio->obj_initialized) {
+ obj_get(aio);
+ return aio->io_len;
+ }
+
+ aio_a = bio_aio_get_aspect(output->brick, aio);
+ CHECK_PTR(aio_a, done);
+ aio_a->output = output;
+ aio_a->bio = NULL;
+
+ if (!aio->io_data) { /* buffered IO. */
+ if (unlikely(aio->io_len <= 0))
+ goto done;
+ status = -ENOMEM;
+ aio->io_data = brick_block_alloc(aio->io_pos, (aio_a->alloc_len = aio->io_len));
+ aio_a->do_dealloc = true;
+ }
+
+ status = make_bio(output->brick, aio->io_data, aio->io_len, aio->io_pos, aio_a, &aio_a->bio);
+ if (unlikely(status < 0 || !aio_a->bio)) {
+ XIO_ERR("could not create bio, status = %d\n", status);
+ goto done;
+ }
+
+ if (unlikely(aio->io_prio < XIO_PRIO_HIGH))
+ aio->io_prio = XIO_PRIO_HIGH;
+ else if (unlikely(aio->io_prio > XIO_PRIO_LOW))
+ aio->io_prio = XIO_PRIO_LOW;
+
+ aio->io_len = status;
+ obj_get_first(aio);
+ status = 0;
+
+done:
+ return status;
+}
+
+static
+void _bio_io_put(struct bio_output *output, struct aio_object *aio)
+{
+ struct bio_aio_aspect *aio_a;
+
+ aio->io_total_size = output->brick->total_size;
+
+ aio_a = bio_aio_get_aspect(output->brick, aio);
+ CHECK_PTR(aio_a, err);
+
+ if (likely(aio_a->bio)) {
+ bio_put(aio_a->bio);
+ aio_a->bio = NULL;
+ }
+ if (aio_a->do_dealloc) {
+ brick_block_free(aio->io_data, aio_a->alloc_len);
+ aio->io_data = NULL;
+ }
+ obj_free(aio);
+
+ goto out_return;
+err:
+ XIO_FAT("cannot work\n");
+out_return:;
+}
+
+#define BIO_AIO_PUT(output, aio) \
+ ({ \
+ if (obj_put(aio)) { \
+ _bio_io_put(output, aio); \
+ } \
+ })
+
+static
+void bio_io_put(struct bio_output *output, struct aio_object *aio)
+{
+ BIO_AIO_PUT(output, aio);
+}
+
+static
+void _bio_io_io(struct bio_output *output, struct aio_object *aio, bool cork)
+{
+ struct bio_brick *brick = output->brick;
+ struct bio_aio_aspect *aio_a = bio_aio_get_aspect(output->brick, aio);
+ struct bio *bio;
+ unsigned long long latency;
+ unsigned long flags;
+ int rw;
+ int status = -EINVAL;
+
+ CHECK_PTR(aio_a, err);
+ bio = aio_a->bio;
+ CHECK_PTR(bio, err);
+
+ obj_get(aio);
+ atomic_inc(&brick->fly_count[PRIO_INDEX(aio)]);
+
+ bio_get(bio);
+
+ rw = aio->io_rw & 1;
+ if (brick->do_noidle && !cork)
+ rw |= REQ_NOIDLE;
+ if (!aio->io_skip_sync) {
+ if (brick->do_sync)
+ rw |= REQ_SYNC;
+ }
+
+ aio_a->start_stamp = cpu_clock(raw_smp_processor_id());
+ spin_lock_irqsave(&brick->lock, flags);
+ list_add_tail(&aio_a->io_head, &brick->submitted_list[rw & 1]);
+ spin_unlock_irqrestore(&brick->lock, flags);
+
+ bio->bi_rw = rw;
+ latency = TIME_STATS(
+ &timings[rw & 1],
+ submit_bio(rw, bio)
+ );
+
+ threshold_check(&bio_submit_threshold, latency);
+
+ status = 0;
+#ifdef BIO_EOPNOTSUPP /* missing since b25de9d6da49b1a8760a89672283128aa8c78345 */
+ if (unlikely(bio_flagged(bio, BIO_EOPNOTSUPP)))
+ status = -EOPNOTSUPP;
+#endif
+
+ if (likely(status >= 0))
+ goto done;
+
+ bio_put(bio);
+ atomic_dec(&brick->fly_count[PRIO_INDEX(aio)]);
+
+err:
+ XIO_ERR("IO error %d\n", status);
+ CHECKED_CALLBACK(aio, status, done);
+ atomic_dec(&xio_global_io_flying);
+
+done:;
+}
+
+static
+void bio_io_io(struct bio_output *output, struct aio_object *aio)
+{
+ CHECK_PTR(aio, fatal);
+
+ obj_get(aio);
+ atomic_inc(&xio_global_io_flying);
+
+ if (aio->io_prio == XIO_PRIO_LOW ||
+ (aio->io_prio == XIO_PRIO_NORMAL && aio->io_rw)) {
+ struct bio_aio_aspect *aio_a = bio_aio_get_aspect(output->brick, aio);
+ struct bio_brick *brick = output->brick;
+ unsigned long flags;
+
+ spin_lock_irqsave(&brick->lock, flags);
+ list_add_tail(&aio_a->io_head, &brick->queue_list[PRIO_INDEX(aio)]);
+ atomic_inc(&brick->queue_count[PRIO_INDEX(aio)]);
+ spin_unlock_irqrestore(&brick->lock, flags);
+ brick->submitted = true;
+
+ wake_up_interruptible(&brick->submit_event);
+ goto out_return;
+ }
+
+ /* realtime IO: start immediately */
+ _bio_io_io(output, aio, false);
+ BIO_AIO_PUT(output, aio);
+ goto out_return;
+fatal:
+ XIO_FAT("cannot handle aio %p on output %p\n", aio, output);
+out_return:;
+}
+
+static
+int bio_response_thread(void *data)
+{
+ struct bio_brick *brick = data;
+
+ XIO_INF("bio response thread has started on '%s'.\n", brick->brick_path);
+
+ for (;;) {
+ LIST_HEAD(tmp_list);
+ unsigned long flags;
+ int thr_limit;
+ int sleeptime;
+ int count;
+ int i;
+
+ thr_limit = bio_io_threshold[0].thr_limit;
+ if (bio_io_threshold[1].thr_limit < thr_limit)
+ thr_limit = bio_io_threshold[1].thr_limit;
+
+ sleeptime = HZ / 10;
+ if (thr_limit > 0) {
+ sleeptime = thr_limit / (1000000 * 2 / HZ);
+ if (unlikely(sleeptime < 2))
+ sleeptime = 2;
+ }
+
+ wait_event_interruptible_timeout(
+ brick->response_event,
+ atomic_read(&brick->completed_count) > 0,
+ sleeptime);
+
+#ifdef CONFIG_MARS_DEBUG
+ if (mars_hang_mode & 2) {
+ brick_msleep(100);
+ continue;
+ }
+#endif
+ spin_lock_irqsave(&brick->lock, flags);
+ list_replace_init(&brick->completed_list, &tmp_list);
+ spin_unlock_irqrestore(&brick->lock, flags);
+
+ count = 0;
+ for (;;) {
+ struct list_head *tmp;
+ struct bio_aio_aspect *aio_a;
+ struct aio_object *aio;
+ unsigned long long latency;
+ int code;
+
+ if (list_empty(&tmp_list)) {
+ if (brick_thread_should_stop() &&
+ atomic_read(&brick->fly_count[0]) +
+ atomic_read(&brick->fly_count[1]) +
+ atomic_read(&brick->fly_count[2]) <= 0)
+ goto done;
+ break;
+ }
+
+ tmp = tmp_list.next;
+ list_del_init(tmp);
+ atomic_dec(&brick->completed_count);
+
+ aio_a = container_of(tmp, struct bio_aio_aspect, io_head);
+ aio = aio_a->object;
+
+ latency = cpu_clock(raw_smp_processor_id()) - aio_a->start_stamp;
+ threshold_check(&bio_io_threshold[aio->io_rw & 1], latency);
+
+ code = aio_a->status_code;
+
+ if (code < 0) {
+ XIO_ERR("IO error %d\n", code);
+ } else {
+ aio_checksum(aio);
+ aio->io_flags |= AIO_UPTODATE;
+ }
+
+ SIMPLE_CALLBACK(aio, code);
+
+ atomic_dec(&brick->fly_count[PRIO_INDEX(aio)]);
+ atomic_inc(&brick->total_completed_count[PRIO_INDEX(aio)]);
+ count++;
+
+ if (likely(aio_a->bio))
+ bio_put(aio_a->bio);
+ BIO_AIO_PUT(aio_a->output, aio);
+
+ atomic_dec(&xio_global_io_flying);
+ }
+
+ /* Try to detect slow requests as early as possible,
+ * even before they have completed.
+ */
+ for (i = 0; i < 2; i++) {
+ unsigned long long eldest = 0;
+
+ spin_lock_irqsave(&brick->lock, flags);
+ if (!list_empty(&brick->submitted_list[i])) {
+ struct bio_aio_aspect *aio_a;
+
+ aio_a = container_of(brick->submitted_list[i].next, struct bio_aio_aspect, io_head);
+ eldest = aio_a->start_stamp;
+ }
+ spin_unlock_irqrestore(&brick->lock, flags);
+
+ if (eldest)
+ threshold_check(&bio_io_threshold[i], cpu_clock(raw_smp_processor_id()) - eldest);
+ }
+
+ if (count) {
+ brick->submitted = true;
+ wake_up_interruptible(&brick->submit_event);
+ }
+ }
+done:
+ XIO_INF("bio response thread has stopped.\n");
+ return 0;
+}
+
+static
+bool _bg_should_run(struct bio_brick *brick)
+{
+ return (atomic_read(&brick->queue_count[2]) > 0 &&
+ atomic_read(&brick->fly_count[0]) + atomic_read(&brick->fly_count[1]) <= brick->bg_threshold &&
+ (brick->bg_maxfly <= 0 || atomic_read(&brick->fly_count[2]) < brick->bg_maxfly));
+}
+
+static
+int bio_submit_thread(void *data)
+{
+ struct bio_brick *brick = data;
+
+ XIO_INF("bio submit thread has started on '%s'.\n", brick->brick_path);
+
+ while (!brick_thread_should_stop()) {
+ int prio;
+
+ wait_event_interruptible_timeout(
+ brick->submit_event,
+ brick->submitted,
+ HZ / 2);
+
+ brick->submitted = false;
+
+ for (prio = 0; prio < XIO_PRIO_NR; prio++) {
+ LIST_HEAD(tmp_list);
+ unsigned long flags;
+
+ if (prio == XIO_PRIO_NR - 1 && !_bg_should_run(brick))
+ break;
+
+ spin_lock_irqsave(&brick->lock, flags);
+ list_replace_init(&brick->queue_list[prio], &tmp_list);
+ spin_unlock_irqrestore(&brick->lock, flags);
+
+ while (!list_empty(&tmp_list)) {
+ struct list_head *tmp = tmp_list.next;
+ struct bio_aio_aspect *aio_a;
+ struct aio_object *aio;
+ bool cork;
+
+ list_del_init(tmp);
+
+ aio_a = container_of(tmp, struct bio_aio_aspect, io_head);
+ aio = aio_a->object;
+ if (unlikely(!aio)) {
+ XIO_ERR("invalid aio\n");
+ continue;
+ }
+
+ atomic_dec(&brick->queue_count[PRIO_INDEX(aio)]);
+ cork = atomic_read(&brick->queue_count[PRIO_INDEX(aio)]) > 0;
+
+ _bio_io_io(aio_a->output, aio, cork);
+
+ BIO_AIO_PUT(aio_a->output, aio);
+ }
+ }
+ }
+
+ XIO_INF("bio submit thread has stopped.\n");
+ return 0;
+}
+
+static int bio_switch(struct bio_brick *brick)
+{
+ int status = 0;
+
+ if (brick->power.button) {
+ if (brick->power.on_led)
+ goto done;
+
+ xio_set_power_off_led((void *)brick, false);
+
+ if (!brick->bdev) {
+ static int index;
+ const char *path = brick->brick_path;
+ int flags = O_RDWR | O_EXCL | O_LARGEFILE;
+ struct address_space *mapping;
+ struct inode *inode = NULL;
+ struct request_queue *q;
+
+ brick->mf = mapfree_get(path, flags);
+ if (unlikely(!brick->mf || !brick->mf->mf_filp)) {
+ status = -ENOENT;
+ XIO_ERR("cannot open file '%s'\n", path);
+ goto done;
+ }
+ mapfree_pages(brick->mf, -1);
+ mapping = brick->mf->mf_filp->f_mapping;
+ if (likely(mapping))
+ inode = mapping->host;
+ if (unlikely(!mapping || !inode)) {
+ XIO_ERR("internal problem with '%s'\n", path);
+ status = -EINVAL;
+ goto done;
+ }
+ if (unlikely(!S_ISBLK(inode->i_mode) || !inode->i_bdev)) {
+ XIO_ERR("sorry, '%s' is not a block device\n", path);
+ status = -ENODEV;
+ goto done;
+ }
+
+ mapping_set_gfp_mask(mapping, mapping_gfp_mask(mapping) & ~(__GFP_IO | __GFP_FS));
+
+ q = bdev_get_queue(inode->i_bdev);
+ if (unlikely(!q)) {
+ XIO_ERR("internal queue '%s' does not exist\n", path);
+ status = -EINVAL;
+ goto done;
+ }
+
+ XIO_INF(
+ "'%s' ra_pages OLD=%lu NEW=%d\n", path, q->backing_dev_info.ra_pages, brick->ra_pages);
+ q->backing_dev_info.ra_pages = brick->ra_pages;
+
+ brick->bvec_max = queue_max_hw_sectors(q) >> (PAGE_SHIFT - 9);
+ if (brick->bvec_max > BIO_MAX_PAGES)
+ brick->bvec_max = BIO_MAX_PAGES;
+ else if (brick->bvec_max <= 1)
+ brick->bvec_max = 1;
+ brick->total_size = i_size_read(inode);
+ XIO_INF(
+ "'%s' size=%lld bvec_max=%d\n",
+ path, brick->total_size, brick->bvec_max);
+
+ brick->response_thread = brick_thread_create(
+ bio_response_thread, brick, "xio_bio_r%d", index);
+ brick->submit_thread = brick_thread_create(bio_submit_thread, brick, "xio_bio_s%d", index);
+ status = -ENOMEM;
+ if (likely(brick->submit_thread && brick->response_thread)) {
+ brick->bdev = inode->i_bdev;
+ brick->mode_ptr = &brick->mf->mf_mode;
+ index++;
+ status = 0;
+ }
+ }
+ }
+
+ xio_set_power_on_led((void *)brick, brick->power.button && brick->bdev);
+
+done:
+ if (status < 0 || !brick->power.button) {
+ if (brick->submit_thread) {
+ brick_thread_stop(brick->submit_thread);
+ brick->submit_thread = NULL;
+ }
+ if (brick->response_thread) {
+ brick_thread_stop(brick->response_thread);
+ brick->response_thread = NULL;
+ }
+ if (brick->mf) {
+ mapfree_put(brick->mf);
+ brick->mf = NULL;
+ }
+ brick->mode_ptr = NULL;
+ brick->bdev = NULL;
+ if (!brick->power.button) {
+ xio_set_power_off_led((void *)brick, true);
+ brick->total_size = 0;
+ }
+ }
+ return status;
+}
+
+/*************** informational * statistics **************/
+
+static noinline
+char *bio_statistics(struct bio_brick *brick, int verbose)
+{
+ char *res = brick_string_alloc(4096);
+ int pos = 0;
+
+ pos += report_timing(&timings[0], res + pos, 4096 - pos);
+ pos += report_timing(&timings[1], res + pos, 4096 - pos);
+
+ snprintf(
+ res + pos, 4096 - pos,
+ "total completed[0] = %d completed[1] = %d completed[2] = %d | queued[0] = %d queued[1] = %d queued[2] = %d flying[0] = %d flying[1] = %d flying[2] = %d completing = %d\n",
+ atomic_read(&brick->total_completed_count[0]),
+ atomic_read(&brick->total_completed_count[1]),
+ atomic_read(&brick->total_completed_count[2]),
+ atomic_read(&brick->fly_count[0]),
+ atomic_read(&brick->queue_count[0]),
+ atomic_read(&brick->queue_count[1]),
+ atomic_read(&brick->queue_count[2]),
+ atomic_read(&brick->fly_count[1]),
+ atomic_read(&brick->fly_count[2]),
+ atomic_read(&brick->completed_count));
+
+ return res;
+}
+
+static noinline
+void bio_reset_statistics(struct bio_brick *brick)
+{
+ atomic_set(&brick->total_completed_count[0], 0);
+ atomic_set(&brick->total_completed_count[1], 0);
+ atomic_set(&brick->total_completed_count[2], 0);
+}
+
+/*************** object * aspect constructors * destructors **************/
+
+static int bio_aio_aspect_init_fn(struct generic_aspect *_ini)
+{
+ struct bio_aio_aspect *ini = (void *)_ini;
+
+ INIT_LIST_HEAD(&ini->io_head);
+ return 0;
+}
+
+static void bio_aio_aspect_exit_fn(struct generic_aspect *_ini)
+{
+ struct bio_aio_aspect *ini = (void *)_ini;
+
+ (void)ini;
+}
+
+XIO_MAKE_STATICS(bio);
+
+/********************* brick constructors * destructors *******************/
+
+static int bio_brick_construct(struct bio_brick *brick)
+{
+ spin_lock_init(&brick->lock);
+ INIT_LIST_HEAD(&brick->queue_list[0]);
+ INIT_LIST_HEAD(&brick->queue_list[1]);
+ INIT_LIST_HEAD(&brick->queue_list[2]);
+ INIT_LIST_HEAD(&brick->submitted_list[0]);
+ INIT_LIST_HEAD(&brick->submitted_list[1]);
+ INIT_LIST_HEAD(&brick->completed_list);
+ init_waitqueue_head(&brick->submit_event);
+ init_waitqueue_head(&brick->response_event);
+ return 0;
+}
+
+static int bio_brick_destruct(struct bio_brick *brick)
+{
+ return 0;
+}
+
+static int bio_output_construct(struct bio_output *output)
+{
+ return 0;
+}
+
+static int bio_output_destruct(struct bio_output *output)
+{
+ return 0;
+}
+
+/************************ static structs ***********************/
+
+static struct bio_brick_ops bio_brick_ops = {
+ .brick_switch = bio_switch,
+ .brick_statistics = bio_statistics,
+ .reset_statistics = bio_reset_statistics,
+};
+
+static struct bio_output_ops bio_output_ops = {
+ .xio_get_info = bio_get_info,
+ .aio_get = bio_io_get,
+ .aio_put = bio_io_put,
+ .aio_io = bio_io_io,
+};
+
+const struct bio_input_type bio_input_type = {
+ .type_name = "bio_input",
+ .input_size = sizeof(struct bio_input),
+};
+
+static const struct bio_input_type *bio_input_types[] = {
+ &bio_input_type,
+};
+
+const struct bio_output_type bio_output_type = {
+ .type_name = "bio_output",
+ .output_size = sizeof(struct bio_output),
+ .master_ops = &bio_output_ops,
+ .output_construct = &bio_output_construct,
+ .output_destruct = &bio_output_destruct,
+};
+
+static const struct bio_output_type *bio_output_types[] = {
+ &bio_output_type,
+};
+
+const struct bio_brick_type bio_brick_type = {
+ .type_name = "bio_brick",
+ .brick_size = sizeof(struct bio_brick),
+ .max_inputs = 0,
+ .max_outputs = 1,
+ .master_ops = &bio_brick_ops,
+ .aspect_types = bio_aspect_types,
+ .default_input_types = bio_input_types,
+ .default_output_types = bio_output_types,
+ .brick_construct = &bio_brick_construct,
+ .brick_destruct = &bio_brick_destruct,
+};
+
+/***************** module init stuff ************************/
+
+int __init init_xio_bio(void)
+{
+ XIO_INF("init_bio()\n");
+ _bio_brick_type = (void *)&bio_brick_type;
+ return bio_register_brick_type();
+}
+
+void exit_xio_bio(void)
+{
+ XIO_INF("exit_bio()\n");
+ bio_unregister_brick_type();
+}
diff --git a/include/linux/xio/xio_bio.h b/include/linux/xio/xio_bio.h
new file mode 100644
index 000000000000..a0d98bed63b5
--- /dev/null
+++ b/include/linux/xio/xio_bio.h
@@ -0,0 +1,85 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef XIO_BIO_H
+#define XIO_BIO_H
+
+#define BIO_SUBMIT_MAX_LATENCY 250 /* 250 us */
+#define BIO_IO_R_MAX_LATENCY 40000 /* 40 ms */
+#define BIO_IO_W_MAX_LATENCY 100000 /* 100 ms */
+
+extern struct threshold bio_submit_threshold;
+extern struct threshold bio_io_threshold[2];
+
+#include <linux/blkdev.h>
+
+struct bio_aio_aspect {
+ GENERIC_ASPECT(aio);
+ struct list_head io_head;
+ struct bio *bio;
+ struct bio_output *output;
+ unsigned long long start_stamp;
+ int status_code;
+ int hash_pos;
+ int alloc_len;
+ bool do_dealloc;
+};
+
+struct bio_brick {
+ XIO_BRICK(bio);
+ /* tunables */
+ int ra_pages;
+ int bg_threshold;
+ int bg_maxfly;
+ bool do_noidle;
+ bool do_sync;
+ bool do_unplug;
+
+ /* readonly */
+ loff_t total_size;
+ atomic_t fly_count[XIO_PRIO_NR];
+ atomic_t queue_count[XIO_PRIO_NR];
+ atomic_t completed_count;
+ atomic_t total_completed_count[XIO_PRIO_NR];
+
+ /* private */
+ spinlock_t lock;
+ struct list_head queue_list[XIO_PRIO_NR];
+ struct list_head submitted_list[2];
+ struct list_head completed_list;
+
+ wait_queue_head_t submit_event;
+ wait_queue_head_t response_event;
+ struct mapfree_info *mf;
+ struct block_device *bdev;
+ struct task_struct *submit_thread;
+ struct task_struct *response_thread;
+ int bvec_max;
+ bool submitted;
+};
+
+struct bio_input {
+ XIO_INPUT(bio);
+};
+
+struct bio_output {
+ XIO_OUTPUT(bio);
+};
+
+XIO_TYPES(bio);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 19/32] mars: add new module xio_client |
| Message-ID | <sUetc-2Cs-21@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/xio_bricks/xio_client.c | 1083 ++++++++++++++++++++++++++
include/linux/xio/xio_client.h | 105 +++
2 files changed, 1188 insertions(+)
create mode 100644 drivers/staging/mars/xio_bricks/xio_client.c
create mode 100644 include/linux/xio/xio_client.h
diff --git a/drivers/staging/mars/xio_bricks/xio_client.c b/drivers/staging/mars/xio_bricks/xio_client.c
new file mode 100644
index 000000000000..209523378660
--- /dev/null
+++ b/drivers/staging/mars/xio_bricks/xio_client.c
@@ -0,0 +1,1083 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/jiffies.h>
+
+#include <linux/xio/xio.h>
+
+/************************ own type definitions ***********************/
+
+#include <linux/xio/xio_client.h>
+
+#define CLIENT_HASH_MAX (PAGE_SIZE / sizeof(struct list_head))
+
+int xio_client_abort = 10;
+
+int max_client_channels = 1;
+
+int max_client_bulk = 16;
+
+/************************ own helper functions ***********************/
+
+static int thread_count;
+
+static
+void _do_resubmit(struct client_channel *ch)
+{
+ struct client_output *output = ch->output;
+ unsigned long flags;
+
+ spin_lock_irqsave(&output->lock, flags);
+ if (!list_empty(&ch->wait_list)) {
+ struct list_head *first = ch->wait_list.next;
+ struct list_head *last = ch->wait_list.prev;
+ struct list_head *old_start = output->aio_list.next;
+
+#define list_connect __list_del /* the original routine has a misleading name: in reality it is more general */
+ list_connect(&output->aio_list, first);
+ list_connect(last, old_start);
+ INIT_LIST_HEAD(&ch->wait_list);
+ }
+ spin_unlock_irqrestore(&output->lock, flags);
+}
+
+static
+void _kill_thread(struct client_threadinfo *ti, const char *name)
+{
+ struct task_struct *thread = ti->thread;
+
+ if (thread) {
+ XIO_DBG("stopping %s thread\n", name);
+ ti->thread = NULL;
+ brick_thread_stop(thread);
+ }
+}
+
+static
+void _kill_channel(struct client_channel *ch)
+{
+ XIO_DBG("channel = %p\n", ch);
+ if (xio_socket_is_alive(&ch->socket)) {
+ XIO_DBG("shutdown socket\n");
+ xio_shutdown_socket(&ch->socket);
+ }
+ _kill_thread(&ch->receiver, "receiver");
+ if (ch->is_open) {
+ XIO_DBG("close socket\n");
+ xio_put_socket(&ch->socket);
+ }
+ ch->recv_error = 0;
+ ch->is_used = false;
+ ch->is_open = false;
+ ch->is_connected = false;
+ /* Re-Submit any waiting requests
+ */
+ _do_resubmit(ch);
+}
+
+static inline
+void _kill_all_channels(struct client_bundle *bundle)
+{
+ int i;
+
+ /* first pass: shutdown in parallel without waiting */
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++) {
+ struct client_channel *ch = &bundle->channel[i];
+
+ if (xio_socket_is_alive(&ch->socket)) {
+ XIO_DBG("shutdown socket %d\n", i);
+ xio_shutdown_socket(&ch->socket);
+ }
+ }
+ /* separate pass (may wait) */
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++)
+ _kill_channel(&bundle->channel[i]);
+}
+
+static int receiver_thread(void *data);
+
+static
+int _setup_channel(struct client_bundle *bundle, int ch_nr)
+{
+ struct client_channel *ch = &bundle->channel[ch_nr];
+ struct sockaddr_storage src_sockaddr;
+ struct sockaddr_storage dst_sockaddr;
+ int status;
+
+ ch->ch_nr = ch_nr;
+ if (unlikely(ch->receiver.thread)) {
+ XIO_WRN("receiver thread %d unexpectedly not dead\n", ch_nr);
+ _kill_thread(&ch->receiver, "receiver");
+ }
+
+ status = xio_create_sockaddr(&src_sockaddr, my_id());
+ if (unlikely(status < 0)) {
+ XIO_DBG("no src sockaddr, status = %d\n", status);
+ goto done;
+ }
+
+ status = xio_create_sockaddr(&dst_sockaddr, bundle->host);
+ if (unlikely(status < 0)) {
+ XIO_DBG("no dst sockaddr, status = %d\n", status);
+ goto done;
+ }
+
+ status = xio_create_socket(&ch->socket, &src_sockaddr, &dst_sockaddr, &device_tcp_params);
+ if (unlikely(status < 0)) {
+ XIO_DBG("no socket, status = %d\n", status);
+ goto really_done;
+ }
+ ch->socket.s_shutdown_on_err = true;
+ ch->socket.s_send_abort = xio_client_abort;
+ ch->socket.s_recv_abort = xio_client_abort;
+ ch->is_open = true;
+
+ ch->receiver.thread = brick_thread_create(
+ receiver_thread, ch, "xio_receiver%d.%d.%d", bundle->thread_count, ch_nr, ch->thread_count++);
+ if (unlikely(!ch->receiver.thread)) {
+ XIO_ERR("cannot start receiver thread %d, status = %d\n", ch_nr, status);
+ status = -ENOENT;
+ goto done;
+ }
+ ch->is_used = true;
+
+done:
+ if (status < 0) {
+ XIO_DBG(
+ "cannot connect channel %d to remote host '%s', retrying, status = %d\n",
+ ch_nr,
+ bundle->host ? bundle->host : "NULL",
+ status);
+ _kill_channel(ch);
+ }
+
+really_done:
+ return status;
+}
+
+static
+void _kill_bundle(struct client_bundle *bundle)
+{
+ _kill_thread(&bundle->sender, "sender");
+ _kill_all_channels(bundle);
+}
+
+static
+void _maintain_bundle(struct client_bundle *bundle)
+{
+ int i;
+
+ /* Re-open _any_ failed channel, even old ones.
+ * Reason: the number of channels might change during operation.
+ */
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++) {
+ struct client_channel *ch = &bundle->channel[i];
+
+ if (!ch->is_used ||
+ (!ch->recv_error && xio_socket_is_alive(&ch->socket)))
+ continue;
+
+ XIO_DBG("killing channel %d\n", i);
+ _kill_channel(ch);
+ /* Re-setup including connect optiona is done later.
+ */
+ }
+}
+
+static
+struct client_channel *_get_channel(struct client_bundle *bundle, int min_channel, int max_channel)
+{
+ struct client_channel *res;
+ long best_space;
+ int best_channel;
+ int i;
+
+ if (unlikely(max_channel <= 0 || max_channel > MAX_CLIENT_CHANNELS))
+ max_channel = MAX_CLIENT_CHANNELS;
+ if (unlikely(min_channel < 0 || min_channel >= max_channel)) {
+ min_channel = max_channel - 1;
+ if (unlikely(min_channel < 0))
+ min_channel = 0;
+ }
+
+ /* Fast path.
+ * Speculate that the next channel is already usable,
+ * and that it has enough room.
+ */
+ best_channel = bundle->old_channel + 1;
+ if (best_channel >= max_channel)
+ best_channel = min_channel;
+ res = &bundle->channel[best_channel];
+ if (res->is_connected && !res->recv_error && xio_socket_is_alive(&res->socket)) {
+ res->current_space = xio_socket_send_space_available(&res->socket);
+ if (res->current_space > (PAGE_SIZE + PAGE_SIZE / 4))
+ goto found;
+ }
+
+ /* Slow path. Do all the teady work.
+ */
+ _maintain_bundle(bundle);
+
+ res = NULL;
+ best_space = -1;
+ best_channel = -1;
+ for (i = min_channel; i < max_channel; i++) {
+ struct client_channel *ch = &bundle->channel[i];
+ long this_space;
+
+ /* create new channels when necessary */
+ if (unlikely(!ch->is_open)) {
+ int status;
+
+ /* only create one new channel at a time */
+ status = _setup_channel(bundle, i);
+ XIO_DBG("setup channel %d status=%d\n", i, status);
+ if (unlikely(status < 0))
+ continue;
+
+ this_space = xio_socket_send_space_available(&ch->socket);
+ ch->current_space = this_space;
+ /* Always prefer the newly opened channel */
+ res = ch;
+ best_channel = i;
+ break;
+ }
+
+ /* select the best usable channel */
+ this_space = xio_socket_send_space_available(&ch->socket);
+ ch->current_space = this_space;
+ if (this_space > best_space) {
+ best_space = this_space;
+ best_channel = i;
+ res = ch;
+ }
+ }
+
+ if (unlikely(!res)) {
+ XIO_WRN(
+ "cannot setup communication channel '%s' @%s\n",
+ bundle->path,
+ bundle->host);
+ goto done;
+ }
+
+ /* send initial connect command */
+ if (unlikely(!res->is_connected)) {
+ struct xio_cmd cmd = {
+ .cmd_code = CMD_CONNECT,
+ .cmd_str1 = bundle->path,
+ };
+ int status;
+
+ if (strstr(bundle->path, "/replay-"))
+ cmd.cmd_code = CMD_CONNECT_LOGGER;
+
+ status = xio_send_struct(&res->socket, &cmd, xio_cmd_meta);
+ XIO_DBG("send CMD_CONNECT status = %d\n", status);
+ if (unlikely(status < 0)) {
+ XIO_WRN(
+ "connect '%s' @%s on channel %d failed, status = %d\n",
+ bundle->path,
+ bundle->host,
+ best_channel,
+ status);
+ _kill_channel(res);
+ res = NULL;
+ goto done;
+ }
+ res->is_connected = true;
+ }
+
+found:
+ bundle->old_channel = best_channel;
+
+done:
+ return res;
+}
+
+static
+int _request_info(struct client_channel *ch)
+{
+ struct xio_cmd cmd = {
+ .cmd_code = CMD_GETINFO,
+ };
+ int status;
+
+ status = xio_send_struct(&ch->socket, &cmd, xio_cmd_meta);
+ XIO_DBG("send CMD_GETINFO status = %d\n", status);
+ if (unlikely(status < 0))
+ XIO_DBG("send of getinfo failed, status = %d\n", status);
+ return status;
+}
+
+static int sender_thread(void *data);
+
+static
+int _setup_bundle(struct client_bundle *bundle, const char *str)
+{
+ int status = -ENOMEM;
+
+ _kill_bundle(bundle);
+ brick_string_free(bundle->path);
+
+ bundle->path = brick_strdup(str);
+
+ status = -EINVAL;
+ bundle->host = strchr(bundle->path, '@');
+ if (unlikely(!bundle->host)) {
+ brick_string_free(bundle->path);
+ bundle->path = NULL;
+ XIO_ERR("parameter string '%s' contains no remote specifier with '@'-syntax\n", str);
+ goto done;
+ }
+ *bundle->host++ = '\0';
+
+ bundle->thread_count = thread_count++;
+ bundle->sender.thread = brick_thread_create(sender_thread, bundle, "xio_sender%d", bundle->thread_count);
+ if (unlikely(!bundle->sender.thread)) {
+ XIO_ERR(
+ "cannot start sender thread for '%s' @%s\n",
+ bundle->path,
+ bundle->host);
+ status = -ENOENT;
+ goto done;
+ }
+
+ status = 0;
+
+done:
+ XIO_DBG("status = %d\n", status);
+ return status;
+}
+
+/***************** own brick * input * output operations *****************/
+
+static int client_get_info(struct client_output *output, struct xio_info *info)
+{
+ int status;
+
+ output->get_info = true;
+ wake_up_interruptible_all(&output->bundle.sender_event);
+
+ wait_event_interruptible_timeout(output->info_event, output->got_info, 20 * HZ);
+ status = -ETIME;
+ if (output->got_info && info) {
+ output->got_info = false;
+ memcpy(info, &output->info, sizeof(*info));
+ status = 0;
+ }
+
+ return status;
+}
+
+static int client_io_get(struct client_output *output, struct aio_object *aio)
+{
+ int maxlen;
+
+ if (aio->obj_initialized) {
+ obj_get(aio);
+ return aio->io_len;
+ }
+
+ /* Limit transfers to page boundaries.
+ * Currently, this is more restrictive than necessary.
+ * TODO: improve performance by doing better when possible.
+ * This needs help from the server in some efficient way.
+ */
+ maxlen = PAGE_SIZE - (aio->io_pos & (PAGE_SIZE - 1));
+ if (aio->io_len > maxlen)
+ aio->io_len = maxlen;
+
+ if (!aio->io_data) { /* buffered IO */
+ struct client_aio_aspect *aio_a = client_aio_get_aspect(output->brick, aio);
+
+ if (!aio_a)
+ return -EILSEQ;
+
+ aio->io_data = brick_block_alloc(aio->io_pos, (aio_a->alloc_len = aio->io_len));
+
+ aio_a->do_dealloc = true;
+ aio->io_flags = 0;
+ }
+
+ obj_get_first(aio);
+ return 0;
+}
+
+static void client_io_put(struct client_output *output, struct aio_object *aio)
+{
+ struct client_aio_aspect *aio_a;
+
+ if (!obj_put(aio))
+ goto out_return;
+ aio_a = client_aio_get_aspect(output->brick, aio);
+ if (aio_a && aio_a->do_dealloc)
+ brick_block_free(aio->io_data, aio_a->alloc_len);
+ obj_free(aio);
+out_return:;
+}
+
+static
+void _hash_insert(struct client_output *output, struct client_aio_aspect *aio_a)
+{
+ struct aio_object *aio = aio_a->object;
+ unsigned long flags;
+ int hash_index;
+
+ spin_lock_irqsave(&output->lock, flags);
+ list_del(&aio_a->io_head);
+ list_add_tail(&aio_a->io_head, &output->aio_list);
+ list_del(&aio_a->hash_head);
+ aio->io_id = ++output->last_id;
+ hash_index = aio->io_id % CLIENT_HASH_MAX;
+ list_add_tail(&aio_a->hash_head, &output->hash_table[hash_index]);
+ spin_unlock_irqrestore(&output->lock, flags);
+}
+
+static void client_io_io(struct client_output *output, struct aio_object *aio)
+{
+ struct client_aio_aspect *aio_a;
+ int error = -EINVAL;
+
+ aio_a = client_aio_get_aspect(output->brick, aio);
+ if (unlikely(!aio_a))
+ goto error;
+
+ while (output->brick->max_flying > 0 && atomic_read(&output->fly_count) > output->brick->max_flying)
+ brick_msleep(1000 * 2 / HZ);
+
+ if (!output->brick->power.on_led)
+ XIO_ERR("IO submission on dead instance\n");
+
+ atomic_inc(&xio_global_io_flying);
+ atomic_inc(&output->fly_count);
+ obj_get(aio);
+
+ aio_a->submit_jiffies = jiffies;
+ _hash_insert(output, aio_a);
+
+ wake_up_interruptible_all(&output->bundle.sender_event);
+
+ goto out_return;
+error:
+ XIO_ERR("IO error = %d\n", error);
+ SIMPLE_CALLBACK(aio, error);
+ client_io_put(output, aio);
+out_return:;
+}
+
+static
+int receiver_thread(void *data)
+{
+ struct client_channel *ch = data;
+ struct client_output *output = ch->output;
+ int status = 0;
+
+ while (!brick_thread_should_stop()) {
+ struct xio_cmd cmd = {};
+ struct list_head *tmp;
+ struct client_aio_aspect *aio_a = NULL;
+ struct aio_object *aio = NULL;
+ unsigned long flags;
+
+ if (ch->recv_error) {
+ /* The protocol may be out of sync.
+ * Consume some data to avoid distributed deadlocks.
+ */
+ (void)xio_recv_raw(&ch->socket, &cmd, 0, sizeof(cmd));
+ brick_msleep(100);
+ status = ch->recv_error;
+ continue;
+ }
+
+ status = xio_recv_struct(&ch->socket, &cmd, xio_cmd_meta);
+ if (status <= 0) {
+ if (!xio_socket_is_alive(&ch->socket)) {
+ XIO_DBG("socket is dead\n");
+ brick_msleep(1000);
+ continue;
+ }
+ goto done;
+ }
+
+ switch (cmd.cmd_code & CMD_FLAG_MASK) {
+ case CMD_NOTIFY:
+ local_trigger();
+ break;
+ case CMD_CONNECT:
+ if (cmd.cmd_int1 < 0) {
+ status = cmd.cmd_int1;
+ XIO_ERR(
+ "remote brick connect '%s' @%s failed, remote status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ goto done;
+ }
+ break;
+ case CMD_CB:
+ {
+ int hash_index = cmd.cmd_int1 % CLIENT_HASH_MAX;
+
+ spin_lock_irqsave(&output->lock, flags);
+ for (
+ tmp = output->hash_table[hash_index].next; tmp != &output->hash_table[hash_index]; tmp = tmp->next) {
+ struct aio_object *tmp_aio;
+
+ aio_a = container_of(tmp, struct client_aio_aspect, hash_head);
+ tmp_aio = aio_a->object;
+ CHECK_PTR(tmp_aio, err);
+ if (tmp_aio->io_id != cmd.cmd_int1)
+ continue;
+ aio = tmp_aio;
+ list_del_init(&aio_a->hash_head);
+ list_del_init(&aio_a->io_head);
+ break;
+
+err:
+ spin_unlock_irqrestore(&output->lock, flags);
+ status = -EBADR;
+ goto done;
+ }
+ spin_unlock_irqrestore(&output->lock, flags);
+
+ if (unlikely(!aio)) {
+ XIO_WRN(
+ "got unknown callback id %d on '%s' @%s\n",
+ cmd.cmd_int1,
+ output->bundle.path,
+ output->bundle.host);
+ /* try to consume the corresponding payload */
+ aio = client_alloc_aio(output->brick);
+ status = xio_recv_cb(&ch->socket, aio, &cmd);
+ obj_free(aio);
+ goto done;
+ }
+
+ status = xio_recv_cb(&ch->socket, aio, &cmd);
+ if (unlikely(status < 0)) {
+ XIO_WRN(
+ "interrupted data transfer during callback on '%s' @%s, status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ _hash_insert(output, aio_a);
+ goto done;
+ }
+
+ if (aio->_object_cb.cb_error < 0)
+ XIO_DBG("ERROR %d\n", aio->_object_cb.cb_error);
+ SIMPLE_CALLBACK(aio, aio->_object_cb.cb_error);
+
+ client_io_put(output, aio);
+
+ atomic_dec(&output->fly_count);
+ atomic_dec(&xio_global_io_flying);
+ break;
+ }
+ case CMD_GETINFO:
+ status = xio_recv_struct(&ch->socket, &output->info, xio_info_meta);
+ if (status < 0) {
+ XIO_WRN(
+ "got bad info from remote '%s' @%s, status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ goto done;
+ }
+ output->got_info = true;
+ wake_up_interruptible_all(&output->info_event);
+ break;
+ default:
+ XIO_ERR(
+ "got bad command %d from remote '%s' @%s, terminating.\n",
+ cmd.cmd_code,
+ output->bundle.path,
+ output->bundle.host);
+ status = -EBADR;
+ goto done;
+ }
+done:
+ brick_string_free(cmd.cmd_str1);
+ if (unlikely(status < 0)) {
+ if (!ch->recv_error) {
+ XIO_DBG("signalling recv_error = %d\n", status);
+ ch->recv_error = status;
+ }
+ brick_msleep(100);
+ }
+ /* wake up sender in any case */
+ wake_up_interruptible_all(&output->bundle.sender_event);
+ }
+
+ if (unlikely(status < 0)) {
+ XIO_WRN(
+ "receiver thread '%s' @%s terminated with status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ }
+
+ xio_shutdown_socket(&ch->socket);
+ return status;
+}
+
+static
+void _do_timeout(struct client_output *output, struct list_head *anchor, int *rounds, bool force)
+{
+ struct client_brick *brick = output->brick;
+ struct list_head *tmp;
+ struct list_head *next;
+ LIST_HEAD(tmp_list);
+ long io_timeout = brick->power.io_timeout;
+ unsigned long flags;
+
+ if (list_empty(anchor))
+ goto out_return;
+ /* When io_timeout is 0, use the global default.
+ * When io_timeout is negative, no timeout will occur.
+ * Exeception: when the brick is forcefully shutting down.
+ */
+ if (!io_timeout)
+ io_timeout = global_net_io_timeout;
+
+ if (!xio_net_is_alive || !brick->power.button)
+ force = true;
+
+ if (!force && io_timeout <= 0)
+ goto out_return;
+ io_timeout *= HZ;
+
+ spin_lock_irqsave(&output->lock, flags);
+ for (tmp = anchor->next, next = tmp->next; tmp != anchor; tmp = next, next = tmp->next) {
+ struct client_aio_aspect *aio_a;
+
+ aio_a = container_of(tmp, struct client_aio_aspect, io_head);
+
+ if (!force &&
+ !time_is_before_jiffies(aio_a->submit_jiffies + io_timeout)) {
+ continue;
+ }
+
+ list_del_init(&aio_a->hash_head);
+ list_del_init(&aio_a->io_head);
+ list_add_tail(&aio_a->tmp_head, &tmp_list);
+ }
+ spin_unlock_irqrestore(&output->lock, flags);
+
+ while (!list_empty(&tmp_list)) {
+ struct client_aio_aspect *aio_a;
+ struct aio_object *aio;
+
+ tmp = tmp_list.next;
+ list_del_init(tmp);
+ aio_a = container_of(tmp, struct client_aio_aspect, tmp_head);
+ aio = aio_a->object;
+
+ if (unlikely(!(*rounds)++)) {
+ XIO_WRN(
+ "'%s' @%s timeout after %ld: signalling IO error at pos = %lld len = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ io_timeout,
+ aio->io_pos,
+ aio->io_len);
+ }
+
+ atomic_inc(&output->timeout_count);
+
+ SIMPLE_CALLBACK(aio, -ESTALE);
+
+ client_io_put(output, aio);
+
+ atomic_dec(&output->fly_count);
+ atomic_dec(&xio_global_io_flying);
+ }
+out_return:;
+}
+
+static
+void _do_timeout_all(struct client_output *output, bool force)
+{
+ int rounds = 0;
+ int i;
+
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++) {
+ struct client_channel *ch = &output->bundle.channel[i];
+
+ if (!ch->is_used)
+ continue;
+ _do_timeout(output, &ch->wait_list, &rounds, force);
+ }
+ _do_timeout(output, &output->aio_list, &rounds, force);
+ if (unlikely(rounds > 0)) {
+ XIO_WRN(
+ "'%s' @%s had %d timeouts, force = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ rounds,
+ force);
+ }
+}
+
+static int sender_thread(void *data)
+{
+ struct client_bundle *bundle = data;
+ struct client_output *output = container_of(bundle, struct client_output, bundle);
+ struct client_brick *brick = output->brick;
+ struct client_channel *ch = NULL;
+ bool do_timeout = false;
+ int ch_skip = max_client_bulk;
+ int status = -ESHUTDOWN;
+ unsigned long flags;
+
+ while (!brick_thread_should_stop()) {
+ struct list_head *tmp = NULL;
+ struct client_aio_aspect *aio_a;
+ struct aio_object *aio;
+ int min_nr;
+ int max_nr;
+
+ /* timeouting is a rather expensive operation, don't do it too often */
+ if (do_timeout) {
+ do_timeout = false;
+ _maintain_bundle(&output->bundle);
+ _do_timeout_all(output, false);
+ }
+
+ wait_event_interruptible_timeout(
+ output->bundle.sender_event,
+ !list_empty(&output->aio_list) ||
+ output->get_info,
+ 2 * HZ);
+
+ if (output->get_info) {
+ ch = _get_channel(bundle, 0, 1);
+ if (unlikely(!ch)) {
+ do_timeout = true;
+ brick_msleep(1000);
+ continue;
+ }
+ status = _request_info(ch);
+ if (unlikely(status < 0)) {
+ XIO_WRN(
+ "cannot send info request '%s' @%s, status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ do_timeout = true;
+ brick_msleep(1000);
+ continue;
+ }
+ output->get_info = false;
+ }
+
+ /* Grab the next aio from the queue
+ */
+ spin_lock_irqsave(&output->lock, flags);
+ tmp = output->aio_list.next;
+ if (tmp == &output->aio_list) {
+ spin_unlock_irqrestore(&output->lock, flags);
+ XIO_DBG("empty %d %d\n", output->get_info, brick_thread_should_stop());
+ do_timeout = true;
+ continue;
+ }
+ list_del_init(tmp);
+ /* notice: hash_head remains in its list! */
+ spin_unlock_irqrestore(&output->lock, flags);
+
+ aio_a = container_of(tmp, struct client_aio_aspect, io_head);
+ aio = aio_a->object;
+
+ if (brick->limit_mode) {
+ int amount = 0;
+
+ if (aio->io_cs_mode < 2)
+ amount = (aio->io_len - 1) / 1024 + 1;
+ rate_limit_sleep(&client_limiter, amount);
+ }
+
+ /* try to spread reads over multiple channels.... */
+ min_nr = 0;
+ max_nr = max_client_channels;
+ if (!aio->io_rw) {
+ /* optionally separate reads from writes */
+ if (brick->separate_reads && max_nr > 1)
+ min_nr = 1;
+ } else if (!brick->allow_permuting_writes) {
+ max_nr = 1;
+ }
+ if (!ch || ch->recv_error ||
+ !xio_socket_is_alive(&ch->socket))
+ do_timeout = true;
+ if (do_timeout || ch->ch_nr >= max_nr || --ch_skip < 0) {
+ ch = _get_channel(bundle, min_nr, max_nr);
+ if (unlikely(!ch)) {
+ /* notice: this will re-assign hash_head without harm */
+ _hash_insert(output, aio_a);
+ brick_msleep(1000);
+ continue;
+ }
+ /* estimate: add some headroom for overhead */
+ ch_skip = ch->current_space / PAGE_SIZE +
+ ch->current_space / (PAGE_SIZE * 8);
+ if (ch_skip > max_client_bulk)
+ ch_skip = max_client_bulk;
+ }
+
+ spin_lock_irqsave(&output->lock, flags);
+ list_add(tmp, &ch->wait_list);
+ /* notice: hash_head is already there! */
+ spin_unlock_irqrestore(&output->lock, flags);
+
+ status = xio_send_aio(&ch->socket, aio);
+ if (unlikely(status < 0)) {
+ _hash_insert(output, aio_a);
+ do_timeout = true;
+ ch = NULL;
+ /* retry submission on next occasion.. */
+ XIO_WRN(
+ "aio send '%s' @%s failed, status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+
+ brick_msleep(100);
+ continue;
+ }
+ }
+
+ if (unlikely(status < 0)) {
+ XIO_WRN(
+ "sender thread '%s' @%s terminated with status = %d\n",
+ output->bundle.path,
+ output->bundle.host,
+ status);
+ }
+
+ _kill_all_channels(bundle);
+
+ /* Signal error on all pending IO requests.
+ * We have no other chance (except probably delaying
+ * this until destruction which is probably not what
+ * we want).
+ */
+ _do_timeout_all(output, true);
+ wake_up_interruptible_all(&output->bundle.sender_event);
+ XIO_DBG("sender terminated\n");
+ return status;
+}
+
+static int client_switch(struct client_brick *brick)
+{
+ struct client_output *output = brick->outputs[0];
+ int status = 0;
+
+ if (brick->power.button) {
+ if (brick->power.on_led)
+ goto done;
+ xio_set_power_off_led((void *)brick, false);
+ status = _setup_bundle(&output->bundle, brick->brick_name);
+ if (likely(status >= 0)) {
+ output->get_info = true;
+ brick->connection_state = 1;
+ xio_set_power_on_led((void *)brick, true);
+ }
+ } else {
+ if (brick->power.off_led)
+ goto done;
+ xio_set_power_on_led((void *)brick, false);
+ _kill_bundle(&output->bundle);
+ _do_timeout_all(output, true);
+ output->got_info = false;
+ brick->connection_state = 0;
+ xio_set_power_off_led((void *)brick, !output->bundle.sender.thread);
+ }
+done:
+ return status;
+}
+
+/*************** informational * statistics **************/
+
+static
+char *client_statistics(struct client_brick *brick, int verbose)
+{
+ struct client_output *output = brick->outputs[0];
+ char *res = brick_string_alloc(1024);
+ int socket_count = 0;
+ int i;
+
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++) {
+ struct client_channel *ch = &output->bundle.channel[i];
+
+ if (xio_socket_is_alive(&ch->socket))
+ socket_count++;
+ }
+ snprintf(
+ res, 1024,
+ "socket_count = %d max_flying = %d io_timeout = %d | timeout_count = %d fly_count = %d\n",
+ socket_count,
+ brick->max_flying,
+ brick->power.io_timeout,
+ atomic_read(&output->timeout_count),
+ atomic_read(&output->fly_count));
+
+ return res;
+}
+
+static
+void client_reset_statistics(struct client_brick *brick)
+{
+ struct client_output *output = brick->outputs[0];
+
+ atomic_set(&output->timeout_count, 0);
+}
+
+/*************** object * aspect constructors * destructors **************/
+
+static int client_aio_aspect_init_fn(struct generic_aspect *_ini)
+{
+ struct client_aio_aspect *ini = (void *)_ini;
+
+ INIT_LIST_HEAD(&ini->io_head);
+ INIT_LIST_HEAD(&ini->hash_head);
+ INIT_LIST_HEAD(&ini->tmp_head);
+ return 0;
+}
+
+static void client_aio_aspect_exit_fn(struct generic_aspect *_ini)
+{
+ struct client_aio_aspect *ini = (void *)_ini;
+
+ CHECK_HEAD_EMPTY(&ini->io_head);
+ CHECK_HEAD_EMPTY(&ini->hash_head);
+}
+
+XIO_MAKE_STATICS(client);
+
+/********************* brick constructors * destructors *******************/
+
+static int client_brick_construct(struct client_brick *brick)
+{
+ return 0;
+}
+
+static int client_output_construct(struct client_output *output)
+{
+ int i;
+
+ output->hash_table = brick_block_alloc(0, PAGE_SIZE);
+
+ for (i = 0; i < CLIENT_HASH_MAX; i++)
+ INIT_LIST_HEAD(&output->hash_table[i]);
+
+ for (i = 0; i < MAX_CLIENT_CHANNELS; i++) {
+ struct client_channel *ch = &output->bundle.channel[i];
+
+ ch->output = output;
+ INIT_LIST_HEAD(&ch->wait_list);
+ }
+
+ init_waitqueue_head(&output->bundle.sender_event);
+
+ spin_lock_init(&output->lock);
+ INIT_LIST_HEAD(&output->aio_list);
+ init_waitqueue_head(&output->info_event);
+ return 0;
+}
+
+static int client_output_destruct(struct client_output *output)
+{
+ brick_string_free(output->bundle.path);
+ output->bundle.path = NULL;
+ brick_block_free(output->hash_table, PAGE_SIZE);
+ return 0;
+}
+
+/************************ static structs ***********************/
+
+static struct client_brick_ops client_brick_ops = {
+ .brick_switch = client_switch,
+ .brick_statistics = client_statistics,
+ .reset_statistics = client_reset_statistics,
+};
+
+static struct client_output_ops client_output_ops = {
+ .xio_get_info = client_get_info,
+ .aio_get = client_io_get,
+ .aio_put = client_io_put,
+ .aio_io = client_io_io,
+};
+
+const struct client_input_type client_input_type = {
+ .type_name = "client_input",
+ .input_size = sizeof(struct client_input),
+};
+
+static const struct client_input_type *client_input_types[] = {
+ &client_input_type,
+};
+
+const struct client_output_type client_output_type = {
+ .type_name = "client_output",
+ .output_size = sizeof(struct client_output),
+ .master_ops = &client_output_ops,
+ .output_construct = &client_output_construct,
+ .output_destruct = &client_output_destruct,
+};
+
+static const struct client_output_type *client_output_types[] = {
+ &client_output_type,
+};
+
+const struct client_brick_type client_brick_type = {
+ .type_name = "client_brick",
+ .brick_size = sizeof(struct client_brick),
+ .max_inputs = 0,
+ .max_outputs = 1,
+ .master_ops = &client_brick_ops,
+ .aspect_types = client_aspect_types,
+ .default_input_types = client_input_types,
+ .default_output_types = client_output_types,
+ .brick_construct = &client_brick_construct,
+};
+
+/***************** module init stuff ************************/
+
+struct rate_limiter client_limiter = {
+ /* Let all be zero */
+};
+
+int global_net_io_timeout = 30;
+
+module_param_named(net_io_timeout, global_net_io_timeout, int, 0);
+
+int __init init_xio_client(void)
+{
+ XIO_INF("init_client()\n");
+ _client_brick_type = (void *)&client_brick_type;
+ return client_register_brick_type();
+}
+
+void exit_xio_client(void)
+{
+ XIO_INF("exit_client()\n");
+ client_unregister_brick_type();
+}
diff --git a/include/linux/xio/xio_client.h b/include/linux/xio/xio_client.h
new file mode 100644
index 000000000000..5accb4898adc
--- /dev/null
+++ b/include/linux/xio/xio_client.h
@@ -0,0 +1,105 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef XIO_CLIENT_H
+#define XIO_CLIENT_H
+
+#include <linux/xio/xio_net.h>
+#include <linux/brick/lib_limiter.h>
+
+extern struct rate_limiter client_limiter;
+extern int global_net_io_timeout;
+extern int xio_client_abort;
+extern int max_client_channels;
+extern int max_client_bulk;
+
+#define MAX_CLIENT_CHANNELS 4
+
+struct client_aio_aspect {
+ GENERIC_ASPECT(aio);
+ struct list_head io_head;
+ struct list_head hash_head;
+ struct list_head tmp_head;
+ unsigned long submit_jiffies;
+ int alloc_len;
+ bool do_dealloc;
+};
+
+struct client_brick {
+ XIO_BRICK(client);
+ /* tunables */
+ int max_flying; /* limit on parallelism */
+ bool limit_mode;
+ bool allow_permuting_writes;
+ bool separate_reads;
+
+ /* readonly from outside */
+ int connection_state; /* 0 = switched off, 1 = not connected, 2 = connected */
+};
+
+struct client_input {
+ XIO_INPUT(client);
+};
+
+struct client_threadinfo {
+ struct task_struct *thread;
+};
+
+struct client_channel {
+ struct xio_socket socket;
+ struct client_threadinfo receiver;
+ struct list_head wait_list;
+ struct client_output *output;
+ long current_space;
+ int thread_count;
+ int recv_error;
+ int ch_nr;
+ bool is_used;
+ bool is_open;
+ bool is_connected;
+};
+
+struct client_bundle {
+ char *host;
+ char *path;
+ int thread_count;
+ int old_channel;
+
+ wait_queue_head_t sender_event;
+ struct client_threadinfo sender;
+ struct client_channel channel[MAX_CLIENT_CHANNELS];
+};
+
+struct client_output {
+ XIO_OUTPUT(client);
+ atomic_t fly_count;
+ atomic_t timeout_count;
+ spinlock_t lock;
+ struct list_head aio_list;
+ int last_id;
+ struct client_bundle bundle;
+ struct xio_info info;
+
+ wait_queue_head_t info_event;
+ bool get_info;
+ bool got_info;
+ struct list_head *hash_table;
+};
+
+XIO_TYPES(client);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 28/32] mars: add new module mars_proc |
| Message-ID | <sUetc-2Cs-37@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/mars/mars_proc.c | 389 ++++++++++++++++++++++++++++++++++
drivers/staging/mars/mars/mars_proc.h | 34 +++
2 files changed, 423 insertions(+)
create mode 100644 drivers/staging/mars/mars/mars_proc.c
create mode 100644 drivers/staging/mars/mars/mars_proc.h
diff --git a/drivers/staging/mars/mars/mars_proc.c b/drivers/staging/mars/mars/mars_proc.c
new file mode 100644
index 000000000000..84b4dfc82211
--- /dev/null
+++ b/drivers/staging/mars/mars/mars_proc.c
@@ -0,0 +1,389 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+
+#include <linux/sysctl.h>
+#include <linux/uaccess.h>
+
+#include "strategy.h"
+#include "mars_proc.h"
+#include <linux/xio/lib_mapfree.h>
+#include <linux/xio/xio_bio.h>
+#include <linux/xio/xio_if.h>
+#include <linux/xio/xio_copy.h>
+#include <linux/xio/xio_client.h>
+#include <linux/xio/xio_server.h>
+#include <linux/xio/xio_trans_logger.h>
+
+xio_info_fn xio_info;
+
+static
+int trigger_sysctl_handler(
+ struct ctl_table *table,
+ int write,
+ void __user *buffer,
+ size_t *length,
+ loff_t *ppos)
+{
+ ssize_t res = 0;
+ size_t len = *length;
+
+ XIO_DBG("write = %d len = %ld pos = %lld\n", write, len, *ppos);
+
+ if (!len || *ppos > 0)
+ goto done;
+
+ if (write) {
+ char tmp[8] = {};
+
+ res = len; /* fake consumption of all data */
+
+ if (len > 7)
+ len = 7;
+ if (!copy_from_user(tmp, buffer, len)) {
+ int code = 0;
+ int status = kstrtoint(tmp, 10, &code);
+
+ /* the return value from ssanf() does not matter */
+ (void)status;
+ if (code > 0)
+ local_trigger();
+ if (code > 1)
+ remote_trigger();
+ }
+ } else {
+ char *answer = "MARS module not operational\n";
+ char *tmp = NULL;
+ int mylen;
+
+ if (xio_info) {
+ answer = "internal error while determining xio_info\n";
+ tmp = xio_info();
+ if (tmp)
+ answer = tmp;
+ }
+
+ mylen = strlen(answer);
+ if (len > mylen)
+ len = mylen;
+ res = len;
+ if (copy_to_user(buffer, answer, len)) {
+ XIO_ERR("write %ld bytes at %p failed\n", len, buffer);
+ res = -EFAULT;
+ }
+ brick_string_free(tmp);
+ }
+
+done:
+ XIO_DBG("res = %ld\n", res);
+ *length = res;
+ if (res >= 0) {
+ *ppos += res;
+ return 0;
+ }
+ return res;
+}
+
+static
+int lamport_sysctl_handler(
+ struct ctl_table *table,
+ int write,
+ void __user *buffer,
+ size_t *length,
+ loff_t *ppos)
+{
+ ssize_t res = 0;
+ size_t len = *length;
+ int my_len = 128;
+ char *tmp = brick_string_alloc(my_len);
+ struct timespec know = CURRENT_TIME;
+ struct timespec lnow;
+
+ XIO_DBG("write = %d len = %ld pos = %lld\n", write, len, *ppos);
+
+ if (!len || *ppos > 0)
+ goto done;
+
+ if (write)
+ return -EINVAL;
+
+ get_lamport(&lnow);
+
+ res = scnprintf(
+ tmp,
+ my_len,
+ "CURRENT_TIME=%ld.%09ld\nlamport_now=%ld.%09ld\n",
+ know.tv_sec, know.tv_nsec,
+ lnow.tv_sec, lnow.tv_nsec
+ );
+
+ if (copy_to_user(buffer, tmp, res)) {
+ XIO_ERR("write %ld bytes at %p failed\n", res, buffer);
+ res = -EFAULT;
+ }
+ brick_string_free(tmp);
+
+done:
+ XIO_DBG("res = %ld\n", res);
+ *length = res;
+ if (res >= 0) {
+ *ppos += res;
+ return 0;
+ }
+ return res;
+}
+
+#ifdef CTL_UNNUMBERED
+#define _CTL_NAME .ctl_name = CTL_UNNUMBERED,
+#define _CTL_STRATEGY(handler) .strategy = &handler,
+#else
+#define _CTL_NAME /*empty*/
+#define _CTL_STRATEGY(handler) /*empty*/
+#endif
+
+#define VEC_ENTRY(NAME, VAR, MODE, COUNT) \
+ { \
+ _CTL_NAME \
+ .procname = NAME, \
+ .data = &(VAR), \
+ .maxlen = sizeof(int) * (COUNT), \
+ .mode = MODE, \
+ .proc_handler = &proc_dointvec, \
+ _CTL_STRATEGY(sysctl_intvec) \
+ }
+
+#define INT_ENTRY(NAME, VAR, MODE) \
+ VEC_ENTRY(NAME, VAR, MODE, 1)
+
+/* checkpatch.pl: no, these complex values cannot be easily enclosed
+ * in parentheses. If { ... } were used inside the macro body, it would
+ * no longer be possible to add additional fields externally.
+ * I could inject further fields externally via parameters, but
+ * that would make it less understandable.
+ */
+#define LIMITER_ENTRIES(VAR, PREFIX, SUFFIX) \
+ INT_ENTRY(PREFIX "_ops_total_" SUFFIX, (VAR)->lim_total_ops, 0400),\
+ INT_ENTRY(PREFIX "_amount_total_" SUFFIX, (VAR)->lim_total_amount, 0400),\
+ INT_ENTRY(PREFIX "_ops_ratelimit_" SUFFIX, (VAR)->lim_max_ops_rate, 0600),\
+ INT_ENTRY(PREFIX "_amount_ratelimit_" SUFFIX, (VAR)->lim_max_amount_rate, 0600),\
+ INT_ENTRY(PREFIX "_maxdelay_ms", (VAR)->lim_max_delay, 0600), \
+ INT_ENTRY(PREFIX "_minwindow_ms", (VAR)->lim_min_window, 0600),\
+ INT_ENTRY(PREFIX "_maxwindow_ms", (VAR)->lim_max_window, 0600),\
+ INT_ENTRY(PREFIX "_ops_cumul_" SUFFIX, (VAR)->lim_ops_cumul, 0600),\
+ INT_ENTRY(PREFIX "_amount_cumul_" SUFFIX, (VAR)->lim_amount_cumul, 0600),\
+ INT_ENTRY(PREFIX "_ops_rate_" SUFFIX, (VAR)->lim_ops_rate, 0400),\
+ INT_ENTRY(PREFIX "_amount_rate_" SUFFIX, (VAR)->lim_amount_rate, 0400)\
+
+#define THRESHOLD_ENTRIES(VAR, PREFIX) \
+ INT_ENTRY(PREFIX "_threshold_us", (VAR)->thr_limit, 0600), \
+ INT_ENTRY(PREFIX "_factor_percent", (VAR)->thr_factor, 0600), \
+ INT_ENTRY(PREFIX "_plus_us", (VAR)->thr_plus, 0600), \
+ INT_ENTRY(PREFIX "_max_ms", (VAR)->thr_max, 0600), \
+ INT_ENTRY(PREFIX "_triggered", (VAR)->thr_triggered, 0400),\
+ INT_ENTRY(PREFIX "_true_hit", (VAR)->thr_true_hit, 0400) \
+
+static
+struct ctl_table traffic_tuning_table[] = {
+ LIMITER_ENTRIES(&client_limiter, "client_role_traffic", "kb"),
+ LIMITER_ENTRIES(&server_limiter, "server_role_traffic", "kb"),
+ {}
+};
+
+static
+struct ctl_table io_tuning_table[] = {
+ LIMITER_ENTRIES(&global_writeback.limiter, "writeback", "kb"),
+ INT_ENTRY("writeback_until_percent", global_writeback.until_percent, 0600),
+ THRESHOLD_ENTRIES(&global_io_threshold, "global_io"),
+ THRESHOLD_ENTRIES(&bio_submit_threshold, "bio_submit"),
+ THRESHOLD_ENTRIES(&bio_io_threshold[0], "bio_io_r"),
+ THRESHOLD_ENTRIES(&bio_io_threshold[1], "bio_io_w"),
+ {}
+};
+
+#define TCP_TUNING_ENTRIES(VAR) \
+ INT_ENTRY("ip_tos", (VAR)->ip_tos, 0600), \
+ INT_ENTRY("tcp_window_size", (VAR)->tcp_window_size, 0600), \
+ INT_ENTRY("tcp_nodelay", (VAR)->tcp_nodelay, 0600), \
+ INT_ENTRY("tcp_timeout", (VAR)->tcp_timeout, 0600), \
+ INT_ENTRY("tcp_keepcnt", (VAR)->tcp_keepcnt, 0600), \
+ INT_ENTRY("tcp_keepintvl", (VAR)->tcp_keepintvl, 0600), \
+ INT_ENTRY("tcp_keepidle", (VAR)->tcp_keepidle, 0600), \
+
+static
+struct ctl_table repl_tuning_table[] = {
+ TCP_TUNING_ENTRIES(&repl_tcp_params)
+ {}
+};
+
+static
+struct ctl_table device_tuning_table[] = {
+ TCP_TUNING_ENTRIES(&device_tcp_params)
+ {}
+};
+
+static
+struct ctl_table mars_table[] = {
+ {
+ _CTL_NAME
+ .procname = "trigger",
+ .mode = 0200,
+ .proc_handler = &trigger_sysctl_handler,
+ },
+ {
+ _CTL_NAME
+ .procname = "info",
+ .mode = 0400,
+ .proc_handler = &trigger_sysctl_handler,
+ },
+ {
+ _CTL_NAME
+ .procname = "lamport_clock",
+ .mode = 0400,
+ .proc_handler = &lamport_sysctl_handler,
+ },
+ INT_ENTRY("show_log_messages", brick_say_logging, 0600),
+ INT_ENTRY("show_debug_messages", brick_say_debug, 0600),
+ INT_ENTRY("show_statistics_global", global_show_statist, 0600),
+ INT_ENTRY("show_statistics_server", server_show_statist, 0600),
+#ifdef CONFIG_MARS_DEBUG
+ INT_ENTRY("debug_crash_mode", mars_crash_mode, 0600),
+ INT_ENTRY("debug_hang_mode", mars_hang_mode, 0600),
+#endif
+ INT_ENTRY("logger_completion_semantics", trans_logger_completion_semantics, 0600),
+ INT_ENTRY("logger_do_crc", trans_logger_do_crc, 0600),
+ INT_ENTRY("syslog_min_class", brick_say_syslog_min, 0600),
+ INT_ENTRY("syslog_max_class", brick_say_syslog_max, 0600),
+ INT_ENTRY("syslog_flood_class", brick_say_syslog_flood_class, 0600),
+ INT_ENTRY("syslog_flood_limit", brick_say_syslog_flood_limit, 0600),
+ INT_ENTRY("syslog_flood_recovery_s", brick_say_syslog_flood_recovery, 0600),
+ INT_ENTRY("delay_say_on_overflow", delay_say_on_overflow, 0600),
+ INT_ENTRY("mapfree_period_sec", mapfree_period_sec, 0600),
+ INT_ENTRY("mapfree_grace_keep_mb", mapfree_grace_keep_mb, 0600),
+ INT_ENTRY("logger_max_interleave", trans_logger_max_interleave, 0600),
+ INT_ENTRY("logger_resume", trans_logger_resume, 0600),
+ INT_ENTRY("logger_replay_timeout_sec", trans_logger_replay_timeout, 0600),
+ INT_ENTRY("mem_limit_percent", mars_mem_percent, 0600),
+ INT_ENTRY("logger_mem_used_kb", trans_logger_mem_usage, 0400),
+ INT_ENTRY("mem_used_raw_kb", brick_global_block_used, 0400),
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ INT_ENTRY("mem_allow_freelist", brick_allow_freelist, 0600),
+ VEC_ENTRY("mem_freelist_max", brick_mem_freelist_max, 0600, BRICK_MAX_ORDER + 1),
+ VEC_ENTRY("mem_alloc_count", brick_mem_alloc_count, 0400, BRICK_MAX_ORDER + 1),
+ VEC_ENTRY("mem_alloc_max", brick_mem_alloc_count, 0600, BRICK_MAX_ORDER + 1),
+#endif
+ INT_ENTRY("io_flying_count", xio_global_io_flying, 0400),
+ INT_ENTRY("copy_overlap", xio_copy_overlap, 0600),
+ INT_ENTRY("copy_read_prio", xio_copy_read_prio, 0600),
+ INT_ENTRY("copy_write_prio", xio_copy_write_prio, 0600),
+ INT_ENTRY("copy_read_max_fly", xio_copy_read_max_fly, 0600),
+ INT_ENTRY("copy_write_max_fly", xio_copy_write_max_fly, 0600),
+ INT_ENTRY("statusfiles_rollover_sec", mars_rollover_interval, 0600),
+ INT_ENTRY("scan_interval_sec", mars_scan_interval, 0600),
+ INT_ENTRY("propagate_interval_sec", mars_propagate_interval, 0600),
+ INT_ENTRY("sync_flip_interval_sec", mars_sync_flip_interval, 0600),
+ INT_ENTRY("peer_abort", mars_peer_abort, 0600),
+ INT_ENTRY("client_abort", xio_client_abort, 0600),
+ INT_ENTRY("do_fast_fullsync", mars_fast_fullsync, 0600),
+ INT_ENTRY("logrot_auto_gb", global_logrot_auto, 0600),
+ INT_ENTRY("remaining_space_kb", global_remaining_space, 0400),
+ INT_ENTRY("required_total_space_0_gb", global_free_space_0, 0600),
+ INT_ENTRY("required_free_space_1_gb", global_free_space_1, 0600),
+ INT_ENTRY("required_free_space_2_gb", global_free_space_2, 0600),
+ INT_ENTRY("required_free_space_3_gb", global_free_space_3, 0600),
+ INT_ENTRY("required_free_space_4_gb", global_free_space_4, 0600),
+ INT_ENTRY("sync_want", global_sync_want, 0400),
+ INT_ENTRY("sync_nr", global_sync_nr, 0400),
+ INT_ENTRY("sync_limit", global_sync_limit, 0600),
+ INT_ENTRY("mars_emergency_mode", mars_emergency_mode, 0600),
+ INT_ENTRY("mars_reset_emergency", mars_reset_emergency, 0600),
+ INT_ENTRY("mars_keep_msg_s", mars_keep_msg, 0600),
+ INT_ENTRY("write_throttle_start_percent", xio_throttle_start, 0600),
+ INT_ENTRY("write_throttle_end_percent", xio_throttle_end, 0600),
+ INT_ENTRY("write_throttle_size_threshold_kb", if_throttle_start_size, 0400),
+ LIMITER_ENTRIES(&if_throttle, "write_throttle", "kb"),
+ /* changing makes no sense because the server will immediately start upon modprobe */
+ INT_ENTRY("xio_port", xio_net_default_port, 0400),
+#ifdef __HAVE_LZO
+ INT_ENTRY("network_compress_data", xio_net_compress_data, 0600),
+#endif
+ INT_ENTRY("net_bind_before_listen", xio_net_bind_before_listen, 0600),
+ INT_ENTRY("net_bind_before_connect", xio_net_bind_before_connect, 0600),
+ INT_ENTRY("network_io_timeout", global_net_io_timeout, 0600),
+ INT_ENTRY("parallel_connections", max_client_channels, 0600),
+ INT_ENTRY("parallel_bulk_feed", max_client_bulk, 0600),
+ {
+ _CTL_NAME
+ .procname = "traffic_tuning",
+ .mode = 0500,
+ .child = traffic_tuning_table,
+ },
+ {
+ _CTL_NAME
+ .procname = "io_tuning",
+ .mode = 0500,
+ .child = io_tuning_table,
+ },
+ {
+ _CTL_NAME
+ .procname = "tcp_tuning_repl",
+ .mode = 0500,
+ .child = repl_tuning_table,
+ },
+ {
+ _CTL_NAME
+ .procname = "tcp_tuning_device",
+ .mode = 0500,
+ .child = device_tuning_table,
+ },
+ {}
+};
+
+static
+struct ctl_table mars_root_table[] = {
+ {
+ _CTL_NAME
+ .procname = "mars",
+ .mode = 0500,
+ .child = mars_table,
+ },
+ {}
+};
+
+/***************** module init stuff ************************/
+
+static struct ctl_table_header *header;
+
+int __init init_xio_proc(void)
+{
+ XIO_INF("init_proc()\n");
+
+ header = register_sysctl_table(mars_root_table);
+
+ return 0;
+}
+
+void exit_xio_proc(void)
+{
+ XIO_INF("exit_proc()\n");
+ if (header) {
+ unregister_sysctl_table(header);
+ header = NULL;
+ }
+}
diff --git a/drivers/staging/mars/mars/mars_proc.h b/drivers/staging/mars/mars/mars_proc.h
new file mode 100644
index 000000000000..883b819eced1
--- /dev/null
+++ b/drivers/staging/mars/mars/mars_proc.h
@@ -0,0 +1,34 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef MARS_PROC_H
+#define MARS_PROC_H
+
+typedef char * (*xio_info_fn)(void);
+
+extern xio_info_fn xio_info;
+
+extern int min_free_kbytes;
+
+/***********************************************************************/
+
+/* init */
+
+extern int init_xio_proc(void);
+extern void exit_xio_proc(void);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 02/32] mars: add new module brick_say |
| Message-ID | <sUetc-2Cs-31@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/brick_say.c | 920 +++++++++++++++++++++++++++++++++++++++
include/linux/brick/brick_say.h | 89 ++++
2 files changed, 1009 insertions(+)
create mode 100644 drivers/staging/mars/brick_say.c
create mode 100644 include/linux/brick/brick_say.h
diff --git a/drivers/staging/mars/brick_say.c b/drivers/staging/mars/brick_say.c
new file mode 100644
index 000000000000..f3bb49a0dfc3
--- /dev/null
+++ b/drivers/staging/mars/brick_say.c
@@ -0,0 +1,920 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+
+#include <linux/brick/brick_say.h>
+#include <linux/brick/lamport.h>
+
+/*******************************************************************/
+
+/* messaging */
+
+#include <linux/fs.h>
+#include <linux/blkdev.h>
+#include <linux/file.h>
+#include <linux/sched.h>
+#include <linux/preempt.h>
+#include <linux/hardirq.h>
+#include <linux/smp.h>
+#include <linux/slab.h>
+#include <linux/kthread.h>
+#include <linux/syscalls.h>
+
+#include <linux/uaccess.h>
+
+#include <linux/brick/vfs_compat.h>
+
+#ifndef GFP_BRICK
+#define GFP_BRICK GFP_NOIO
+#endif
+
+#define SAY_ORDER 0
+#define SAY_BUFMAX (PAGE_SIZE << SAY_ORDER)
+#define SAY_BUF_LIMIT (SAY_BUFMAX - 1500)
+#define MAX_FILELEN 16
+#define MAX_IDS 1000
+
+const char *say_class[MAX_SAY_CLASS] = {
+ [SAY_DEBUG] = "debug",
+ [SAY_INFO] = "info",
+ [SAY_WARN] = "warn",
+ [SAY_ERROR] = "error",
+ [SAY_FATAL] = "fatal",
+ [SAY_TOTAL] = "total",
+};
+
+int brick_say_logging = 1;
+
+module_param_named(say_logging, brick_say_logging, int, 0);
+int brick_say_debug;
+
+module_param_named(say_debug, brick_say_debug, int, 0);
+
+int brick_say_syslog_min = 1;
+int brick_say_syslog_max = -1;
+int brick_say_syslog_flood_class = 3;
+int brick_say_syslog_flood_limit = 20;
+int brick_say_syslog_flood_recovery = 300;
+
+int delay_say_on_overflow =
+#ifdef CONFIG_MARS_DEBUG
+ 1;
+#else
+ 0;
+#endif
+
+static atomic_t say_alloc_channels = ATOMIC_INIT(0);
+static atomic_t say_alloc_names = ATOMIC_INIT(0);
+static atomic_t say_alloc_pages = ATOMIC_INIT(0);
+
+static unsigned long flood_start_jiffies;
+static int flood_count;
+
+struct say_channel {
+ char *ch_name;
+ struct say_channel *ch_next;
+
+ /* protect against concurrent writes */
+ spinlock_t ch_lock[MAX_SAY_CLASS];
+ char *ch_buf[MAX_SAY_CLASS][2];
+
+ short ch_index[MAX_SAY_CLASS];
+ struct file *ch_filp[MAX_SAY_CLASS][2];
+ int ch_overflow[MAX_SAY_CLASS];
+ bool ch_written[MAX_SAY_CLASS];
+ bool ch_rollover;
+ bool ch_must_exist;
+ bool ch_is_dir;
+ bool ch_delete;
+ int ch_status_written;
+ int ch_id_max;
+ void *ch_ids[MAX_IDS];
+
+ wait_queue_head_t ch_progress;
+};
+
+struct say_channel *default_channel;
+
+static struct say_channel *channel_list;
+
+static rwlock_t say_lock = __RW_LOCK_UNLOCKED(say_lock);
+
+static struct task_struct *say_thread;
+
+static DECLARE_WAIT_QUEUE_HEAD(say_event);
+
+bool say_dirty;
+
+#define use_atomic() \
+ ((preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK | HARDIRQ_MASK | NMI_MASK)) != 0 || irqs_disabled())
+
+static
+void wait_channel(struct say_channel *ch, int class)
+{
+ if (delay_say_on_overflow && ch->ch_index[class] > SAY_BUF_LIMIT) {
+ if (!use_atomic()) {
+ say_dirty = true;
+ wake_up_interruptible(&say_event);
+ wait_event_interruptible_timeout(
+ ch->ch_progress, ch->ch_index[class] < SAY_BUF_LIMIT, HZ / 10);
+ }
+ }
+}
+
+static
+struct say_channel *find_channel(const void *id)
+{
+ struct say_channel *res = default_channel;
+ struct say_channel *ch;
+
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ int i;
+
+ for (i = 0; i < ch->ch_id_max; i++) {
+ if (ch->ch_ids[i] == id) {
+ res = ch;
+ goto found;
+ }
+ }
+ }
+found:
+ read_unlock(&say_lock);
+ return res;
+}
+
+static
+void _remove_binding(struct task_struct *whom)
+{
+ struct say_channel *ch;
+ int i;
+
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ for (i = 0; i < ch->ch_id_max; i++) {
+ if (ch->ch_ids[i] == whom)
+ ch->ch_ids[i] = NULL;
+ }
+ }
+}
+
+void bind_to_channel(struct say_channel *ch, struct task_struct *whom)
+{
+ int i;
+
+ write_lock(&say_lock);
+ _remove_binding(whom);
+ for (i = 0; i < ch->ch_id_max; i++) {
+ if (!ch->ch_ids[i]) {
+ ch->ch_ids[i] = whom;
+ goto done;
+ }
+ }
+ if (likely(ch->ch_id_max < MAX_IDS - 1))
+ ch->ch_ids[ch->ch_id_max++] = whom;
+ else
+ goto err;
+done:
+ write_unlock(&say_lock);
+ goto out_return;
+err:
+ write_unlock(&say_lock);
+
+ say_to(default_channel, SAY_ERROR, "ID overflow for thread '%s'\n", whom->comm);
+out_return:;
+}
+
+struct say_channel *get_binding(struct task_struct *whom)
+{
+ struct say_channel *ch;
+ int i;
+
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ for (i = 0; i < ch->ch_id_max; i++) {
+ if (ch->ch_ids[i] == whom)
+ goto found;
+ }
+ }
+ ch = NULL;
+found:
+ read_unlock(&say_lock);
+ return ch;
+}
+
+void remove_binding_from(struct say_channel *ch, struct task_struct *whom)
+{
+ bool found = false;
+ int i;
+
+ write_lock(&say_lock);
+ for (i = 0; i < ch->ch_id_max; i++) {
+ if (ch->ch_ids[i] == whom) {
+ ch->ch_ids[i] = NULL;
+ found = true;
+ break;
+ }
+ }
+ if (!found)
+ _remove_binding(whom);
+ write_unlock(&say_lock);
+}
+
+void remove_binding(struct task_struct *whom)
+{
+ write_lock(&say_lock);
+ _remove_binding(whom);
+ write_unlock(&say_lock);
+}
+
+void rollover_channel(struct say_channel *ch)
+{
+ if (!ch)
+ ch = find_channel(current);
+ if (likely(ch))
+ ch->ch_rollover = true;
+}
+
+void rollover_all(void)
+{
+ struct say_channel *ch;
+
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next)
+ ch->ch_rollover = true;
+ read_unlock(&say_lock);
+}
+
+void del_channel(struct say_channel *ch)
+{
+ if (unlikely(!ch))
+ goto out_return;
+ if (unlikely(ch == default_channel)) {
+ say_to(default_channel, SAY_ERROR, "thread '%s' tried to delete the default channel\n", current->comm);
+ goto out_return;
+ }
+
+ ch->ch_delete = true;
+out_return:;
+}
+
+static
+void _del_channel(struct say_channel *ch)
+{
+ struct say_channel *tmp;
+ struct say_channel **_tmp;
+ int i, j;
+
+ if (!ch)
+ goto out_return;
+ write_lock(&say_lock);
+ for (_tmp = &channel_list; (tmp = *_tmp) != NULL; _tmp = &tmp->ch_next) {
+ if (tmp == ch) {
+ *_tmp = tmp->ch_next;
+ break;
+ }
+ }
+ write_unlock(&say_lock);
+
+ for (i = 0; i < MAX_SAY_CLASS; i++) {
+ for (j = 0; j < 2; j++) {
+ if (ch->ch_filp[i][j]) {
+ filp_close(ch->ch_filp[i][j], NULL);
+ ch->ch_filp[i][j] = NULL;
+ }
+ }
+ for (j = 0; j < 2; j++) {
+ char *buf = ch->ch_buf[i][j];
+
+ if (buf) {
+ __free_pages(virt_to_page((unsigned long)buf), SAY_ORDER);
+ atomic_dec(&say_alloc_pages);
+ }
+ }
+ }
+ if (ch->ch_name) {
+ atomic_dec(&say_alloc_names);
+ kfree(ch->ch_name);
+ }
+ kfree(ch);
+ atomic_dec(&say_alloc_channels);
+out_return:;
+}
+
+static
+struct say_channel *_make_channel(const char *name, bool must_exist)
+{
+ struct say_channel *res = NULL;
+ struct kstat kstat = {};
+ int i, j;
+ unsigned long mode = use_atomic() ? GFP_ATOMIC : GFP_BRICK;
+
+ mm_segment_t oldfs;
+ bool is_dir = false;
+ int status;
+
+ oldfs = get_fs();
+ set_fs(get_ds());
+ status = vfs_stat((char *)name, &kstat);
+ set_fs(oldfs);
+
+ if (unlikely(status < 0)) {
+ if (must_exist) {
+ say(SAY_ERROR, "cannot create channel '%s', status = %d\n", name, status);
+ goto done;
+ }
+ } else {
+ is_dir = S_ISDIR(kstat.mode);
+ }
+
+restart:
+ res = kzalloc(sizeof(*res), mode);
+ if (unlikely(!res)) {
+ schedule();
+ goto restart;
+ }
+ atomic_inc(&say_alloc_channels);
+ res->ch_must_exist = must_exist;
+ res->ch_is_dir = is_dir;
+ init_waitqueue_head(&res->ch_progress);
+restart2:
+ res->ch_name = kstrdup(name, mode);
+ if (unlikely(!res->ch_name)) {
+ schedule();
+ goto restart2;
+ }
+ atomic_inc(&say_alloc_names);
+ for (i = 0; i < MAX_SAY_CLASS; i++) {
+ spin_lock_init(&res->ch_lock[i]);
+ for (j = 0; j < 2; j++) {
+ char *buf;
+
+restart3:
+ buf = (void *)__get_free_pages(mode, SAY_ORDER);
+ if (unlikely(!buf)) {
+ schedule();
+ goto restart3;
+ }
+ atomic_inc(&say_alloc_pages);
+ res->ch_buf[i][j] = buf;
+ }
+ }
+done:
+ return res;
+}
+
+struct say_channel *make_channel(const char *name, bool must_exist)
+{
+ struct say_channel *res = NULL;
+ struct say_channel *ch;
+
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ if (!strcmp(ch->ch_name, name)) {
+ res = ch;
+ break;
+ }
+ }
+ read_unlock(&say_lock);
+
+ if (unlikely(!res)) {
+ res = _make_channel(name, must_exist);
+ if (unlikely(!res))
+ goto done;
+
+ write_lock(&say_lock);
+
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ if (ch != res && unlikely(!strcmp(ch->ch_name, name))) {
+ _del_channel(res);
+ res = ch;
+ goto race_found;
+ }
+ }
+
+ res->ch_next = channel_list;
+ channel_list = res;
+
+race_found:
+ write_unlock(&say_lock);
+ }
+
+done:
+ return res;
+}
+
+/* tell gcc to check for varargs errors */
+static
+void _say(struct say_channel *ch, int class, va_list args, bool use_args, const char *fmt, ...) __printf(5, 6);
+
+static
+void _say(struct say_channel *ch, int class, va_list args, bool use_args, const char *fmt, ...)
+{
+ char *start;
+ int offset;
+ int rest;
+ int written;
+
+ if (unlikely(!ch))
+ goto out_return;
+ if (unlikely(ch->ch_delete && ch != default_channel)) {
+ say_to(default_channel, SAY_ERROR, "thread '%s' tried to write on deleted channel\n", current->comm);
+ goto out_return;
+ }
+
+ offset = ch->ch_index[class];
+ start = ch->ch_buf[class][0] + offset;
+ rest = SAY_BUFMAX - 1 - offset;
+ if (unlikely(rest <= 0)) {
+ ch->ch_overflow[class]++;
+ goto out_return;
+ }
+
+ if (use_args) {
+ va_list args2;
+
+ va_start(args2, fmt);
+ written = vscnprintf(start, rest, fmt, args2);
+ va_end(args2);
+ } else {
+ written = vscnprintf(start, rest, fmt, args);
+ }
+
+ if (likely(rest > written)) {
+ start[written] = '\0';
+ ch->ch_index[class] += written;
+ say_dirty = true;
+ } else {
+ /* indicate overflow */
+ start[0] = '\0';
+ ch->ch_overflow[class]++;
+ }
+out_return:;
+}
+
+void say_to(struct say_channel *ch, int class, const char *fmt, ...)
+{
+ va_list args;
+ unsigned long flags;
+
+ if (!class && !brick_say_debug)
+ goto out_return;
+ if (!ch)
+ ch = find_channel(current);
+
+ if (likely(ch)) {
+ if (!ch->ch_is_dir)
+ class = SAY_TOTAL;
+ if (likely(class >= 0 && class < MAX_SAY_CLASS)) {
+ wait_channel(ch, class);
+ spin_lock_irqsave(&ch->ch_lock[class], flags);
+
+ va_start(args, fmt);
+ _say(ch, class, args, false, fmt);
+ va_end(args);
+
+ spin_unlock_irqrestore(&ch->ch_lock[class], flags);
+ }
+ }
+
+ ch = default_channel;
+ if (likely(ch)) {
+ class = SAY_TOTAL;
+ wait_channel(ch, class);
+ spin_lock_irqsave(&ch->ch_lock[class], flags);
+
+ va_start(args, fmt);
+ _say(ch, class, args, false, fmt);
+ va_end(args);
+
+ spin_unlock_irqrestore(&ch->ch_lock[class], flags);
+
+ wake_up_interruptible(&say_event);
+ }
+out_return:;
+}
+
+void brick_say_to(
+struct say_channel *ch,
+int class,
+bool dump,
+const char *prefix,
+const char *file,
+int line,
+const char *func,
+const char *fmt,
+...)
+{
+ const char *channel_name = "-";
+ struct timespec s_now;
+ struct timespec l_now;
+ int filelen;
+ int orig_class;
+ va_list args;
+ unsigned long flags;
+
+ if (!class && !brick_say_debug)
+ goto out_return;
+ s_now = CURRENT_TIME;
+ get_lamport(&l_now);
+
+ if (!ch)
+ ch = find_channel(current);
+
+ orig_class = class;
+
+ /* limit the filename */
+ filelen = strlen(file);
+ if (filelen > MAX_FILELEN)
+ file += filelen - MAX_FILELEN;
+
+ if (likely(ch)) {
+ channel_name = ch->ch_name;
+ if (!ch->ch_is_dir)
+ class = SAY_TOTAL;
+ if (likely(class >= 0 && class < MAX_SAY_CLASS)) {
+ wait_channel(ch, class);
+ spin_lock_irqsave(&ch->ch_lock[class], flags);
+
+ _say(
+ ch, class, NULL, true,
+ "%ld.%09ld %ld.%09ld %s %s[%d] %s:%d %s(): ",
+ s_now.tv_sec, s_now.tv_nsec,
+ l_now.tv_sec, l_now.tv_nsec,
+ prefix,
+ current->comm, (int)smp_processor_id(),
+ file, line,
+ func);
+
+ va_start(args, fmt);
+ _say(ch, class, args, false, fmt);
+ va_end(args);
+
+ spin_unlock_irqrestore(&ch->ch_lock[class], flags);
+ }
+ }
+
+ ch = default_channel;
+ if (likely(ch)) {
+ wait_channel(ch, SAY_TOTAL);
+ spin_lock_irqsave(&ch->ch_lock[SAY_TOTAL], flags);
+
+ _say(
+ ch, SAY_TOTAL, NULL, true,
+ "%ld.%09ld %ld.%09ld %s_%-5s %s %s[%d] %s:%d %s(): ",
+ s_now.tv_sec, s_now.tv_nsec,
+ l_now.tv_sec, l_now.tv_nsec,
+ prefix, say_class[orig_class],
+ channel_name,
+ current->comm, (int)smp_processor_id(),
+ file, line,
+ func);
+
+ va_start(args, fmt);
+ _say(ch, SAY_TOTAL, args, false, fmt);
+ va_end(args);
+
+ spin_unlock_irqrestore(&ch->ch_lock[SAY_TOTAL], flags);
+ }
+#ifdef CONFIG_MARS_DEBUG
+ if (dump)
+ brick_dump_stack();
+#endif
+ wake_up_interruptible(&say_event);
+out_return:;
+}
+
+static
+void try_open_file(struct file **file, char *filename, bool creat)
+{
+ struct address_space *mapping;
+ int flags = O_APPEND | O_WRONLY | O_LARGEFILE;
+ int prot = 0600;
+
+ if (creat)
+ flags |= O_CREAT;
+
+ *file = filp_open(filename, flags, prot);
+ if (unlikely(IS_ERR(*file))) {
+ *file = NULL;
+ goto out_return;
+ }
+ mapping = (*file)->f_mapping;
+ if (likely(mapping))
+ mapping_set_gfp_mask(mapping, mapping_gfp_mask(mapping) & ~(__GFP_IO | __GFP_FS));
+out_return:;
+}
+
+static
+void out_to_file(struct file *file, char *buf, int len)
+{
+ loff_t log_pos = 0;
+
+ mm_segment_t oldfs;
+
+ if (file) {
+ oldfs = get_fs();
+ set_fs(get_ds());
+ (void)vfs_write(file, buf, len, &log_pos);
+ set_fs(oldfs);
+ }
+}
+
+static inline
+void reset_flood(void)
+{
+ if (flood_start_jiffies &&
+ time_is_before_jiffies(flood_start_jiffies + brick_say_syslog_flood_recovery * HZ)) {
+ flood_start_jiffies = 0;
+ flood_count = 0;
+ }
+}
+
+static
+void printk_with_class(int class, char *buf)
+{
+ switch (class) {
+ case SAY_INFO:
+ printk(KERN_INFO "%s", buf);
+ break;
+ case SAY_WARN:
+ printk(KERN_WARNING "%s", buf);
+ break;
+ case SAY_ERROR:
+ case SAY_FATAL:
+ printk(KERN_ERR "%s", buf);
+ break;
+ default:
+ printk(KERN_DEBUG "%s", buf);
+ }
+}
+
+static
+void out_to_syslog(int class, char *buf, int len)
+{
+ reset_flood();
+ if (class >= brick_say_syslog_min && class <= brick_say_syslog_max) {
+ buf[len] = '\0';
+ printk_with_class(class, buf);
+ } else if (class >= brick_say_syslog_flood_class && brick_say_syslog_flood_class >= 0 && class != SAY_TOTAL) {
+ flood_start_jiffies = jiffies;
+ if (++flood_count <= brick_say_syslog_flood_limit) {
+ buf[len] = '\0';
+ printk_with_class(class, buf);
+ }
+ }
+}
+
+static inline
+char *_make_filename(struct say_channel *ch, int class, int transact, int add_tmp)
+{
+ char *filename;
+
+restart:
+ filename = kmalloc(1024, GFP_KERNEL);
+ if (unlikely(!filename)) {
+ schedule();
+ goto restart;
+ }
+ atomic_inc(&say_alloc_names);
+ if (ch->ch_is_dir) {
+ snprintf(
+ filename,
+ 1023,
+ "%s/%d.%s.%s%s",
+ ch->ch_name,
+ class,
+ say_class[class],
+ transact ? "status" : "log",
+ add_tmp ? ".tmp" : "");
+ } else {
+ snprintf(filename, 1023, "%s.%s%s", ch->ch_name, transact ? "status" : "log", add_tmp ? ".tmp" : "");
+ }
+ return filename;
+}
+
+static
+void _rollover_channel(struct say_channel *ch)
+{
+ int start = 0;
+ int class;
+
+ ch->ch_rollover = false;
+ ch->ch_status_written = 0;
+
+ if (!ch->ch_is_dir)
+ start = SAY_TOTAL;
+
+ for (class = start; class < MAX_SAY_CLASS; class++) {
+ char *old = _make_filename(ch, class, 1, 1);
+ char *new = _make_filename(ch, class, 1, 0);
+
+ if (likely(old && new)) {
+ int i;
+
+ mm_segment_t oldfs;
+
+ for (i = 0; i < 2; i++) {
+ if (ch->ch_filp[class][i]) {
+ filp_close(ch->ch_filp[class][i], NULL);
+ ch->ch_filp[class][i] = NULL;
+ }
+ }
+
+ oldfs = get_fs();
+ set_fs(get_ds());
+#ifdef __USE_COMPAT
+ _compat_rename(old, new);
+#else
+ sys_rename(old, new);
+#endif
+ set_fs(oldfs);
+ }
+
+ if (likely(old)) {
+ kfree(old);
+ atomic_dec(&say_alloc_names);
+ }
+ if (likely(new)) {
+ kfree(new);
+ atomic_dec(&say_alloc_names);
+ }
+ }
+}
+
+static
+void treat_channel(struct say_channel *ch, int class)
+{
+ int len;
+ int overflow;
+ int transact;
+ int start;
+ char *buf;
+ char *tmp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&ch->ch_lock[class], flags);
+
+ buf = ch->ch_buf[class][0];
+ tmp = ch->ch_buf[class][1];
+ ch->ch_buf[class][1] = buf;
+ ch->ch_buf[class][0] = tmp;
+ len = ch->ch_index[class];
+ ch->ch_index[class] = 0;
+ overflow = ch->ch_overflow[class];
+ ch->ch_overflow[class] = 0;
+
+ spin_unlock_irqrestore(&ch->ch_lock[class], flags);
+
+ wake_up_interruptible(&ch->ch_progress);
+
+ ch->ch_status_written += len;
+ out_to_syslog(class, buf, len);
+ start = 0;
+ if (!brick_say_logging)
+ start++;
+ for (transact = start; transact < 2; transact++) {
+ if (unlikely(!ch->ch_filp[class][transact])) {
+ char *filename = _make_filename(ch, class, transact, transact);
+
+ if (likely(filename)) {
+ try_open_file(&ch->ch_filp[class][transact], filename, transact);
+ kfree(filename);
+ atomic_dec(&say_alloc_names);
+ }
+ }
+ out_to_file(ch->ch_filp[class][transact], buf, len);
+ }
+
+ if (unlikely(overflow > 0)) {
+ struct timespec s_now = CURRENT_TIME;
+ struct timespec l_now;
+
+ get_lamport(&l_now);
+ len = scnprintf(
+ buf,
+ SAY_BUFMAX,
+ "%ld.%09ld %ld.%09ld %s %d OVERFLOW %d times\n",
+ s_now.tv_sec, s_now.tv_nsec,
+ l_now.tv_sec, l_now.tv_nsec,
+ ch->ch_name,
+ class,
+ overflow);
+ ch->ch_status_written += len;
+ out_to_syslog(class, buf, len);
+ for (transact = 0; transact < 2; transact++)
+ out_to_file(ch->ch_filp[class][transact], buf, len);
+ }
+}
+
+static
+int _say_thread(void *data)
+{
+ while (!kthread_should_stop()) {
+ struct say_channel *ch;
+ int i;
+
+ wait_event_interruptible_timeout(say_event, say_dirty, HZ);
+ say_dirty = false;
+
+restart_rollover:
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ if (ch->ch_rollover && ch->ch_status_written > 0) {
+ read_unlock(&say_lock);
+ _rollover_channel(ch);
+ goto restart_rollover;
+ }
+ }
+ read_unlock(&say_lock);
+
+restart:
+ read_lock(&say_lock);
+ for (ch = channel_list; ch; ch = ch->ch_next) {
+ int start = 0;
+
+ if (!ch->ch_is_dir)
+ start = SAY_TOTAL;
+ for (i = start; i < MAX_SAY_CLASS; i++) {
+ if (ch->ch_index[i] > 0) {
+ read_unlock(&say_lock);
+ treat_channel(ch, i);
+ goto restart;
+ }
+ }
+ if (ch->ch_delete) {
+ read_unlock(&say_lock);
+ _del_channel(ch);
+ goto restart;
+ }
+ }
+ read_unlock(&say_lock);
+ }
+
+ return 0;
+}
+
+void init_say(void)
+{
+ default_channel = make_channel(CONFIG_MARS_LOGDIR, true);
+ say_thread = kthread_create(_say_thread, NULL, "brick_say");
+ if (IS_ERR(say_thread)) {
+ say_thread = NULL;
+ } else {
+ get_task_struct(say_thread);
+ wake_up_process(say_thread);
+ }
+}
+
+void exit_say(void)
+{
+ int memleak_channels;
+ int memleak_names;
+ int memleak_pages;
+
+ if (say_thread) {
+ kthread_stop(say_thread);
+ put_task_struct(say_thread);
+ say_thread = NULL;
+ }
+
+ default_channel = NULL;
+ while (channel_list)
+ _del_channel(channel_list);
+
+ memleak_channels = atomic_read(&say_alloc_channels);
+ memleak_names = atomic_read(&say_alloc_names);
+ memleak_pages = atomic_read(&say_alloc_pages);
+ if (unlikely(memleak_channels || memleak_names || memleak_pages))
+ printk("MEMLEAK: channels=%d names=%d pages=%d\n", memleak_channels, memleak_names, memleak_pages);
+}
+
+#ifdef CONFIG_MARS_DEBUG
+
+static int dump_max = 5;
+
+void brick_dump_stack(void)
+{
+ if (dump_max > 0) {
+ dump_max--; /* racy, but does no harm */
+ dump_stack();
+ }
+}
+
+#endif
diff --git a/include/linux/brick/brick_say.h b/include/linux/brick/brick_say.h
new file mode 100644
index 000000000000..13a28c80081f
--- /dev/null
+++ b/include/linux/brick/brick_say.h
@@ -0,0 +1,89 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef BRICK_SAY_H
+#define BRICK_SAY_H
+
+/***********************************************************************/
+
+extern int brick_say_logging;
+extern int brick_say_debug;
+extern int brick_say_syslog_min;
+extern int brick_say_syslog_max;
+extern int brick_say_syslog_flood_class;
+extern int brick_say_syslog_flood_limit;
+extern int brick_say_syslog_flood_recovery;
+extern int delay_say_on_overflow;
+
+/* printk() replacements */
+
+enum {
+ SAY_DEBUG,
+ SAY_INFO,
+ SAY_WARN,
+ SAY_ERROR,
+ SAY_FATAL,
+ SAY_TOTAL,
+ MAX_SAY_CLASS
+};
+
+extern const char *say_class[MAX_SAY_CLASS];
+
+struct say_channel;
+
+extern struct say_channel *default_channel;
+
+extern struct say_channel *make_channel(const char *name, bool must_exit);
+
+extern void del_channel(struct say_channel *ch);
+
+extern void bind_to_channel(struct say_channel *ch, struct task_struct *whom);
+
+#define bind_me(_name) \
+ bind_to_channel(make_channel(_name), current)
+
+extern struct say_channel *get_binding(struct task_struct *whom);
+
+extern void remove_binding_from(struct say_channel *ch, struct task_struct *whom);
+extern void remove_binding(struct task_struct *whom);
+
+extern void rollover_channel(struct say_channel *ch);
+extern void rollover_all(void);
+
+extern void say_to(struct say_channel *ch, int class, const char *fmt, ...) __printf(3, 4);
+
+#define say(_class, _fmt, _args...) \
+ say_to(NULL, _class, _fmt, ##_args)
+
+extern void brick_say_to(
+struct say_channel *ch, int class, bool dump, const char *prefix, const char *file, int line, const char *func, const char *fmt, ...) __printf(
+
+8, 9);
+
+#define brick_say(_class, _dump, _prefix, _file, _line, _func, _fmt, _args...)\
+ brick_say_to(NULL, _class, _dump, _prefix, _file, _line, _func, _fmt, ##_args)
+
+extern void init_say(void);
+extern void exit_say(void);
+
+#ifdef CONFIG_MARS_DEBUG
+extern void brick_dump_stack(void);
+#else /* CONFIG_MARS_DEBUG */
+#define brick_dump_stack() /*empty*/
+#endif /* CONFIG_MARS_DEBUG */
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 03/32] mars: add new module brick_mem |
| Message-ID | <sUetd-2Cs-43@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/brick_mem.c | 1080 ++++++++++++++++++++++++++++++++++++++
include/linux/brick/brick_mem.h | 218 ++++++++
2 files changed, 1298 insertions(+)
create mode 100644 drivers/staging/mars/brick_mem.c
create mode 100644 include/linux/brick/brick_mem.h
diff --git a/drivers/staging/mars/brick_mem.c b/drivers/staging/mars/brick_mem.c
new file mode 100644
index 000000000000..232dbf6cb0ca
--- /dev/null
+++ b/drivers/staging/mars/brick_mem.c
@@ -0,0 +1,1080 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/mm.h>
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+
+#include <linux/atomic.h>
+
+#include <linux/brick/brick_mem.h>
+#include <linux/brick/brick_say.h>
+#include <linux/brick/lamport.h>
+
+#define USE_KERNEL_PAGES /* currently mandatory (vmalloc does not work) */
+
+#define MAGIC_BLOCK 0x8B395D7B
+#define MAGIC_BEND 0x8B395D7C
+#define MAGIC_MEM1 0x8B395D7D
+#define MAGIC_MEM2 0x9B395D8D
+#define MAGIC_MEND1 0x8B395D7E
+#define MAGIC_MEND2 0x9B395D8E
+#define MAGIC_STR 0x8B395D7F
+#define MAGIC_SEND 0x9B395D8F
+
+#define INT_ACCESS(ptr, offset) (*(int *)(((char *)(ptr)) + (offset)))
+
+#define _BRICK_FMT(_fmt, _class) \
+ "%ld.%09ld %ld.%09ld MEM_%-5s %s[%d] %s:%d %s(): " \
+ _fmt, \
+ _s_now.tv_sec, _s_now.tv_nsec, \
+ _l_now.tv_sec, _l_now.tv_nsec, \
+ say_class[_class], \
+ current->comm, (int)smp_processor_id(), \
+ __BASE_FILE__, \
+ __LINE__, \
+ __func__
+
+#define _BRICK_MSG(_class, _dump, _fmt, _args...) \
+ do { \
+ struct timespec _s_now = CURRENT_TIME; \
+ struct timespec _l_now; \
+ get_lamport(&_l_now); \
+ say(_class, _BRICK_FMT(_fmt, _class), ##_args); \
+ if (_dump) \
+ dump_stack(); \
+ } while (0)
+
+#define BRICK_ERR(_fmt, _args...) _BRICK_MSG(SAY_ERROR, true, _fmt, ##_args)
+#define BRICK_WRN(_fmt, _args...) _BRICK_MSG(SAY_WARN, false, _fmt, ##_args)
+#define BRICK_INF(_fmt, _args...) _BRICK_MSG(SAY_INFO, false, _fmt, ##_args)
+
+/***********************************************************************/
+
+/* limit handling */
+
+#include <linux/swap.h>
+
+long long brick_global_memavail;
+long long brick_global_memlimit;
+
+atomic64_t brick_global_block_used = ATOMIC64_INIT(0);
+
+void get_total_ram(void)
+{
+ struct sysinfo i = {};
+
+ si_meminfo(&i);
+ /* si_swapinfo(&i); */
+ brick_global_memavail = (long long)i.totalram * (PAGE_SIZE / 1024);
+ BRICK_INF("total RAM = %lld [KiB]\n", brick_global_memavail);
+}
+
+/***********************************************************************/
+
+/* small memory allocation (use this only for len < PAGE_SIZE) */
+
+#ifdef BRICK_DEBUG_MEM
+static atomic_t phys_mem_alloc = ATOMIC_INIT(0);
+static atomic_t mem_redirect_alloc = ATOMIC_INIT(0);
+static atomic_t mem_count[BRICK_DEBUG_MEM];
+static atomic_t mem_free[BRICK_DEBUG_MEM];
+static int mem_len[BRICK_DEBUG_MEM];
+
+#define PLUS_SIZE (6 * sizeof(int))
+#else
+#define PLUS_SIZE (2 * sizeof(int))
+#endif
+
+static inline
+void *__brick_mem_alloc(int len)
+{
+ void *res;
+
+ if (len >= PAGE_SIZE) {
+#ifdef BRICK_DEBUG_MEM
+ atomic_inc(&mem_redirect_alloc);
+#endif
+ res = _brick_block_alloc(0, len, 0);
+ } else {
+ for (;;) {
+ res = kmalloc(len, GFP_BRICK);
+ if (likely(res))
+ break;
+ msleep(1000);
+ }
+#ifdef BRICK_DEBUG_MEM
+ atomic_inc(&phys_mem_alloc);
+#endif
+ }
+ return res;
+}
+
+static inline
+void __brick_mem_free(void *data, int len)
+{
+ if (len >= PAGE_SIZE) {
+ _brick_block_free(data, len, 0);
+#ifdef BRICK_DEBUG_MEM
+ atomic_dec(&mem_redirect_alloc);
+#endif
+ } else {
+ kfree(data);
+#ifdef BRICK_DEBUG_MEM
+ atomic_dec(&phys_mem_alloc);
+#endif
+ }
+}
+
+void *_brick_mem_alloc(int len, int line)
+{
+ void *res;
+
+#ifdef CONFIG_MARS_DEBUG
+ might_sleep();
+#endif
+
+ res = __brick_mem_alloc(len + PLUS_SIZE);
+
+#ifdef BRICK_DEBUG_MEM
+ if (unlikely(line < 0))
+ line = 0;
+ else if (unlikely(line >= BRICK_DEBUG_MEM))
+ line = BRICK_DEBUG_MEM - 1;
+ INT_ACCESS(res, 0 * sizeof(int)) = MAGIC_MEM1;
+ INT_ACCESS(res, 1 * sizeof(int)) = len;
+ INT_ACCESS(res, 2 * sizeof(int)) = line;
+ INT_ACCESS(res, 3 * sizeof(int)) = MAGIC_MEM2;
+ res += 4 * sizeof(int);
+ INT_ACCESS(res, len + 0 * sizeof(int)) = MAGIC_MEND1;
+ INT_ACCESS(res, len + 1 * sizeof(int)) = MAGIC_MEND2;
+ atomic_inc(&mem_count[line]);
+ mem_len[line] = len;
+#else
+ INT_ACCESS(res, 0 * sizeof(int)) = len;
+ res += PLUS_SIZE;
+#endif
+ return res;
+}
+
+void _brick_mem_free(void *data, int cline)
+{
+#ifdef BRICK_DEBUG_MEM
+ void *test = data - 4 * sizeof(int);
+ int magic1 = INT_ACCESS(test, 0 * sizeof(int));
+ int len = INT_ACCESS(test, 1 * sizeof(int));
+ int line = INT_ACCESS(test, 2 * sizeof(int));
+ int magic2 = INT_ACCESS(test, 3 * sizeof(int));
+
+ if (unlikely(magic1 != MAGIC_MEM1)) {
+ BRICK_ERR(
+ "line %d memory corruption: magix1 %08x != %08x, len = %d\n", cline, magic1, MAGIC_MEM1, len);
+ goto _out_return;
+ }
+ if (unlikely(magic2 != MAGIC_MEM2)) {
+ BRICK_ERR(
+ "line %d memory corruption: magix2 %08x != %08x, len = %d\n", cline, magic2, MAGIC_MEM2, len);
+ goto _out_return;
+ }
+ if (unlikely(line < 0 || line >= BRICK_DEBUG_MEM)) {
+ BRICK_ERR("line %d memory corruption: alloc line = %d, len = %d\n", cline, line, len);
+ goto _out_return;
+ }
+ INT_ACCESS(test, 0) = 0xffffffff;
+ magic1 = INT_ACCESS(data, len + 0 * sizeof(int));
+ if (unlikely(magic1 != MAGIC_MEND1)) {
+ BRICK_ERR(
+ "line %d memory corruption: magix1 %08x != %08x, len = %d\n", cline, magic1, MAGIC_MEND1, len);
+ goto _out_return;
+ }
+ magic2 = INT_ACCESS(data, len + 1 * sizeof(int));
+ if (unlikely(magic2 != MAGIC_MEND2)) {
+ BRICK_ERR(
+ "line %d memory corruption: magix2 %08x != %08x, len = %d\n", cline, magic2, MAGIC_MEND2, len);
+ goto _out_return;
+ }
+ INT_ACCESS(data, len) = 0xffffffff;
+ atomic_dec(&mem_count[line]);
+ atomic_inc(&mem_free[line]);
+#else
+ void *test = data - PLUS_SIZE;
+ int len = INT_ACCESS(test, 0 * sizeof(int));
+
+#endif
+ data = test;
+ __brick_mem_free(data, len + PLUS_SIZE);
+#ifdef BRICK_DEBUG_MEM
+_out_return:;
+#endif
+}
+
+/***********************************************************************/
+
+/* string memory allocation */
+
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+# define STRING_CANARY \
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
+ "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" \
+ "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" \
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
+ "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" \
+ "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" \
+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
+ "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy" \
+ "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" \
+ " FILE = " __FILE__ \
+ " VERSION = " __VERSION__ \
+ " xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx STRING_error xxx\n"
+# define STRING_PLUS (sizeof(int) * 3 + sizeof(STRING_CANARY))
+#elif defined(BRICK_DEBUG_MEM)
+# define STRING_PLUS (sizeof(int) * 4)
+#else
+# define STRING_PLUS 0
+#endif
+
+#ifdef BRICK_DEBUG_MEM
+static atomic_t phys_string_alloc = ATOMIC_INIT(0);
+static atomic_t string_count[BRICK_DEBUG_MEM];
+static atomic_t string_free[BRICK_DEBUG_MEM];
+
+#endif
+
+char *_brick_string_alloc(int len, int line)
+{
+ char *res;
+
+#ifdef CONFIG_MARS_DEBUG
+ might_sleep();
+ if (unlikely(len > PAGE_SIZE))
+ BRICK_WRN("line = %d string too long: len = %d\n", line, len);
+#endif
+ if (len <= 0)
+ len = BRICK_STRING_LEN;
+
+ for (;;) {
+ res = kzalloc(len + STRING_PLUS, GFP_BRICK);
+ if (likely(res))
+ break;
+ msleep(1000);
+ }
+
+#ifdef BRICK_DEBUG_MEM
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ memset(res + 1, '?', len - 1);
+#endif
+ atomic_inc(&phys_string_alloc);
+ if (unlikely(line < 0))
+ line = 0;
+ else if (unlikely(line >= BRICK_DEBUG_MEM))
+ line = BRICK_DEBUG_MEM - 1;
+ INT_ACCESS(res, 0) = MAGIC_STR;
+ INT_ACCESS(res, sizeof(int)) = len;
+ INT_ACCESS(res, sizeof(int) * 2) = line;
+ res += sizeof(int) * 3;
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ strcpy(res + len, STRING_CANARY);
+#else
+ INT_ACCESS(res, len) = MAGIC_SEND;
+#endif
+ atomic_inc(&string_count[line]);
+#endif
+ return res;
+}
+
+void _brick_string_free(const char *data, int cline)
+{
+#ifdef BRICK_DEBUG_MEM
+ int magic;
+ int len;
+ int line;
+ char *orig = (void *)data;
+
+ data -= sizeof(int) * 3;
+ magic = INT_ACCESS(data, 0);
+ if (unlikely(magic != MAGIC_STR)) {
+ BRICK_ERR("cline %d stringmem corruption: magix %08x != %08x\n", cline, magic, MAGIC_STR);
+ goto _out_return;
+ }
+ len = INT_ACCESS(data, sizeof(int));
+ line = INT_ACCESS(data, sizeof(int) * 2);
+ if (unlikely(len <= 0)) {
+ BRICK_ERR("cline %d stringmem corruption: line = %d len = %d\n", cline, line, len);
+ goto _out_return;
+ }
+ if (unlikely(len > PAGE_SIZE))
+ BRICK_ERR("cline %d string too long: line = %d len = %d string='%s'\n", cline, line, len, orig);
+ if (unlikely(line < 0 || line >= BRICK_DEBUG_MEM)) {
+ BRICK_ERR("cline %d stringmem corruption: line = %d (len = %d)\n", cline, line, len);
+ goto _out_return;
+ }
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ if (unlikely(strcmp(orig + len, STRING_CANARY))) {
+ BRICK_ERR(
+ "cline %d stringmem corruption: bad canary '%s', line = %d len = %d\n",
+ cline, STRING_CANARY, line, len);
+ goto _out_return;
+ }
+ orig[len]--;
+ memset(orig, '!', len);
+#else
+ magic = INT_ACCESS(orig, len);
+ if (unlikely(magic != MAGIC_SEND)) {
+ BRICK_ERR(
+ "cline %d stringmem corruption: end_magix %08x != %08x, line = %d len = %d\n",
+ cline, magic, MAGIC_SEND, line, len);
+ goto _out_return;
+ }
+ INT_ACCESS(orig, len) = 0xffffffff;
+#endif
+ atomic_dec(&string_count[line]);
+ atomic_inc(&string_free[line]);
+ atomic_dec(&phys_string_alloc);
+#endif
+ kfree(data);
+#ifdef BRICK_DEBUG_MEM
+_out_return:;
+#endif
+}
+
+/***********************************************************************/
+
+/* block memory allocation */
+
+static
+int len2order(int len)
+{
+ int order = 0;
+
+ if (unlikely(len <= 0)) {
+ BRICK_ERR("trying to use %d bytes\n", len);
+ return 0;
+ }
+
+ while ((PAGE_SIZE << order) < len)
+ order++;
+
+ if (unlikely(order > BRICK_MAX_ORDER)) {
+ BRICK_ERR("trying to use %d bytes (oder = %d, max = %d)\n", len, order, BRICK_MAX_ORDER);
+ return BRICK_MAX_ORDER;
+ }
+ return order;
+}
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+static atomic_t _alloc_count[BRICK_MAX_ORDER + 1];
+int brick_mem_alloc_count[BRICK_MAX_ORDER + 1] = {};
+int brick_mem_alloc_max[BRICK_MAX_ORDER + 1] = {};
+int brick_mem_freelist_max[BRICK_MAX_ORDER + 1] = {};
+
+#endif
+
+#ifdef BRICK_DEBUG_MEM
+static atomic_t phys_block_alloc = ATOMIC_INIT(0);
+
+/* indexed by line */
+static atomic_t block_count[BRICK_DEBUG_MEM];
+static atomic_t block_free[BRICK_DEBUG_MEM];
+static int block_len[BRICK_DEBUG_MEM];
+
+/* indexed by order */
+static atomic_t op_count[BRICK_MAX_ORDER + 1];
+static atomic_t raw_count[BRICK_MAX_ORDER + 1];
+static int alloc_line[BRICK_MAX_ORDER + 1];
+static int alloc_len[BRICK_MAX_ORDER + 1];
+
+#endif
+
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+
+#define MAX_INFO_LISTS 1024
+
+#define INFO_LIST_HASH(addr) ((unsigned long)(addr) / (PAGE_SIZE * 2) % MAX_INFO_LISTS)
+
+struct mem_block_info {
+ struct list_head inf_head;
+ void *inf_data;
+ int inf_len;
+ int inf_line;
+ bool inf_used;
+};
+
+static struct list_head inf_anchor[MAX_INFO_LISTS];
+static rwlock_t inf_lock[MAX_INFO_LISTS];
+
+static
+void _new_block_info(void *data, int len, int cline)
+{
+ struct mem_block_info *inf;
+ int hash;
+
+ for (;;) {
+ inf = kmalloc(sizeof(*inf), GFP_BRICK);
+ if (likely(inf))
+ break;
+ msleep(1000);
+ }
+ inf->inf_data = data;
+ inf->inf_len = len;
+ inf->inf_line = cline;
+ inf->inf_used = true;
+
+ hash = INFO_LIST_HASH(data);
+
+ write_lock(&inf_lock[hash]);
+ list_add(&inf->inf_head, &inf_anchor[hash]);
+ write_unlock(&inf_lock[hash]);
+}
+
+static
+struct mem_block_info *_find_block_info(void *data, bool remove)
+{
+ struct mem_block_info *res = NULL;
+ struct list_head *tmp;
+ int hash = INFO_LIST_HASH(data);
+
+ if (remove)
+ write_lock(&inf_lock[hash]);
+ else
+ read_lock(&inf_lock[hash]);
+ for (tmp = inf_anchor[hash].next; tmp != &inf_anchor[hash]; tmp = tmp->next) {
+ struct mem_block_info *inf = container_of(tmp, struct mem_block_info, inf_head);
+
+ if (inf->inf_data != data)
+ continue;
+ if (remove)
+ list_del_init(tmp);
+ res = inf;
+ break;
+ }
+ if (remove)
+ write_unlock(&inf_lock[hash]);
+ else
+ read_unlock(&inf_lock[hash]);
+ return res;
+}
+
+#endif /* CONFIG_MARS_DEBUG_MEM_STRONG */
+
+static inline
+void *__brick_block_alloc(gfp_t gfp, int order, int cline)
+{
+ void *res;
+
+ for (;;) {
+#ifdef USE_KERNEL_PAGES
+ res = (void *)__get_free_pages(gfp, order);
+#else
+ res = __vmalloc(PAGE_SIZE << order, gfp, PAGE_KERNEL_IO);
+#endif
+ if (likely(res))
+ break;
+ msleep(1000);
+ }
+
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ _new_block_info(res, PAGE_SIZE << order, cline);
+#endif
+#ifdef BRICK_DEBUG_MEM
+ atomic_inc(&phys_block_alloc);
+ atomic_inc(&raw_count[order]);
+#endif
+ atomic64_add((PAGE_SIZE / 1024) << order, &brick_global_block_used);
+
+ return res;
+}
+
+static inline
+void __brick_block_free(void *data, int order, int cline)
+{
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ struct mem_block_info *inf = _find_block_info(data, true);
+
+ if (likely(inf)) {
+ int inf_len = inf->inf_len;
+ int inf_line = inf->inf_line;
+
+ kfree(inf);
+ if (unlikely(inf_len != (PAGE_SIZE << order))) {
+ BRICK_ERR(
+ "line %d: address %p: bad freeing size %d (correct should be %d, previous line = %d)\n", cline, data, (
+
+ int)(PAGE_SIZE << order), inf_len, inf_line);
+ goto err;
+ }
+ } else {
+ BRICK_ERR("line %d: trying to free non-existent address %p (order = %d)\n", cline, data, order);
+ goto err;
+ }
+#endif
+#ifdef USE_KERNEL_PAGES
+ __free_pages(virt_to_page((unsigned long)data), order);
+#else
+ vfree(data);
+#endif
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+err:
+#endif
+#ifdef BRICK_DEBUG_MEM
+ atomic_dec(&phys_block_alloc);
+ atomic_dec(&raw_count[order]);
+#endif
+ atomic64_sub((PAGE_SIZE / 1024) << order, &brick_global_block_used);
+}
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+int brick_allow_freelist = 1;
+
+int brick_pre_reserve[BRICK_MAX_ORDER + 1] = {};
+
+/* Note: we have no separate lists per CPU.
+ * This should not hurt because the freelists are only used
+ * for higher-order pages which should be rather low-frequency.
+ */
+static spinlock_t freelist_lock[BRICK_MAX_ORDER + 1];
+static void *brick_freelist[BRICK_MAX_ORDER + 1];
+static atomic_t freelist_count[BRICK_MAX_ORDER + 1];
+
+static
+void *_get_free(int order, int cline)
+{
+ void *data;
+ unsigned long flags;
+
+ spin_lock_irqsave(&freelist_lock[order], flags);
+ data = brick_freelist[order];
+ if (likely(data)) {
+ void *next = *(void **)data;
+
+#ifdef BRICK_DEBUG_MEM /* check for corruptions */
+ long pattern = *(((long *)data) + 1);
+ void *copy = *(((void **)data) + 2);
+
+ if (unlikely(pattern != 0xf0f0f0f0f0f0f0f0 || next != copy)) { /* found a corruption */
+ /* prevent further trouble by leaving a memleak */
+ brick_freelist[order] = NULL;
+ spin_unlock_irqrestore(&freelist_lock[order], flags);
+ BRICK_ERR(
+ "line %d:freelist corruption at %p (pattern = %lx next %p != %p, murdered = %d), order = %d\n",
+ cline, data, pattern, next, copy, atomic_read(&freelist_count[order]), order);
+ return NULL;
+ }
+#endif
+ brick_freelist[order] = next;
+ atomic_dec(&freelist_count[order]);
+ }
+ spin_unlock_irqrestore(&freelist_lock[order], flags);
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ if (data) {
+ struct mem_block_info *inf = _find_block_info(data, false);
+
+ if (likely(inf)) {
+ if (unlikely(inf->inf_len != (PAGE_SIZE << order))) {
+ BRICK_ERR(
+ "line %d: address %p: bad freelist size %d (correct should be %d, previous line = %d)\n",
+ cline, data, (int)(PAGE_SIZE << order), inf->inf_len, inf->inf_line);
+ }
+ inf->inf_line = cline;
+ inf->inf_used = true;
+ } else {
+ BRICK_ERR("line %d: freelist address %p is invalid (order = %d)\n", cline, data, order);
+ }
+ }
+#endif
+ return data;
+}
+
+static
+void _put_free(void *data, int order)
+{
+ void *next;
+ unsigned long flags;
+
+#ifdef BRICK_DEBUG_MEM /* fill with pattern */
+ memset(data, 0xf0, PAGE_SIZE << order);
+#endif
+
+ spin_lock_irqsave(&freelist_lock[order], flags);
+ next = brick_freelist[order];
+ *(void **)data = next;
+#ifdef BRICK_DEBUG_MEM /* insert redundant copy for checking */
+ *(((void **)data) + 2) = next;
+#endif
+ brick_freelist[order] = data;
+ spin_unlock_irqrestore(&freelist_lock[order], flags);
+ atomic_inc(&freelist_count[order]);
+}
+
+static
+void _free_all(void)
+{
+ int order;
+
+ for (order = BRICK_MAX_ORDER; order >= 0; order--) {
+ for (;;) {
+ void *data = _get_free(order, __LINE__);
+
+ if (!data)
+ break;
+ __brick_block_free(data, order, __LINE__);
+ }
+ }
+}
+
+int brick_mem_reserve(void)
+{
+ int order;
+ int status = 0;
+
+ for (order = BRICK_MAX_ORDER; order >= 0; order--) {
+ int max = brick_pre_reserve[order];
+ int i;
+
+ brick_mem_freelist_max[order] += max;
+ BRICK_INF(
+ "preallocating %d at order %d (new maxlevel = %d)\n", max, order, brick_mem_freelist_max[order]);
+
+ max = brick_mem_freelist_max[order] - atomic_read(&freelist_count[order]);
+ if (max >= 0) {
+ for (i = 0; i < max; i++) {
+ void *data = __brick_block_alloc(GFP_KERNEL, order, __LINE__);
+
+ if (likely(data))
+ _put_free(data, order);
+ else
+ status = -ENOMEM;
+ }
+ } else {
+ for (i = 0; i < -max; i++) {
+ void *data = _get_free(order, __LINE__);
+
+ if (likely(data))
+ __brick_block_free(data, order, __LINE__);
+ }
+ }
+ }
+ return status;
+}
+#else
+int brick_mem_reserve(struct mem_reservation *r)
+{
+ BRICK_INF("preallocation is not compiled in\n");
+ return 0;
+}
+#endif
+
+void *_brick_block_alloc(loff_t pos, int len, int line)
+{
+ void *data;
+ int count;
+
+#ifdef BRICK_DEBUG_MEM
+#ifdef BRICK_DEBUG_ORDER0
+ const int plus0 = PAGE_SIZE;
+
+#else
+ const int plus0 = 0;
+
+#endif
+ const int plus = len <= PAGE_SIZE ? plus0 : PAGE_SIZE * 2;
+
+#else
+ const int plus = 0;
+
+#endif
+ int order = len2order(len + plus);
+
+ if (unlikely(order < 0)) {
+ BRICK_ERR("trying to allocate %d bytes (max = %d)\n", len, (int)(PAGE_SIZE << order));
+ return NULL;
+ }
+
+#ifdef CONFIG_MARS_DEBUG
+ might_sleep();
+#endif
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ count = atomic_add_return(1, &_alloc_count[order]);
+ brick_mem_alloc_count[order] = count;
+ if (count > brick_mem_alloc_max[order])
+ brick_mem_alloc_max[order] = count;
+#endif
+
+#ifdef BRICK_DEBUG_MEM
+ atomic_inc(&op_count[order]);
+ /* statistics */
+ alloc_line[order] = line;
+ alloc_len[order] = len;
+#endif
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ /* Dynamic increase of limits, in order to reduce
+ * fragmentation on higher-order pages.
+ * This comes on cost of higher memory usage.
+ */
+ if (order > 0 && count > brick_mem_freelist_max[order])
+ brick_mem_freelist_max[order] = count;
+#endif
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ data = _get_free(order, line);
+ if (!data)
+#endif
+ data = __brick_block_alloc(GFP_BRICK, order, line);
+
+#ifdef BRICK_DEBUG_MEM
+ if (order > 0) {
+ if (unlikely(line < 0))
+ line = 0;
+ else if (unlikely(line >= BRICK_DEBUG_MEM))
+ line = BRICK_DEBUG_MEM - 1;
+ atomic_inc(&block_count[line]);
+ block_len[line] = len;
+ if (order > 1) {
+ INT_ACCESS(data, 0 * sizeof(int)) = MAGIC_BLOCK;
+ INT_ACCESS(data, 1 * sizeof(int)) = line;
+ INT_ACCESS(data, 2 * sizeof(int)) = len;
+ data += PAGE_SIZE;
+ INT_ACCESS(data, -1 * sizeof(int)) = MAGIC_BLOCK;
+ INT_ACCESS(data, len) = MAGIC_BEND;
+ } else if (order == 1) {
+ INT_ACCESS(data, PAGE_SIZE + 0 * sizeof(int)) = MAGIC_BLOCK;
+ INT_ACCESS(data, PAGE_SIZE + 1 * sizeof(int)) = line;
+ INT_ACCESS(data, PAGE_SIZE + 2 * sizeof(int)) = len;
+ }
+ }
+#endif
+ return data;
+}
+
+void _brick_block_free(void *data, int len, int cline)
+{
+ int order;
+
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ struct mem_block_info *inf;
+ char *real_data;
+
+#endif
+#ifdef BRICK_DEBUG_MEM
+ int prev_line = 0;
+
+#ifdef BRICK_DEBUG_ORDER0
+ const int plus0 = PAGE_SIZE;
+
+#else
+ const int plus0 = 0;
+
+#endif
+ const int plus = len <= PAGE_SIZE ? plus0 : PAGE_SIZE * 2;
+
+#else
+ const int plus = 0;
+
+#endif
+
+ order = len2order(len + plus);
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ real_data = data;
+ if (order > 1)
+ real_data -= PAGE_SIZE;
+ inf = _find_block_info(real_data, false);
+ if (likely(inf)) {
+ prev_line = inf->inf_line;
+ if (unlikely(inf->inf_len != (PAGE_SIZE << order))) {
+ BRICK_ERR(
+ "line %d: address %p: bad freeing size %d (correct should be %d, previous line = %d)\n",
+ cline, data, (int)(PAGE_SIZE << order), inf->inf_len, prev_line);
+ goto _out_return;
+ }
+ if (unlikely(!inf->inf_used)) {
+ BRICK_ERR(
+ "line %d: address %p: double freeing (previous line = %d)\n", cline, data, prev_line);
+ goto _out_return;
+ }
+ inf->inf_line = cline;
+ inf->inf_used = false;
+ } else {
+ BRICK_ERR("line %d: trying to free non-existent address %p (order = %d)\n", cline, data, order);
+ goto _out_return;
+ }
+#endif
+#ifdef BRICK_DEBUG_MEM
+ if (order > 1) {
+ void *test = data - PAGE_SIZE;
+ int magic = INT_ACCESS(test, 0);
+ int line = INT_ACCESS(test, sizeof(int));
+ int oldlen = INT_ACCESS(test, sizeof(int) * 2);
+ int magic1 = INT_ACCESS(data, -1 * sizeof(int));
+ int magic2;
+
+ if (unlikely(magic1 != MAGIC_BLOCK)) {
+ BRICK_ERR(
+ "line %d memory corruption: %p magix1 %08x != %08x (previous line = %d)\n",
+ cline,
+ data,
+ magic1,
+ MAGIC_BLOCK,
+ prev_line);
+ goto _out_return;
+ }
+ if (unlikely(magic != MAGIC_BLOCK)) {
+ BRICK_ERR(
+ "line %d memory corruption: %p magix %08x != %08x (previous line = %d)\n",
+ cline,
+ data,
+ magic,
+ MAGIC_BLOCK,
+ prev_line);
+ goto _out_return;
+ }
+ if (unlikely(line < 0 || line >= BRICK_DEBUG_MEM)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: alloc line = %d (previous line = %d)\n",
+ cline,
+ data,
+ line,
+ prev_line);
+ goto _out_return;
+ }
+ if (unlikely(oldlen != len)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: len != oldlen (%d != %d, previous line = %d))\n",
+ cline,
+ data,
+ len,
+ oldlen,
+ prev_line);
+ goto _out_return;
+ }
+ magic2 = INT_ACCESS(data, len);
+ if (unlikely(magic2 != MAGIC_BEND)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: magix %08x != %08x (previous line = %d)\n",
+ cline,
+ data,
+ magic,
+ MAGIC_BEND,
+ prev_line);
+ goto _out_return;
+ }
+ INT_ACCESS(test, 0) = 0xffffffff;
+ INT_ACCESS(data, len) = 0xffffffff;
+ data = test;
+ atomic_dec(&block_count[line]);
+ atomic_inc(&block_free[line]);
+ } else if (order == 1) {
+ void *test = data + PAGE_SIZE;
+ int magic = INT_ACCESS(test, 0 * sizeof(int));
+ int line = INT_ACCESS(test, 1 * sizeof(int));
+ int oldlen = INT_ACCESS(test, 2 * sizeof(int));
+
+ if (unlikely(magic != MAGIC_BLOCK)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: magix %08x != %08x (previous line = %d)\n",
+ cline,
+ data,
+ magic,
+ MAGIC_BLOCK,
+ prev_line);
+ goto _out_return;
+ }
+ if (unlikely(line < 0 || line >= BRICK_DEBUG_MEM)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: alloc line = %d (previous line = %d)\n",
+ cline,
+ data,
+ line,
+ prev_line);
+ goto _out_return;
+ }
+ if (unlikely(oldlen != len)) {
+ BRICK_ERR(
+ "line %d memory corruption %p: len != oldlen (%d != %d, previous line = %d))\n",
+ cline,
+ data,
+ len,
+ oldlen,
+ prev_line);
+ goto _out_return;
+ }
+ atomic_dec(&block_count[line]);
+ atomic_inc(&block_free[line]);
+ }
+#endif
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ if (
+ order > 0 && brick_allow_freelist && atomic_read(&freelist_count[order]) <= brick_mem_freelist_max[order]) {
+ _put_free(data, order);
+ } else
+#endif
+ __brick_block_free(data, order, cline);
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ brick_mem_alloc_count[order] = atomic_dec_return(&_alloc_count[order]);
+#endif
+#ifdef BRICK_DEBUG_MEM
+_out_return:;
+#endif
+}
+
+struct page *brick_iomap(void *data, int *offset, int *len)
+{
+ int _offset = ((unsigned long)data) & (PAGE_SIZE - 1);
+ struct page *page;
+
+ *offset = _offset;
+ if (*len > PAGE_SIZE - _offset)
+ *len = PAGE_SIZE - _offset;
+ if (is_vmalloc_addr(data))
+ page = vmalloc_to_page(data);
+ else
+ page = virt_to_page(data);
+ return page;
+}
+
+/***********************************************************************/
+
+/* module */
+
+void brick_mem_statistics(bool final)
+{
+#ifdef BRICK_DEBUG_MEM
+ int i;
+ int count = 0;
+ int places = 0;
+
+ BRICK_INF("======== page allocation:\n");
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ for (i = 0; i <= BRICK_MAX_ORDER; i++) {
+ BRICK_INF(
+ "pages order = %2d operations = %9d freelist_count = %4d / %3d raw_count = %5d alloc_count = %5d alloc_len = %5d line = %5d max_count = %5d\n",
+ i,
+ atomic_read(&op_count[i]),
+ atomic_read(&freelist_count[i]),
+ brick_mem_freelist_max[i],
+ atomic_read(&raw_count[i]),
+ brick_mem_alloc_count[i],
+ alloc_len[i],
+ alloc_line[i],
+ brick_mem_alloc_max[i]);
+ }
+#endif
+ for (i = 0; i < BRICK_DEBUG_MEM; i++) {
+ int val = atomic_read(&block_count[i]);
+
+ if (val) {
+ count += val;
+ places++;
+ BRICK_INF("line %4d: %6d allocated (last size = %4d, freed = %6d)\n",
+ i,
+ val,
+ block_len[i],
+ atomic_read(&block_free[i]));
+ }
+ }
+ if (!final || !count) {
+ BRICK_INF("======== %d block allocations in %d places (phys=%d)\n",
+ count, places, atomic_read(&phys_block_alloc));
+ } else {
+ BRICK_ERR("======== %d block allocations in %d places (phys=%d)\n",
+ count, places, atomic_read(&phys_block_alloc));
+ }
+ count = 0;
+ places = 0;
+ for (i = 0; i < BRICK_DEBUG_MEM; i++) {
+ int val = atomic_read(&mem_count[i]);
+
+ if (val) {
+ count += val;
+ places++;
+ BRICK_INF("line %4d: %6d allocated (last size = %4d, freed = %6d)\n",
+ i,
+ val,
+ mem_len[i],
+ atomic_read(&mem_free[i]));
+ }
+ }
+ if (!final || !count) {
+ BRICK_INF("======== %d memory allocations in %d places (phys=%d,redirect=%d)\n",
+ count, places,
+ atomic_read(&phys_mem_alloc), atomic_read(&mem_redirect_alloc));
+ } else {
+ BRICK_ERR("======== %d memory allocations in %d places (phys=%d,redirect=%d)\n",
+ count, places,
+ atomic_read(&phys_mem_alloc), atomic_read(&mem_redirect_alloc));
+ }
+ count = 0;
+ places = 0;
+ for (i = 0; i < BRICK_DEBUG_MEM; i++) {
+ int val = atomic_read(&string_count[i]);
+
+ if (val) {
+ count += val;
+ places++;
+ BRICK_INF("line %4d: %6d allocated (freed = %6d)\n",
+ i,
+ val,
+ atomic_read(&string_free[i]));
+ }
+ }
+ if (!final || !count) {
+ BRICK_INF("======== %d string allocations in %d places (phys=%d)\n",
+ count, places, atomic_read(&phys_string_alloc));
+ } else {
+ BRICK_ERR("======== %d string allocations in %d places (phys=%d)\n",
+ count, places, atomic_read(&phys_string_alloc));
+ }
+#endif
+}
+
+/* module init stuff */
+
+int __init init_brick_mem(void)
+{
+ int i;
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ for (i = BRICK_MAX_ORDER; i >= 0; i--)
+ spin_lock_init(&freelist_lock[i]);
+#endif
+#ifdef CONFIG_MARS_DEBUG_MEM_STRONG
+ for (i = 0; i < MAX_INFO_LISTS; i++) {
+ INIT_LIST_HEAD(&inf_anchor[i]);
+ rwlock_init(&inf_lock[i]);
+ }
+#else
+ (void)i;
+#endif
+
+ get_total_ram();
+
+ return 0;
+}
+
+void exit_brick_mem(void)
+{
+ BRICK_INF("deallocating memory...\n");
+#ifdef CONFIG_MARS_MEM_PREALLOC
+ _free_all();
+#endif
+
+ brick_mem_statistics(true);
+}
diff --git a/include/linux/brick/brick_mem.h b/include/linux/brick/brick_mem.h
new file mode 100644
index 000000000000..cb812da83877
--- /dev/null
+++ b/include/linux/brick/brick_mem.h
@@ -0,0 +1,218 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef BRICK_MEM_H
+#define BRICK_MEM_H
+
+#include <linux/mm_types.h>
+
+#define BRICK_DEBUG_MEM 4096
+
+#ifndef CONFIG_MARS_DEBUG_MEM
+#undef BRICK_DEBUG_MEM
+#endif
+#ifdef CONFIG_MARS_DEBUG_ORDER0
+#define BRICK_DEBUG_ORDER0
+#endif
+
+#define CONFIG_MARS_MEM_PREALLOC /* this is VITAL - disable only for experiments! */
+
+#define GFP_BRICK GFP_NOIO
+
+extern long long brick_global_memavail;
+extern long long brick_global_memlimit;
+extern atomic64_t brick_global_block_used;
+
+/* All brick memory allocations are guaranteed to succeed.
+ * In case of low memory, they will just retry (forever).
+ *
+ * We always prefer threads for concurrency.
+ * Therefore, in_interrupt() code does not occur, and we can
+ * always sleep in case of memory pressure.
+ *
+ * Resource deadlocks are avoided by the above memory limits.
+ * When exceeded, new memory is simply not allocated any more
+ * (except for vital memory, such as IO memory for which a
+ * low_mem_reserve must always exist, anyway).
+ */
+
+/***********************************************************************/
+
+/* compiler tweaking */
+
+/* Some functions are known to return non-null pointer values,
+ * at least under some Kconfig conditions.
+ *
+ * In code like...
+ *
+ * void *ptr = myfunction();
+ * if (unlikely(!ptr)) {
+ * printk("ERROR: this should not happen\n");
+ * goto fail;
+ * }
+ *
+ * ... the dead code elimination of gcc will not remove the if clause
+ * because the function might return a NULL value, even if a human
+ * would know that myfunction() does not return a NULL value.
+ *
+ * Unfortunately, the __attribute__((nonnull)) can only be applied
+ * to input parameters, but not to the return value.
+ *
+ * More unfortunately, a small inline wrapper does not help,
+ * because it seems that together with the elimination of the wrapper,
+ * its nonnull attribute seems to be eliminated alltogether.
+ * I don't know whether this is a bug or a feature (or just a weakness).
+ *
+ * Following is a small hack which solves the problem at least for gcc 4.7.
+ *
+ * In order to be useful, the -fdelete-null-pointer-checks must be set.
+ * Since BRICK is superuser-only anyway, enabling this for MARS should not
+ * be a security risk
+ * (c.f. upstream kernel commit a3ca86aea507904148870946d599e07a340b39bf)
+ */
+extern inline
+void *brick_mark_nonnull(void *_ptr)
+{
+ char *ptr = _ptr;
+
+ /* fool gcc to believe that the pointer were dereferenced... */
+ asm("" : : "X" (*ptr));
+ return ptr;
+}
+
+/***********************************************************************/
+
+/* small memory allocation (use this only for len < PAGE_SIZE) */
+
+#define brick_mem_alloc(_len_) \
+ ({ \
+ void *_res_ = _brick_mem_alloc(_len_, __LINE__); \
+ brick_mark_nonnull(_res_); \
+ })
+
+#define brick_zmem_alloc(_len_) \
+ ({ \
+ void *_res_ = _brick_mem_alloc(_len_, __LINE__); \
+ _res_ = brick_mark_nonnull(_res_); \
+ memset(_res_, 0, _len_); \
+ _res_; \
+ })
+
+#define brick_mem_free(_data_) \
+ do { \
+ if (_data_) { \
+ _brick_mem_free(_data_, __LINE__); \
+ } \
+ } while (0)
+
+/* don't use the following directly */
+extern void *_brick_mem_alloc(int len, int line) __attribute__((malloc)) __attribute__((alloc_size(1)));
+extern void _brick_mem_free(void *data, int line);
+
+/***********************************************************************/
+
+/* string memory allocation */
+
+#define BRICK_STRING_LEN 1024 /* default value when len == 0 */
+
+#define brick_string_alloc(_len_) \
+ ({ \
+ char *_res_ = _brick_string_alloc((_len_), __LINE__); \
+ (char *)brick_mark_nonnull(_res_); \
+ })
+
+#define brick_strndup(_orig_, _len_) \
+ ({ \
+ char *_res_ = _brick_string_alloc((_len_) + 1, __LINE__);\
+ _res_ = brick_mark_nonnull(_res_); \
+ strncpy(_res_, (_orig_), (_len_)); \
+ /* always null-terminate for safety */ \
+ _res_[_len_] = '\0'; \
+ (char *)brick_mark_nonnull(_res_); \
+ })
+
+#define brick_strdup(_orig_) \
+ ({ \
+ int _len_ = strlen(_orig_); \
+ char *_res_ = _brick_string_alloc((_len_) + 1, __LINE__);\
+ _res_ = brick_mark_nonnull(_res_); \
+ strncpy(_res_, (_orig_), (_len_) + 1); \
+ (char *)brick_mark_nonnull(_res_); \
+ })
+
+#define brick_string_free(_data_) \
+ do { \
+ if (_data_) { \
+ _brick_string_free(_data_, __LINE__); \
+ } \
+ } while (0)
+
+/* don't use the following directly */
+extern char *_brick_string_alloc(int len, int line) __attribute__((malloc));
+extern void _brick_string_free(const char *data, int line);
+
+/***********************************************************************/
+
+/* block memory allocation (for aligned multiples of 512 resp PAGE_SIZE) */
+
+#define brick_block_alloc(_pos_, _len_) \
+ ({ \
+ void *_res_ = _brick_block_alloc((_pos_), (_len_), __LINE__);\
+ brick_mark_nonnull(_res_); \
+ })
+
+#define brick_block_free(_data_, _len_) \
+ do { \
+ if (_data_) { \
+ _brick_block_free((_data_), (_len_), __LINE__); \
+ } \
+ } while (0)
+
+extern struct page *brick_iomap(void *data, int *offset, int *len);
+
+/* don't use the following directly */
+extern void *_brick_block_alloc(loff_t pos, int len, int line) __attribute__((malloc)) __attribute__((alloc_size(2)));
+extern void _brick_block_free(void *data, int len, int cline);
+
+/***********************************************************************/
+
+/* reservations / preallocation */
+
+#define BRICK_MAX_ORDER 11
+
+#ifdef CONFIG_MARS_MEM_PREALLOC
+extern int brick_allow_freelist;
+
+extern int brick_pre_reserve[BRICK_MAX_ORDER+1];
+extern int brick_mem_freelist_max[BRICK_MAX_ORDER+1];
+extern int brick_mem_alloc_count[BRICK_MAX_ORDER+1];
+extern int brick_mem_alloc_max[BRICK_MAX_ORDER+1];
+
+extern int brick_mem_reserve(void);
+
+#endif
+
+extern void brick_mem_statistics(bool final);
+
+/***********************************************************************/
+
+/* init */
+
+extern int init_brick_mem(void);
+extern void exit_brick_mem(void);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 13/32] mars: add new module xio |
| Message-ID | <sUetc-2Cs-39@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/xio_bricks/xio.c | 227 ++++++++++++++++++++++++
include/linux/xio/xio.h | 319 ++++++++++++++++++++++++++++++++++
2 files changed, 546 insertions(+)
create mode 100644 drivers/staging/mars/xio_bricks/xio.c
create mode 100644 include/linux/xio/xio.h
diff --git a/drivers/staging/mars/xio_bricks/xio.c b/drivers/staging/mars/xio_bricks/xio.c
new file mode 100644
index 000000000000..e58f11f497f9
--- /dev/null
+++ b/drivers/staging/mars/xio_bricks/xio.c
@@ -0,0 +1,227 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+
+#include <linux/xio/xio.h>
+
+/************************************************************/
+
+/* infrastructure */
+
+struct banning xio_global_ban = {};
+atomic_t xio_global_io_flying = ATOMIC_INIT(0);
+
+/************************************************************/
+
+/* object stuff */
+
+const struct generic_object_type aio_type = {
+ .object_type_name = "aio",
+ .default_size = sizeof(struct aio_object),
+ .object_type_nr = OBJ_TYPE_AIO,
+};
+
+/************************************************************/
+
+/* brick stuff */
+
+/*******************************************************************/
+
+/* meta descriptions */
+
+const struct meta xio_info_meta[] = {
+ META_INI(current_size, struct xio_info, FIELD_INT),
+ META_INI(tf_align, struct xio_info, FIELD_INT),
+ META_INI(tf_min_size, struct xio_info, FIELD_INT),
+ {}
+};
+
+const struct meta xio_aio_user_meta[] = {
+ META_INI(_object_cb.cb_error, struct aio_object, FIELD_INT),
+ META_INI(io_pos, struct aio_object, FIELD_INT),
+ META_INI(io_len, struct aio_object, FIELD_INT),
+ META_INI(io_may_write, struct aio_object, FIELD_INT),
+ META_INI(io_prio, struct aio_object, FIELD_INT),
+ META_INI(io_cs_mode, struct aio_object, FIELD_INT),
+ META_INI(io_timeout, struct aio_object, FIELD_INT),
+ META_INI(io_total_size, struct aio_object, FIELD_INT),
+ META_INI(io_checksum, struct aio_object, FIELD_RAW),
+ META_INI(io_flags, struct aio_object, FIELD_INT),
+ META_INI(io_rw, struct aio_object, FIELD_INT),
+ META_INI(io_id, struct aio_object, FIELD_INT),
+ META_INI(io_skip_sync, struct aio_object, FIELD_INT),
+ {}
+};
+
+const struct meta xio_timespec_meta[] = {
+ META_INI_TRANSFER(tv_sec, struct timespec, FIELD_UINT, 8),
+ META_INI_TRANSFER(tv_nsec, struct timespec, FIELD_UINT, 4),
+ {}
+};
+
+/************************************************************/
+
+/* crypto stuff */
+
+#include <linux/scatterlist.h>
+#include <linux/crypto.h>
+
+/* 896545098777564212b9e91af4c973f094649aa7 */
+#ifndef crt_hash
+#define HAS_NEW_CRYPTO
+#endif
+
+#ifdef HAS_NEW_CRYPTO
+
+/* Nor now, use shash.
+ * Later, asynchronous support should be added for full exploitation
+ * of crypto hardware.
+ */
+#include <crypto/hash.h>
+
+static struct crypto_shash *xio_tfm;
+int xio_digest_size;
+
+struct mars_sdesc {
+ struct shash_desc shash;
+ char ctx[];
+};
+
+void xio_digest(unsigned char *digest, void *data, int len)
+{
+ int size = sizeof(struct mars_sdesc) + crypto_shash_descsize(xio_tfm);
+ struct mars_sdesc *sdesc = brick_mem_alloc(size);
+ int status;
+
+ sdesc->shash.tfm = xio_tfm;
+ sdesc->shash.flags = 0;
+
+ memset(digest, 0, xio_digest_size);
+ status = crypto_shash_digest(&sdesc->shash, data, len, digest);
+ if (unlikely(status < 0))
+ XIO_ERR(
+ "cannot calculate cksum on %p len=%d, status=%d\n",
+ data, len,
+ status);
+
+ brick_mem_free(sdesc);
+}
+
+#else /* HAS_NEW_CRYPTO */
+
+/* Old implementation, to disappear.
+ * Was a quick'n dirty lab prototype with unnecessary
+ * global variables and locking.
+ */
+
+static struct crypto_hash *xio_tfm;
+static struct semaphore tfm_sem;
+int xio_digest_size;
+
+void xio_digest(unsigned char *digest, void *data, int len)
+{
+ struct hash_desc desc = {
+ .tfm = xio_tfm,
+ .flags = 0,
+ };
+ struct scatterlist sg;
+
+ memset(digest, 0, xio_digest_size);
+
+ /* TODO: use per-thread instance, omit locking */
+ down(&tfm_sem);
+
+ crypto_hash_init(&desc);
+ sg_init_table(&sg, 1);
+ sg_set_buf(&sg, data, len);
+ crypto_hash_update(&desc, &sg, sg.length);
+ crypto_hash_final(&desc, digest);
+ up(&tfm_sem);
+}
+
+#endif /* HAS_NEW_CRYPTO */
+
+void aio_checksum(struct aio_object *aio)
+{
+ unsigned char checksum[xio_digest_size];
+ int len;
+
+ if (aio->io_cs_mode <= 0 || !aio->io_data)
+ goto out_return;
+ xio_digest(checksum, aio->io_data, aio->io_len);
+
+ len = sizeof(aio->io_checksum);
+ if (len > xio_digest_size)
+ len = xio_digest_size;
+ memcpy(&aio->io_checksum, checksum, len);
+out_return:;
+}
+
+/*******************************************************************/
+
+/* init stuff */
+
+int __init init_xio(void)
+{
+ XIO_INF("init_xio()\n");
+
+ sema_init(&tfm_sem, 1);
+
+#ifdef HAS_NEW_CRYPTO
+ xio_tfm = crypto_alloc_shash("md5", 0, 0);
+ if (unlikely(!xio_tfm) || IS_ERR(xio_tfm)) {
+ XIO_ERR(
+ "cannot alloc crypto hash, status=%ld\n",
+ PTR_ERR(xio_tfm));
+ return -ELIBACC;
+ }
+ xio_digest_size = crypto_shash_digestsize(xio_tfm);
+#else /* HAS_NEW_CRYPTO */
+ xio_tfm = crypto_alloc_hash("md5", 0, CRYPTO_ALG_ASYNC);
+ if (!xio_tfm) {
+ XIO_ERR("cannot alloc crypto hash\n");
+ return -ENOMEM;
+ }
+ if (IS_ERR(xio_tfm)) {
+ XIO_ERR("alloc crypto hash failed, status = %d\n", (int)PTR_ERR(xio_tfm));
+ return PTR_ERR(xio_tfm);
+ }
+ xio_digest_size = crypto_hash_digestsize(xio_tfm);
+#endif /* HAS_NEW_CRYPTO */
+ XIO_INF("digest_size = %d\n", xio_digest_size);
+
+ return 0;
+}
+
+void exit_xio(void)
+{
+ XIO_INF("exit_xio()\n");
+
+ if (xio_tfm) {
+#ifdef HAS_NEW_CRYPTO
+ crypto_free_shash(xio_tfm);
+#else /* HAS_NEW_CRYPTO */
+ crypto_free_hash(xio_tfm);
+#endif /* HAS_NEW_CRYPTO */
+ }
+}
diff --git a/include/linux/xio/xio.h b/include/linux/xio/xio.h
new file mode 100644
index 000000000000..d26a1c761ee3
--- /dev/null
+++ b/include/linux/xio/xio.h
@@ -0,0 +1,319 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef XIO_H
+#define XIO_H
+
+#include <linux/semaphore.h>
+#include <linux/rwsem.h>
+#include <linux/major.h>
+
+#if defined(CONFIG_CRYPTO_LZO) || defined(CONFIG_CRYPTO_LZO_MODULE)
+#define __HAVE_LZO
+#endif
+
+#ifdef __enabled_CONFIG_CRYPTO_LZO
+#if __enabled_CONFIG_CRYPTO_LZO
+#define __HAVE_LZO
+#endif
+#endif
+
+#ifdef __enabled_CONFIG_CRYPTO_LZO_MODULE
+#if __enabled_CONFIG_CRYPTO_LZO_MODULE
+#define __HAVE_LZO
+#endif
+#endif
+
+/* TRANSITIONAL compatibility to BOTH the old prepatch
+ * and the new wrapper around vfs_*(). Both will be replaced
+ * for kernel upstream.
+ */
+#include <linux/brick/vfs_compat.h>
+#ifndef MARS_MAJOR
+#define __USE_COMPAT
+#endif
+
+/***********************************************************************/
+
+/* include the generic brick infrastructure */
+
+#define OBJ_TYPE_AIO 0
+#define OBJ_TYPE_MAX 1
+
+#include <linux/brick/brick.h>
+#include <linux/brick/brick_mem.h>
+#include <linux/brick/lamport.h>
+#include <linux/brick/lib_timing.h>
+
+/***********************************************************************/
+
+/* XIO-specific debugging helpers */
+
+#define _XIO_MSG(_class, _dump, _fmt, _args...) \
+ brick_say(_class, _dump, "XIO", __BASE_FILE__, __LINE__, __func__, _fmt, ##_args)
+
+#define XIO_FAT(_fmt, _args...) _XIO_MSG(SAY_FATAL, true, _fmt, ##_args)
+#define XIO_ERR(_fmt, _args...) _XIO_MSG(SAY_ERROR, false, _fmt, ##_args)
+#define XIO_WRN(_fmt, _args...) _XIO_MSG(SAY_WARN, false, _fmt, ##_args)
+#define XIO_INF(_fmt, _args...) _XIO_MSG(SAY_INFO, false, _fmt, ##_args)
+
+#ifdef XIO_DEBUGGING
+#define XIO_DBG(_fmt, _args...) _XIO_MSG(SAY_DEBUG, false, _fmt, ##_args)
+#else
+#define XIO_DBG(_args...) /**/
+#endif
+
+/***********************************************************************/
+
+/* XIO-specific definitions */
+
+#define XIO_PRIO_HIGH -1
+#define XIO_PRIO_NORMAL 0 /* this is automatically used by memset() */
+#define XIO_PRIO_LOW 1
+#define XIO_PRIO_NR 3
+
+/* object stuff */
+
+/* aio */
+
+#define AIO_UPTODATE 1
+#define AIO_READING 2
+#define AIO_WRITING 4
+
+extern const struct generic_object_type aio_type;
+
+#define XIO_CHECKSUM_SIZE 16
+
+#define AIO_OBJECT(OBJTYPE) \
+ CALLBACK_OBJECT(OBJTYPE); \
+ /* supplied by caller */ \
+ void *io_data; /* preset to NULL for buffered IO */ \
+ loff_t io_pos; \
+ int io_len; \
+ int io_may_write; \
+ int io_prio; \
+ int io_timeout; \
+ int io_cs_mode; /* 0 = off, 1 = checksum + data, 2 = checksum only */\
+ /* maintained by the aio implementation, readable for callers */\
+ loff_t io_total_size; /* just for info, need not be implemented */\
+ unsigned char io_checksum[XIO_CHECKSUM_SIZE]; \
+ int io_flags; \
+ int io_rw; \
+ int io_id; /* not mandatory; may be used for identification */\
+ bool io_skip_sync /* skip sync for this particular aio */
+
+struct aio_object {
+ AIO_OBJECT(aio);
+};
+
+/* internal helper structs */
+
+struct xio_info {
+ loff_t current_size;
+
+ int tf_align; /* transfer alignment constraint */
+ int tf_min_size; /* transfer is only possible in multiples of this */
+};
+
+/* brick stuff */
+
+#define XIO_BRICK(BRITYPE) \
+ GENERIC_BRICK(BRITYPE); \
+ struct generic_object_layout aio_object_layout; \
+ struct list_head global_brick_link; \
+ struct list_head dent_brick_link; \
+ const char *brick_name; \
+ const char *brick_path; \
+ void *private_ptr; \
+ void **kill_ptr; \
+ int *mode_ptr; \
+ int kill_round; \
+ bool killme; \
+ void (*show_status)(struct xio_brick *brick, bool shutdown)
+
+struct xio_brick {
+ XIO_BRICK(xio);
+};
+
+#define XIO_INPUT(BRITYPE) \
+ GENERIC_INPUT(BRITYPE)
+
+struct xio_input {
+ XIO_INPUT(xio);
+};
+
+#define XIO_OUTPUT(BRITYPE) \
+ GENERIC_OUTPUT(BRITYPE)
+
+struct xio_output {
+ XIO_OUTPUT(xio);
+};
+
+#define XIO_BRICK_OPS(BRITYPE) \
+ GENERIC_BRICK_OPS(BRITYPE); \
+ char *(*brick_statistics)(struct BRITYPE##_brick *brick, int verbose);\
+ void (*reset_statistics)(struct BRITYPE##_brick *brick)
+
+#define XIO_OUTPUT_OPS(BRITYPE) \
+ GENERIC_OUTPUT_OPS(BRITYPE); \
+ int (*xio_get_info)(struct BRITYPE##_output *output, struct xio_info *info);\
+ /* aio */ \
+ int (*aio_get)(struct BRITYPE##_output *output, struct aio_object *aio);\
+ void (*aio_io)(struct BRITYPE##_output *output, struct aio_object *aio);\
+ void (*aio_put)(struct BRITYPE##_output *output, struct aio_object *aio)
+
+/* all non-extendable types */
+
+#define _XIO_TYPES(BRITYPE) \
+ \
+struct BRITYPE##_brick_ops { \
+ XIO_BRICK_OPS(BRITYPE); \
+}; \
+ \
+struct BRITYPE##_output_ops { \
+ XIO_OUTPUT_OPS(BRITYPE); \
+}; \
+ \
+struct BRITYPE##_brick_type { \
+ GENERIC_BRICK_TYPE(BRITYPE); \
+}; \
+ \
+struct BRITYPE##_input_type { \
+ GENERIC_INPUT_TYPE(BRITYPE); \
+}; \
+ \
+struct BRITYPE##_output_type { \
+ GENERIC_OUTPUT_TYPE(BRITYPE); \
+}; \
+ \
+struct BRITYPE##_callback { \
+ GENERIC_CALLBACK(BRITYPE); \
+}; \
+ \
+DECLARE_BRICK_FUNCTIONS(BRITYPE)
+
+#define XIO_TYPES(BRITYPE) \
+ \
+_XIO_TYPES(BRITYPE); \
+ \
+DECLARE_ASPECT_FUNCTIONS(BRITYPE, aio) \
+extern int init_xio_##BRITYPE(void); \
+extern void exit_xio_##BRITYPE(void)
+
+/* instantiate pseudo base-classes */
+
+DECLARE_OBJECT_FUNCTIONS(aio)
+_XIO_TYPES(xio);
+DECLARE_ASPECT_FUNCTIONS(xio, aio)
+
+/***********************************************************************/
+
+/* XIO-specific helpers */
+
+#define XIO_MAKE_STATICS(BRITYPE) \
+ \
+int BRITYPE##_brick_nr = -EEXIST; \
+ \
+static const struct generic_aspect_type BRITYPE##_aio_aspect_type = { \
+ .aspect_type_name = #BRITYPE "_aio_aspect_type", \
+ .object_type = &aio_type, \
+ .aspect_size = sizeof(struct BRITYPE##_aio_aspect), \
+ .init_fn = BRITYPE##_aio_aspect_init_fn, \
+ .exit_fn = BRITYPE##_aio_aspect_exit_fn, \
+}; \
+ \
+static const struct generic_aspect_type *BRITYPE##_aspect_types[OBJ_TYPE_MAX] = {\
+ [OBJ_TYPE_AIO] = &BRITYPE##_aio_aspect_type, \
+}
+
+extern const struct meta xio_info_meta[];
+extern const struct meta xio_aio_user_meta[];
+extern const struct meta xio_timespec_meta[];
+
+/***********************************************************************/
+
+/* Some minimal upcalls from generic IO layer to the strategy layer.
+ * TODO: abstract away.
+ */
+
+extern void xio_set_power_on_led(struct xio_brick *brick, bool val);
+extern void xio_set_power_off_led(struct xio_brick *brick, bool val);
+
+/* this should disappear!
+ */
+extern void (*_local_trigger)(void);
+extern void (*_remote_trigger)(void);
+#define local_trigger() do { if (_local_trigger) { XIO_DBG("trigger...\n"); _local_trigger(); } } while (0)
+#define remote_trigger() \
+do { if (_remote_trigger) { XIO_DBG("remote_trigger...\n"); _remote_trigger(); } } while (0)
+
+/***********************************************************************/
+
+/* Some global stuff.
+ */
+
+extern struct banning xio_global_ban;
+
+extern atomic_t xio_global_io_flying;
+
+extern int xio_throttle_start;
+extern int xio_throttle_end;
+
+/***********************************************************************/
+
+/* Some special brick types for avoidance of cyclic references.
+ *
+ * The client/server network bricks use this for independent instantiation
+ * from the main instantiation logic (separate modprobe for xio_server
+ * is possible).
+ */
+extern const struct generic_brick_type *_client_brick_type;
+extern const struct generic_brick_type *_bio_brick_type;
+extern const struct generic_brick_type *_sio_brick_type;
+
+/***********************************************************************/
+
+/* Crypto stuff
+ */
+
+extern int xio_digest_size;
+extern void xio_digest(unsigned char *digest, void *data, int len);
+extern void aio_checksum(struct aio_object *aio);
+
+/***********************************************************************/
+
+/* Crash-testing instrumentation.
+ * Only for debugging. Never use this for production.
+ * Simulate a crash at the "wrong moment".
+ */
+
+#ifdef CONFIG_MARS_DEBUG
+extern int mars_crash_mode;
+extern int mars_hang_mode;
+extern void _crashme(int mode, bool do_sync);
+#else
+extern inline void _crashme(int mode, bool do_sync) {}
+#endif
+
+/***********************************************************************/
+
+/* init */
+
+extern int init_xio(void);
+extern void exit_xio(void);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 26/32] mars: add new module net |
| Message-ID | <sUetc-2Cs-41@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/mars/net.c | 109 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 109 insertions(+)
create mode 100644 drivers/staging/mars/mars/net.c
diff --git a/drivers/staging/mars/mars/net.c b/drivers/staging/mars/mars/net.c
new file mode 100644
index 000000000000..d1b9715c0a93
--- /dev/null
+++ b/drivers/staging/mars/mars/net.c
@@ -0,0 +1,109 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/string.h>
+
+#include "strategy.h"
+#include <linux/xio/xio_net.h>
+
+static
+char *_xio_translate_hostname(const char *name)
+{
+ char *res = brick_strdup(name);
+ char *test;
+ char *tmp;
+
+ for (tmp = res; *tmp; tmp++) {
+ if (*tmp == ':') {
+ *tmp = '\0';
+ break;
+ }
+ }
+
+ tmp = path_make("/mars/ips/ip-%s", res);
+ if (unlikely(!tmp))
+ goto done;
+
+ test = mars_readlink(tmp);
+ if (test && test[0]) {
+ XIO_DBG("'%s' => '%s'\n", tmp, test);
+ brick_string_free(res);
+ res = test;
+ } else {
+ brick_string_free(test);
+ XIO_WRN("no hostname translation for '%s'\n", tmp);
+ }
+ brick_string_free(tmp);
+
+done:
+ return res;
+}
+
+int xio_send_dent_list(struct xio_socket *sock, struct list_head *anchor)
+{
+ struct list_head *tmp;
+ struct mars_dent *dent;
+ int status = 0;
+
+ for (tmp = anchor->next; tmp != anchor; tmp = tmp->next) {
+ dent = container_of(tmp, struct mars_dent, dent_link);
+ status = xio_send_struct(sock, dent, mars_dent_meta);
+ if (status < 0)
+ break;
+ }
+ if (status >= 0) { /* send EOR */
+ status = xio_send_struct(sock, NULL, mars_dent_meta);
+ }
+ return status;
+}
+
+int xio_recv_dent_list(struct xio_socket *sock, struct list_head *anchor)
+{
+ int status;
+
+ for (;;) {
+ struct mars_dent *dent = brick_zmem_alloc(sizeof(struct mars_dent));
+
+ INIT_LIST_HEAD(&dent->dent_link);
+ INIT_LIST_HEAD(&dent->brick_list);
+
+ status = xio_recv_struct(sock, dent, mars_dent_meta);
+ if (status <= 0) {
+ xio_free_dent(dent);
+ goto done;
+ }
+ list_add_tail(&dent->dent_link, anchor);
+ }
+done:
+ return status;
+}
+
+/***************** module init stuff ************************/
+
+int __init init_sy_net(void)
+{
+ XIO_INF("init_sy_net()\n");
+ xio_translate_hostname = _xio_translate_hostname;
+ return 0;
+}
+
+void exit_sy_net(void)
+{
+ XIO_INF("exit_sy_net()\n");
+}
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 00:10 +0100 |
| Subject | [RFC 10/32] mars: add new module lib_limiter |
| Message-ID | <sUetd-2Cs-45@gated-at.bofh.it> |
| In reply to | #1548765 |
Signed-off-by: Thomas Schoebel-Theuer <tst@schoebel-theuer.de>
---
drivers/staging/mars/lib/lib_limiter.c | 163 +++++++++++++++++++++++++++++++++
include/linux/brick/lib_limiter.h | 52 +++++++++++
2 files changed, 215 insertions(+)
create mode 100644 drivers/staging/mars/lib/lib_limiter.c
create mode 100644 include/linux/brick/lib_limiter.h
diff --git a/drivers/staging/mars/lib/lib_limiter.c b/drivers/staging/mars/lib/lib_limiter.c
new file mode 100644
index 000000000000..e77b74a0eae7
--- /dev/null
+++ b/drivers/staging/mars/lib/lib_limiter.c
@@ -0,0 +1,163 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/brick/lib_limiter.h>
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/jiffies.h>
+#include <linux/sched.h>
+
+#define LIMITER_TIME_RESOLUTION NSEC_PER_SEC
+
+int rate_limit(struct rate_limiter *lim, int amount)
+{
+ int delay = 0;
+ long long now;
+
+ now = cpu_clock(raw_smp_processor_id());
+
+ /* Compute the maximum delay along the path
+ * down to the root of the hierarchy tree.
+ */
+ while (lim) {
+ long long window = now - lim->lim_stamp;
+
+ /* Sometimes, raw CPU clocks may do weired things...
+ * Smaller windows in the denominator than 1s could fake unrealistic rates.
+ */
+ if (unlikely(lim->lim_min_window <= 0))
+ lim->lim_min_window = 1000;
+ if (unlikely(lim->lim_max_window <= lim->lim_min_window))
+ lim->lim_max_window = lim->lim_min_window + 8000;
+ if (unlikely(window < (long long)lim->lim_min_window * (LIMITER_TIME_RESOLUTION / 1000)))
+ window = (long long)lim->lim_min_window * (LIMITER_TIME_RESOLUTION / 1000);
+
+ /* Update total statistics.
+ * They will intentionally wrap around.
+ * Userspace must take care of that.
+ */
+ if (likely(amount > 0)) {
+ lim->lim_total_amount += amount;
+ lim->lim_total_ops++;
+ }
+
+ /* Only use incremental accumulation at repeated calls, but
+ * never after longer pauses.
+ */
+ if (likely(lim->lim_stamp &&
+ window < (long long)lim->lim_max_window * (LIMITER_TIME_RESOLUTION / 1000))) {
+ long long rate_raw;
+ int rate;
+ int max_rate;
+
+ /* Races are possible, but taken into account.
+ * There is no real harm from rarely lost updates.
+ */
+ if (likely(amount > 0)) {
+ lim->lim_amount_accu += amount;
+ lim->lim_amount_cumul += amount;
+ lim->lim_ops_accu++;
+ lim->lim_ops_cumul++;
+ }
+
+ /* compute amount values */
+ rate_raw = lim->lim_amount_accu * LIMITER_TIME_RESOLUTION / window;
+ rate = rate_raw;
+ if (unlikely(rate_raw > INT_MAX))
+ rate = INT_MAX;
+ lim->lim_amount_rate = rate;
+
+ /* amount limit exceeded? */
+ max_rate = lim->lim_max_amount_rate;
+ if (max_rate > 0 && rate > max_rate) {
+ int this_delay = (
+
+ window * rate / max_rate - window) / (LIMITER_TIME_RESOLUTION / 1000);
+ /* compute maximum */
+ if (this_delay > delay && this_delay > 0)
+ delay = this_delay;
+ }
+
+ /* compute ops values */
+ rate_raw = lim->lim_ops_accu * LIMITER_TIME_RESOLUTION / window;
+ rate = rate_raw;
+ if (unlikely(rate_raw > INT_MAX))
+ rate = INT_MAX;
+ lim->lim_ops_rate = rate;
+
+ /* ops limit exceeded? */
+ max_rate = lim->lim_max_ops_rate;
+ if (max_rate > 0 && rate > max_rate) {
+ int this_delay = (
+
+ window * rate / max_rate - window) / (LIMITER_TIME_RESOLUTION / 1000);
+ /* compute maximum */
+ if (this_delay > delay && this_delay > 0)
+ delay = this_delay;
+ }
+
+ /* Try to keep the next window below min_window
+ */
+ window -= lim->lim_min_window * (LIMITER_TIME_RESOLUTION / 1000);
+ if (window > 0) {
+ long long used_up = (long long)lim->lim_amount_rate * window / LIMITER_TIME_RESOLUTION;
+
+ if (used_up > 0) {
+ lim->lim_stamp += window;
+ lim->lim_amount_accu -= used_up;
+ if (unlikely(lim->lim_amount_accu < 0))
+ lim->lim_amount_accu = 0;
+ }
+ used_up = (long long)lim->lim_ops_rate * window / LIMITER_TIME_RESOLUTION;
+ if (used_up > 0) {
+ lim->lim_stamp += window;
+ lim->lim_ops_accu -= used_up;
+ if (unlikely(lim->lim_ops_accu < 0))
+ lim->lim_ops_accu = 0;
+ }
+ }
+ } else { /* reset, start over with new measurement cycle */
+ if (unlikely(amount < 0))
+ amount = 0;
+ lim->lim_ops_accu = 1;
+ lim->lim_amount_accu = amount;
+ lim->lim_stamp = now - lim->lim_min_window * (LIMITER_TIME_RESOLUTION / 1000);
+ lim->lim_ops_rate = 0;
+ lim->lim_amount_rate = 0;
+ }
+ lim = lim->lim_father;
+ }
+ return delay;
+}
+
+void rate_limit_sleep(struct rate_limiter *lim, int amount)
+{
+ int sleep = rate_limit(lim, amount);
+
+ if (sleep > 0) {
+ unsigned long timeout;
+
+ if (unlikely(lim->lim_max_delay <= 0))
+ lim->lim_max_delay = 1000;
+ if (sleep > lim->lim_max_delay)
+ sleep = lim->lim_max_delay;
+ timeout = msecs_to_jiffies(sleep);
+ while ((long)timeout > 0)
+ timeout = schedule_timeout_uninterruptible(timeout);
+ }
+}
diff --git a/include/linux/brick/lib_limiter.h b/include/linux/brick/lib_limiter.h
new file mode 100644
index 000000000000..fab0c0e1858f
--- /dev/null
+++ b/include/linux/brick/lib_limiter.h
@@ -0,0 +1,52 @@
+/*
+ * MARS Long Distance Replication Software
+ *
+ * Copyright (C) 2010-2014 Thomas Schoebel-Theuer
+ * Copyright (C) 2011-2014 1&1 Internet AG
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef LIB_LIMITER_H
+#define LIB_LIMITER_H
+
+#include <linux/utsname.h>
+
+struct rate_limiter {
+ /* hierarchy tree */
+ struct rate_limiter *lim_father;
+
+ /* tunables */
+ int lim_max_ops_rate;
+ int lim_max_amount_rate;
+ int lim_max_delay;
+ int lim_min_window;
+ int lim_max_window;
+
+ /* readable */
+ int lim_ops_rate;
+ int lim_amount_rate;
+ int lim_ops_cumul;
+ int lim_amount_cumul;
+ int lim_total_ops;
+ int lim_total_amount;
+ long long lim_stamp;
+
+ /* internal */
+ long long lim_ops_accu;
+ long long lim_amount_accu;
+};
+
+extern int rate_limit(struct rate_limiter *lim, int amount);
+
+extern void rate_limit_sleep(struct rate_limiter *lim, int amount);
+
+#endif
--
2.11.0
[toc] | [prev] | [next] | [standalone]
| From | Thomas Schoebel-Theuer <tst@schoebel-theuer.de> |
|---|---|
| Date | 2016-12-31 07:50 +0100 |
| Message-ID | <sUlEl-7QR-3@gated-at.bofh.it> |
| In reply to | #1548765 |
Typo correction: On 12/30/2016 11:57 PM, Thomas Schoebel-Theuer wrote: > standalone servers with local hardware RAIDs. They are hosting about > 500 MARS resources (originally DRBD resources) just for the web servers; This must read 2500. Somehow the leading "2" was eaten at wraparound.
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web