Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.misc > #549 > unrolled thread

Can Reactive Programming Handle Complexity?

Back to article view | Back to comp.misc

Contents

  Can Reactive Programming Handle Complexity? Oregonian Haruspex <bob_davis_retired@yahoo.com> - 2014-02-18 16:33 -0800
    Re: Can Reactive Programming Handle Complexity? dialmove <dialmove@gmail.com> - 2014-02-20 16:17 +0000
      Re: Can Reactive Programming Handle Complexity? dialmove <dialmove@gmail.com> - 2014-02-20 16:36 +0000
        Re: Can Reactive Programming Handle Complexity? Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 20:09 +0200
          Re: Can Reactive Programming Handle Complexity? wjwlsn <ibillwilson@gmail.com> - 2014-02-20 13:05 -0800
            Re: Can Reactive Programming Handle Complexity? Marko Rauhamaa <marko@pacujo.net> - 2014-02-20 23:18 +0200
              Re: Can Reactive Programming Handle Complexity? wjwlsn <ibillwilson@gmail.com> - 2014-02-20 13:40 -0800
              Re: Can Reactive Programming Handle Complexity? dialmove <dialmove@gmail.com> - 2014-02-21 13:56 +0000
            Re: Can Reactive Programming Handle Complexity? dialmove <dialmove@gmail.com> - 2014-02-21 13:34 +0000
              Re: Can Reactive Programming Handle Complexity? Marko Rauhamaa <marko@pacujo.net> - 2014-02-21 16:57 +0200
                Re: Can Reactive Programming Handle Complexity? dialmove <dialmove@gmail.com> - 2014-02-21 15:49 +0000
                  Re: Can Reactive Programming Handle Complexity? Marko Rauhamaa <marko@pacujo.net> - 2014-02-21 18:22 +0200

#549 — Can Reactive Programming Handle Complexity?

Unrepentantly ripped off from the obsolete site:

"A recent article on Reactive Programming 
("http://www.espressologic.com/reactive-programming-database-logic"), 
which suggested that five lines of Reactive could solve a problem that 
required 500 lines using Java or 200 lines using triggers, led many 
readers to question (passionately) whether Reactive enables you to 
address not just typical problems, but complex ones as well. In a 
follow-up column, Espresso Logic CTO Val Huber argues that, while it 
certainly can't solve all use cases, Reactive Programming is very 
capable of addressing many complex problems 
("http://www.espressologic.com/example-of-reactive-programming-vs-conventional-programming-for-database"), 
and can address all other scenarios via a transparent integration with 
procedural languages. He shows how Reactive can handle complexity using 
two different scenarios: a classically complicated database application 
(a bill of materials price rollup) and procedural integration (to 
address external systems such as email and transactions not limited by 
a database update). Take a look at his work; do you agree?"

It seems that every few years somebody tries to re-invent true MVC OOP. 
 "Reactive Programming™" seems to be the attempt being pushed on the 
poor public by Dice Media, no doubt in exchange for a healthy sum.  If 
you look at the examples given, and are familiar with Self or 
Smalltalk, you'll see nothing new, and both languages are now over two 
decades old (Smalltalk is well over three).

What I can't quite figure out is, are Slashdot's promotions of 
"Reactive Programming™" aimed at convincing programmers to move to the 
"Reactive Programming™" platform, or are they aimed at convincing the 
PHBs, or are they just plain old SEO spam?

[toc] | [next] | [standalone]

#619

El Tue, 18 Feb 2014 16:33:28 -0800, Oregonian Haruspex escribió:
> He shows how Reactive can handle complexity using
> two different scenarios: a classically complicated database application
> (a bill of materials price rollup) and procedural integration (to
> address external systems such as email and transactions not limited by a
> database update). Take a look at his work; do you agree?"

The Functional Reactive paradigm provides better abstractions for 
concurrency than imperative code - futures, promises and observables are 
at a way higher level than locks and semaphores, and in many cases allow 
writing complex asynchronous code in a pure functional way. For the rest 
of cases, you can add explicit state and transform it into an agent-based 
model, which is essentially OOP parallelism done right.

So yes, I think a high-level paradigm that simplifies synchronization can 
make complex parallel tasks easier to program and reason about. I've seen 
some examples in action, of which Espresso Logic is an awesome case.



> It seems that every few years somebody tries to re-invent true MVC OOP.
>  "Reactive Programming™" seems to be the attempt being pushed on the
> poor public by Dice Media, no doubt in exchange for a healthy sum.  If
> you look at the examples given, and are familiar with Self or Smalltalk,
> you'll see nothing new, and both languages are now over two decades old
> (Smalltalk is well over three).

That's like saying that building a library for P2P communication is trying 
to reinvent the TCP/IP protocol. Sometimes you need to look your old tools 
from a new perspective to adapt it for new, complex use cases that were 
impossible to tackle from a low level perspective.

Several mayor players as well as multiple small ones are adopting tools 
from FRP for their libraries and software stacks, and knowledge about the 
paradigm is spreading through official documentation, MMO courses and the 
blogosphere - which makes corner cases shallow, and deepens our 
understanding of the paradigm.

[toc] | [prev] | [next] | [standalone]

#621

On Thu, 20 Feb 2014 16:17:36 +0000, dialmove wrote:

> The Functional Reactive paradigm provides better abstractions for
> concurrency than imperative code - futures, promises and observables are
> at a way higher level than locks and semaphores, and in many cases allow
> writing complex asynchronous code in a pure functional way. For the rest
> of cases, you can add explicit state and transform it into an
> agent-based model, which is essentially OOP parallelism done right.
> 
> So yes, I think a high-level paradigm that simplifies synchronization
> can make complex parallel tasks easier to program and reason about. I've
> seen some examples in action, of which Espresso Logic is an awesome
> case.

...aaaaand, this writer at the old site explains it way more eloquently 
that I ever could:

http://developers.slashdot.org/comments.pl?sid=4527735&cid=45625371

[toc] | [prev] | [next] | [standalone]

#629

dialmove <dialmove@gmail.com>:

> ...aaaaand, this writer at the old site explains it way more eloquently 
> that I ever could:
>
> http://developers.slashdot.org/comments.pl?sid=4527735&cid=45625371

I'm sorry, I didn't get anything out of the linked response. The
Wikipedia entry on Reactive programming is very slim as well.


Marko

[toc] | [prev] | [next] | [standalone]

#634

On Thursday, February 20, 2014 1:09:45 PM UTC-5, Marko Rauhamaa wrote:
>> dialmove <dial...@gmail.com>:
>> ...aaaaand, this writer at the old site explains it way more eloquently
>> that I ever could:
>>
>> http://developers.slashdot.org/comments.pl?sid=4527735&cid=45625371
>
> I'm sorry, I didn't get anything out of the linked response. The
> Wikipedia entry on Reactive programming is very slim as well.
>
> Marko

OK, so I read the old /. post, dialmove's explanation, and Wikipedia. It
took a while, but something just clicked for me (I think). My mental model
is of a very complicated spreadsheet with very many interconnected formulae,
a few input cells, and a large number of output cells. A change to any one
input cell might result in changes to a few output cells (lets say 10 to 20%
of them), while the rest remain unchanged. Upon changing any input, recalc
is automatic and immediate, as long as the program (the spreadsheet) is
active.

If I was going to program this in a traditional fashion, I could either 
write it so that upon a change to any single input value, ALL intermediate
formulae are recalculated to ensure all affected output values are updated...
OR I could write a whole bunch of conditionals so that only the dependent
formulae are recalculated and only the dependent outputs are updated. (Or,
I could write some mish-mash combination of these two extremes.)

One of those ways is efficient for the programmer (fewer lines of code) but
potentially much less efficient at runtime (many unnecessary calculations).

The other way is much less efficient for the programmer (many more lines of
code) but potentially much more efficient at runtime (no unnecessary calcs
performed).

So, my idea of a reactive programming language is that I can write a program
by specifying the input and output variables and coding the formulae without
concerning myself about data dependencies. The compilation and runtime system
will automatically handle dependencies to ensure that only the required
recalculations occur, and only the dependent outputs are updated.

Now I can see how such a thing would be useful for a database system... you
could potentially reduce database updates for result values by a significant
amount.

Of course, this all assumes that my mental model is anywhere close to being
correct!

--
Usenet 2014 ... Go nntp!

[toc] | [prev] | [next] | [standalone]

#635

wjwlsn <ibillwilson@gmail.com>:

> My mental model is of a very complicated spreadsheet with very many
> interconnected formulae, a few input cells, and a large number of
> output cells. A change to any one input cell might result in changes
> to a few output cells (lets say 10 to 20% of them), while the rest
> remain unchanged. Upon changing any input, recalc is automatic and
> immediate, as long as the program (the spreadsheet) is active.

Ok, I get spreadsheets, but I have a hard time seeing how I could use
this reactive programming in any of the problems I've encountered during
my lengthy career. I can do functional programming in Python, I can do
object-oriented programming in C, I can do logic programming in Lisp,
but how would I do reactive programming in any programming language?

Let's say I have a network device driver and I need to maintain two
simple statistics: a packet count and a byte count. I might naively
write something like this (in C):

   void update_traffic_stats(struct stats *stats, size_t pktlen)
   {
       stats->packet_count++;
       stats->byte_count += pktlen;
   }

What would be the "reactive" way of maintaining those statistics?


Marko

[toc] | [prev] | [next] | [standalone]

#636

On Thursday, February 20, 2014 4:18:10 PM UTC-5, Marko Rauhamaa wrote:
> Let's say I have a network device driver and I need to maintain two
> simple statistics: a packet count and a byte count. I might naively
> write something like this (in C):
>
> void update_traffic_stats(struct stats *stats, size_t pktlen)
> {
>   stats->packet_count++;
>   stats->byte_count += pktlen;
> }
>
> What would be the "reactive" way of maintaining those statistics?
>
> Marko

Hmmm... I think you would put a "watch" on the struct variable
that's updated when a new packet comes in. The watch would define
that the stats struct variable is dependent on the packet struct
variable, and point to whatever code is needed to update stats.
You won't call update_traffic_stats directly; the watch will do
that when a new packet comes in.

I think.

:)

--
Usenet 2014 ... Go nntp!

[toc] | [prev] | [next] | [standalone]

#658

On Thu, 20 Feb 2014 23:18:10 +0200, Marko Rauhamaa wrote:

> Ok, I get spreadsheets, but I have a hard time seeing how I could use
> this reactive programming in any of the problems I've encountered during
> my lengthy career. I can do functional programming in Python, I can do
> object-oriented programming in C, I can do logic programming in Lisp,
> but how would I do reactive programming in any programming language?

The programming language has to provide capabilities for it, that's for 
sure. Those can be native to the language, or can be added as a library; 
there are several popular reactive libraries for Javascript for example, 
and of course Microsoft is pushing the paradigm forward with its Rx 
extensions.


This morning I've found this list of examples in the Elm language that 
shows how it's done. Elm is a FRP language that compiles to HTML+CSS
+Javascript:

http://elm-lang.org/Examples.elm


The Intermediate section include a bunch of examples that are considered 
quite difficult in classic functional programming (video games & 
animation), where the global program state needs to be passed around in 
parameters as a whole, but FRP seem to handle those very well.

The Mario and Turtle examples follow a rather intuitive structure - 
declare your variable structures, then define your methods for behavior 
(with a very imperative flavor to them - first find the image for the 
canvas, then rotate it, then place it on its coordinates), and connect the 
input devices to the model and logic.


The trick to understand the different paradigm is to think that behaviors 
don't really destroy the previous value of their variables. You can use a 
variable as a scalar (e.g. "the x and y coordinates *right now*") but it's 
often useful to think of them as collections.

Each variable contains an infinite vector with all values that the 
variable takes during runtime, generated from one stream or a combination 
of them ("all mouse clicks or keyboard presses"). You can access past and 
future values or apply filters over the whole collection, and the 
efficient language VM will make sure that the current value of the stream 
is propagated to whatever part of the program where it's used (such as 
updating the screen). 

Expressing the same on an imperative language would require building a 
state machine that keeps track of what has happened before in the sequence 
of values for the variable, or storing and updating in memory the 
collection of previous values by yourself. With FRP, those common 
programming aspects are encapsulated and easy to express in the language. 

Of course, the glue code will take a totally different shape; in the 
imperative paradigm, you think of the language virtual machines as very 
complex state machines that consume their inputs, update their internal 
state by destroying the previous one, and spit new generated values by 
changing the state of the output devices (i.e. changing the world itself). 
Glue code takes the form of sequential subroutines that give orders to the 
machine as to what state to go next, and the pass of time is represented 
by moving to the next instruction in the sequence.

Instead, in dataflow languages, the virtual machine is a spreadsheet-like 
environment that allows you to draw directed graphs that connect 
components together (in a spreadsheet you can see the graph by drawing 
arrows between each cell and all its dependencies). Program execution 
consists of connecting the inputs to a data storage and waiting for the 
results to appear at the opposite side, and the pass of time is 
represented by accessing values that are positioned later and later in 
their streams.

[toc] | [prev] | [next] | [standalone]

#656

On Thu, 20 Feb 2014 13:05:27 -0800, wjwlsn wrote:

> OK, so I read the old /. post, dialmove's explanation, and Wikipedia. It
> took a while, but something just clicked for me (I think). My mental
> model is of a very complicated spreadsheet with very many interconnected
> formulae, a few input cells, and a large number of output cells. A
> change to any one input cell might result in changes to a few output
> cells (lets say 10 to 20% of them), while the rest remain unchanged.
> Upon changing any input, recalc is automatic and immediate, as long as
> the program (the spreadsheet) is active.

I have arrived to the following intuitive way to explain why FRP and other 
dataflow paradigms matter.

Think first of GUIs and object-orientation: class inheritance allows you 
to create a hierarchy of widgets with more and more specialized behavior, 
and reuse the parts of code that handle common tasks like layout and 
redrawing. Now, you could think of this as a very complicated way to draw 
squares on a canvas, or realize that it breaks the GUI problem in 1) 
simple encapsulated components plus 2) simple glue code to bind them.

Now, I've come to believe that dataflow programming allows you to do the 
same thing with distributed computations and asynchronous communication. 
Change in the system is seen as a stream of values instead of a chain of 
events, which allows you to create compositions and transformations of 
events - in the same way that you would build complex dialogs by composing 
and transforming widgets in a GUI.

This complex composition of events is very difficult to reason about in 
imperative code, but in FRP it takes the form of reusable "behavior 
components" that are programmed in a simple way (clasic stateless 
functional composition). The hardwired inputs and outputs between static 
widgets and/or dynamic behaviors are automatically updated by the language 
runtime in the lazy, efficient way you describe above.

Does all of this makes any sense?



> One of those ways is efficient for the programmer (fewer lines of code)
> but potentially much less efficient at runtime (many unnecessary
> calculations).
> 
> The other way is much less efficient for the programmer (many more lines
> of code) but potentially much more efficient at runtime (no unnecessary
> calcs performed).

The best thing is that, thanks to recent theoretical advances (combined 
with the ever increasing computational power of new hardware), FRP doesn't 
seem to be that inefficient any more; and I think that's why it's 
beginning to spread everywhere you look at. Though it will require a lot 
of training on old-school developers, that's for sure!

[toc] | [prev] | [next] | [standalone]

#663

dialmove <dialmove@gmail.com>:

> Though it will require a lot of training on old-school developers,
> that's for sure!

Since nobody here seems to know enough about Reactive Programming to
train us old-school developers, I had to do some digging around. Here's
a PDF tutorial:

  <URL: http://go.microsoft.com/fwlink/?LinkId=208528>

It seems to be a big name for the concept of event streaming.
Specifically, C#/.NET has added standard classes to convert callbacks to
event streams.

Streaming definitely has its place and has been in active use for
decades (half-duplex Unix pipelines, full-duplex Sys V network drivers
etc), but it's a smallish paradigm that hardly merits the term "Reactive
Programming." It's nowhere near the fundamental nature of Logic
Programming or even threads.

By all means, use streams wherever applicable. Note, however, that when
addressing events, the devil is in the state machine. Very few things in
the event world are so streamlined that you can deal with them with a
nifty pipeline. And in the few cases where such a half-duplex pipeline
can be set up, a classic, prosaic *wrapper class* can manage your
reactive programming needs without special support from the standard
library.

Based on the Microsoft tutorial, it seems the main push behind Reactive
Programming is to get Visual Studio users to use a fancy new set of
classes available and maybe buy an upgrade while they are at that.


Marko

[toc] | [prev] | [next] | [standalone]

#667

On Fri, 21 Feb 2014 16:57:28 +0200, Marko Rauhamaa wrote:


> It seems to be a big name for the concept of event streaming.
> Specifically, C#/.NET has added standard classes to convert callbacks to
> event streams.
> 
> Streaming definitely has its place and has been in active use for
> decades (half-duplex Unix pipelines, full-duplex Sys V network drivers
> etc), but it's a smallish paradigm that hardly merits the term "Reactive
> Programming." It's nowhere near the fundamental nature of Logic
> Programming or even threads.

It depends on perspective. If you see it as a tacked-down library on top 
of an interactive language, of course it's smallish. But from that 
perspective, OOP is just high-order functional programming plus cells, so 
it's not a big thing either, true? It's "just a big name for the concept 
of state".

The thing with programming paradigms is not that they introduce whole new 
constructs, it's the way they make you think about them. Any paradigm will 
reuse the same old basic elements available in all previous programming 
styles, and explain them in a new way that makes sense.


> 
> By all means, use streams wherever applicable. Note, however, that when
> addressing events, the devil is in the state machine. Very few things in
> the event world are so streamlined that you can deal with them with a
> nifty pipeline. And in the few cases where such a half-duplex pipeline
> can be set up, a classic, prosaic *wrapper class* can manage your
> reactive programming needs without special support from the standard
> library.

But that's the thing, if you program in RFP you *don't have* state 
machines, it's streams all the way down. If you study RFP as a 
mathematical way to describe program execution, like PL people do, it's a 
whole new theoretical construct, with its own set of axioms. The formal 
semantics of an RFP language will look different than those from an 
imperative language, which look different than those from a functional or 
a logic language. 

Surely you *can* think of the language ''implementation'' as being a 
program written in a different, old paradigm (which negates any benefit 
provided by the new, btw), but you don't *need* to; any Turing-complete 
language can be transformed into any other. I've seen languages for 
programming micro-controllers based on the lambda calculus, with no notion 
of state, if that's your thing.

You wouldn't use dataflow programming for calculating a concurrent 
constraint satisfaction problem (or maybe you would, but it will be more 
difficult), as you wouldn't use a constraint logic solver to program a 
platformer game. When you find a paradigm that is well suited to the 
problem to solve, there's little use in thinking of how that paradigm 
translates into a different, less adequate paradigm.


See [1] for how different paradigms relate to each other (taken from [2]); 
the difference from one paradigm to another is always minimal, but they 
make the language feel completely different.


[1] http://en.wikipedia.org/wiki/File:Programming_paradigms.svg
[2]  "Programming Paradigms for Dummies: What Every Programmer Should 
Know", http://www.info.ucl.ac.be/~pvr/VanRoyChapter.pdf

[toc] | [prev] | [next] | [standalone]

#669

dialmove <dialmove@gmail.com>:

> But that's the thing, if you program in RFP you *don't have* state
> machines, it's streams all the way down.

I challenge you to write the tiniest RFC 821 SMTP server in such a way.

Note that Microsoft's tutorial mentioned the example of a throttling
filter (a perfectly sound streaming example). You won't be able to
implement that throttling filter without a state machine.

> When you find a paradigm that is well suited to the problem to solve,
> there's little use in thinking of how that paradigm translates into a
> different, less adequate paradigm.

Streaming as a *technique* is the perfect tool in many situations. I use
it all the time.

I understand state machines quickly get bewilderingly complicated.
However, don't sweep them under the rug because you are only going to
get into bigger trouble that way (I've learned my lessons). Instead, I
recommend putting The State on a pedestal in your classes. That gives it
the right visibility and you a chance to get it right.


Marko

[toc] | [prev] | [standalone]

Back to top | Article view | comp.misc

csiph-web