What have you found for these years?

2012-06-12

rest-core 2.0 roadmap, thunk based response

So I am going to make a few incompatible changes in rest-core-2.0 branch,
in order to cleanup some internal implementation and introduce a new mechanism
to handle multiple requests concurrently, avoiding needless block.

Before we go into detail, here's who can upgrade without changing anything,
and who should make a few adjustments in their code:

* If you're only using rest-more, e.g. RC::Facebook or RC::Twitter, etc.,
you don't have to change anything. This won't affect rest-more users.

* If you're only using rest-core's built in middlewares to build your own clients,
you don't have to change anything as well. All the hard works are done by me.

* If you're building your own middlewares, then you are the ones who need to
make changes. RC::ASYNC would be removed, and you're supposed to read
the response from the block you provide to the app.call(env){ |response| },
instead of reading the response directly from calling response = app.call(env).

This change is for simplifying the internal structure of middlewares which need
to handle both synchronous and asynchronous requests. Previously, I put this check:
if env[ASYNC] to determine if it's asynchronous or not, and I never thought
it's a good idea to do that... Finally I decided to unify it by forcing middleware
writers to provide their callback block.

This is decided because the new EmHttpRequestThunk HTTP client is synchronous
on the client code, but asynchronous internally, unlike EmHttpRequestFiber is also
synchronous internally, for middlewares.

* Let me know if you're using cool.io. Otherwise, I am going to remove it in 2.0.

That's it! And Let's see the detail:



Previously, this would block *twice* using the fiber approach:
(e.g. with EmHttpRequestFiber)

facebook = RC::Facebook.new
id0 = facebook.get(4)['id']
id1 = facebook.get(5)['id']
puts id0.to_i + id1.to_i # 9

In rest-core 2.0, the new mechanism is called thunk-based, EmHttpRequestThunk,
which would only block *once* in the above code, assuming two requests cost
the same time. The idea was from Favonia.

In rest-core 1.0, the only way to avoid two blocks is using EmHttpRequestAsync
with callback based client codes along with a fiber:
facebook = RC::Facebook.new
ids = []
f = Fiber.current
Fiber.new{
  facebook.get(4){ |data| ids << data['id']; f.resume if ids.size == 2 }.
           get(5){ |data| ids << data['id']; f.resume if ids.size == 2 }
}.resume
Fiber.yield
puts ids.map(&:to_i).inject(&:+) # 9

Which is quite ugly and mechanical, requiring a lot of boilerplate codes.
You would need to do similar things for other computations as well,
thus I did write sync-defer for those kinds of computations, wrapping
threads and call Fiber.yield whenever all computations are done.
I haven't done this for rest-core, because I didn't get an idea for the
interface. Now I guess thunk based implementation would be the best,
maybe I should also write this for sync-defer, but only after rest-core 2.0
is done I guess.

And I would like to take this opportunity to make all those incompatible
changes I have in my mind. Here's the list:

* Remove cool.io support. I don't think anyone is really using it.
Please let me know if you're using it.

* Make RC::Auto be EmHttpRequestAsync and EmHttpRequestThunk based,
instead of EmHttpRequestAsync and EmHttpRequestFiber based.

* Remove RC::ASYNC approach, and force everyone pass a
callback block, unifying synchronous and asynchronous protocol.
This is somehow similar to CPS, Continuous Passing Style. We all know
that callbacks are hard to use, but I guess it's acceptable here because
that's the only callback we would be dealing with in a middleware.

That's probably all of them right now in my mind. I'll talk more
implementation detail after above changes are mostly done.
You can take a look at thunk branch for the current status, or
em-http-request-thunk.rb and response_thunk.rb directly.

0 retries:

Post a Comment

Note: Only a member of this blog may post a comment.



All texts are licensed under CC Attribution 3.0