[ANN] rest-core 3.0.0 and rest-more 3.0.0 just released

It's been a long while! Roughly a year. rest-more itself does not really change too much,
as you can see from the CHANGES.md:

* Adopted latest rest-core
* Added RC::Instagram, thanks @khoan
* Added RC::Firebase

P.S. I believe RC::Firebase might be the best Firebase client in Ruby currently.

However, it's really a hard work to make rest-core advanced to 3.0.0, here's the
CHANGES.md:

Highlights:

* Hijack for streaming responses
* EventSource for SSE (server-sent events)
* Thread pool
* Keep-alive connections from httpclient

See the above CHANGES.md for the complete list, here we only list the highlights.
Hijack and EventSource are roughly the same work. As long as we have streaming
support, it's fairly easy to get EventSource work.

Switching from rest-client to httpclient did not really take long, but
survey and experiments did take some time. At first I was trying net-http-persistent
because I trust the author, and think it's merely a thin layer on top of net/http,
so it should be fast and reliable. However, it turns out it's actually... quite slow.
rest-core's test suites were running 20% slower when switched to net-http-persistent
from rest-client. This is... not very desirable. Not to mention I really spend so
much time figuring out how to use net-http-persistent correctly. net/http family
really has a bad taste of API, feeling like Java.

So why I want to switch? rest-client is still quite good, though it needs some patches.
It does not really support persistent connections, which could definitely boost the
performance if we're hitting the API server quite frequently. That's exactly why I
tried net-http-persistent in the first place. It's claiming it's thread-safe, too.

But do I have other alternatives? I don't really want to implement this if there's one.
Then I found this post again: ruby HTTP client performance shootout redux
I believe I had read it before, but I totally forgot this.

It turns out httpclient performs quite well, therefore I want to give it a try.
The code is actually quite complicated comparing to rest-client and net-http-persistent,
so I doubt if that would really run faster. However indeed it's only roughly a bit slower
than rest-client, running rest-core's test suites.

And since httpclient does support persistent connections, and only a bit slower than
rest-client given the simplest case. It's definitely the way to go. Also, I could easily
grab the socket underneath to implement hijacking for streaming response and EventSource.
Which... I am too lazy to implement for rest-client and net-http-persistent. I would
probably drop support for rest-client and net-http-persistent in the future.

* Thread pool

The hardest part is of course in implementing the thread pool. Oh well, no, timeout is
the hardest part, but I'll talk about this later. I spent quite some time finding an
existing implementation, just to realize if I want to make it the best, I needed to do
on my own, i.e. a customized thread pool, not a general purpose one.

The reason I began looking in this was a ticket from Github.
#7 Threads Error on Heroku, How to Limit Threads?

Apparently a thread pool was what he needed... then I looked into several thread pool
implementation, finally taking puma's thread pool implementation as a reference
implementation, because it's a quite simple and complete one.

It's actually quite an interesting process to implement it. At first, of course I
tried to use the built-in ::Queue to do this. However, the built-in ::Queue does not
support timing out on the pop operation... Therefore I need to roll my own Queue,
which would pop a lambda{false} upon timing out to tell the worker that
it has been idled for too long, and it should shut itself down. Otherwise the task
would return a true to tell the worker keep working.

I can't make it that simple if I picked a general purpose thread pool.

* Timing out

Then, I realized that Thread#kill or Thread#raise is not safe.
There's no safe way to interrupt a thread, because we might kill or raise an exception
at the point target thread is running inside an ensure block, risking it from not
releasing the resource the thread was grabbing.

Checkout Charles Nutter's post for the detail:
Ruby's Thread#raise, Thread#kill, timeout.rb, and net/protocol.rb libraries are broken

Although killing a new thread as before might be ok if we're not using a thread pool,
since the newly spawned thread might not grab any external or shared resources anyway.
But nah, I don't really want to make it so complicated though. It's already quite
complicated, and still we're risking even we're spawning new threads.

I guess this is why I can't make this run reliably at first. I almost wanted to give up
timing it out, only to cancel the task if it hasn't been running. However it might be
fine if we're spawning a new thread each time, but it would be much more problematic
if we're blocking the threads in the pool, blocking for other tasks from running.

Therefore, we still need timing it out somehow without killing it or raising some asynchronous
exceptions. This is probably why Eric Wong always emphasizes that we need to use the timeout
facilities provided in the client libraries, instead of using an outsider to handle timeout.

However, all HTTP clients I was trying, didn't provide an overall timeout time, including
net/http, net-http-persistent and httpclient. They have open_timeout, read_timeout, and
various other timeouts... It makes sense to separate them, but what I really want is a
total timeout, since I don't care where it blocks.

For now, I just treat all timeout time as the total timeout time, setting both open_timeout
and read_timeout to the timeout I want. In the worst case, it means it could spend twice more
time than expected. But I guess since Thread#kill and Thread#raise can't be safe, it might
be the only hope we could rely on.

I stopped working on this for a while because I found no way to fix this at that moment,
feeling a bit frustrated that why Ruby can't do this right. I guess this is because it's
all depending on how we schedule the threads. If we can't give some hints to the scheduling,
then we can't make sure the critical sections would have never been interrupted.

I believe Haskell did this right, with their own scheduling.
Checkout: Cancellation and Timeouts: Masking Asynchronous Exceptions
Maybe all programming languages runtime should have their own scheduling to solve this.
i.e. Provide maskAsyncExceptions and unmaskAsyncExceptions primitives in order to implement
the mask or bracket function.

At last, I just picked Tony Arcieri's timers, hoping it would do it right, and I stopped
raising or killing the threads, simply set the flag to tell it stop working if you haven't
started. If you started already, fine, but I would ignore your work, and rely on timeout
from the HTTP clients to stop blocking the thread.

We have no way but desperately to depend on the libraries to do the right thing, unless we
could fix this scheduling issues in Ruby, providing those masking facilities.