What have you found for these years?

Showing posts with label 程式. Show all posts
Showing posts with label 程式. Show all posts

2018-07-06

Let's Encrypt for *.godfat.org

This is a note to myself so that if my server breaks I could easier recover it.

* Use dehydrated (renamed from letsencrypt.sh)
* Use letsencrypt-manual-hook
* Set domains.txt as godfat.org *.godfat.org > godfat.org
* Run: ./dehydrated --cron --challenge dns-01 --hook 'hooks/manual/manual_hook.rb'
* Go to my DNS provider and setup TXT records manually. Twice Two TXT records in the same time.
* sudo cp certs/godfat.org/fullchain.pem /etc/nginx/ssl.crt
* sudo cp certs/godfat.org/privkey.pem /etc/nginx/ssl.key
* sudo systemctl restart nginx

2018-03-16

Backup Google Authenticator secrets

So I finally rooted my new Pixel 2, transferring over my data,
game saves, and setting up everything including Google Authenticator.
Now the most important step! The very reason why I want to root
my Android, to backup Google Authenticator secrets...

What an awful idea to make it only able to backup with a rooted device.

Here are the steps:

# Enable Developer options on the phone
# Enable USB debugging

# Connect the phone with the computer
$ /usr/local/share/android-sdk/platform-tools/adb shell

walleye:/ $ su
# Grant superuser privilege from the phone screen
# If it's not popping up, and it shows permission denied,
# then go to Magisk app which we used to root, and find
# the option to grant superuser privilege to Shell manually
walleye:/ # cp /data/data/com.google.android.apps.authenticator2/databases/databases /sdcard/Download

# Ctrl-D to leave phone console

$ /usr/local/share/android-sdk/platform-tools/adb pull /sdcard/Download/databases .
$ sqlite3 databases
sqlite> select * from accounts;

# You should see all your secrets!

Of course, remove /sdcard/Download/databases on the phone afterward.
Keep that sqlite3 database in a safe place. (or just copy the secrets and remove it)

References:

* Manually Extract Your Credentials [Root Only]

2018-01-18

How to X with/in programming [languages]

這系列真的都很好笑,而且都很酸 XD 不過很多都要有點經驗才看得懂。

How to Save the Princess in 8 Programming Languages
Lisp 和 PHP 讓我笑最多。

How to Kill the Dragon with 9 Programming Languages
Haskell 超好笑 XD Swift 也不錯。

How To Build A Horse With Programming
還是 PHP 讓我噴笑 XDD

If The World Was Created By A Programmer
酸斃了 XD neck.js!

2016-09-08

Finally fixed my iTerm2

Updated: I just realized that the offending config is exactly
setting the tab theme to Light (High Contrast).
Apparently this is broken for a very, very long time...
So I have no other choice but just set it to Dark (High Contrast).
I do not like dark tab theme but without "High Contrast" the gradient
colors are driving me crazy because it's just unreadable.



This seriously drove me crazy, and it's been for so long.
Probably a few months ago, every time after I upgraded iTerm2,
it would be broken. It would just keep opening blank unusable
windows. I doubted that has something to do with session
restoration, because it worked a few times if I just download
a fresh new version.

But not this time... and seriously I don't want to reconfigure
it at all, because I have a ton of customization which I don't
quite remember and I don't want to go over with it again!

So I tried to load my backup settings for iTerm2, but it turned
out that if I just restored it, the problem would persist. However,
of course not all the settings were the offenders! If I tried to
restore settings piece by piece, it would actually work.

Finally I found the offenders were below. Just remove them and
iTerm2 could work properly. I think it's probably some settings
which in a newer version they're removed, but it's not handled
properly therefore would cause issues.

Anyway, I am so glad this is fixable.

<key>TabStyle</key>
<integer>2</integer>
<key>TabViewType</key>
<integer>0</integer>

Also note that the config was located at:
~/Library/Preferences/com.googlecode.iterm2.plist
Apparently I still lose some settings, e.g. tab title is barely
readable, and I am not sure if that could be fixed. But at least
it's working now.

2016-02-25

Finally got letsencrypt done

Finally got letsencrypt done.

I don't quite understand why the official scripts were so complicated,
but finally I got it done with letsencrypt.sh (Thanks Ash for
recommending it) along with some scripts from letsencrypt-nosudo.

This is a note for what I've done in case I need to do this again.
First let's clone it first:

git clone git@github.com:lukas2511/letsencrypt.sh.git
cd letsencrypt.sh

Then let's generate a private key and a signing request:
(This was from letsencrypt-nosudo)
openssl genrsa 4096 > domain.key
openssl req -new -sha256 -key domain.key -subj "/" -reqexts SAN -config <(cat /etc/ssl/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS:example.com,DNS:www.example.com")) > domain.csr
Note that the second command cannot run under Fish,
and I would like to know if there's any way to create a wildcard certificate?
I tried to use *.godfat.org however it seems letencrypt doesn't allow it.

Let's setup the other configs:
echo example.com www.example.com > domains.txt
mkdir .acme-challenges

Finally setup Nginx for acme-challenge:
(Note that letsencrypt.sh would write files into it and clear it after done)
# /etc/nginx/nginx.conf
location /.well-known/acme-challenge {
  alias /home/example/letsencrypt.sh/.acme-challenges;
}

Begin hitting letencrypt:
./letsencrypt.sh --signcsr domain.csr

You'll get the certificate printed on stdout. Not sure if there's a way to
output it to a file, but anyway you could just copy it and put it in a file.
I'll assume you put it in domain.crt and we could continue
setting up Nginx:
mkdir dist
# Download the other certificate from letsencrypt (source)
wget https://letsencrypt.org/certs/lets-encrypt-x1-cross-signed.pem
# Bundle it for the final certificate:
cat domain.crt lets-encrypt-x1-cross-signed.pem > dist/ssl.crt
# Setting up the final files
cp domain.key dist/ssl.key
chmod 600 dist/ssl.key
chmod 644 dist/ssl.crt
sudo chown root:root dist/ssl.*
sudo cp dist/ssl.* /etc/nginx/

And here's the relevant Nginx config:
# /etc/nginx/nginx.conf
server {
  listen 80;
  listen 443 ssl http2;
  ssl_certificate ssl.crt;
  ssl_certificate_key ssl.key;
  ssl_session_timeout 5m;
  ssl_prefer_server_ciphers on;
  ssl_protocols TLSv1.2;
  ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
}

After restarting Nginx, you could verify it with ssltest.

P.S. Thanks to StartSSL started involving something with China
pushing me forward this. I was so lazy figuring out how it works
before. The world deserves free/cheap secure connections for
independent developers/organizations.

2016-02-16

Whenever Sublime Text updates...

I'll need to do:

cd '/Applications/Sublime Text 3.app/Contents/MacOS/Packages'
ln -s Ruby.sublime-package Ruby.sublime-package.zip
# Find:
# Snippets/require-..-(req).sublime-snippet
# Change " to '

# Find:
# Ruby.sublime-syntax
# Search:
# require
# Remove `push:` clause

ln -s 'Color Scheme - Default.sublime-package' 'Color Scheme - Default.sublime-package.zip'
# Find:
# Twilight.tmTheme
# Add color scheme for GitGutter
# Add just before <plist><dict></array>
# Change color:
# deleted -> #FF0000
# inserted -> #00FF00
# changed -> #FFFF00

ln -s 'Default.sublime-package' 'Default.sublime-package.zip'
# Find:
# block.py
# Find:
# def indented_block(view, r):
# Insert:
#         return False
# To avoid inserting } in the wrong block

2016-01-19

A few thoughts on "nil Is Not NULL, and Other Tales"

A few thoughts on: nil Is Not NULL, and Other Tales.

nil is an Object, NULL is a memory pointer
[...]
To be absolutely clear, a segmentation fault and a runtime exception are radically different things. But the consequence, your program aborts, makes them seem quite similar.
Yes, I agree. However:
and if its "nil" doesn't support that, something must be done. This characteristic of a statically-typed language makes nil seem dangerous.
No, it's not. It's called NullPointerException in Java, it's still just an exception. This is not the characteristic of a statically-typed language. They could be the same in both statically/dynamically typed language. My point is that nils are not superior in a dynamically typed language.

nil is mathematically realistic
[...]
A value like nil signals that "no mapping is available for that input". It's a very useful concept.
The concept is useful but are we talking about the concept or nil in Ruby in particular? You're talking about null set here but it's not the nil in Ruby. The nil in Ruby represents "nothing" not "null set".

nil is mistreated in Ruby
[...]
In the first case, developers tend to see nil as an error, and hence, it is quite unwelcomed. "Oh look, that method returned a nil, something must be wrong!"

No, not at all! Nothing need be wrong. The method returned a nil to say, "no value here!"
This really depends on the API. If an API claimed that it should never return nil, then it should never return nil and if nil was returned, surely something must be wrong! On the other hand, of course we could design an API which could potentially return nil. In Haskell, this would be called Maybe. A value of Maybe would either be Nothing (nil in Ruby) or Just something else. In this case, indeed nothing need to be wrong. It's all depending on the specification. The question is, could the specification really enforce the requirements/constraints?
In Rubinius, this is such a problem that we had to introduce a special value we called "undefined" to be able to implement the Ruby core library in Ruby. The problem is that some methods take nil as a value, but also have a default value. So, we were unable to distinguish between some_method() and some_method(nil).
Indeed this is really annoying. The concept of undefined was there but Ruby doesn't have a value for it. I often wrote Undefined = Object.new for default arguments. I hate that JavaScript has null and undefined, but seriously sometimes we do want "undefined" in Ruby. Hash is another example which has this difference.
h = {}
h[0] = nil
h[0] # => nil
h[1] # => nil
h.has_key?(0) # => true
h.has_key?(1) # => false
h.keys # => [0]
h.values # => [nil]

The "lonely operator" is an unnecessary mistake
The "lonely operator" supposedly solves the problem of calling a method on nil that then raises a runtime exception causing the program to abort.

Unfortunately, it only partially solves this problem
I totally agree that this is a mistake and it does not partially solve the problem, because it does not solve the problem at all. It's merely making it easier for us to write sloppy codes which could hinder real bugs, encouraging the use of unstructured data (i.e. hash tables) over the places, spreading nils around the world.

It's somehow a curse, from the `try` method from Rails, from people abusing JSON and hash tables, using them like JavaScript objects without classes.

It's backward.

Not to mention a special syntax for it. Ruby's syntax is already horribly complex.

nil is A Good Thing™
Well, I can't say it's a good thing or bad thing. We need the concept, but not necessarily the current nil. The problem is that, there's no way to enforce an API that would never return nil. Granted, actually this is the same issue lacking static types. It's just that nil is the most possible value which could sneak in because THERE ARE SO MANY METHODS could return nil. If we made that most of the API, the objects and methods from standard library would seldom return nil, then we are probably saved.

For example, h[0] could give me HashMissingValue instead of nil, and other methods could return other similar values, and the point is, they should be distinguishable.

Yes, and that's actually very similar to that Traceable nils in Rubinius which mentioned in the bottom of the post. That, yes, could probably solve the issue.

The simple alternative to the "lonely operator"
Defining method_missing on nil can't be the alternative to that lonely operator, unless we're using "refinements" to localize the changes. I am not saying refinements are good, but I think we all agreed that global behaviour modification is 99% bad, unless we're talking about a small script.

Ruby developers need to see where nil is
[...]
This problem of knowing where a value comes from is much bigger than nil. It is a result of the fundamental tradeoff that a late-bound language (usually called a dynamically-typed language) makes relative to an eagerly-bound language (usually called a statically-typed language).
Yes, exactly, though even a statically typed language could still be plagued by nil. See NullPointerException again. Some talked about Option type (again, Maybe in Haskell) could be the answer, for example Scala does return this for some of the methods. However, I don't feel this solve all the problems because there are still nulls somewhere else. Guess what, Option value could also be null. So actually there are 3 possible values: Some something, None, and our beloved null.

Well.
Late binding provides a malleable system that easily manages the complexity of high-uncertainty contexts. Objects can interact with other objects that provide certain behaviors. They do not have to be specific kinds of objects. The lessening of the constraints that the developer's assumptions impose on the system can increase the utility and resilience of the system.
This is certainly true though. While now I strongly believe in static typing much more, this is the reason why I still cannot give up Ruby and still prefer Ruby in some cases. Playing around it, interact with it. That's sometimes much more pleasant, until, I need to scale the complexity...

Traceable nils in Rubinius
[...]
In the past, we have only ever used that precise value. In other words, all the other bits are zero. But nothing requires this, and those other bits don't need to be wasted. On 64bit architectures, this gives Rubinius approximately 259 values of "nil". That's more than enough for a typical Rails app, I'm sure.
I was like, what the hell was this? when I read this paragraph. But to think about it, indeed we could use this trick to provide insights about where the nil was generated from. I highly hope this would work well and debugging nils could be much easier, and then, perhaps we could really afford returning nil everywhere. I don't know, but maybe.

Cheers,

2015-07-28

Using PostgreSQL JSONB with Rails 4.1

So we just upgraded PostgreSQL to 9.4 and I was so excited to migrate to
JSONB in order to use various querying functions which don't have
equivalents for JSON.

After migration was written, all API tests were passed, then I realized
that this won't work for Rails 4.1 because it didn't recognize JSONB.
(We run API tests first, then Rails tests)

Doh. We can't upgrade Rails because we're locked at an older version of
devise (3.0.4 to be specific), which won't work with Rails 4.2.

Then I finally made my mind to remove devise, rewrite everything, and
upgrade Rails. This was a very hard decision because then I need to figure
out the black magic inside devise, which would be a very painful process.

Fortunately, @wildjcrt sent me a quote from @boriscy:

One note, Rails 4.1 supports json but not jsonb, you need to add

serialize :col, JSON

to make work jsonb in rails 4.1
p.s. I was too lazy to search around because I am so tired of fixing black magic
and I just want to rewrite everything to make things under control.

That's not a full solution but an hint that it's not that broken,
and it could probably be easily patched to work. After some time
playing around, I was able to make JSONB work for Rails 4.1 without
touching all the models we have. Because it would also be painful to
edit all the models using JSONB.

This is the patch to make JSONB work with Rails 4.1.

Basically, register the OID for jsonb and after all the models were
loaded, walk through all the columns which data type is jsonb and
default to '{}::jsonb' (because we only use it this way and the others
could come from que which I shouldn't be patching with)

Sequel was used here because we have it and it works great. If you
want to work with ActiveRecord to fight with this, go ahead, I think
it should also work.

After this patch was applied, all tests passed.

Here's the other thread on twitter.

2014-12-29

fish like up/down history search for readline (or bash)

I am not sure if everyone already knows this,
but I just hardly found this on the Internet.
Seriously, this should really be the default!

finally!! fish like up/down arrow for readline!

Here's the .inputrc you would use:

"\e[A": history-substring-search-backward
"\e[B": history-substring-search-forward

I don't care about bash because I am using fish,
but for things like rib or other interactive shells
are all using readline. This should give them
the power of full-text search for history.

Fish still did a better job to highlight the matched
text, but it's probably good enough for now.

2014-12-27

ThreadGroup local variables

Just a copy from: ThreadGroup local variables

Here's the story. I wrote a testing framework which could run test
cases in parallel. To accumulate the number of assertions, I could
just use a shared number and lock it for each testing threads.

However, I would also like to detect if a single test case didn't
make any assertions, and raise an error. That means I can't just
lock the number, otherwise the thread won't have any idea if the
number was touched by the other threads. That means I need to lock
around each test cases, which defeats the purpose of running test
cases in parallel.

Then we could try to store the number inside the instance of the
running worker, something like this:

    def expect obj
      Expect.new(obj) do
        @assertions += 1
      end
    end

    would 'test 1 == 1' do
      @assertions = 0
      expect(1).eq 1
    end
This works fine, but what if we want to make the other matcher,
such as `Kernel#should`, which has no idea about the worker?
    would 'test 1 == 1' do
      @assertions = 0
      1.should.eq 1
    end
Here 1 has absolutely no idea about the worker, how could it increment
the number then? We could try to use a thread local to accumulate the
assertions, and after all threads are done, accumulate all the numbers
from each threads. This way each numbers won't be interfering with each
other, and each objects could have the access to the corresponding
number from the running thread local.

However this has an issue. What if a test case would spawn several
threads in a worker thread? Those threads would have no access to
the worker thread local variable! Shown as below:
    would 'test 1 == 1' do
      @assertions = 0
      Thread.new do
        1.should.eq 1
      end.join
    end
ThreadGroup to the rescue. Since a newly spawn thread would share the
same group from the parent thread, we could create a thread group for
each worker thread, and all objects should just find the corresponding
number by checking the thread group local. It should be protected by
a mutex, of course. Here's a demonstration:
    module Kernel
      def should
        Should.new(self) do
          Thread.current.group.synchronize do |group|
            group[:assertions] += 1
          end
          # P.S. in the real code, it's a thread-safe Stat object
        end
      end
    end
Some alternative solutions:

* Just use instance_variable_set and instance_variable_get on ThreadGroup
* What I was doing before: Assume ThreadGroup#list.first is the owner of
the group, thus the worker thread, and use that thread to store the number.
Something like:
Thread.current.group.list.first[:assertions] += 1
This works for Ruby 1.9, 2.0, 2.1, but not for 2.2.
This also works for Rubinius. I thought this is somehow an expected behaviour,
therefore did a patch for JRuby to make this work:
https://github.com/jruby/jruby/pull/2221
Until now it failed on Ruby 2.2, did I know the order was not preserved...

* What I am doing right now: Find the worker thread through the list from the
group by checking the existence of the data from thread locals. Like:
Thread.current.group.list.find{ |t| t[:assertions] }[:assertions] += 1
At any rate, if we ever have thread group locals, the order won't be an issue,
at least for this use case.

Any idea?

2014-09-10

Authorizing with rest-more (actually, random thoughts)

Before talking about authorizing with rest-more, let's talk about
omniauth first. I appreciate the work of omniauth, but it didn't
work for me. To be short, it didn't work for me because it's
trying to be very generalized amongst all the parties, however
the reality is, they are different in cases.

Things are getting better I believe, as more parties are choosing
OAuth 2.0 and stabilizing the specification. But given the current
situation, they just implemented OAuth 2.0 differently. Moreover,
since they already did that, before everyone agree on the
specification, they are not going to make things the same and
break stuffs.

In this regard, OAuth 1.0c is much more stable, though it's not
good enough for today's use cases.

omniauth tries to mimic those difference, but only if we only need
the very basic stuff. If we need a very specific stuff, we'll
probably end up with skipping omniauth, or only using it for
retrieving the access token.

I didn't look into omniauth carefully, and things are changing
all the time. I don't know if this still applies, but that's
what I've observed in the past few years.

Honestly, I've tried to do something similar to omniauth,
which is the RailsUtilUtil in rest-more. Why UtilUtil?
Because it's an utility to build the utilities. As we always know,
there are very similar stuffs in authorization and retrieving
information from a 3rd party. So of course everyone tries to
generalize them and keeps the maintenance minimum.

However I would probably call it a failure because they are
still different. I paid so much and ended up with a very complicated
code base for minimizing the difference, and make customization
as easy as possible.

They actually worked, and worked well. But they were outdated now,
and the problem is, I am too lazy to update them, because the
effort to do so is too expensive for me. Benefit is too little.

If I ever just did it straightforwardly, it would be much simpler.
In this case, I could be ending up with a ton of copypasta, but
it's actually much easier to maintain them, because I won't be
touching everything if I only need to fix one of the parties.

Frankly speaking, omniauth might not have this issue because
that's not it is sacrificing. I was sacrificing simplicity,
but omniauth was sacrificing performance, and now I am
sacrificing generalization for simplicity.

If you're going to optimize the authorization flow, you must
build the flow tighten to the framework you're using. In my
case, which is Rails. So I built RailsUtilUtil, instead of
something on top of Rack. Building on Rack means you're going to
be paying redirecting multiple times unless you're calling
API instead of doing redirects.

I love Rack and all my stuffs except RailsUtilUtil are based on
Rack if they are web based, but Rack won't work well in Rails,
at least to my requirement.

I am quite satisfied to what I'd done, that is simply doing what
really needs. They might look long, and some of the codes look
very similar but different, but they are very simple and
very easy to be maintained.

I could understand this could be tough for some people though.
Because you'll need to understand what's underneath very clearly.
Look at those session handling. Some people don't even understand
how session works. They are not going to know how to build from
scratch.

I still think DRY somehow made people done things much more
complicated than it needs. Copy and edit may mean more works
initially, but you'll appreciate it when you start trying to
treat different parties differently. Only generalize things
if they are really the same.

devise could be the other example which I refuse to work with.
(though unfortunately we're using it and I am maintaining it)
We are not going to have the same user system and working flow,
and even if we're initially, it would become a burden to change to
something different lately. The same applies to some "cloud"
user system.

I still think "people are using it" and "they should have done it"
are very bad reasons to choose a tool. To me, they are merely
excuses to avoid thinking what we really want. Use the tools to
build what we want, not see what we could do with the tools.

--
Well, I guess I still don't get it.
Not sure if I would ever get it.
Different people just do things differently,
and since they do things differently,
of course they would see different results.
If we don't try everything, we won't see everything.
Well, we just can't do everything, and even if we did,
I guess we still can't see everything.
I am still wondering and see, and
do what I think it should be done.

2014-08-20

Spellbook 重啟動

以下是在回航的飛機上寫的:

2014-08-13

Spellbook 重啟動

終於又開始想要繼續製作 spellbook 了!只能說東西看來看去,
最後只會讓我覺得要做什麼東西的話,還是做遊戲吧。但要做什麼
遊戲呢?想了很久很多,覺得單單只是做夢是沒有意義的,最終如果
沒做出來的話,做夢也只是做夢而已。因此還是得考慮實現的可能性。

以現況而言,對我來說最容易跟工作結合的大概還是網頁技術吧。
最早曾經用 c++ 寫過一些蜂窩格的實作,不過那些沒什麼,只是
著迷於 template 的走火入魔罷了。後來有用 scala 和 jruby
做了一些實驗,這有放到 github 上面。基本上可以做到不錯的效果,
讓 scala 跟 jruby 之間互相溝通。做完這些實驗後,就放在那了...

為什麼仍然要回頭挑 spellbook 來做?我想過先做簡化版,例如單純
做成適合在網頁上面玩的,或是簡單的卡牌遊戲,或是在 terminal
上面玩的,或是乾脆做成像 Radical Dreamers 那樣的小說式遊戲。
不過怎麼說呢,我後來仔細想想,覺得說要做那些,不過都只是想要逃避
實作上的困難而已,所以挑好做的東西做。可是這樣其實也還滿沒意思的,
自己的動力就不是很夠強烈。畢竟,我本來就不是具有強烈動機的人,
就算已經是比一般的事情要來得具有動機了,卻還不足夠到真的開始動手。

這樣好像也只是無限往後延而已。

那不如一開始就把 spellbook 做好算了。要寫故事,也什麼遊戲都嘛
能寫。只寫故事也很無趣,而且表現手法不明的話,也很難決定要怎麼
實際寫出。表現手法跟具體內容,終究是一體兩面呀。靈魂與軀體是難以
切割的!這是攻殼告訴我們的...。

這次我的實驗是 frege + jruby + livescript + pixi.js. 原本
scala + jruby 是打算套 GUI toolkit 或 LWJGL, 不過試試現在
web game engine 做得如何也是不錯。畢竟如果做在 web 上的話,
我現有的許多 open source library 都能用上。

挑 frege 的理由也很簡單。我不太想再弄 scala 了,我覺得那像是
另一個 c++, 這條路不太對。我想用 haskell, 可是 haskell 我確實
還不是那麼熟,而且感覺 toolchain 也都不是那麼成熟。我甚至不知道
haskell toolchain 會不會有成熟的一天。因此如果能有什麼東西跑在
JVM 上,那就很有利了。frege 接 jruby 沒有 scala 接 jruby 要
來得順利,因為 haskell 跟 ruby 的概念差距比較遠,溝通上沒那麼
直接。大概是不用肖想能跟 jruby 互相物件繼承了,可能要完全用基本
型別來溝通。溝通成本應該會滿高的,但暫時先這樣看看吧。

跑在 JVM 上我也可以看苗頭不對,再回去試 LWJGL.

是想過用純 ruby, 不過總覺得很無趣呀 :P haskell 寫純運算的東西,
還是漂亮太多了,不能比。ruby 的優勢只有整合容易而已。

因此目前變成前端用 livescript + pixi.js, 透過 websocket
跟後端 jruby 溝通,接著 jruby 再呼叫 frege 寫好的核心程式。

我原本有打算把許多邏輯做在前端,因為很多東西在前端上處理,本來就
比較合理。可是這又會碰上兩個問題,一個是 livescript 我還是寫不慣,
另一個是老問題,前後端很可能會需要一樣的功能,這豈不是要我寫兩遍?
休想叫我使用 nodejs, 門都沒有,就算 livescript 寫慣還是沒有。
javascript runtime 根本就是個不可能能接受的東西。

所以目前變成非常極端,前端只處理跟繪圖有關的東西,其他所有運算一率
透過 websocket 問後端。這就避開 javascript runtime 的問題,
同時也可以用任意語言撰寫後端。搞不好還多上防作弊的功能...
至於單機嘛,單機也能跑一台 local server 起來,沒問題的。

我原本還想做一個銜接 terminal 跟 web 的東西,後來覺得要處理 tty
可能太麻煩了。還是直接一點用繪圖引擎直上吧。

*

那麼具體上應該怎麼執行呢?我發覺我仍然必須同步進行,實作跟設計必須
同時,不然我會覺得有太多東西還處於不明狀態,使得有點不知道從何著手,
進而降低動力。我的設計停很久了,現在應該是實作應該追上,然後設計的
細節才能開始進行。

第一階段,可能是以 hot seat 的方式實作兩人遊戲。而完成這個階段的
第一階段則是做出可以在地圖上行走的棋子。接著則是實作核心規則。
我不確定第二階段會是什麼,幾個可能是:

* 遊戲選單
* 良好的擴充架構
* 連線功能
* 遊戲美化
* 單人劇本 (含 RPG 成分)
* 擴充規則

不確定接下來會怎麼樣。反正我向來也只是說說。一邊工作一邊做遊戲的
可能性近乎於零呀。要以做這維生也太過困難了。spellbook 本身沒
什麼收費空間吧,我也不覺得能做到像是 Braid 那樣。一開始我也會
以 open source 的形式開發。不是我期待 open source 會給我什麼,
也不是覺得應該要 open source, 只是反正沒有眼球關注的
open source project, 跟不是 open source 的 project 之間,
也不會有太大的差距 :P 這只是方便我開發罷了。

2014-08-10

edit last git commit

git reset --mixed HEAD~1
git commit -c ORIG_HEAD

2014-07-08

hackerdojo

There are a few posters on the wall in hackerdojo,
saying that "Done is better than perfect."
While I am not disagreeing with this, I dislike
this kind of saying a lot. I would rather say:
"Still doing it is better than done it wrong."
No one is perfect, so is work.

2014-05-28

web app performance

Personal feels.

<50ms: very fast
50ms~100ms: fast
100ms~150ms: acceptable
150ms~200ms: a bit slow
200ms~300ms: slow
300ms~500ms: quite slow
>500ms: not acceptable

2014-05-04

[ANN] rest-core 3.0.0 and rest-more 3.0.0 just released

It's been a long while! Roughly a year. rest-more itself does not really change too much,
as you can see from the CHANGES.md:

* Adopted latest rest-core
* Added RC::Instagram, thanks @khoan
* Added RC::Firebase

P.S. I believe RC::Firebase might be the best Firebase client in Ruby currently.

However, it's really a hard work to make rest-core advanced to 3.0.0, here's the
CHANGES.md:

Highlights:

* Hijack for streaming responses
* EventSource for SSE (server-sent events)
* Thread pool
* Keep-alive connections from httpclient

See the above CHANGES.md for the complete list, here we only list the highlights.
Hijack and EventSource are roughly the same work. As long as we have streaming
support, it's fairly easy to get EventSource work.

Switching from rest-client to httpclient did not really take long, but
survey and experiments did take some time. At first I was trying net-http-persistent
because I trust the author, and think it's merely a thin layer on top of net/http,
so it should be fast and reliable. However, it turns out it's actually... quite slow.
rest-core's test suites were running 20% slower when switched to net-http-persistent
from rest-client. This is... not very desirable. Not to mention I really spend so
much time figuring out how to use net-http-persistent correctly. net/http family
really has a bad taste of API, feeling like Java.

So why I want to switch? rest-client is still quite good, though it needs some patches.
It does not really support persistent connections, which could definitely boost the
performance if we're hitting the API server quite frequently. That's exactly why I
tried net-http-persistent in the first place. It's claiming it's thread-safe, too.

But do I have other alternatives? I don't really want to implement this if there's one.
Then I found this post again: ruby HTTP client performance shootout redux
I believe I had read it before, but I totally forgot this.

It turns out httpclient performs quite well, therefore I want to give it a try.
The code is actually quite complicated comparing to rest-client and net-http-persistent,
so I doubt if that would really run faster. However indeed it's only roughly a bit slower
than rest-client, running rest-core's test suites.

And since httpclient does support persistent connections, and only a bit slower than
rest-client given the simplest case. It's definitely the way to go. Also, I could easily
grab the socket underneath to implement hijacking for streaming response and EventSource.
Which... I am too lazy to implement for rest-client and net-http-persistent. I would
probably drop support for rest-client and net-http-persistent in the future.

* Thread pool

The hardest part is of course in implementing the thread pool. Oh well, no, timeout is
the hardest part, but I'll talk about this later. I spent quite some time finding an
existing implementation, just to realize if I want to make it the best, I needed to do
on my own, i.e. a customized thread pool, not a general purpose one.

The reason I began looking in this was a ticket from Github.
#7 Threads Error on Heroku, How to Limit Threads?

Apparently a thread pool was what he needed... then I looked into several thread pool
implementation, finally taking puma's thread pool implementation as a reference
implementation, because it's a quite simple and complete one.

It's actually quite an interesting process to implement it. At first, of course I
tried to use the built-in ::Queue to do this. However, the built-in ::Queue does not
support timing out on the pop operation... Therefore I need to roll my own Queue,
which would pop a lambda{false} upon timing out to tell the worker that
it has been idled for too long, and it should shut itself down. Otherwise the task
would return a true to tell the worker keep working.

I can't make it that simple if I picked a general purpose thread pool.

* Timing out

Then, I realized that Thread#kill or Thread#raise is not safe.
There's no safe way to interrupt a thread, because we might kill or raise an exception
at the point target thread is running inside an ensure block, risking it from not
releasing the resource the thread was grabbing.

Checkout Charles Nutter's post for the detail:
Ruby's Thread#raise, Thread#kill, timeout.rb, and net/protocol.rb libraries are broken

Although killing a new thread as before might be ok if we're not using a thread pool,
since the newly spawned thread might not grab any external or shared resources anyway.
But nah, I don't really want to make it so complicated though. It's already quite
complicated, and still we're risking even we're spawning new threads.

I guess this is why I can't make this run reliably at first. I almost wanted to give up
timing it out, only to cancel the task if it hasn't been running. However it might be
fine if we're spawning a new thread each time, but it would be much more problematic
if we're blocking the threads in the pool, blocking for other tasks from running.

Therefore, we still need timing it out somehow without killing it or raising some asynchronous
exceptions. This is probably why Eric Wong always emphasizes that we need to use the timeout
facilities provided in the client libraries, instead of using an outsider to handle timeout.

However, all HTTP clients I was trying, didn't provide an overall timeout time, including
net/http, net-http-persistent and httpclient. They have open_timeout, read_timeout, and
various other timeouts... It makes sense to separate them, but what I really want is a
total timeout, since I don't care where it blocks.

For now, I just treat all timeout time as the total timeout time, setting both open_timeout
and read_timeout to the timeout I want. In the worst case, it means it could spend twice more
time than expected. But I guess since Thread#kill and Thread#raise can't be safe, it might
be the only hope we could rely on.

I stopped working on this for a while because I found no way to fix this at that moment,
feeling a bit frustrated that why Ruby can't do this right. I guess this is because it's
all depending on how we schedule the threads. If we can't give some hints to the scheduling,
then we can't make sure the critical sections would have never been interrupted.

I believe Haskell did this right, with their own scheduling.
Checkout: Cancellation and Timeouts: Masking Asynchronous Exceptions
Maybe all programming languages runtime should have their own scheduling to solve this.
i.e. Provide maskAsyncExceptions and unmaskAsyncExceptions primitives in order to implement
the mask or bracket function.

At last, I just picked Tony Arcieri's timers, hoping it would do it right, and I stopped
raising or killing the threads, simply set the flag to tell it stop working if you haven't
started. If you started already, fine, but I would ignore your work, and rely on timeout
from the HTTP clients to stop blocking the thread.

We have no way but desperately to depend on the libraries to do the right thing, unless we
could fix this scheduling issues in Ruby, providing those masking facilities.

2014-02-17

Rails + Bower on Heroku

Tested with Rails 3.2.15 and Bower 1.2.8.

First we need both Heroku's NodeJS and Ruby buildpacks,
using a multi-buildpack which could combine buildpacks.
Run this command to enable this multi-buildpack:

heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git

Then we need to create a .buildpacks file to
specify which buildpacks we want to use.

$ cat .buildpacks
https://github.com/heroku/heroku-buildpack-nodejs
https://github.com/heroku/heroku-buildpack-ruby

Next, we need a package.json for Heroku to
install npm dependencies, and run bower install.

$ cat package.json
{
  "name": "example",
  "version": "0.0.1",
  "repository": {
    "type": "git"
  },
  "dependencies": {
    "bower": "*"
  },
  "engines": {
    "node": "0.10.x"
  },
  "scripts": {
    "postinstall": "./node_modules/bower/bin/bower install"
  }
}

Note that repository is for silencing warnings from npm,
and scripts is where we specify how to run bower install.

Here's the bower.json to specify dependencies:
$ cat bower.json
{
  "name": "example",
  "version": "0.0.1",
  "dependencies": {
    "normalize-css": "*",
  },
}

We also need to specify the installation path for bower:
$ cat .bowerrc
{
  "directory": "app/assets/components"
}

We tried to install to this path because it seems Rails is
aware of this path, and people are using it. (Honestly, I don't
yet really understand what's Bower doing, just want to make it work)

Hopefully the last one, telling Rails to precompile assets from Bower.
We need to put this into our config/application.rb:
# Precompile assets from Bower
prefix = "#{Rails.root}/app/assets/components"
config.assets.precompile.push(*Dir["#{prefix}/*"].map{ |path|
  "#{path[/^#{Regexp.escape(prefix)}\/(.*)/, 1]}/*"
})

Updated 2014-08-12 00:04
This might work better:
config.assets.precompile = [lambda do |path|
  !path.start_with?('jquery/src') &&  # exclude broken jquery
    %w[.js .css .svg .png .jpg .gif .ttf .woff .eot .wav .mp3 .ogg].
      include?(File.extname(path)) && # compile only them
    Dir['app/assets/*'].find do |base|
      File.exist?("#{base}/#{path}")  # compile only from app/assets
    end
end]

I don't really understand why we need to do this, but I guess
this is how Rails designed, and I am too tired to argue with this.

That's basically everything we made Rails and Bower work on Heroku.

Example logs from Heroku:
-----> Fetching custom git buildpack... done
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-nodejs
=====> Detected Framework: Node.js
-----> Requested node range:  0.10.x
-----> Resolved node version: 0.10.25
-----> Downloading and installing node
-----> Restoring node_modules directory from cache
-----> Pruning cached dependencies not specified in package.json
-----> Exporting config vars to environment
-----> Installing dependencies

       > example@0.0.1 postinstall /tmp/build_random-uuid
       > ./node_modules/bower/bin/bower install
[...]
-----> Caching node_modules directory for future builds
-----> Cleaning up node-gyp and npm artifacts
-----> Building runtime environment
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-ruby
=====> Detected Framework: Ruby
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.1.0
-----> Installing dependencies using 1.5.2
       Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
[...]
       Your bundle is complete!
       Gems in the groups development and test were not installed.
       It was installed into ./vendor/bundle
       Bundle completed (1.29s)
       Cleaning up the bundler cache.
-----> Writing config/database.yml to read from DATABASE_URL
-----> Preparing app for Rails asset pipeline
       Running: rake assets:precompile
[...]
-----> WARNINGS:
       Injecting plugin 'rails_log_stdout'
       Injecting plugin 'rails3_serve_static_assets'
       Add 'rails_12factor' gem to your Gemfile to skip plugin injection
       Using release configuration from last framework Ruby:
       ---
       addons:
       - heroku-postgresql:dev
       default_process_types:
         rake: bundle exec rake
         console: bundle exec rails console
         web: bundle exec rails server -p $PORT
         worker: bundle exec rake jobs:work
-----> Discovering process types
       Procfile declares types     -> web, worker
       Default types for Multipack -> console, rake

-----> Compressing... done, 12.93MB
-----> Launching... done, v1293
-----> Deploy hooks scheduled, check output in your logs
       http://example.herokuapp.com deployed to Heroku

To git@heroku.com:example.git
   fee1dead...deadbeef master -> master

2014-01-24

Think about a talk for rubyconf.tw/2014

Doh, I believe this description is far too long though...
Still thinking.

RubyQC -- A conceptual QuickCheck library for Ruby.

How do we make sure our programs work as expected?
Taking it to the extreme, of course we prove it formally.
We have Agda for Haskell people, or Coq for OCaml people.

However, in most cases we don't really care if they are
100% correct. Do we care PRNGs are really random
in games? Some people might care, we don't. Can we
prove halting problem? Of course not, but we do need
termination check at times.

For most cases, an army of tests is far good enough.
Usually we write scenario based tests. We first assume
things, and then do things, finally verify results. This is
quite simple, but cannot really cover most of the inputs
without great effort.

QuickCheck took another approach. Instead of writing
scenario, we think about what properties do our programs,
or functions have, giving a range of inputs. Suppose
we want to test the reverse function, instead of testing
against a fixed set of lists and verify a fixed set of results,
we think about what properties does reverse have.

For example, if we reverse and reverse a list, the result
should be equal to the original list. With QuickCheck, it
could then generate arbitrary random lists to the property
function you just wrote, and verify if the property holds.
By default, it would generate 100 test cases.

This approach would force you think more about the
precondition and postcondition, eliminating unusual
corner cases you might never think of, and force you
think what are the functions we're really writing. We
could also raise the number of test cases by configuring
it and raise our level of confidence about correctness.

2014-01-21

Why Bother to Create? Why Not Just Use?

I forgot to post this slide after the talk. Here it is.

2014-01-07: Why Bother to Create? Why Not Just Use?

p.s. Now I feel much easier with HTML slides, however
can't really post HTML slides on sites like slideshare
or speakerdeck is really a disadvantage. Any solution?
I can't really convert/print my slides into PDF.
The output was simply awful and not readable.


Ruby Tuesday #28
為什麼要再造輪?現有的輪子不好用嗎?什麼時候我們甘願用方形的輪子?
如果我們都會飛了,為什麼還要輪子?跟大家分享我一路走來用了造了各種
輪子的一些心路歷程。

2014-01-06

Muack as a development runtime static typing system

Muack as a development runtime static typing system

Ever consider a static type system in Ruby? You could actually see a lot of
asserts inserted in the beginning of some methods in some libraries. For
example, there are assert_valid_key_size, assert_kind_of, etc, in
dm-core, and assert_valid_keys, assert_valid_transaction_action,
and various random asserts in activerecord.

You could find them by searching against raise ArgumentError because
rails is much less consistent and sometimes it's hard to find a pattern in
rails. But you get the idea, those ArgumentError would much help us debug
our code from misusing the API, and that's exactly the point of type system,
or more specifically, static type system.

We could also use some static analysis tools to do something like this, for
example, there's ruby-lint. However, as you might already know, since
Ruby is so dynamic, static analysis tools cannot really do a great job if
our code is quite dynamic. Of course we could write it more statically,
and treat our static analysis tools better, but that might not be the spirit
of Ruby somehow.

Alternatively, it would be great to do this static type checking
dynamically... I mean, in the runtime rather than compile time. This
means it would be much more accurate, just like those asserts in the
above examples.

However, if we're doing those checks in a hot path, for example, right
inside a loop looping over a million times, this would definitely slow
things down if we're checking them in the runtime. Even if we put $DEBUG
guards around those check, we're still suffering from checking the flag.

It would be great if we could actually just remove those checks in
production, while turn it on when we're developing or debugging.
Muack could actually fulfill this desire, as it could inject codes
externally and seamlessly, and we could remove them anytime when we
call Muack.reset, or, simply don't do any stubs in production config.

Consider we have two classes:

Food = Class.new
User = Class.new{ attr_accessor :food }

And we could make sure User#food is always a kind of Food by putting this
into a development config or so:

Muack::API.module_eval do
  any_instance_of(User) do |user|
    stub(user).food = is_a(Food)
  end
end

And then if we're trying to set a food other than a Food...

u, f = User.new, Food.new
u.food = f # ok
u.food = 1 # raise Muack::Unexpected

This could go wild and we could customize our own domain specific argument
verifiers. For example, we could do this to check if the food is frozen:

Food = Class.new
User = Class.new{ attr_accessor :food }

FoodFrozen = Class.new(Muack::Satisfy) do
  def initialize
    super lambda{ |actual_arg| actual_arg.frozen? }
  end
end

Muack::API.module_eval do
  any_instance_of(User) do |user|
    stub(user).food = FoodFrozen.new
  end
end

u = User.new
p u.food = Food.new.freeze # ok
p u.food = Food.new        # raise Muack::Unexpected

Please check Arguments Verifiers (Satisfy) section for more argument
verifiers details.



All texts are licensed under CC Attribution 3.0