leastfixedpoint

JSON and JSON-RPC for Erlang

This page is a mirrored copy of an article originally posted on the LShift blog; see the archive index here.

About a month ago, I wrote an implementation of RFC 4627, the JSON RFC, for Erlang. I also implemented JSON-RPC over HTTP, in the form of mod_jsonrpc, a plugin for Erlang’s built-in inets httpd. This makes accessing Erlang services from in-browser Javascript very comfortable and easy indeed.

Downloading the code:

  • you can browse the code here on github
  • a tarball is available here (note: this is dynamically generated from the HEAD revision in the git repository)
  • the git repository holding the code can be retrieved with the command
    git clone git://github.com/tonyg/erlang-rfc4627.git

Documentation is available, including notes on how to write a service and how to access it from javascript, and the curious may wish to see the code for an example Erlang JSON-RPC service and its corresponding javascript client.

The JSON codec uses a data type mapping suggested by Joe Armstrong, where strings map to binaries and arrays map to lists.

Coincidentally, on the very same day I started writing my JSON codec, Eric Merritt released his new JSON codec, Ktuo. If I’d seen that, I probably wouldn’t have started writing my own. At the time, the only other implementation I knew of was the json.erl included with yaws, which uses an awkward (to me) encoding and was, at the time I was using it, a bit buggy (decoding “[]” returned an incorrect value - it seems to have been fixed somewhere between yaws 1.64 and 1.68). To an extent, Eric’s rationale for a new JSON codec applies to mine, too, and my other excuse is that the data type mapping where strings become Erlang binaries is much more useful to my application. Your mileage may vary!

Comments

On 5 September, 2007 at 6:26 pm, Sam Ruby wrote:

I’m interested in proving patches to support utf-8. For more background (and a link to the first patch), see here.

On 6 September, 2007 at 10:40 am, tonyg wrote:

Sam, that’s great stuff! Thank you. I’m very interested in integrating your utf-8 work - I’ll get in touch via email.

(Update: Er, or I would if I could find an email address for you! :-) I’ll leave a comment on your blog, instead.)

On 14 September, 2007 at 2:01 pm, Sam Ruby wrote:

Email is required to post a comment, but even you can’t see it?

In any case, rubys at my weblog’s host.

On 11 October, 2007 at 10:50 am, Mikl Kurkov wrote:

Hi!
Thanks for very good work. I’m interested in your jsonrpc implementation. It seems to work pretty good. But I don’t througly understand is it synchronyse or not. I mean if some rpc request takes long time to proceed will over requests wait for it or they can be proceed in parallel.
As I can see in current implementation you use genserver:call so it is not possible to proceed other request until previous will proceed. Also with function call - all system will wait current call to return.
Do you plan implementing async requests in this system. Or may be I misiing something.

On 23 October, 2007 at 6:07 pm, rich wrote:

there’s also ejson

On 8 January, 2008 at 11:53 am, Daniel Kwiecinski wrote:

Hi,

How easy would be to convert mod_json into yaws appmod? Could you make any suggestion - clues to make this task easier?

Cheers,
Daniel

On 14 January, 2008 at 12:41 pm, tonyg wrote:

Hi Daniel,

It ought not to be too difficult. It may need a bit of refactoring. The key differences will be separating out the inets-httpd specific way of retrieving the URL, query parameters, and POST body from the HTTP request. The main entry point that inets-httpd uses (after a few trivial calls) is do_rpc/1, and the main workers are parse_jsonrpc/2 and invoke_service_method/6. See also the section “Invoking JSON-RPC procedures from Erlang” in the manual.

On 18 April, 2009 at 2:49 am, tonyg wrote:

(By the way, a bunch of the necessary refactorings have been done, and the current codebase works with both mochiweb and inets httpd. Adding yaws support might be straightforward!)

On 28 July, 2009 at 9:49 pm, n8 wrote:

Hi,

Thanks for sharing the code! It would be helpful to have installation instructions (or a “make install” target) for those unfamiliar with erlang. I need to install this package as a dependency of rabbitmq-http2 and I’m really not sure that I’ve done it correctly.

Cheers,
-n8

On 2 September, 2009 at 1:03 am, Joseph Wayne Norton wrote:

Tony - I’m not sure where to submit bug reports. If there is a better place, please let me know.

There appears to be a bug in rfc426.erl. Here is the report from dialyzer:

rfc4627.erl:153: The call dict:to_list(Dict::tuple()) does not have an opaque term of type dict() in position 1

It seems the when clause is missing a is_tuple(Dict) guard.

On 2 September, 2009 at 2:50 pm, tonyg wrote:

Hi Joe,

Thanks for the report — here’s fine, or you can email me directly. Dialyzer has found a place where we’re violating what little encapsulation Erlang provides. The code it’s complaining about is examining the tuple-based representation of dicts using element/2 as a guard. The dict module doesn’t provide an is_dict/1, so we kind of have to examine the structure itself; and using element/2 as a guard is safe even when the examined object is not a tuple, or is a 0-ary tuple, because guards don’t throw exceptions, they just “fail”, causing evaluation to proceed to the next clause in the function.

Eshell V5.6.5  (abort with ^G)
1> rfc4627:encode(dict:new()).
"{}"
2> rfc4627:encode({obj, []}).
“{}”
3> rfc4627:encode([1, 2]).
“[1,2]”
4>

I’m not sure there’s any easy way of telling dialyzer that what it has found is a non-error — maybe some explicit typespecs would help?

On 18 September, 2009 at 2:38 pm, Erlang tap to the Twitter stream @ jebu.net wrote:

[...] a code fragment from the code behind nowwhat.in which taps into the Twitter data stream. I use the erlang json parser from Lshift for processing the tweets. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 [...]

On 15 October, 2009 at 9:29 pm, RobW wrote:

I believe there is another bug in the code, unless I am simply using it wrong…

There doesn’t seem to be a way to nest mappings. Because Erlang doesn’t have a true dictionary data type, it seems it’s simulated using a list of tagged tuples. If you create a JSON “object” (i.e. mapping) that is flat, there is no problem. But there seems to be no way to nest mappings.

For example, this works:
rfc4627:encode( {obj, [{a,1}, {b,1}]} ).

But this won’t work:
rfc4627:encode( {obj, [{a,1}, {b,[{c,3},{d,4}]}]} ).

Yet I believe nested mappings are legal in JSON. They certainly work fine with Python 2.6 and the json module.

Both of these would work for Python:
json.dumps({’a':1, ‘b’:2})
json.dumps({’a':1, ‘b’: {’c':3, ‘d’:4}})

On 22 October, 2009 at 8:18 am, mikeb wrote:

@RobW, you want

rfc4627:encode( {obj, [{a,1}, {b, {obj, [{c,3},{d,4}]}}]} )

On 7 May, 2010 at 8:08 am, [erlang]基于yaws的jsonrpc模块 | 熊窝[春眠不觉晓] wrote:

[...] 某个项目里用到的一些代码。最初是使用 LShift 提供的一个基于inets的 rfc4627jsonrpc的application的,不过经过一段时间的试用发现不少问题,比如错误处理不是很好经常就吧处理进程挂起。还有json编码解码效率不是很高。并且inets也有各种各样的麻烦事比如mimetype我怎么也映射不对。后来发现yaws里也自带了一个jsonrpc模块,不过那个模块为了兼容Haxe和SOAP功能上就十分简陋了。于是决定自己写一个jsonrpc处理模块用。 [...]

On 15 September, 2010 at 3:46 pm, We’re releasing eCouch - tarpipe blog wrote:

[...] for their RFC 4627 (JSON) Erlang [...]