Let's make a web service and client in Rust
So I'm working on this project Crater for doing Rust regression testing. Looking into the feasibility of writing parts in Rust. I need an HTTP server that speaks JSON, along with a corresponding client. I don't see a lot of docs on how to do this so I'm recording my investigation.
I'm going to use Iron and Hyper, neither of which I have experience with.
Each commit in this repo corresponds with a chapter, so follow along if you want.
Edit: Some inaccuracies within! Thanks /r/rust!
1. Preparing to serve some JSON
I start by asking Cargo to give me a new executable project called 'httptest'. Passing --bin says
to create a source file in src/main.rs that will be compiled to an application.
$ cargo new httptest --bin
I'll use Iron for the server, so following the instructions in their docs add the following
to my Cargo.toml file.
[dependencies]
iron = "*"Then into main.rs I just copy their example.
extern crate iron;
use iron::prelude::*;
use iron::status;
fn main() {
fn hello_world(_: &mut Request) -> IronResult<Response> {
Ok(Response::with((status::Ok, "Hello World!")))
}
Iron::new(hello_world).http("localhost:3000").unwrap();
println!("On 3000");
}And type cargo build.
$ cargo build
Compiling hyper v0.3.13
Compiling iron v0.1.16
Compiling httptest v0.2.0 (file:///opt/dev/httptest)
Pure success so far. Damn, Rust is smooth. Let's try running the server with cargo run.
$ cargo run
Running `target/debug/httptest`
I sit here waiting a while expecting it to print "On 3000" but it never does. Cargo must be capturing output. Let's see if we're serving something.
$ curl http://localhost:3000
Hello World!
Oh, that's super cool. We know how to build a web server now. Good starting point.
2. Serving a struct as JSON
Is rustc-serialize still the easiest way to convert to and from JSON? Maybe I should use serde, but then you really want serde_macros, but that only works on Rust nightlies. Should I just use nightlies? Nobody else is going to need to use this.
Let's just go with the tried-and-true rustc-serialize. Now my Cargo.toml 'dependencies' section looks like the following.
[dependencies]
iron = "*"
rustc-serialize = "*"Based on the rustc-serialize docs I update main.rs
to look like this:
extern crate iron;
extern crate rustc_serialize;
use iron::prelude::*;
use iron::status;
use rustc_serialize::json;
#[derive(RustcEncodable)]
struct Greeting {
msg: String
}
fn main() {
fn hello_world(_: &mut Request) -> IronResult<Response> {
let greeting = Greeting { msg: "Hello, World".to_string() };
let payload = json::encode(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
Iron::new(hello_world).http("localhost:3000").unwrap();
println!("On 3000");
}And then run cargo build.
$ cargo build
Updating registry `https://github.com/rust-lang/crates.io-index`
Downloading unicase v1.1.1
<... more downloading here ...>
Compiling lazy_static v0.1.15
<... more compiling here ...>
Compiling httptest v0.2.0 (file:///opt/dev/httptest)
src/main.rs:17:12: 17:47 error: this function takes 1 parameter but 2 parameters were supplied [E0061]
src/main.rs:17 Ok(Response::with(status::Ok, payload))
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:17:12: 17:47 help: run `rustc --explain E0061` to see a detailed explanation
error: aborting due to previous error
Could not compile `httptest`.
To learn more, run the command again with --verbose.
Lot's of new dependencies now. But an error. Response::with has this definition:
fn with<M: Modifier<Response>>(m: M) -> ResponseI don't know what a Modifier is. Some
docs don't
help. I don't know what to do here but I notice that the original
example passed a tuple to Response::with whereas my update treated
Response::with as taking two parameters. It seems a tuple is a
Modifier.
Add the tuple to Ok(Response::with((status::Ok, payload))), execute cargo run, curl some JSON.
$ curl http://localhost:3000
{"msg":"Hello, World"}
Blam! We're sending JSON. Time for another break.
3. Routes
Next I want to POST some JSON, but before I do that I need a proper URL to post to, so I guess I need to learn how to set up routes.
I look at the Iron docs and don't see anything obvious in the main text, but there's a crate called router that might be interesting.
The module docs are "Router provides fast and flexible routing for
Iron", but not much else. How do I use a Router?! After I lot of
sleuthing I discover an
example.
OK, let's try to adapt that to our evolving experiment.
I add router = "*" to my Cargo.toml [dependencies] section,
and begin writing. The following is what I come up with
before getting stuck reading the POST data.
extern crate iron;
extern crate router;
extern crate rustc_serialize;
use iron::prelude::*;
use iron::status;
use router::Router;
use rustc_serialize::json;
#[derive(RustcEncodable, RustcDecodable)]
struct Greeting {
msg: String
}
fn main() {
let mut router = Router::new();
router.get("/", hello_world);
router.post("/set", set_greeting);
fn hello_world(_: &mut Request) -> IronResult<Response> {
let greeting = Greeting { msg: "Hello, World".to_string() };
let payload = json::encode(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn set_greeting(request: &mut Request) -> IronResult<Response> {
let payload = request.body.read_to_string();
let request: Greeting = json::decode(payload).unwrap();
let greeting = Greeting { msg: request.msg };
let payload = json::encode(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
Iron::new(router).http("localhost:3000").unwrap();
}This uses Router to control handler dispatch. It builds and still
responds to curl http://localhost:3000, but the handler for the
/set route is yet unimplemented.
Now to read the POST body into a string. The docs for
Request
say the field body is an iterator, so we just need to collect
that iterator into a string.
I first try let payload = request.body.read_to_string(); because I
know it used to work.
It does not work.
$ cargo build
Compiling httptest v0.2.0 (file:///opt/dev/httptest)
src/main.rs:29:36: 29:52 error: no method named `read_to_string` found for type `iron::request::Body<'_, '_>` in the current scope
src/main.rs:29 let payload = request.body.read_to_string();
^~~~~~~~~~~~~~~~
src/main.rs:29:36: 29:52 help: items from traits can only be used if the trait is in scope; the following trait is implemented but not in scope, perhaps add a `use` for it:
src/main.rs:29:36: 29:52 help: candidate #1: use `std::io::Read`
error: aborting due to previous error
Could not compile `httptest`.
To learn more, run the command again with --verbose.
I throw my hands up in disgust. 'Why does this method no longer exist?
The Rust team is always playing tricks on us!' Then I notice the
compiler has - at some length - explained that the method does exist
and that I should import std::io::Read.
I add the import and discover that read_to_string behaves
differently than I thought.
101 $ cargo build
Compiling httptest v0.2.0 (file:///opt/dev/httptest)
src/main.rs:30:36: 30:52 error: this function takes 1 parameter but 0 parameters were supplied [E0061]
src/main.rs:30 let payload = request.body.read_to_string();
^~~~~~~~~~~~~~~~
Ok, yeah the signature of Read::read_to_string is now fn read_to_string(&mut self, buf: &mut String) -> Result<usize, Error>,
so that the buffer is supplied and errors handled. Rewrite the
set_greeting method.
// Receive a message by POST and play it back.
fn set_greeting(request: &mut Request) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
let request: Greeting = json::decode(&payload).unwrap();
let greeting = Greeting { msg: request.msg };
let payload = json::encode(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}Let's run this and give it a curl.
$ curl -X POST -d '{"msg":"Just trust the Rust"}' http://localhost:3000/set
{"msg":"Just trust the Rust"}
Oh, Rust. You're just too bad.
4. Mutation
Hey, I know all those .unwrap()s are wrong. I don't care. We're prototyping.
Before we continue on to writing a client, I want to modify this toy example
to store some state on POST to /set and report it later. I'll make greeting
a local, capture it in some closures, then see how the compiler complains.
Here's my new main function, before attempting to compile:
fn main() {
let mut greeting = Greeting { msg: "Hello, World".to_string() };
let mut router = Router::new();
router.get("/", |r| hello_world(r, &greeting));
router.post("/set", |r| set_greeting(r, &mut greeting));
fn hello_world(_: &mut Request, greeting: &Greeting) -> IronResult<Response> {
let payload = json::encode(&greeting).unwrap();
Ok(Response::with((status::Ok, payload)))
}
// Receive a message by POST and play it back.
fn set_greeting(request: &mut Request, greeting: &mut Greeting) -> IronResult<Response> {
let mut payload = String::new();
request.body.read_to_string(&mut payload).unwrap();
*greeting = json::decode(&payload).unwrap();
Ok(Response::with(status::Ok))
}
Iron::new(router).http("localhost:3000").unwrap();
}src/main.rs:21:12: 21:51 error: type mismatch resolving `for<'r, 'r, 'r> <[closure@src/main.rs:21:21: 21:50 greeting:_] as core::ops::FnOnce<(&'r mut iron::request::Request<'r, 'r>,)>>::Output == core::result::Result<iron::response::Response, iron::error::IronError>`:
expected bound lifetime parameter ,
found concrete lifetime [E0271]
src/main.rs:21 router.get("/", |r| hello_world(r, &greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:21:12: 21:51 help: run `rustc --explain E0271` to see a detailed explanation
src/main.rs:22:12: 22:60 error: type mismatch resolving `for<'r, 'r, 'r> <[closure@src/main.rs:22:25: 22:59 greeting:_] as core::ops::FnOnce<(&'r mut iron::request::Request<'r, 'r>,)>>::Output == core::result::Result<iron::response::Response, iron::error::IronError>`:
expected bound lifetime parameter ,
found concrete lifetime [E0271]
src/main.rs:22 router.post("/set", |r| set_greeting(r, &mut greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:22:12: 22:60 help: run `rustc --explain E0271` to see a detailed explanation
src/main.rs:21:12: 21:51 error: type mismatch: the type `[closure@src/main.rs:21:21: 21:50 greeting:&Greeting]` implements the trait `core::ops::Fn<(&mut iron::request::Request<'_, '_>,)>`, but the trait `for<'r, 'r, 'r> core::ops::Fn<(&'r mut iron::request::Request<'r, 'r>,)>` is required (expected concrete lifetime, found bound lifetime parameter ) [E0281]
src/main.rs:21 router.get("/", |r| hello_world(r, &greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:21:12: 21:51 help: run `rustc --explain E0281` to see a detailed explanation
src/main.rs:22:12: 22:60 error: the trait `for<'r, 'r, 'r> core::ops::Fn<(&'r mut iron::request::Request<'r, 'r>,)>` is not implemented for the type `[closure@src/main.rs:22:25: 22:59 greeting:&mut Greeting]` [E0277]
src/main.rs:22 router.post("/set", |r| set_greeting(r, &mut greeting));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/main.rs:22:12: 22:60 help: run `rustc --explain E0277` to see a detailed explanation
error: aborting due to 4 previous errors
Could not compile `httptest`.
rustc is not happy. But I expected that. I'm throwing types at it just to get a response. Tell me what to do rustc.
Those error messages are confusing, but clearly the closure is the wrong type. The get and post methods of Router take a Handler, and from the doc page I see there's an impl defined as
pub trait Handler: Send + Sync + Any {
fn handle(&self, &mut Request) -> IronResult<Response>;
}That's a mouthful, but Handler is defined for Fn, not FnOnce or
FnMut, and it has to be Send + Sync. Since it needs to be send,
we're not going to be capturing any references, and since the
environment isn't mutable we have to use interior mutability to mutate
the greeting. So I'm going to use a sendable smart pointer, Arc, and
to make it mutable, put a Mutex inside it. For that we need to import
both from the standard library like this: use std::sync::{Mutex, Arc};.
I'm also going to have to move the captures with move |r| ... to avoid
capturing by reference.
Updating my code like so yields the same error messages.
let greeting = Arc::new(Mutex::new(Greeting { msg: "Hello, World".to_string() }));
let greeting_clone = greeting.clone();
let mut router = Router::new();
router.get("/", move |r| hello_world(r, &greeting.lock().unwrap()));
router.post("/set", move |r| set_greeting(r, &mut greeting_clone.lock().unwrap()));rustc doesn't like the lifetimes of my closure. Why? I don't know. I ask reem in #rust if he knows what to do.
Several hours later reem says
16:02 < reem> brson: Partially hint the type of the request art, rustc has trouble inferring HRTBs
HRTB means 'higher-ranked trait bounds', which means roughly 'complicated lifetimes'.
I change those same lines to hint the type of r: &mut Request and everything works...
let greeting = Arc::new(Mutex::new(Greeting { msg: "Hello, World".to_string() }));
let greeting_clone = greeting.clone();
let mut router = Router::new();
router.get("/", move |r: &mut Request| hello_world(r, &greeting.lock().unwrap()));
router.post("/set", move |r: &mut Request| set_greeting(r, &mut greeting_clone.lock().unwrap()));It was seemingly a bug in Rust's inferencer. That's lame.
Now it builds again, so we can test with curl.
$ curl http://localhost:3000
{"msg":"Hello, World"}
$ curl -X POST -d '{"msg":"Just trust the Rust"}' http://localhost:3000/set
$ curl http://localhost:3000
{"msg":"Just trust the Rust"}
Now we're playing with power.
5. The client
We've got a little JSON server going. Now let's write the client. This time we're going to use Hyper directly.
I add it to my Cargo.toml: hyper = "*", then create src/bin/client.rs:
extern crate hyper;
fn main() { }Source files in src/bin/ are automatically built as executables by cargo. Run cargo build
and the target/debug/client program appears. Good, universe is sane. Now figure out Hyper.
Cribbing off the Hyper client example I come up with this snippet that just makes a request for "/" and prints the body:
extern crate hyper;
use hyper::*;
use std::io::Read;
fn main() {
let client = Client::new();
let mut res = client.get("http://localhost:3000/").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut s = String::new();
res.read_to_string(&mut s).unwrap();
println!("{}", s);
}But now cargo run no longer works.
$ cargo run
`cargo run` requires that a project only have one executable; use the `--bin` option to specify which one to run
I must type cargo run --bin httptest to start the server. I do so, then cargo run --bin client and see
$ cargo run --bin client
Running `target/debug/client`
{"msg":"Hello, World"}
Oh, man, I'm a Rust wizard. One last thing I want to do, make the POST
request to set the message. Obvious thing to do is change client.get
to
client.post.
This returns a
RequestBuilder,
so I'm looking for a builder method that sets the payload. How about body?
My new creation:
extern crate hyper;
use hyper::*;
use std::io::Read;
fn main() {
let client = Client::new();
let res = client.post("http://localhost:3000/set").body("Just trust the Rust").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut res = client.get("http://localhost:3000/").send().unwrap();
assert_eq!(res.status, hyper::Ok);
let mut s = String::new();
res.read_to_string(&mut s).unwrap();
println!("{}", s);
}But running it is disappointing.
101 $ cargo run --bin client
Compiling httptest v0.2.0 (file:///Users/danieleesposti/workspace/httptest)
Running `target/debug/client`
thread '<main>' panicked at 'assertion failed: `(left == right)` (left: `InternalServerError`, right: `Ok`)', src/bin/client.rs:9
And simultaneously I see that the server has also errored:
$ cargo run --bin httptest
Running `target/debug/httptest`
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: ParseError(SyntaxError("invalid syntax", 1, 1))', src/libcore/result.rs:741
It's because of my bad error handling! I didn't pass valid JSON to the /set route. Fixing the body to be .body(r#"{ "msg": "Just trust the Rust" }"#) lets the client succeed:
$ cargo run --bin client
Compiling httptest v0.2.0 (file:///opt/dev/httptest)
Running `target/debug/client`
{"msg":"Just trust the Rust"}
And just like that we've created a web service and client in Rust. Looks like the future is near. Go build something with Iron and Hyper.