loading...

JavaScript – Asynchronous I/O with Node

Node is a fast C++-based JavaScript interpreter with bindings to
the low-level Unix APIs for working with processes, files, network
sockets, etc., and also to HTTP client and server APIs. Except for
some specially named synchronous methods, Node’s bindings are all
asynchronous, and by default Node programs never block, which means
that they typically scale well and handle high loads effectively.
Because the APIs are asynchronous, Node relies on event handlers,
which are often implemented using nested functions and
closures.[23]

This section highlights some of Node’s most important APIs and
events, but the documentation is by no means complete. See Node’s
online documentation at http://nodejs.org/api/.

Obtaining Node

Node is free software that you can download from http://nodejs.org. At the time of this writing, Node
is still under active development, and binary distributions are not
available—you have to build your own copy from source. The examples
in this section were written and tested using Node version 0.4. The
API is not yet frozen, but the fundamentals illustrated here are
unlikely to change very much in the future.

Node is built on top of Google’s V8 JavaScript engine. Node
0.4 uses V8 version 3.1, which implements all of ECMAScript 5 except
for strict mode.

Once you have downloaded, compiled, and installed Node, you
can run node programs with commands like this:

node program.js

We began the explanation of Rhino with its print() and load() functions. Node has similar features
under different names:

// Node defines console.log() for debugging output like browsers do.
console.log("Hello Node"); // Debugging output to console

// Use require() instead of load().  It loads and executes (only once) the
// named module, returning an object that contains its exported symbols.
var fs = require("fs");    // Load the "fs" module and return its API object

Node implements all of the standard ECMAScript 5 constructors,
properties, and functions in its global object. In addition, however,
it also supports the client-side timer functions set setTimeout(), setInterval(), clearTimeout(), and clearInterval():

// Say hello one second from now.
setTimeout(function() { console.log("Hello World"); }, 1000);

These client-side globals are covered in Timers. Node’s implementation is compatible with web
browser implementations.

Node defines other important globals under the process namespace. These are some of the
properties of that object:

process.version   // Node version string
process.argv      // Command-line args as an array argv[0] is "node"
process.env       // Enviroment variables as an object. e.g.: process.env.PATH
process.pid       // Process id
process.getuid()  // Return user id
process.cwd()     // Return current working directory
process.chdir()   // Change directory
process.exit()    // Quit (after running shutdown hooks)

Because Node’s functions and methods are asynchronous, they do
not block while waiting for operations to complete. The return value
of a nonblocking method cannot return the result of an asynchronous
operation to you. If you need to obtain results, or just need to know
when an operation is complete, you have to provide a function that
Node can invoke when the results are ready or when the operation is
complete (or when an error occurs). In some cases (as in the call to
setTimeout() above), you simply
pass the function as an argument and Node will call it at the
appropriate time. In other cases, you can rely on Node’s event
infrastructure. Node objects that generate events (known as event
emitters) define an on() method for registering handlers. Pass
the event type (a string) as the first argument, and pass the handler
function as the second argument. Different types of events pass
different arguments to the handler function, and you may need to refer
to the API documentation to know exactly how to write your
handlers:

emitter.on(name, f)          // Register f to handle name events from emitter
emitter.addListener(name, f) // Ditto: addListener() is a synonym for on()
emitter.once(name, f)        // One-time only, then f is automatically removed
emitter.listeners(name)      // Return an array of handler functions 
emitter.removeListener(name, f)  // Deregister event handler f
emitter.removeAllListeners(name) // Remove all handlers for name events

The process object shown
above is an event emitter. Here are example handlers for some of its
events:

// The "exit" event is sent before Node exits.
process.on("exit", function() { console.log("Goodbye"); });

// Uncaught exceptions generate events, if any handlers are registered.
// Otherwise, the exception just makes Node print an error and exit.
process.on("uncaughtException", function(e) { console.log(Exception, e); });// POSIX signals like SIGINT, SIGHUP and SIGTERM generate events 
process.on("SIGINT", function() { console.log("Ignored Ctrl-C"); });

Since Node is designed for high-performance I/O, its stream API
is a commonly used one. Readable streams trigger events when data is
ready. In the code below, assume s
is a readable stream, obtained elsewhere. We’ll see how to get stream
objects for files and network sockets below:

// Input stream s:
s.on("data", f);    // When data is available, pass it as an argument to f()
s.on("end", f);     // "end" event fired on EOF when no more data will arrive
s.on("error", f);   // If something goes wrong, pass exception to f()
s.readable          // => true if it is a readable stream that is still open
s.pause();          // Pause "data" events.  For throttling uploads, e.g.
s.resume();         // Resume again

// Specify an encoding if you want strings passed to "data" event handler
s.setEncoding(enc); // How to decode bytes: "utf8", "ascii", or "base64"

Writable streams are less event-centric than readable streams.
Use the write() method to send data
and use the end() method to close
the stream when all the data has been written. The write() method never blocks. If Node cannot
write the data immediately and has to buffer it internally, the
write() method returns false. Register a handler for “drain” events
if you need to know when Node’s buffer has been flushed and the data
has actually been written:

// Output stream s:
s.write(buffer);          // Write binary data
s.write(string, encoding) // Write string data. encoding defaults to "utf-8"
s.end()                   // Close the stream.
s.end(buffer);            // Write final chunk of binary data and close.
s.end(str, encoding)      // Write final string and close all in one
s.writeable;              // true if the stream is still open and writeable
s.on("drain", f)          // Call f() when internal buffer becomes empty

As you can see in the code above, Node’s streams can work with
binary data or textual data. Text is transferred using regular
JavaScript strings. Bytes are manipulated using a Node-specific type
known as a Buffer. Node’s buffers are fixed-length array-like objects
whose elements must be numbers between 0 and 255. Node programs can
often treat buffers as opaque chunks of data, reading them from one
stream and writing them to another. But the bytes in a buffer can be
accessed as array elements, and there are methods for copying bytes
from one buffer to another, for obtaining slices of an underlying
buffer, for writing strings into a buffer using a specified encoding,
and for decoding a buffer or a portion of a buffer back to a
string:

var bytes = new Buffer(256);          // Create a new buffer of 256 bytes
for(var i = 0; i < bytes.length; i++) // Loop through the indexes
    bytes[i] = i;                     // Set each element of the buffer
var end = bytes.slice(240, 256);      // Create a new view of the buffer
end[0]                                // => 240: end[0] is bytes[240]
end[0] = 0;                           // Modify an element of the slice
bytes[240]                            // => 0: underlying buffer modified, too
var more = new Buffer(8);             // Create a separate new buffer
end.copy(more, 0, 8, 16);             // Copy elements 8-15 of end[] into more[]
more[0]                               // => 248

// Buffers also do binary <=> text conversion
// Valid encodings: "utf8", "ascii" and "base64". "utf8" is the default.
var buf = new Buffer("2πr", "utf8");  // Encode text to bytes using UTF-8
buf.length                            // => 3 characters take 4 bytes
buf.toString()                        // => "2πr": back to text
buf = new Buffer(10);                 // Start with a new fixed-length buffer
var len = buf.write("πr²", 4);        // Write text to it, starting at byte 4
buf.toString("utf8",4, 4+len)         // => "πr²": decode a range of bytes

Node’s file and filesystem API is in the “fs” module:

var fs = require("fs"); // Load the filesystem API

This module provides synchronous versions of most of its
methods. Any method whose name ends with “Sync” is a blocking method
that returns a value or throws an exception. Filesystem methods that
do not end with “Sync” are nonblocking methods that pass their result
or error to the callback function you specify. The following code
shows how to read a text file using a blocking method and how to read
a binary file using the nonblocking method:

// Synchronously read a file. Pass an encoding to get text instead of bytes.
var text = fs.readFileSync("config.json", "utf8");

// Asynchronously read a binary file.  Pass a function to get the data
fs.readFile("image.png", function(err, buffer) {
    if (err) throw err;  // If anything went wrong
    process(buffer);     // File contents are in buffer
});

Similar writeFile() and
writeFileSync() functions exist for
writing files:

fs.writeFile("config.json", JSON.stringify(userprefs));

The functions shown above treat the contents of the file as a
single string or Buffer. Node also defines a streaming API for reading
and writing files. The function below copies one file to
another:

// File copy with streaming API.
// Pass a callback if you want to know when it is done
function fileCopy(filename1, filename2, done) {
    var input = fs.createReadStream(filename1);          // Input stream
    var output = fs.createWriteStream(filename2);        // Output stream
    
    input.on("data", function(d) { output.write(d); });  // Copy in to out
    input.on("error", function(err) { throw err; });     // Raise errors
    input.on("end", function() {                         // When input ends
        output.end();                                    // close output
        if (done) done();                                // And notify callback
    });
}

The “fs” module also includes a number of methods for listing
directories, querying file attributes, and so on. The Node program
below uses synchronous methods to list the contents of a directory,
along with file size and modification date:

#! /usr/local/bin/node
var fs = require("fs"), path = require("path");     // Load the modules we need
var dir = process.cwd();                            // Current directory
if (process.argv.length > 2) dir = process.argv[2]; // Or from the command line
var files = fs.readdirSync(dir);                    // Read directory contents
process.stdout.write("Name\tSize\tDate\n");         // Output a header
files.forEach(function(filename) {                  // For each file name
    var fullname = path.join(dir,filename);         // Join dir and name 
    var stats = fs.statSync(fullname);              // Get file attributes
    if (stats.isDirectory()) filename += "/";       // Mark subdirectories
    process.stdout.write(filename + "\t" +          // Output file name plus
                         stats.size + "\t" +        //   file size plus
                         stats.mtime + "\n");       //   modification time
});

Note the #! comment on the
first line above. This is a Unix “shebang” comment used to make a
script file like this self-executable by specifying what language
interpreter to run it with. Node ignores lines like this when they
appear as the first line of the file.

The “net” module is an API for TCP-based networking. (See the
“dgram” module for datagram-based networking.) Here’s a very simple
TCP server in Node:

// A simple TCP echo server in Node: it listens for connections on port 2000
// and echoes the client's data back to it.
var net = require('net');
var server = net.createServer();
server.listen(2000, function() { console.log("Listening on port 2000"); });
server.on("connection", function(stream) {
    console.log("Accepting connection from", stream.remoteAddress);
    stream.on("data", function(data) { stream.write(data); });
    stream.on("end", function(data) { console.log("Connection closed"); });
});

In addition to the basic “net” module, Node has built-in support
for the HTTP protocol using the “http” module. The examples that
follow demonstrate it in more detail.

Node Example: HTTP Server

Example 12-2 is a simple HTTP server in
Node. It serves files from the current directory and also implements
two special-purpose URLs that it handles specially. It uses Node’s
“http” module and also uses the file and stream APIs demonstrated
earlier. Example 18-17 in Chapter 18
is a similar specialized HTTP server example.

Example 12-2. An HTTP server in Node

// This is a simple NodeJS HTTP server that can serve files from the current
// directory and also implements two special URLs for testing.
// Connect to the server at http://localhost:8000 or http://127.0.0.1:8000

// First, load the modules we'll be using
var http = require('http');      // HTTP server API
var fs = require('fs');          // For working with local files

var server = new http.Server();  // Create a new HTTP server
server.listen(8000);             // Run it on port 8000.

// Node uses the "on()" method to register event handlers.
// When the server gets a new request, run this function to handle it.
server.on("request", function (request, response) {
    // Parse the requested URL
    var url = require('url').parse(request.url);

    // A special URL that just makes the server wait before sending the 
    // response. This can be useful to simulate a slow network connection.
    if (url.pathname === "/test/delay") {
        // Use query string for delay amount, or 2000 milliseconds
        var delay = parseInt(url.query) || 2000;
        // Set the response status code and headers
        response.writeHead(200, {"Content-Type": "text/plain; charset=UTF-8"});
        // Start writing the response body right away
        response.write("Sleeping for " + delay + " milliseconds...");
        // And then finish it in another function invoked later.
        setTimeout(function() { 
            response.write("done.");
            response.end();
        }, delay);
    }
    // If the request was for "/test/mirror", send back the request verbatim.
    // Useful when you need to see the request headers and body.
    else if (url.pathname === "/test/mirror") {
        // Response status and headers
        response.writeHead(200, {"Content-Type": "text/plain; charset=UTF-8"});
        // Begin the response body with the request 
        response.write(request.method + " " + request.url + 
                       " HTTP/" + request.httpVersion + "\r\n");
        // And the request headers
        for(var h in request.headers) {
            response.write(h + ": " + request.headers[h] + "\r\n");
        }
        response.write("\r\n");  // End headers with an extra blank line

        // We complete the response in these event handler functions:
        // When a chunk of the request body, add it to the response.
        request.on("data", function(chunk) { response.write(chunk); });
        // When the request ends, the response is done, too.
        request.on("end", function(chunk) { response.end(); });
    }
    // Otherwise, serve a file from the local directory.
    else {
        // Get local filename and guess its content type based on its extension.
        var filename = url.pathname.substring(1); // strip leading /
        var type;  
        switch(filename.substring(filename.lastIndexOf(".")+1))  { // extension
        case "html":
        case "htm":      type = "text/html; charset=UTF-8"; break;
        case "js":       type = "application/javascript; charset=UTF-8"; break;
        case "css":      type = "text/css; charset=UTF-8"; break;
        case "txt" :     type = "text/plain; charset=UTF-8"; break;
        case "manifest": type = "text/cache-manifest; charset=UTF-8"; break;
        default:         type = "application/octet-stream"; break;
        }
                
        // Read the file asynchronously and pass the content as a single
        // chunk to the callback function. For really large files, using the
        // streaming API with fs.createReadStream() would be better.
        fs.readFile(filename, function(err, content) {
            if (err) {  // If we couldn't read the file for some reason
                response.writeHead(404, {    // Send a 404 Not Found status
                    "Content-Type": "text/plain; charset=UTF-8"});
                response.write(err.message); // Simple error message body
                response.end();              // Done
            }
            else {      // Otherwise, if the file was read successfully.
                response.writeHead(200,  // Set the status code and MIME type
                                   {"Content-Type": type});
                response.write(content); // Send file contents as response body
                response.end();          // And we're done
            }
        });
    }
});

Node Example: HTTP Client Utilities Module

Example 12-3 uses the “http” module to
define utility functions for issuing HTTP GET and POST requests. The
example is structured as an “httputils” module, which you might use
in your own code like this:

var httputils = require("./httputils");  // Note no ".js" suffix
httputils.get(url, function(status, headers, body) { console.log(body); });

The require() function does not execute module
code with an ordinary eval().
Modules are evaluated in a special environment so that they cannot
define any global variables or otherwise alter the global namespace.
This special module evaluation environment always includes a global
object named exports. Modules
export their API by defining properties in this object.[24]

Example 12-3. Node “httputils” module

//
// An "httputils" module for Node.
//

// Make an asynchronous HTTP GET request for the specified URL and pass the
// HTTP status, headers and response body to the specified callback function.
// Notice how we export this method through the exports object.
exports.get = function(url, callback) {  
    // Parse the URL and get the pieces we need from it
    url = require('url').parse(url);
    var hostname = url.hostname, port = url.port || 80;
    var path = url.pathname, query = url.query;
    if (query) path += "?" + query;

    // Make a simple GET request
    var client = require("http").createClient(port, hostname);
    var request = client.request("GET", path, { 
        "Host": hostname    // Request headers
    }); 
    request.end();

    // A function to handle the response when it starts to arrive
    request.on("response", function(response) {
        // Set an encoding so the body is returned as text, not bytes
        response.setEncoding("utf8");
        // Save the response body as it arrives
        var body = ""
        response.on("data", function(chunk) { body += chunk; });
        // When response is complete, call the callback
        response.on("end", function() {
            if (callback) callback(response.statusCode, response.headers, body);
        });
    });
};

// Simple HTTP POST request with data as the request body
exports.post = function(url, data, callback) {
    // Parse the URL and get the pieces we need from it
    url = require('url').parse(url);
    var hostname = url.hostname, port = url.port || 80;
    var path = url.pathname, query = url.query;
    if (query) path += "?" + query;

    // Figure out the type of data we're sending as the request body
    var type;
    if (data == null) data = "";
    if (data instanceof Buffer)             // Binary data
        type = "application/octet-stream";
    else if (typeof data === "string")      // String data
        type = "text/plain; charset=UTF-8";
    else if (typeof data === "object") {    // Name=value pairs
        data = require("querystring").stringify(data);
        type = "application/x-www-form-urlencoded";
    }

    // Make a POST request, including a request body
    var client = require("http").createClient(port, hostname);
    var request = client.request("POST", path, {
        "Host": hostname,       
        "Content-Type": type
    });
    request.write(data);                        // Send request body
    request.end();       
    request.on("response", function(response) { // Handle the response
        response.setEncoding("utf8");           // Assume it is text
        var body = ""                           // To save the response body
        response.on("data", function(chunk) { body += chunk; });
        response.on("end", function() {         // When done, call the callback
            if (callback) callback(response.statusCode, response.headers, body);
        });
    });
};


[23] Client-side JavaScript is also highly asynchronous and
event-based, and the examples in this section may be easier to
understand once you have read Part II and have
been exposed to client-side JavaScript programs.

[24] Node implements the CommonJS module contract, which you
can read about at http://www.commonjs.org/specs/modules/1.0/.

Comments are closed.

loading...